NullPointerException: "frame" is null #41

cacheflowe · 2023-10-13T15:26:44Z

Hello!

I'm getting this error consistently on Windows 11 with the latest Processing, the new realsense-processing library (2.5.0), and the D415 with the latest firmware. Is there any way to override and protect against this error?

I'm running the Realsense update in a Thread, described below. I only start a thread if the previous frame has completed. I've used this code for a long time across projects but am seeing lots of crashes on my current project.

java.lang.NullPointerException: Cannot invoke "org.intel.rs.frame.DepthFrame.release()" because "frame" is null
        at ch.bildspur.realsense.RealSenseCamera.getDepthData(RealSenseCamera.java:625)
        at com.haxademic.core.hardware.depthcamera.cameras.RealSenseWrapper$1.run(RealSenseWrapper.java:141)
        at java.base/java.lang.Thread.run(Thread.java:833)

Here's my code:

Thread updateThread = new Thread(new Runnable() { public void run() {
	threadBusy = true;
	try {
		camera.readFrames();
	} catch (NullPointerException e) {
		P.out("RealSenseWrapper failed to update");
	}
	data = camera.getDepthData();
	threadBusy = false;
}});
updateThread.start();

I also start seeing other errors after a while:

org.intel.rs.util.RealSenseException: Object does not support "librealsense::composite_frame" interface!
        at org.intel.rs.util.RealSenseError.checkError(RealSenseError.java:26)
        at org.intel.rs.util.RealSenseError.checkError(RealSenseError.java:17)
        at org.intel.rs.frame.FrameList.count(FrameList.java:26)
        at org.intel.rs.util.NativeListIterator.<init>(NativeListIterator.java:13)
        at org.intel.rs.frame.FrameList.iterator(FrameList.java:111)
        at org.intel.rs.frame.FrameList.getFirstOrDefault(FrameList.java:72)
        at org.intel.rs.frame.FrameList.getFirstOrDefault(FrameList.java:68)
        at org.intel.rs.frame.FrameList.getColorFrame(FrameList.java:88)
        at ch.bildspur.realsense.RealSenseCamera.readFrames(RealSenseCamera.java:339)
        at com.haxademic.core.hardware.depthcamera.cameras.RealSenseWrapper$1.run(RealSenseWrapper.java:135)
        at java.base/java.lang.Thread.run(Thread.java:833)

Thank you!

cansik · 2023-10-13T15:31:32Z

Thanks for reporting, is this only happening in the most recent version?

cacheflowe · 2023-10-13T15:40:38Z

@cansik It was happening with the last version, so I checked here and saw the update, so I tried upgrading the realsense-processing jar. But the behavior is similar. The first error always seems to be the null frame in the library code here:

   public short[][] getDepthData() {
      this.checkRunning();
      DepthFrame frame = this.frames.getDepthFrame(); // <- this is null
      this.depthStream.updateDepthData(frame);
      frame.release(); // <- and NPE happens here
      return this.depthStream.getData();
   }

But the second error noted above has been happening more with the new library. I might be seeing less hard crashing now, which was coming with an error log, found below (this was with the older version - I'm testing the new one right now)

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffb186d4067, pid=5176, tid=5568
#
# JRE version: OpenJDK Runtime Environment Temurin-17.0.8.1+1 (17.0.8.1+1) (build 17.0.8.1+1)
# Java VM: OpenJDK 64-Bit Server VM Temurin-17.0.8.1+1 (17.0.8.1+1, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
# Problematic frame:
# C  0x00007ffb186d4067
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# If you would like to submit a bug report, please visit:
#   https://github.com/adoptium/adoptium-support/issues
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

---------------  S U M M A R Y ------------

Command Line: -Xmx16G -Xms8G -Djava.library.path=lib\processing-4\core\library\windows-amd64;lib\processing-4\libraries\video\library\windows-amd64 com.client.Game

Host: 12th Gen Intel(R) Core(TM) i7-12700K, 20 cores, 31G,  Windows 11 , 64 bit Build 22621 (10.0.22621.2361)
Time: Fri Oct 13 07:54:48 2023 Pacific Daylight Time elapsed time: 187.742690 seconds (0d 0h 3m 7s)

---------------  T H R E A D  ---------------

Current thread (0x000001ebd958ca70):  JavaThread "Thread-5934" [_thread_in_native, id=5568, stack(0x00000050c5200000,0x00000050c5300000)]

Stack: [0x00000050c5200000,0x00000050c5300000],  sp=0x00000050c52ff090,  free space=1020k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  0x00007ffb186d4067

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 4610  org.bytedeco.librealsense2.global.realsense2.rs2_release_frame(Lorg/bytedeco/librealsense2/rs2_frame;)V (0 bytes) @ 0x000001eb4381397a [0x000001eb43813920+0x000000000000005a]
J 6784 c2 org.intel.rs.frame.FrameList.getFirstOrDefault(Lorg/intel/rs/types/Stream;Lorg/intel/rs/types/Format;)Lorg/intel/rs/frame/Frame; (70 bytes) @ 0x000001eb437ad410 [0x000001eb437aca60+0x00000000000009b0]
J 7055 c2 ch.bildspur.realsense.RealSenseCamera.readFrames()V (338 bytes) @ 0x000001eb43c0ac08 [0x000001eb43c09380+0x0000000000001888]
J 7048 c2 java.lang.Thread.run()V java.base@17.0.8.1 (17 bytes) @ 0x000001eb43c03748 [0x000001eb43c03680+0x00000000000000c8]
v  ~StubRoutines::call_stub

siginfo: EXCEPTION_ACCESS_VIOLATION (0xc0000005), reading address 0x0000000000000000

cansik · 2023-10-13T15:45:37Z

Ok, I hope I can have a look at it this weekend.

cacheflowe · 2023-10-13T16:53:24Z

This may seem a little crazy, but I've added a bunch of defensive checks for null frame data in this commit. This has made a big difference so far! After switching to my updated RealSenseProcessing.jar, It still seems like there can be an occasional error noted in my original Issue ☝️ , but they're almost completely gone, especially for how many errors I was seeing!

Thank you for making this library so easy to build, and for all of your work 🙏 Everything seems to be working normally - let me know if you'd want a PR.

Maybe I'm only seeing these errors because I run the Realsense updates in a thread? I believe that I have to use a Thread to keep my app running at 60fps. For some reason this is mostly only happening on the PC that I've launched the project on - I don't see many of these errors on my own PC, but I have noticed them occasionally on past projects.

cacheflowe · 2023-10-13T16:56:07Z

Also, I wish I knew which null check solved my problem. I started small, in the getDepthData() function, but it kept happening, and I needed to quickly find a solution for my project, so I checked for all of the frame references 🙈 So maybe this wouldn't be a great PR 🤷

cacheflowe · 2023-10-13T19:01:49Z

Okay, after a bit of observing, the frame == null exception hasn't happened again 🙌 I've seen a few more of the following error, but I haven't had any hard crashes 🤞

org.intel.rs.util.RealSenseException: Object does not support "librealsense::composite_frame" interface!

I've also caught several NullPointerExceptions using this portion of my code, but my app keeps running 🙌 I'll try to find a stack trace for these and maybe add another null check in the library code.

try {
	camera.readFrames();
} catch (NullPointerException e) {
	P.out("RealSenseWrapper failed to update");
}

To be continued...

cansik · 2023-10-13T21:46:11Z

Wow thanks for the in-depth investigation, a PR would be great of course. I would be happy to review and merge it.

cacheflowe · 2023-10-20T17:39:52Z

I was still getting this error relatively frequently, but was handled without problem by a try/catch:

java.lang.NullPointerException: Cannot invoke "org.intel.rs.frame.VideoFrame.getWidth()" because "frame" is null
        at ch.bildspur.realsense.stream.VideoRSStream.copyPixels(VideoRSStream.java:32)
        at ch.bildspur.realsense.RealSenseCamera.readFrames(RealSenseCamera.java:340)
        at com.haxademic.core.hardware.depthcamera.cameras.RealSenseWrapper$1.run(RealSenseWrapper.java:135)
        at java.base/java.lang.Thread.run(Thread.java:833)

I've looked into it and added better null checks. I'm testing now and haven't seen any errors yet. I've pushed another commit and started a PR. Let me know what you think, and thank you!

cansik · 2023-10-21T08:55:50Z

Thanks for the PR, could you give the released version a try and tell me if it works now as expected?
https://github.com/cansik/realsense-processing/releases/tag/v2.5.1

cacheflowe · 2023-10-22T11:53:53Z

I have switched to this official release in my project that was previously seeing many errors each day. I've now run for 24 hours with zero errors 🎉

I've also tested some of my local demos just to make sure other features look right. I'll let you know if any issues come up.

Thank you for this library and your continued support! I couldn't build my projects without it 🥇

cansik · 2023-10-23T19:42:20Z

@cacheflowe That sounds great, I have updated the contribution link as well 🙌🏻 Looking forward to your new projects ✨

cansik added the bug Something isn't working label Oct 13, 2023

cacheflowe mentioned this issue Oct 20, 2023

Add defensive checks for null frame #42

Merged

cansik self-assigned this Oct 21, 2023

cansik closed this as completed Oct 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NullPointerException: "frame" is null #41

NullPointerException: "frame" is null #41

cacheflowe commented Oct 13, 2023 •

edited

Loading

cansik commented Oct 13, 2023

cacheflowe commented Oct 13, 2023

cansik commented Oct 13, 2023

cacheflowe commented Oct 13, 2023

cacheflowe commented Oct 13, 2023

cacheflowe commented Oct 13, 2023

cansik commented Oct 13, 2023

cacheflowe commented Oct 20, 2023

cansik commented Oct 21, 2023

cacheflowe commented Oct 22, 2023

cansik commented Oct 23, 2023

NullPointerException: "frame" is null #41

NullPointerException: "frame" is null #41

Comments

cacheflowe commented Oct 13, 2023 • edited Loading

cansik commented Oct 13, 2023

cacheflowe commented Oct 13, 2023

cansik commented Oct 13, 2023

cacheflowe commented Oct 13, 2023

cacheflowe commented Oct 13, 2023

cacheflowe commented Oct 13, 2023

cansik commented Oct 13, 2023

cacheflowe commented Oct 20, 2023

cansik commented Oct 21, 2023

cacheflowe commented Oct 22, 2023

cansik commented Oct 23, 2023

cacheflowe commented Oct 13, 2023 •

edited

Loading