Frames freezing and dropping when recording with Realsense Viewer

[realsens01]

Required Info
Camera Model	D435
Firmware Version	5.11.1
Operating System & Version	Windows 10
Kernel Version (Linux Only)	NA
Platform	PC
SDK Version	2.20.0
Language	Python
Segment	NA

Issue Description

This issue was closed but never resolved for me. I've tested it with the latest firmware and Realsense Viewer. I still see the frame drops when using Realsense Viewer, but not when recording via the Python API. The Realsense Viewer

I disabled autoexposure and still see the freezing of the frame displays in Realsense Viewer.

This might be related to #3708.

[gunhaxxor]

I have the same issue. Dropping frames on recording. I have not had the issue before. It came recently.

[gunhaxxor]

I was able to verify this by also compiling and running the record-playback sample from the c++ sdk. I added two lines to get specific settings from the sensors:

cfg.enable_stream(RS2_STREAM_DEPTH, -1, 1280, 720, RS2_FORMAT_Z16, 30);
cfg.enable_stream(RS2_STREAM_COLOR, -1, 1280, 720, RS2_FORMAT_RGB8, 30);

This build records fine without any sign of dropping frames. I can also playback the recorded file in real sense viewer without any problem.
Using the same settings and recording in the real sense viewer starts dropping frames after a few seconds.

So I can verify that this is completely or partially a software related problem and not a limitation in performance on the machine.

[realsens01]

Thanks @Dealerpriest. This issue was never resolved. I abandoned use of Realsense Viewer.

[gunhaxxor]

I can also verify that this happens when using the demo scenes in the Unity package.

Please address this issue Intel. As of now I'm completely unable to record reliably using c# and unity.
My best option right now seems be to create two separate programs, in different programming environments. One c++ solution for recording and one unity/c# project for playback in VR. That's not what I would consider a smooth developer experience 😢
@RealSense-Customer-Engineering @RealSenseCustomerSupport @dorodnic
Grateful for any feedback on this!

[RealSenseCustomerSupport]

---

@realsens01 We found that this issue might be related to the PC performance. We tried several laptops and did get frame freezing issue on some lower performance PCs, but didn't get such issue on some better performance PCs. Is it possible for you to try on some better performance PCs to see if get this through?

@Dealerpriest If you're getting issue in Unity, I suggest to create another issue to address it. Thanks!

[realsens01]

@RealSenseCustomerSupport: Define "better performance PCs". If you can tell me what needs to perform better, and how much better, then this seems like a reasonable response.

[RealSenseCustomerSupport]

---

@realsens01 We did some tests on i7-8665U CPU@1.90GHz with 4 cores and didn't get freezing issue. Could you please find any PC with comparable configuration to try again? Thanks!

[realsens01]

My original post thread (you'll have to look back a year or two) listed the stats of the computer having the issue. 8th Generation Core i7-8700K. 6 cores. Dell. Pretty standard computer. None of the resource monitors show any issues.

[RealSenseCustomerSupport]

---

@realsens01 Did you get any chance to try https://github.com/IntelRealSense/librealsense/tree/master/tools/recorder to see if any frames freezing or dropping?

[realsens01]

To what end?

Have you looked through the post history?

[RealSenseCustomerSupport]

---

@realsens01 The clue is a bit long, sorry if I missed some information. So I understood you didn't have issue with the https://github.com/IntelRealSense/librealsense/tree/master/tools/recorder but with RealSenseViewer. Could you please check if you enable compressed mode or not in RealSenseViewer? You can click the setting button on the upper right of RealSenseViewer GUI and check "Playback&Record" -> ROS-bag compression. When compressed mode enabled, then it might cause frame drops. Thanks!

[RealSenseCustomerSupport]

---

@realsens01 Any update from your side? Looking forward to your update.

[realsens01]

I do not currently have access to the data collection equipment (including the cameras), due to a global pandemic.

Yes. The post history is long and convoluted. It's a bit hard to follow. Mostly, I think, that is because my issues were repeatedly closed or deflected. This issue dates back to November of 2018. I've done a lot of work to debug it, and it's generally pretty safe to assume that I tested the suggestions in the history.

I saw the posts about compression. I'm about 99% certain that I ruled this out at the time it was posted, in February of 2019, but I am not going to track down any of my posts about compression. You are welcome to if you want to verify. I'll note that the post about compression suggest that it is tied to CPU utilization, and my post in the same thread notes that CPU utilization is only about 30% in my case.

[eidetic-av]

I was struggling with this problem today, and for me the issue was related to the SSD not the CPU. I was using an NVMe SSD but every attempt at recording to a bag file resulted in arbitrary dropped frames.

I solved it by freeing up some space on the SSD! I have a 256gb SSD to record to and the frame drops only occur when the free space on the drive is less than ~20gb. Even though my bag recordings are less than 4gb, it seems like the performance of the drive suffers greatly when the free space is limited.

It might not solve your issue but it's something to note for anyone else experiencing the same thing.

[RealSenseCustomerSupport]

---

@eidetic-av Thanks very much for your sharing!
@realsens01 Could you please check if eidetic-av's sharing help on the issue? Thanks!

[RealSenseCustomerSupport]

---

@realsens01 Any update from your side? Thanks!

[realsens01]

No. The issue did not seem to be related to hard drive space.

Just curious: Any idea why it would be?

[eidetic-av]

When you’re working with a file, and you’re ready to save it, the solid-state drive finds the necessary empty blocks and writes your data for long term storage. The process of filling an empty block with data is the fastest way to write to the storage...
solid-state drives can only write data in pages of 4KB or 8KB inside of a 256KB block, you end with blocks that are not completely filled...
The solution to this problem is a write operation that loads the data inside of a block into cache, modifies its content adding the new data, and then write pages back to the block.

From https://pureinfotech.com/why-solid-state-drive-ssd-performance-slows-down/

So I guess when the storage is low it turns one operation into three to perform the same write.

[RealSenseCustomerSupport]

---

@realsens01 From our understanding, this issue is more related to PC host itself, not in librealsense scope. Can we close it accordingly? Thanks!

[realsens01]

Can you explain how it is related to the PC hardware? My PC is a standard PC from a standard manufacturer -- brand new at the time the project started and purchased solely to record data. If you dig through my extensive comment history, you'll see that I've considered the possibility that the problem is hardware related, but that evidence suggests otherwise. For example, from my original post here:

I still see the frame drops when using Realsense Viewer, but not when recording via the Python API

I've sunk a lot of time into debugging the Realsense product. It has had a lot of problems.

[RealSenseCustomerSupport]

---

@realsens01 Sorry for the trouble. Did you also start image preview when record with Python API? And according to #2216 (comment) The user confirmed that the frame freezing and dropping issue happened when compression mode enabled in RealSenseViewer. So could you please double check if you still have issue when compression mode disabled? Thanks!

[RealSenseSupport]

@realsens01 Could you please try with the latest RealSenseViewer to see if this is still issue for you? Thanks!

[RealSenseSupport]

@realsens01 Sorry that we didn't hear from you for weeks. Will close this ticket in a week if no other questions. Thanks!