-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
imu data hangs on Raspberry pi 4B #11089
Comments
Hi @Yoki-pjx Your accel and gyro frequency problem is unusual, as your frequency values for each are correct. Couldn't resolve requests means that the requested configuration could not be provided, either because of a typing mistake in the instruction or because the requested configuration was not supported by the camera at the time that it was requested. Neither of those possibilities seems to apply here. 63 accel and 200 gyro should be the default values applied by an IMU-equipped 400 Series camera anyway though when custom values are not defined, so leaving out the frequencies should be fine in this particular case. In regard to the hanging of the program after a couple of minutes, may I ask which method you used to install the RealSense SDK on your Pi 4, please? Kernel 5.10 is not officially supported by the SDK and whilst unsupported kernels can work, there may be unpredictable consequences in regards to stability. I would therefore recommend installing the SDK using the RSUSB backend method if you have not done so already. This method bypasses the Linux kernel and so is not dependent on Linux versions or kernel versions and does not require patching. You could also try increasing the size of the swapfile on your Pi so that it has more 'virtual memory' to use when its real memory is used up. |
I have tried reinstalling the SDK by the RSUSB backend method as https://github.com/IntelRealSense/librealsense/issues/6940#issuecomment-665713929. However, it doesn't work. Both issues still exist. I got 'RuntimeError: Frame didn't arrive within 5000' after a couple of minutes and 'RuntimeError: Couldn't resolve requests with pipeline.start(config)' with python. Just double check with you showing how I installed by the RSUSB backend method below:
I tried to increase the size of the swapfile to 2048 as well. |
Thanks very much for the updates. Your RSUSB installation looks fine to me. It is unusual that your Viewer is showing the supported IMU frequencies as 100 / 200, as those modes are usually provided on the L515 camera model which also has an IMU. The camera is clearly detected by the Viewer as a D455 model though. From my own cameras: D455 L515 In regard to the error RuntimeError: Frame didn't arrive within 5000', this indicates that new frames have not been received from the camera for a period of 5 seconds and so the program has 'timed out', generating the error. Does it still occur in the Viewer if you disable the Global Time Enabled option? Global Time is enabled by default on 400 Series cameras, so would likely be active on your Python script too. |
|
I will consult with my Intel RealSense colleagues about your IMU settings on D455. Thanks very much for your patience! |
Okay, thank you Marty. I have tried without the Global Time Enabled option in the viewer. The problem seems still occur but seems it happens later. What's the code to disable the Global Time Enabled option in python? I also would like to have a try in python. |
An example of Python code for disabling Global Time is at #9172 (comment) |
Thanks Marty. |
I checked the data sheet document for the 400 Series cameras. https://dev.intelrealsense.com/docs/intel-realsense-d400-series-product-family-datasheet The D455 model was originally equipped with the BMI055 IMU component with 63/250 frequencies but it later changed to the BMI085 IMU component which supports 100/200 frequencies. So your D455 hardware is okay and the time-out issue is likely due to either a software-related problem or with using accel / gyro on Pi 4. |
I have tried to disable the Global Time Enabled in python, but it still got a hanging-on problem. I also tried to use There is no frame arriving at variable The result of So I am thinking why the imu_frames is NULL here? |
Does your Python IMU script assign fixed index numbers to accel and gyro, such as 0 for accel and 1 for gyro? For example:
If it does assign fixed index numbers to accel and gyro then it is recommended that the streams are assigned dynamically - as described at #4018 (comment) and #4018 (comment) - otherwise the output of the streams can become reversed so that accel values are output for gyro and gyro values are output for accel. |
No, they are dynamically assigned as the following codes:
|
poll_for_frames() could in itself contribute to performance problems, as it is recommended to manually control when the CPU is put to sleep and for how long when using this instruction. Otherwise, the CPU percentage utilization may rise towards 100% as described at #2422 (comment) This link also describes how using try_wait_for_frames() causes a 'false' state to be returned and the program being allowed to continue to run if a timeout occurs instead of a program-exiting error being generated. Does the problem still occur if you use 200 accel and 400 gyro instead of 100 / 200? |
I got error
|
#9800 (comment) has an example of using poll_for_frames() and time.sleep. In the header of the script beneath 'import pyrealsense2 as rs', include the instruction import time |
I am also considering whether I can count the number of null frames, then stop the pipeline, and start it again. Is there any function like restarting the pipeline? |
Hi @Yoki-pjx My apologies for the delay in responding further. The pipeline can be restarted using the hardware_reset() instruction, and its activation can be made conditional on a certain condition being satisfied (for example: If null frame count is greater than a certain value then reset). A Python example of hardware_reset() for a single camera is below.
Here is another example that resets a camera with a particular serial number.
|
I made a variable to count all the NULL frames during the period. When the variable is over 50, I stop the pipeline, then reset the camera, and restart the pipelines, But I got the error after several minutes:
Does Too many open files mean the pipeline creates a new USB connection to the Pi each time? |
In general, when a hardware reset occurs the camera disconnects and then reconnects, which takes 2-3 seconds for the entire disconnect-reconnect process from start to finish. If the reconnection occurs within 5 seconds of the disconnect then the current pipeline should remain open and continue from the point that it left off. So you should not need a pipeline start instruction after |
Is there any way to avoid the 2-3 seconds reconnection? Because the system only withstands 1-second reconnection. |
There is no way to make the camera reset process complete faster, unfortunately. Even if the entire USB port was reset with a Linux bash script instead of just the camera with the SDK's hardware_reset() instruction, it would take around the same time for the camera to reboot (since a hardware reset acts in the same way as physically unplugging the camera and plugging it back in). |
I'm a bit broken up about this device. I also install Ubuntu 20.04.5 LTS (Kernel version 5.4) (which is mentioned in the note of the SDK) from the Raspberry official imager on my Pi today. It still occurs the error that does not receive frames in the realsense-viewer. Does it means there is compatibility problem with Pi and the camera? I also found it works (no error) with the action of stop and start the imu pipeline when I only open the imu pipeline. When two pipelines are working, the error |
The librealsense SDK can be installed for Ubuntu on Raspberry Pi but there can sometimes be complications compared to installing on a PC. The link below has an example of a guide for installing on Pi 4 and Ubuntu 20.04. https://ramith.fyi/setting-up-raspberry-pi-4-with-ubuntu-20-04-ros-intel-realsense/ A RealSense user shared at the link below the method that worked for them with 20.04. Another guide: Yet another approach is to install on the Raspbian / Raspberry Pi OS instead of Ubuntu, like in the official Intel guide here: https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_raspbian.md The install method that has been the most successful for Pi boards in the past has been to build librealsense from source code with the LibUVC backend or the RSUSB backend method, which bypasses the kernel and so is not dependent on Linux versions or kernel versions and does not require kernel patching. |
I installed the SDK by the LibUVC backend or the RSUSB backend method in Ubuntu 20.04. The RGB and depth streams work, and the imu still has problem as I mentioned in the last comment. So I am thinking does it means there is a compatibility problem between Pi and the camera? I also found it works (no error) with the action of stop and start the imu pipeline when I only open the imu pipeline. When two pipelines are working, the error libusb: error [_get_usbfs_fd] libusb couldn't open USB device /dev/bus/usb/002/008: Too many open files occurs when I restart the imu pipeline. I think I should go in this direction to fix the problem..Why it works when I stop and start the imu pipeline? |
In regard to opening one pipeline or two pipelines, it sounds as though you are referring to the different types of streams, and all of the streams are opened in the same pipeline. When three types of streams are enabled in the same pipeline and one of those stream types is IMU, it can cause one of the stream types to stop receiving frames. It does not matter which order the streams are enabled in, as the trigger for the problem is the enabling of IMU. A workaround for this in Python is to actually create two separate pipelines, with the depth and color streams placed on one pipeline and the IMU placed alone on the other pipeline. A Python example script for doing so is at #5628 (comment) |
I apologise if I make you unclear. I list my codes here:
This one above occurs the error (
This one works without any errors, but I cannot use the image stream. I think I should go in this direction to fix the problem.. |
I have not seen that 'Too many files' error before with RealSense and so do not have advice to offer about it unfortunately, though it does also occur with non-RealSense cases, as shown by googling for the term raspberry pi too many open files In the second script that does work but does not provide image data, that is because you do not have any code for accessing the depth and color streams in it, right? In the first script, it seems unusual to me to have two imshow instructions one after another, What happens if you comment out the infrared image line?
|
What happens if you reverse the order of the stop() instructions so pipeline stops first and imu_pipeline stops second? Does it also make a difference if you put a sleep instruction between the stop() instructions to delay the stopping of the second pipeline for a short time? |
Hi @Yoki-pjx Do you require further assistance with this case, please? Thanks! |
Hi @MartyG-RealSense , |
At #10238 (comment) I tracked down another Python IMU + depth + color script that uses threading as its approach to resolving the problem of enabling three streams. |
Hi @Yoki-pjx Do you have an update about this case that you can provide, please? Thanks! |
Hi @MartyG-RealSense, |
I have highlighted this case to my Intel RealSense colleagues. Thanks very much @Yoki-pjx |
After discussion of the Raspberry Pi IMU issue with my Intel RealSense colleagues, it was decided that we will keep this issue open to give further Raspberry Pi users who have this problem the opportunity to add comments about their own experiences. Thanks very much for your report! |
Issue Description
Hi Marty @MartyG-RealSense,
I have the same problem as (https://github.com/IntelRealSense/librealsense/issues/7391) and https://github.com/IntelRealSense/librealsense/issues/7979 IMU data hangs on after a couple of minutes even though I have only one pipeline to gather IMU data.
Both of the issues actually did not fix the problem, I would like to enquire about any solutions for this.
Also, I have another problem with setting the acceleration and acceleration frequency.
config.enable_stream(rs.stream.accel, rs.format.motion_xyz32f, 63)
config.enable_stream(rs.stream.gyro, rs.format.motion_xyz32f, 200)
pipeline.start(config)
I got RuntimeError: Couldn't resolve requests with pipeline.start(config), BUT, it works when I deleted the frequency as
config.enable_stream(rs.stream.accel, rs.format.motion_xyz32f)
config.enable_stream(rs.stream.gyro, rs.format.motion_xyz32f)
pipeline.start(config)
Could you please provide some solutions?
The text was updated successfully, but these errors were encountered: