USB, hwmon, fatal and inconsistent errors using D415, D435i and T265 configurations

[hperik1]

Multiple issues are keep popping up using single and mixed configurations of T265, D415 and D435i. Even not stable starting up using same launch files in a row experienced.
Below a summary of received errors seen on multiple PCs and different librealsense, wrapper and even D415 and D435i firmware versions (5.11.4, 5.11.6.200).
I throw some errors I encountered, but more specific data (screen logs) are available for different configurations ... on request.
Hopefully somebody can already indicate if there is an obvious solution or give me a pointer on getting this resolved as for now a reliable system can not be configured ....

System setup

Ubuntu .... 16.04.1
Kernel .... 4.15.0-47-generic
CPU .... x86_64 / i7-8700K (3.70GHzx12)
D415, D435i .... 5.11.6.200
T265 .... 0.0.18.5448
librealsense .... 2.20.0
realsense wrapper .... 2.2.3

USB related:

[8024] [E] Device-CF40: Error: Failed to claim USB interface. LIBUSB_ERROR_CODE: 0xFFFFFFFA (LIBUSB_ERROR_BUSY)

[ERROR] [1555053223.386618234]: The requested device with serial number 845412111141 is NOT found!

When a launch failed (e.g. rs_t265.launch), running usb-devices in a different terminal and restarting the launch results in a working configuration ... despite hwmon errors and a "Frame meta data isn't available!" warning.

After plugin, the t265 typically enumerates as USB2 device. After a launch the t265 is set as a USB3 device, however no driver is assigned.

hwmon related:

1st-case:

WARNING [139736838567808] (types.cpp:59) hwmon command 0x10 failed. Error type: (48).
WARNING [139736838567808] (rs.cpp:279) null pointer passed for argument "device"
[ERROR] [1555053638.241224667]: An exception has been thrown: hwmon command 0x10 failed. Error type: (48).
[FATAL] [1555053638.241654547]: Failed to load nodelet '/D415/realsense2_cameraof typerealsense2_camera/RealSenseNodeFactoryto managerrealsense2_camera_manager'
WARNING [140485033314176] (types.cpp:59) hwmon command 0x4f failed. Error type: No data to return (-21).

2nd-case:

WARNING [140632521561984] (types.cpp:59) hwmon command 0x30 failed. Error type: (16).
WARNING [140632521561984] (rs.cpp:279) null pointer passed for argument "device"
[ERROR] [1555053223.170069523]: An exception has been thrown: hwmon command 0x30 failed. Error type: (16).
[FATAL] [1555053223.170589101]: Failed to load nodelet '/D435i/realsense2_cameraof typerealsense2_camera/RealSenseNodeFactoryto managerrealsense2_camera_manager'

3rd-case:

WARNING [140631724783360] (types.cpp:59) hwmon command 0x4f failed. Error type: No data to return (-21).

media format related:

WARNING [140485033314176] (sensor.cpp:366) Unregistered Media formats : [ UYVY ]; Supported: [ ]

WARNING [140048991586176] (sensor.cpp:366) Unregistered Media formats : [ W10 ]; Supported: [ ]

Sudden dead of processes:

[D415/realsense2_camera_manager-2] process has died [pid 15863, exit code -11, cmd /opt/ros/kinetic/lib/nodelet/nodelet manager __name:=realsense2_camera_manager __log:=.../D415-realsense2_camera_manager-2.log].
log file: .../D415-realsense2_camera_manager-2*.log
[D415/realsense2_camera-3] process has died [pid 15864, exit code 255, cmd /opt/ros/kinetic/lib/nodelet/nodelet load realsense2_camera/RealSenseNodeFactory realsense2_camera_manager __name:=realsense2_camera __log:=.../D415-realsense2_camera-3.log].
log file: .../D415-realsense2_camera-3*.log

Not found devices:

[ERROR] [1555053223.386618234]: The requested device with serial number 845412111141 is NOT found!

Note that lsusb and usb-devices list the connected devices:
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 5000M
|__ Port 3: Dev 2, If 0, Class=Video, Driver=uvcvideo, 5000M
|__ Port 3: Dev 2, If 1, Class=Video, Driver=uvcvideo, 5000M
|__ Port 3: Dev 2, If 2, Class=Video, Driver=uvcvideo, 5000M
|__ Port 3: Dev 2, If 3, Class=Video, Driver=uvcvideo, 5000M
|__ Port 3: Dev 2, If 4, Class=Video, Driver=uvcvideo, 5000M
|__ Port 3: Dev 2, If 5, Class=Human Interface Device, Driver=usbhid, 5000M
|__ Port 4: Dev 5, If 0, Class=Vendor Specific Class, Driver=, 5000M
|__ Port 5: Dev 3, If 0, Class=Hub, Driver=hub/4p, 5000M
|__ Port 4: Dev 6, If 3, Class=Video, Driver=uvcvideo, 5000M
|__ Port 4: Dev 6, If 1, Class=Video, Driver=uvcvideo, 5000M
|__ Port 4: Dev 6, If 4, Class=Video, Driver=uvcvideo, 5000M
|__ Port 4: Dev 6, If 2, Class=Video, Driver=uvcvideo, 5000M
|__ Port 4: Dev 6, If 0, Class=Video, Driver=uvcvideo, 5000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/16p, 480M
|__ Port 5: Dev 3, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 1: Dev 5, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
|__ Port 1: Dev 5, If 1, Class=Human Interface Device, Driver=usbhid, 1.5M
|__ Port 2: Dev 7, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
|__ Port 8: Dev 4, If 0, Class=Vendor Specific Class, Driver=cp210x, 12M
|__ Port 10: Dev 6, If 0, Class=Wireless, Driver=btusb, 12M
|__ Port 10: Dev 6, If 1, Class=Wireless, Driver=btusb, 12M

During working with realsense-viewer:

WARNING [139912287127296] (types.cpp:59) hwmon command 0x4f failed. Error type: No data to return (-21).
WARNING [139912287127296] (sensor.cpp:366) Unregistered Media formats : [ UYVY ]; Supported: [ ]
WARNING [139912562133568] (sensor.cpp:366) Unregistered Media formats : [ W10 ]; Supported: [ ]
WARNING [139911688603392] (backend-hid.cpp:662) HID set_power 1 failed for /sys/devices/pci0000:00/0000:00:14.0/usb2/2-3/2-3:1.5/0003:8086:0B3A.0001/HID-SENSOR-200073.1.auto/iio:device1/buffer/enable
WARNING [139911671817984] (backend-hid.cpp:662) HID set_power 1 failed for /sys/devices/pci0000:00/0000:00:14.0/usb2/2-3/2-3:1.5/0003:8086:0B3A.0001/HID-SENSOR-200076.2.auto/iio:device0/buffer/enable

[ev-mp]

@hperik1 hello,
There is a lot mentioned, nevertheless -

1st-case: seem to be a duplicate of #3697. Can you check that you have the udev-rules installed
2nd-case: Please start realsense viewer and check that the advanced mode is turned on:

3rd-case: The notification tells that D435i has no IMU calibration, which is the default. There is a white paper published that describes how users can calibrate the IMU by themselves. Note that this is optionally. and there are lot of discussions about IMU calibration on the forum to decide whether it suits your needs.
Media-format-related : can be disregarded - notifications for internal use.
Sudden dead of processes: this is ROS-specific, though it still be Librealsense-rooted. Since there are lots of reasons why the process crashes it shall be dealt on by-case basis, including description of the actual setup and the launch file, how the crash occurred and Librealsense log file.
Not found devices What is the output of lsusb and rs-enumerate-devices when the device cannot be found ?
During working with realsense-viewer: common warnings that are expected on Linux.
a- Similar to 3rd-case above
b - Linux-specific configuration issue.
Again none of these should affect the device(s) functionality, or in other words - it is normal to see the same warnings when running realsense-viewer as well

[hperik1]

@ev-mp Thank you for the response. Herewith some feedback.
1st-case: seen the thread before submission and yes, the /etc/udev/rules.d/99-realsense-librules.rules file is present.
2nd-case: While using the D435i, the advanced mode was already switched on in the realsense-viewer. By redoing the same this time I got also a warning as in the 3rd-case, with addition that the UYVY media format is unregistered (for completeness added). This for both cases: D435i only andD435i-T265 combo.
3rd-case: OK, clear. This and your media format related feedback closes this 3rd, but also the 2nd case.
Not found devices: actually this is the most annoying part as it seems you need to be lucky getting the system up, requiring sometimes multiple consecutive launches. I have added a zip-file with logs (lsusb, lsusb-t, usb-devices) executed in the next sequence and with D435i only, T265 only and D435i-T265 combo configuration:
log-after-boot > log-rs-enumerate-devices > log-after-enumeration > roslaunch-log > log-after-enum-launch
190406_logfiles.zip

[ev-mp]

Hi @hperik1 ,

1. You should check that the /etc/udev/rules.d/99-realsense-librules.rules is up to date - it is often the case that once installed users forget to update so its content could probably be not consistent with the newly introduced devices (T265)
   2-3. I took a brief scan of the attached files but couldn't find anything unusual, so I'll leave it to the repo maintainer for a in-depth review
   One more thing - can you add dmesg log when the device is not recognized ?

[hperik1]

Hi @ev-mp ,

Rules file
The rules file was update but just to be sure, I have double checked the rules file against the 2.20.0 repository, which is the same.

dmesg logging
Again an elaborate set of log files with the following sequences:
log-after-boot > log-rs-enumerate-devices > log-after-enumeration > roslaunch-log > log-after-launch1 > log-after-launch#
Configurations:

1. both devices d435i and t265 connected during boot
2. after boot, 1st connect d435i, than t265 connected and then go for rs-enumeration-devices. Note 1: the t265 had to be plugged in twice to get detected. Note 2: t265 'disapears' as usb device after rs-enumerate-devices, thus the launch is this time correct not finding the t265.

Applying different sequences bring up new issues, which are hopefully related. Though it makes it hard to reproduce the exact cases. But hopefully this set of logs brings us a step further.
190417_logfiles.zip

[hperik1]

Hi @ev-mp,

I like to add some new information into the discussion, hopefully being able to give some new thoughts.
This time I took a clean PC with the attached CPU and HW configuration: lscpu.log, lshw.log and uname.log.

The problem is persistent and with this test, one should conclude that NO other software is interfering.
Looking forward receiving further suggestions, maybe including PC platform suggestions.

Next steps were executed in given order

1. Installed the SDK v2.21.0 as a package
2. plugged in a t265 and d435i
3. started realsense-viewer
4. console indicates to have recognized the t265
5. no notion of the d435i
6. the listing of usb-devices gave equal output as in earlier posted logs: both are listed
7. in the terminal 'DS5 group_devices is empty' errors popped up
8. stopped the realsense-viewer
9. ran rs-enumerate-devices: only the t265 was enumerated
10. downloaded the v2.21.0 SDK source and copied the rules file to /etc/udev/rules.d/, as this file was not present
11. reran realsense-viewer and rs-enumerate-devices with the same results and effects.
12. rebooted the PC with the t265 and d435i still plugged in
13. reran realsense-viewer and both are now detected and functional
14. in the terminal the same warnings as in earlier posts were shown: calibration file missing and internal notification regarding media formats.
15. rs-enumerate-devices now showed both the t265 and d435i.

OK, besides the strong dependency on the udev rules file and a subsequent reboot, a first step is made.

Now I installed ROS-kinetic and installed ROS wrapper v2.2.3.

Then I did run T265_D435i.launch.txt and got the following results out of subsequent launches:

• 1st run t265 node died during the start up: 12-27.log

• 2nd run - d435i node partially died during start up, some topics remained listed (sorry no topic logging): 12-30.log

• _n_th run - both nodes are running: 12-33.log
  And finally a dmesg logging till this point: dmesg-190424-1236.log

[AdriannaGmz]

same messy and lucky situation where the cameras are randomly found, happens on my side. pending on the reply or some other suggestion

[manomitbal]

#752 solves this, give it a try

[hperik1]

@manomitbal, @ev-mp, I picked up this project again and tested the new ROS v2.2.4 release which indicates to solve this issue. I must say that after several trials bringing up D435i and T265, both devices are working. There seems to be stability now, and that is good!
However there is one note ... initially the T265 is not detected (ERROR flag) but saying it will try it again. Then it starts setting up the D435i, screen logging is pausing and after seconds logging starts again indicating warnings, and finally it sets up the T265.
I am not sure if this is normal behavior ... therefore added a screen log (screenlog-190522.log).

[doronhi]

For now, this is the expected behavior. It reflects the current implementation of the T265 device in librealsense as a singleton. While the D435 node scans for devices it initially holds the T265's handle - hence the errors 'Failed to claim USB interface' from the T265's node. Since it is expected, the T265 node waits and tries again. By this time the D435 node already found it's own device and released the unneeded T265 device which is now available for the T265 node.
That's also explains why currently only one T265 can be used.

[hperik1]

@dorodnic, @ev-mp Thanks for the explanation.
I consider this issue closed.
Great support and keep up the good work!