Recommended Setup for multiple D435s

[reyes73a]

Hi guys,

I was wondering what would be a recommended companion computer for interfacing with multiple D435s. Right now, I have been testing a D435 with a Jetson Nano and at full res @30fps and the ROS node seems to be dropping frames frequently. My intention is try to interface with up to 4 D435s at the same time. I know I can try to reduce the resolution and FPS on each device, but I was wondering what is, in your opinion, a good companion computer that could run ROS Kinetic and support 4 D435s (TX1, TX2, Intel Aero?). If I run the same experiment on an i7 it does not drop any frames, but I am not aware of a small companion computer (the size of a Jetson nano) that could pack that much processing power? I have the feeling that the Jetson nano will not be able to handle this much workload. Any help would be greatly appreciated!!!

Thanks!

[MartyG-RealSense]

If you have not seen it already, you may find Intel's guide on using multiple cameras with ROS useful.

https://www.intelrealsense.com/how-to-multiple-camera-setup-with-ros/

The guide gives suggestions for configuring a 3-camera setup in its "Adding More Cameras" section. It too acknowledges the affect of the hardware spec on multi-cam performance, saying: "There are a couple of reasons why the number of cameras connected to a single computer is limited: CPU usage, USB bandwidth, power supply and also, the length of the USB cable. These are all covered in length in the Multiple camera configuration white paper".

In regard to recommendations, there are plenty of options, given the extreme flexibility of the 400 Series cameras in regard to the hardware it will work with. It should be noted though that in the case of Intel Aero, Intel have retired the Aero platform and so whilst the Aero board is still in stock in Intel's online Click store, there is not a long-term future for that product.

The Up Squared board (a more powerful version of the original Up Board with three full USB 3 ports) has been shown to work fine with up to two 400 Series cameras connected to it, though I have not heard how it performs with a greater number of cameras than that. Its size may rule it out for you though if you are aiming for Nano-scale boards.

https://up-board.org/upsquared/specifications/

You might also like to investigate the new Raspberry Pi 4 Model B, which is small and has a low price and yet has a USB 3 port, 4K display support and performance claimed to be equal to a desktop PC.

https://www.raspberrypi.org/products/raspberry-pi-4-model-b/

[realsens01]

The recommendations from @MartyG-RealSense seem overly optimistic, to me, based on my experience in the past year. Things might have improved, but I still advise caution. Problems with the Realsense cameras have unquestionably been the largest source of delays in my project. My project requires recording from a pair of D435s, and I struggled to obtain reliable data, without frame drops. It took a lot of trial-and-error, and independent experimentation, to find a relatively stable setup... and even that required compromise.

Notes:

• The computer I am recording on: Dell with 8th Generation Intel® Core™ i7-8700K, 8GB RAM, and GeForce GTX 1050Ti. The hard drive is an SSD. The computer was purchased new, for this project. USB expansion cards and powered hubs were added during the debugging process.
• My Intel Forums post is my earliest attempt to seek help (Nov 2018), and probably contains the most information.
• How to connect two D415 devices? #2754
• Save two Streams to bag-files at the same time #2882

Good luck.

[MartyG-RealSense]

@realsense01 If you are seeking a more industrial level of stability, the own-brand version of the D435 being released this month by FRAMOS may be of interest to you. It uses ethernet for its cabling and claims greater transmission reliability by doing packet resends.

#4769

Edit: it has just been released in the FRAMOS store, though it has a price premium over the standard D435 due to its industrial nature.

https://shop-us.framos.com/FRAMOS-Industrial-Depth-Camera-D435e-Starter-Kit-p147712085

[reyes73a]

Cool thanks for the replies guys! I now have two D435s with me so I will be testing them both in the next few days on an i7, Jetson Nano and UpBoard. I will let you know how it goes. I am hoping I can at least do 2 D435s with the Up board because the Jetson struggles with only one resulting in late point cloud packets...

[reyes73a]

The recommendations from @MartyG-RealSense seem overly optimistic, to me, based on my experience in the past year. Things might have improved, but I still advise caution. Problems with the Realsense cameras have unquestionably been the largest source of delays in my project. My project requires recording from a pair of D435s, and I struggled to obtain reliable data, without frame drops. It took a lot of trial-and-error, and independent experimentation, to find a relatively stable setup... and even that required compromise.

Notes:

• The computer I am recording on: Dell with 8th Generation Intel® Core™ i7-8700K, 8GB RAM, and GeForce GTX 1050Ti. The hard drive is an SSD. The computer was purchased new, for this project. USB expansion cards and powered hubs were added during the debugging process.
• My Intel Forums post is my earliest attempt to seek help (Nov 2018), and probably contains the most information.
• How to connect two D415 devices? #2754
• Save two Streams to bag-files at the same time #2882

Good luck.

I have not been seeing these issues on my i7 (4770K). I can run a single D435 or together with a T265 at full res without packet drops (maybe an occasional one) and going through ROS. On the Jetson Nano however, I get like 3-5 warnings every second.

[jokla]

Hi @reyes73a,

I am wondering if I could use 4 D435i with a Jetson TX2 (Adding a PCI USB 3 card). I am worried about the CPU workload.

Have you managed to connect 2 cameras on the Jetson Nano with good performance?

Thanks!

[reyes73a]

Yes. I have been able to get up to 4 to a Jetson Nano. Good performance? Depends on what is good performance for you. The Nano is less capable than a standard i5/i7 CPU so even with one camera and using default parameters on the D435 the performance is affected in the form of frame drops. In a nutshell, the D435 generates a huge amount of data making the CPU work too hard to grab it from the camera and broadcast it through ROS (I am using ROS). At full res and fps (max res and 30fps) I see on avg. about 5-10 frames getting dropped per minute with a single D435 and a single client listening on the data. This number will increase as more cameras and clients are added. The TX2 should perform much better I suppose so you should have more flexibility. You can reduce the resolution and fps on the cameras too which should help as well as limit the amount of topics being broadcasted - it depends on what you need. Add the PCI card if possible, but you will have to find a way to externally power the cameras or else you will be reaching the amp limit on the USB bus and the computer, most likely leading to the TX2 under performing even more. I went around this by adding a dedicated USB hub with external power instead of the PCI card. I hope this helps. Daniel.

…

On Mon, Oct 21, 2019 at 9:04 AM Giovanni Claudio ***@***.***> wrote: Hi @reyes73a <https://github.com/reyes73a>, I am wondering if I could use 4 D435i with a Jetson TX2 (Adding a PCI USB 3 card). I am worried about the CPU workload. Have you managed to connect 2 cameras on the Jetson Nano with good performance? Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4792?email_source=notifications&email_token=AEB5L4GSJRRFO7YJEN4KX2TQPWZFZA5CNFSM4ITH2G3KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEB2N23I#issuecomment-544529773>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEB5L4H6UYBQO2ZJAAJ3CKLQPWZFZANCNFSM4ITH2G3A> .

[RealSenseCustomerSupport]

---

@reyes73a Glad to hear you have got it through and thanks for your share! Can we close this accordingly. Thanks!