Determine height "y" of ground debris such as gravel or snow along a depth 'z" line, center "x" in front of robot

[richardrye]

I'm currently using a D435i depth camera, mounted on a wheel loader type robot, with realsence2 and rtabmap_ros loaded. I would like to remove ground debris such as snow or gravel and to verify it is removed would like to know the average height of material, just in a line directly in front of the robot.

To clarify, I would like to specify a z distance, x would be fixed at center line of robot, and I would like to know the heights y from the ground, starting at the robot to the z distance at regular intervals up to z.

If you can provide the C++ code to determine the height y of one position along this line, given a depth and x position would be extremely helpful.

I've added a pic of the robot for context.

Thanks

[MartyG-RealSense]

Hi @richardrye Is the camera mounted inside the white box on the rear of the vehicle, please?

Also, is the camera pointed downwards facing towards the ground instead of facing straight ahead, and you want to set a maximum distance for z so that only depth data under a defined maximum distance from the camera is registered?

If that is your intention then you could set a maximum depth distance in the RealSense ROS wrapper with the clip_distance instruction.

The rs_launch launch file provides an example of defining clip_distance:

https://github.com/IntelRealSense/realsense-ros/blob/development/realsense2_camera/launch/rs_camera.launch#L58

If your camera is not facing downwards towards the ground, please provide information about how the camera is set up on the vehicle.

In regard to getting the distance, there are more resources for Python than for C++. An approach that may work for you is to subscribe to /camera/aligned_depth_to_color/image_raw as described in the links below.

#637 (comment)

#1785 (comment)

Alternatively, a discussion in the link below provides C++ scripting for obtaining the distance in the ROS wrapper using the depth map.

https://answers.ros.org/question/299888/using-a-depth-map/

[richardrye]

Good Morning Marty, The camera is mounted at the front, outside of the white box, flush, in a similar position to an operator. See drawing below. ***@***.***D74D54.F5568A40] The camera is facing straight ahead. I haven’t as yet determined how best to set the depth z but I am now considering the ROS wrapper with clip_distance as you’ve mentioned and will review the rs_launch file as suggested. And to ensure I’ve explained it correctly, I’ve added an illustration below indicating it’s the height from ground y at regular intervals that’s needed. ***@***.***D74D56.B8D2ADE0] I’ll also look into the other three sources you mentioned. #637 (comment)<#637 (comment)> #1785 (comment)<#1785 (comment)> https://answers.ros.org/question/299888/using-a-depth-map/ Thanks again. Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10 From: ***@***.***> Sent: May 20, 2021 4:18 AM To: ***@***.***> Cc: ***@***.***>; ***@***.***> Subject: Re: [IntelRealSense/realsense-ros] Determine height "y" of ground debris such as gravel or snow along a depth 'z" line, center "x" in front of robot (#1884) Hi @richardrye<https://github.com/richardrye> Is the camera mounted inside the white box on the rear of the vehicle, please? Also, is the camera pointed downwards facing towards the ground instead of facing straight ahead, and you want to set a maximum distance for z so that only depth data under a defined maximum distance from the camera is registered? If that is your intention then you could set a maximum depth distance in the RealSense ROS wrapper with the clip_distance instruction. The rs_launch launch file provides an example of defining clip_distance: https://github.com/IntelRealSense/realsense-ros/blob/development/realsense2_camera/launch/rs_camera.launch#L58 If your camera is not facing downwards towards the ground, please provide information about how the camera is set up on the vehicle. In regard to getting the distance, an option that may work for you is to subscribe to /camera/aligned_depth_to_color/image_raw as described in the links below. #637 (comment)<#637 (comment)> #1785 (comment)<#1785 (comment)> Alternatively, a discussion in the link below provides C++ scripting for obtaining the distance in the ROS wrapper using the depth map. https://answers.ros.org/question/299888/using-a-depth-map/ — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#1884 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUAMDKDLLEVQXPF34DYZCHTTOTAV3ANCNFSM45F5KTHA>.

[MartyG-RealSense]

Hi @richardrye The drawings did not appear, as emailed posts tend not to generate images on the GitHub. Can you add them later when you have a chance to access a web browser please? I will wait for the drawings before commenting further. Thanks!

[richardrye]

I've attached a descriptive picture as requested.

[MartyG-RealSense]

Thanks very much. This is actually my first impression of what you were describing before the facing-downwards theory that I posted in the end. It was not clear to me how the camera would be able to see what is on the ground from the straight-ahead viewpoint if the vehicle was in front of the camera and potentially obscuring the lower half of its field of view.

The other point that I considered was how the camera could observe the 'y' height of the snow or gravel in front of the vehicle if looking straight ahead from the white box, since it was looking over the top of the snow / gravel instead of havig a top-down view towards the ground. In that situation, it seemed as though the snow / gravel would be detected if it was high enough to enter the bottom of the field of view of the facing-ahead camera.

I hope that I've explained my thoughts clearly. If not, pease feel free to add futher information. Thanks!

[richardrye]

Thanks Marty, I'll review the points you've outlined and let you know which approach works best. Your help is much appreciated.

…

Sent from my Galaxy Tab A

-------- Original message -------- From: MartyG-RealSense ***@***.***> Date: 2021-05-20 9:53 AM (GMT-05:00) To: IntelRealSense/realsense-ros ***@***.***> Cc: richardrye ***@***.***>, Mention ***@***.***> Subject: Re: [IntelRealSense/realsense-ros] Determine height "y" of ground debris such as gravel or snow along a depth 'z" line, center "x" in front of robot (#1884) Thanks very much. This is actually my first impression of what you were describing before the facing-downwards theory that I posted in the end. It was not clear to me how the camera would be able to see what is on the ground from the straight-ahead viewpoint if the vehicle was in front of the camera and potentially obscuring the lower half of its field of view. The other point that I considered was how the camera could observe the 'y' height of the snow or gravel in front of the vehicle if looking straight ahead from the white box, since it was looking over the top of the snow / gravel. In that situation, it seemed as though the snow / gravel would be detected if it was high enough to enter the bottom of the field of view of the facing-ahead camera. I hope that I've explained my thoughts clearly. If not, pease feel free to add futher information. Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#1884 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUAMDKBU75LADMS2MU3ELL3TOUH7LANCNFSM45F5KTHA>.

[MartyG-RealSense]

Hi @richardrye Do you have an update about this case that you can provide, please? Thanks!

[richardrye]

Hi Marty,
I've found a suitable piece of code on "https://answers.ros.org/question/90696/get-depth-from-kinect-sensor-in-gazebo-simulator/" that enables one to select any pixel and it gives the depth of that pixel. With this and some basic trigonometry I am able to find the height of ground covering.
Thanks,

[MartyG-RealSense]

It's excellent to hear that you found a solution @richardrye - thanks very much for the update!

[richardrye]

Oh one more thing, to work with the D435i I had to change one parameter, specifically "/camera/depth/image_rect_raw" and located in this portion of the code:

image_transport::Subscriber sub = it.subscribe("/camera/depth/image_rect_raw", 1, imageCallback);

May save someone a bit of time.

[MartyG-RealSense]

@richardrye Thanks!