Colorizing image relative from ground for machine learning

[dream-guide]

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

---

Required Info
Camera Model	D435
Firmware Version	05.12.05.00
Operating System & Version	Ubuntu 18.04.5 LTS
Platform	NVIDIA Jetson Nano
SDK Version	2.0
Language	Python 3.6.9; Opencv 3.2.0
Segment	others

Issue Description

I am trying to manipulate the depth image so that the colorized image is relative to the ground.
After that i want to train a deep neural network on these images.

I know the distance to the ground but the camera height can vary from 3m to 10m.
In the final image i want the ground to be always be a color, then up to 0,7m always orange and from there to 2m always a red gradient.
Just a gradient starting at the ground up to 2m would be ok too.

Here an example:

I tried the colorizer but if i set the min and max distance nothing changes.

 colorizer = rs.colorizer()
 colorizer.set_option(rs.option.min_distance, value_min)
 colorizer.set_option(rs.option.max_distance, value_max)
 depth_colormap = np.asanyarray(colorizer.colorize(depth_frame).get_data())

I also tried with opencv and manipulating alpha and beta but have not found a way to translate that to real world measurements.

depth_colormap = cv.applyColorMap(cv.convertScaleAbs(depth_image, alpha=value_alpha, beta=value_beta), cv.COLORMAP_JET)

Does anyone have an idea that could help?

[MartyG-RealSense]

Hi @dream-guide You can have some control over the coloring style and the color grading of depth with the colorizer by setting a color scheme and a visual preset for it.

The color schemes are selected with a number code (Jet is '0'). The full list of numbers is in the link below:

https://github.com/IntelRealSense/librealsense/blob/master/include/librealsense2/hpp/rs_processing.hpp#L717

Below is a link to a discussion that contains a script in the Python language that includes setting the colorizer color scheme with rs.option.color_scheme

#3363

The script in the above link also contains information on setting the colorizer visual preset with Python using rs.option.visual_preset

Further information on setting the colorizer visual preset can be found on pages 12 and 13 of Intel's RealSense Viewer User Guide.

https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/Intel-RealSense-Viewer-User-Guide.pdf

For example, setting the colorizer's visual preset to Fixed causes the image to be colored on a scale from the minimum distance (minz) up to a maximum of 6 meters.

[dream-guide]

Hi @MartyG-RealSense Thank you that helped me with colorizer not using the min_distance and max_distance values.
I did not know visual_preset works for the depth sensor and colorizer.

Should i start a new issue for the question regarding the custom color scheme?
I found out were they are defined in colorizer but i'm not sure how to define my own in python:
https://github.com/IntelRealSense/librealsense/blob/master/src/proc/colorizer.cpp

This is the code that worked. Noteworthy is that the visual preset needs to be set before min and max.

colorizer = rs.colorizer()
colorizer.set_option(rs.option.visual_preset, 1) # 0=Dynamic, 1=Fixed, 2=Near, 3=Far
colorizer.set_option(rs.option.min_distance, value_min)
colorizer.set_option(rs.option.max_distance, value_max)

depth_colormap = np.asanyarray(colorizer.colorize(depth_frame).get_data())

[MartyG-RealSense]

It is fine to discuss custom color schemes in this question, since it is a related subject.

Looking at the color scale in your opening comment, it looks very similar to the SDK's color scale bar. So is it correct that you are hoping to change the distances at which one color shifts to another, please?

[dream-guide]

@MartyG-RealSense yes like in my example image the floor is solid green, objects smaller than 0.7m are orange and then it is a gradient.

I could go over the gradient image with numpy but if i use the colorizer anyways i thought i could do it in one go.
As i want to use this on runtime less operations are desirable.

[MartyG-RealSense]

Apologies for the delay in responding further. I carefully researched your question about a custom colorization scale but was unable to find an in-built way to do it in the RealSense SDK, unfortunately.

[MartyG-RealSense]

Hi @dream-guide Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.