Floating-point image data range seems to be assumed to be at least 0-255.

[kpreid]

Describe the bug
When an archetypes::Image is logged, if the pixel values do not reach at least 255.0, then none of them will be displayed with full brightness in the viewer. It seems that there is an auto-ranging/auto-exposure feature (logging very large values causes the rest of the image to darken) but it never uses a maximum less than 255.0.

This is contrary to typical usage (in my personal experience in computer graphics and GIS) of floating-point pixel formats, where if there is an assumed 100% brightness value, then that value is 1.0.

To Reproduce
Steps to reproduce the behavior:

1. Log an archetypes::Image whose data contains a datatypes::TensorBuffer::F16 with values in the 0.0 - 1.0 range.
2. Observe that in the viewer it appears black, yet the pixel data is displayed on hover.
3. Try again with a larger range such as 0.0 - 100.0.
4. Observe that in the viewer the image appears dim.

Expected behavior
The auto-ranging should select a range that either matches the data, or has an assumed maximum of 1.0.

I'm not sure what would be best to do with the alpha component — which should generally not be auto-scaled since "opaque" is semantically different from "all X% transparent". I haven't investigated Rerun's current behavior with it.

Screenshots

Desktop (please complete the following information):

• OS: macOS Sonoma 14.2.1 (23C71)

Rerun version
rerun-cli 0.11.0 [rustc 1.75.0 (82e1608df 2023-12-21), LLVM 17.0.6] x86_64-apple-darwin

[emilk]

The heuristics here might need tweaking, and should probably be done in conjunction with actually exposing the image range in the UI, i.e.:

• Configurable ranges / (spatial) extent in image and tensor view #1510

The reason for the current default is that it is also quite common to load an image and then convert it to float (perhaps as a side-effect of something else). It would then be weird if Rerun showed the same image differently just because you converted bytes to floats. For instance:

  img = cv2.cvtColor(cv2.imread("img.png", cv2.IMREAD_UNCHANGED), cv2.COLOR_BGRA2RGB)

  rr.log("img_1", rr.Image(img))
  rr.log("img_2", rr.Image(img.astype("float32")))
  rr.log("img_3", rr.Image(1.0 * img)
  rr.log("img_4", rr.Image(0.5 * img)

It's reasonable to expect the first three images to have the same appearance in Rerun. It it is also quite reasonable to expect the last two images to have a different apprearance (img_4 being darker than img_3).

[kpreid]

The reason for the current default is that it is also quite common to load an image and then convert it to float (perhaps as a side-effect of something else). …

That makes sense.

The heuristics here might need tweaking, and should probably be done in conjunction with actually exposing the image range in the UI, i.e.:

Several thoughts:

• But any heuristic is sometimes wrong, so will there be a way to specify a recommended range when logging?

• In the case I was experimenting with, the data in question is natively HDR, so the optimal data display is not necessarily scaling to fit the entire min-max range, but might clip some extremes. That might be reasonably handled solely by interactive adjustment, though.

• Once there are range controls in the viewer, it could detect when an image is displaying very-low-contrast values of any kind and display a “data may be illegible, change scale?” prompt somewhere. Actually, that might be a nice feature to apply to many domains: consider “all your 3D data is currently off this side of the viewport →".

[emilk]

will there be a way to specify a recommended range when logging?

There will! This is part of our Blueprint work, where ultimately everything that can be specified in the GUI can also be specified at log time.