Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for bayer images to camera sensor #336

Merged

Conversation

tejalbarnwal
Copy link
Contributor

@tejalbarnwal tejalbarnwal commented Mar 24, 2023

🎉 New feature

Closes #299

Sub-Tasks

  • Support simulation for BAYER_RGGB8 with OGRE
  • Extend the functionality to support all bayer types with OGRE
  • Extend the functionality to support with OGRE2

Summary and Related PRs

The functionality reads the user input and renders an RGB image, which is later converted into a single channel 8bit Bayer image using ConvertRGBToBayer() added to Utils.cc inside gz-rendering.

  • gz-sensors : Adds a switch case for RGGB bayer format inside CameraSensor.cc and passes the R8G8B8 format to render the image.
  • gz-redering : Adds ConvertRGBToBayer() to Utils.cc, modifies OgreRenderTarget.cc to call the conversion function, and handles image format conversion functions with if-else statements.
  • gz-gui : Adds switch cases to display Bayer images in Gazebo GUI inside. It treats them as single-channel 8-bit images.
  • gz-common : Modifies Image::SetFromData inside Image.cc to support saving of Bayer images. In order to save it It treats them as single-channel 8-bit images.

Test it

In order to test this, one can modify camera_sensor.sdf inside gz-sim here. Would have just to replace the part with the following snippet.

<image>
   <width>320</width>
   <height>240</height>
   <format>BAYER_RGGB8</format>
</image>
<clip>
   <near>0.1</near>
   <far>100</far>
</clip>
<save enabled="true">
    <path>path to the folder</path>
</save>

Checklist

  • Signed all commits for DCO
  • Added tests
  • Added example and/or tutorial
  • Updated documentation (as needed)
  • Updated migration guide (as needed)
  • Consider updating Python bindings (if the library has them)
  • codecheck passed (See contributing)
  • All tests passed (See test coverage)
  • While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

Signed-off-by: tejalbarnwal <tejalbarnwal@gmail.com>
Signed-off-by: tejalbarnwal <tejalbarnwal@gmail.com>
@tejalbarnwal tejalbarnwal requested a review from iche033 as a code owner March 24, 2023 18:34
@github-actions github-actions bot added the 🌱 garden Ignition Garden label Mar 24, 2023
@tejalbarnwal
Copy link
Contributor Author

tejalbarnwal commented Mar 24, 2023

Hey @iche033 ,I couldn't perform make codecheck from the build directory as mentioned here. Therefore, I use a workaround by doing cppcheck file.cpp on each file I had modified.

Updates:
A community member said cppcheck has to be installed before building with colcon packages. I will do so.
I also found that I would need to update ca-certificates on my system according to failed checks on the PRs I have created. I will do so and commit again.

@tejalbarnwal tejalbarnwal changed the title Pr gzsensors7 support bayer images Add support for bayer images to Ogre and Ogre2 Mar 24, 2023
@tejalbarnwal tejalbarnwal changed the title Add support for bayer images to Ogre and Ogre2 Add support for bayer images to camera sensor Mar 24, 2023
@iche033 iche033 added the needs upstream release Blocked by a release of an upstream library label Mar 24, 2023
tejalbarnwal and others added 3 commits March 28, 2023 15:07
Signed-off-by: tejalbarnwal <tejalbarnwal@gmail.com>
Signed-off-by: tejalbarnwal <tejalbarnwal@gmail.com>
Copy link
Contributor

@iche033 iche033 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs gz-rendering7 release

@iche033 iche033 removed the needs upstream release Blocked by a release of an upstream library label May 18, 2023
@codecov
Copy link

codecov bot commented May 18, 2023

Codecov Report

Merging #336 (ba0be93) into gz-sensors7 (f88ac60) will decrease coverage by 0.50%.
The diff coverage is 0.00%.

❗ Current head ba0be93 differs from pull request most recent head eab8420. Consider uploading reports for the commit eab8420 to get more accurate results

@@               Coverage Diff               @@
##           gz-sensors7     #336      +/-   ##
===============================================
- Coverage        70.18%   69.69%   -0.50%     
===============================================
  Files               36       36              
  Lines             3928     3956      +28     
===============================================
  Hits              2757     2757              
- Misses            1171     1199      +28     
Impacted Files Coverage Δ
src/CameraSensor.cc 75.92% <0.00%> (-5.61%) ⬇️

Copy link
Contributor

@ahcorde ahcorde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MacOS sompilation is failing

/Users/jenkins/jenkins-agent/workspace/ignition_sensors-ci-pr_any-homebrew-amd64/ign-sensors/src/CameraSensor.cc:288:56: error: no member named 'PF_BAYER_GBRG8' in namespace 'gz::rendering'
      this->dataPtr->camera->SetImageFormat(rendering::PF_BAYER_GBRG8);
                                            ~~~~~~~~~~~^
/Users/jenkins/jenkins-agent/workspace/ignition_sensors-ci-pr_any-homebrew-amd64/ign-sensors/src/CameraSensor.cc:291:56: error: no member named 'PF_BAYER_GRBG8' in namespace 'gz::rendering'
      this->dataPtr->camera->SetImageFormat(rendering::PF_BAYER_GRBG8);
                                            ~~~~~~~~~~~^
/Users/jenkins/jenkins-agent/workspace/ignition_sensors-ci-pr_any-homebrew-amd64/ign-sensors/src/CameraSensor.cc:628:23: error: no member named 'PF_BAYER_GBRG8' in namespace 'gz::rendering'
      case rendering::PF_BAYER_GBRG8:
           ~~~~~~~~~~~^
/Users/jenkins/jenkins-agent/workspace/ignition_sensors-ci-pr_any-homebrew-amd64/ign-sensors/src/CameraSensor.cc:632:23: error: no member named 'PF_BAYER_GRBG8' in namespace 'gz::rendering'
      case rendering::PF_BAYER_GRBG8:
           ~~~~~~~~~~~^
4 errors generated.
make[2]: *** [src/CMakeFiles/gz-sensors7-camera.dir/CameraSensor.cc.o] Error 1

@iche033
Copy link
Contributor

iche033 commented May 19, 2023

MacOS sompilation is failing

I think the bottle was not ready when the build was triggered. I retriggered the build and should be fixed now

@iche033 iche033 requested a review from ahcorde May 19, 2023 18:33
@ahcorde ahcorde merged commit 60dac3d into gazebosim:gz-sensors7 May 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🌱 garden Ignition Garden
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Bayer images not supported
3 participants