In this release, several runtime optimizations increased the efficiency of nvblox. These optimizations are leveraged in Isaac Perceptor and Isaac Manipulator which rely on the 3D reconstruction provided by nvblox.
In the following sections we first list nvblox runtime optimization and then showcase how they are applied to Isaac Perceptor and Isaac Manipulator.
Runtime optimizations increase the efficiency of nvblox and allow it to operate on low-power, multi-camera systems with higher voxel resolution.
Optimization points:
- Viewpoint cache to reduce redundant ray-cast operations.
- Pre-allocating of voxels in a memory pool to avoid costly on-the-fly memory allocations.
- Bandwidth-limited voxel streaming to allow transmission of voxel maps to remote machines.
- Bulk initialization of voxels to eliminate significant launch overhead when new areas are added to the map.
- Device compaction added to GPU->CPU voxel streaming pipeline to eliminate fragmented global memory reads.
- Parallel kernel launches to maximize utilization of available GPU resources.
- Eliminating all default-stream synchronization points and reducing the amount of local-stream synchronizations. A new pre-merge unit will now ensure no unintended synchronizations are introduced.
- Ensure that all calls to third-party CUDA libraries are made asynchronously. This required an upgrade of the STDGPU library which forms the backend of the voxel map.
- Add user-provided workspace bounds to reduce unnecessary compute.
- Support ESDF/visualization on request for reduced latency and compute.
For a more details, please refer to the CHANGELOG.
Dynamic object detection allows nvblox to handle moving obstacles that would otherwise corrupt the cost map. Dynamic detection is now enabled by default in Perceptor.
Reconstructing an office environment. Dynamic voxels are highlighted in red.
Support for multi-RealSense reconstruction (up to 4 cameras on Jetson AGX Orin).
Reconstruction integrating data from 4 RealSense cameras simultaneously.
Support for Jetson Nano brings nvblox to a lower price point,
which will enable 3D perception for consumer-grade products.
Live reconstruction running on Nano with visualization data streamed to Foxglove over WiFi.
Nvblox enables collision avoidance in manipulation use-cases:
Performance improvements allow reconstruction at 1 cm voxel resolution in workspaces up to 8 m^3 and depth integration at 30 Hz on Jetson AGX Orin.
Support of multi-camera RealSense integration reduces occlusions and increase reconstruction fidelity.
Adding support for pick and place by enabling contact with objects by selective exclusion from the collision field:
