-```
-
-#### ROS Parameters
-
-| ROS Parameter | Type | Default | Description |
-| ------------------------- | ------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
-| `model_file_path` | `string` | `model.onnx` | The absolute path to your model file in the local file system (the model file must be .onnx)
E.g. `model.onnx` |
-| `engine_file_path` | `string` | `/tmp/trt_engine.plan` | The absolute path to either where you want your TensorRT engine plan to be generated (from your model file) or where your pre-generated engine plan file is located
E.g. `model.plan` |
-| `force_engine_update` | `bool` | `true` | If set to true, the node will always try to generate a TensorRT engine plan from your model file and needs to be set to false to use the pre-generated TensorRT engine plan |
-| `input_tensor_names` | `string list` | `['input_tensor']` | A list of tensor names to be bound to specified input bindings names. Bindings occur in sequential order, so the first name here will be mapped to the first name in input_binding_names |
-| `input_binding_names` | `string list` | `['']` | A list of input tensor binding names specified by model
E.g. `['input_2:0']` |
-| `input_tensor_formats` | `string list` | `['']` | A list of input tensor nitros formats. This should be given in sequential order
E.g. `['nitros_tensor_list_nchw_rgb_f32']` |
-| `output_tensor_names` | `string list` | `['output_tensor']` | A list of tensor names to be bound to specified output binding names |
-| `output_binding_names` | `string list` | `['']` | A list of tensor names to be bound to specified output binding names
E.g. `['argmax_1']` |
-| `output_tensor_formats` | `string list` | `['']` | A list of input tensor nitros formats. This should be given in sequential order
E.g. `[nitros_tensor_list_nchw_rgb_f32]` |
-| `verbose` | `bool` | `true` | If set to true, the node will enable verbose logging to console from the internal TensorRT execution |
-| `max_workspace_size` | `int64_t` | `67108864l` | The size of the working space in bytes |
-| `max_batch_size` | `int32_t` | `1` | The maximum possible batch size in case the first dimension is dynamic and used as the batch size |
-| `dla_core` | `int64_t` | `-1` | The DLA Core to use. Fallback to GPU is always enabled. The default setting is GPU only |
-| `enable_fp16` | `bool` | `true` | Enables building a TensorRT engine plan file which uses FP16 precision for inference. If this setting is false, the plan file will use FP32 precision |
-| `relaxed_dimension_check` | `bool` | `true` | Ignores dimensions of 1 for the input-tensor dimension check |
-
-#### ROS Topics Subscribed
-
-| ROS Topic | Type | Description |
-| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- |
-| `tensor_pub` | [isaac_ros_tensor_list_interfaces/TensorList](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/isaac_ros_tensor_list_interfaces/msg/TensorList.msg) | The input tensor stream |
-
-#### ROS Topics Published
-
-| ROS Topic | Type | Description |
-| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- |
-| `tensor_sub` | [isaac_ros_tensor_list_interfaces/TensorList](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/isaac_ros_tensor_list_interfaces/msg/TensorList.msg) | The tensor list of output tensors from the model inference |
+---
-## Troubleshooting
+## Documentation
-### Isaac ROS Troubleshooting
+Please visit the [Isaac ROS Documentation](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_dnn_inference/index.html) to learn how to use this repository.
-For solutions to problems with Isaac ROS, please check [here](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/troubleshooting.md).
+---
-### Deep Learning Troubleshooting
+## Packages
-For solutions to problems with using DNN models, please check [here](docs/troubleshooting.md).
+* [`isaac_ros_dnn_image_encoder`](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_dnn_inference/isaac_ros_dnn_image_encoder/index.html)
+ * [API](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_dnn_inference/isaac_ros_dnn_image_encoder/index.html#api)
+* [`isaac_ros_tensor_rt`](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_dnn_inference/isaac_ros_tensor_rt/index.html)
+ * [Quickstart](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_dnn_inference/isaac_ros_tensor_rt/index.html#quickstart)
+ * [Troubleshooting](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_dnn_inference/isaac_ros_tensor_rt/index.html#troubleshooting)
+ * [API](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_dnn_inference/isaac_ros_tensor_rt/index.html#api)
+* [`isaac_ros_triton`](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_dnn_inference/isaac_ros_triton/index.html)
+ * [Quickstart](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_dnn_inference/isaac_ros_triton/index.html#quickstart)
+ * [Troubleshooting](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_dnn_inference/isaac_ros_triton/index.html#troubleshooting)
+ * [API](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_dnn_inference/isaac_ros_triton/index.html#api)
-## Updates
+## Latest
-| Date | Changes |
-| ---------- | ---------------------------------------------------------------------------------------------------------------------------- |
-| 2023-05-25 | Performance improvements |
-| 2023-04-05 | Source available GXF extensions |
-| 2022-10-19 | Updated OSS licensing |
-| 2022-08-31 | Update to be compatible with JetPack 5.0.2 |
-| 2022-06-30 | Added format string parameter in Triton/TensorRT, switched to NITROS implementation, removed parameters in DNN Image Encoder |
-| 2021-11-03 | Split DOPE and U-Net into separate repositories |
-| 2021-10-20 | Initial release |
+Update 2023-10-18: Updated for Isaac ROS 2.0.0.
diff --git a/docs/model-preparation.md b/docs/model-preparation.md
deleted file mode 100644
index d4cc918..0000000
--- a/docs/model-preparation.md
+++ /dev/null
@@ -1,65 +0,0 @@
-# Preparing Deep Learning Models for Isaac ROS
-
-## Obtaining a Pre-trained Model from NGC
-
-The NVIDIA GPU Cloud hosts a [catalog](https://catalog.ngc.nvidia.com/models) of Deep Learning pre-trained models that are available for your development.
-
-1. Use the **Search Bar** to find a pre-trained model that you are interested in working with.
-
-2. Click on the model's card to view an expanded description, and then click on the **File Browser** tab along the navigation bar.
-
-3. Using the **File Browser**, find a deployable `.etlt` file for the model you are interested in.
-
- > **Note:** The `.etlt` file extension indicates that this model has pre-trained but **encrypted** weights, which means one needs to use the `tao-converter` utility to decrypt the model, as described [below](#using-tao-converter-to-decrypt-the-encrypted-tlt-model-etlt-format).
-
-4. Under the **Actions** heading, click on the **...** icon for the file you selected in the previous step, and then click **Copy `wget` command**.
-5. **Paste** the copied command into a terminal to download the model in the current working directory.
-
-## Using `tao-converter` to decrypt the Encrypted TLT Model (`.etlt`) Format
-
-As discussed above, models distributed with the `.etlt` file extension are encrypted and must be decrypted before use via NVIDIA's [`tao-converter`](https://developer.nvidia.com/tao-toolkit-get-started).
-
-`tao-converter` is already included in the Docker images available as part of the standard [Isaac ROS Development Environment](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/dev-env-setup.md).
-
-The per-platform installation paths are described below:
-
-| Platform | Installation Path | Symlink Path |
-| --------------- | ------------------------------------------------------------- | ----------------------------------- |
-| x86_64 | `/opt/nvidia/tao/tao-converter-x86-tensorrt8.0/tao-converter` | **`/opt/nvidia/tao/tao-converter`** |
-| Jetson(aarch64) | `/opt/nvidia/tao/jp5` | **`/opt/nvidia/tao/tao-converter`** |
-
-### Converting `.etlt` to a TensorRT Engine Plan
-
-Here are some examples for generating the TensorRT engine file using `tao-converter`. In this example, we will use the [`PeopleSemSegnet Shuffleseg` model](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/peoplesemsegnet/files?version=deployable_shuffleseg_unet_v1.0):
-
-#### Generate an engine file for the `fp16` data type
-
- ```bash
- mkdir -p /workspaces/isaac_ros-dev/models && \
- /opt/nvidia/tao/tao-converter -k tlt_encode -d 3,544,960 -p input_2:0,1x3x544x960,1x3x544x960,1x3x544x960 -t fp16 -e /workspaces/isaac_ros-dev/models/peoplesemsegnet_shuffleseg.engine -o argmax_1 peoplesemsegnet_shuffleseg_etlt.etlt
- ```
-
- > **Note:** The specific values used in the command above are retrieved from the **PeopleSemSegnet** page under the **Overview** tab. The model input node name and output node name can be found in `peoplesemsegnet_shuffleseg_cache.txt` from `File Browser`. The output file is specified using the `-e` option. The tool needs write permission to the output directory.
- >
- > A detailed explanation of the input parameters is available [here](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/resources/tao-converter).
-
-#### Generate an engine file for the data type `int8`
-
- Create the models directory:
-
- ```bash
- mkdir -p /workspaces/isaac_ros-dev/models
- ```
-
- Download the calibration cache file:
- > **Note:** Check the model's page on NGC for the latest `wget` command.
-
- ```bash
- wget https://api.ngc.nvidia.com/v2/models/nvidia/tao/peoplesemsegnet/versions/deployable_shuffleseg_unet_v1.0/files/peoplesemsegnet_shuffleseg_cache.txt
- ```
-
- ```bash
- /opt/nvidia/tao/tao-converter -k tlt_encode -d 3,544,960 -p input_2:0,1x3x544x960,1x3x544x960,1x3x544x960 -t int8 -c peoplesemsegnet_shuffleseg_cache.txt -e /workspaces/isaac_ros-dev/models/peoplesemsegnet_shuffleseg.engine -o argmax_1 peoplesemsegnet_shuffleseg_etlt.etlt
- ```
-
- > **Note**: The calibration cache file (specified using the `-c` option) is required to generate the `int8` engine file. This file is provided in the **File Browser** tab of the model's page on NGC.
diff --git a/docs/tensorrt-and-triton-info.md b/docs/tensorrt-and-triton-info.md
deleted file mode 100644
index 4d2c449..0000000
--- a/docs/tensorrt-and-triton-info.md
+++ /dev/null
@@ -1,45 +0,0 @@
-# Isaac ROS Triton and TensorRT Nodes for DNN Inference
-
-NVIDIA's Isaac ROS suite of packages provides two separate nodes for performing DNN inference: Triton and TensorRT.
-
-Our benchmarks show comparable performance and inference speed with both nodes, so a decision should be based on other characteristics of the overall model being deployed.
-
-## NVIDIA Triton
-
-The NVIDIA Triton Inference Server is an [open-source inference serving software](https://developer.nvidia.com/nvidia-triton-inference-server) that provides a uniform interface for deploying AI models. Crucially, Triton supports a wide array of compute devices like NVIDIA GPUs and both x86 and ARM CPUs, and also operates with all major frameworks such as TensorFlow, TensorRT, and PyTorch.
-
-Because Triton can take advantage of additional compute devices beyond just the GPU, Triton can be a better choice in situations where there is GPU resource contention from other model inference or processing tasks. However, in order to provide for this flexibility, Triton requires the creation of a model repository and additional configuration files before deployment.
-
-## NVIDIA TensorRT
-
-NVIDIA TensorRT is a specific CUDA-based, on-GPU inference framework that performs a number of optimizations to deliver extremely performant model execution. TensorRT only supports ONNX and TensorRT Engine Plans, providing less flexibility than Triton but also requiring less initial configuration.
-
-## Using either Triton or TensorRT Nodes
-
-Both nodes use the Isaac ROS [Tensor List message](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/isaac_ros_tensor_list_interfaces/msg/TensorList.msg) for input data and output inference result.
-
-Users can either prepare a custom model or download pre-trained models from NGC as described [here](./model-preparation.md#obtaining-a-pre-trained-model-from-ngc). Models should be converted to the TensorRT Engine File format using the `tao-converter` tool as described [here](./model-preparation.md#using-tao-converter-to-decrypt-the-encrypted-tlt-model-etlt-format).
-
-> **Note:** While the TensorRT node can automatically convert ONNX plans to the TensorRT Engine Plan format if configured to use a `.onnx` file, this conversion step will considerably extend the node's per-launch initial setup time.
->
-> As a result, we recommend converting any ONNX models to TensorRT Engine Plans first, and configuring the TensorRT node to use the Engine Plan instead.
-
-## Pre- and Post-Processing Nodes
-
-In order to be a useful component of a ROS graph, both Isaac ROS Triton and TensorRT inference nodes will require application-specific `pre-processor` (`encoder`) and `post-processor` (`decoder`) nodes to handle type conversion and other necessary steps.
-
-A `pre-processor` node should take in a ROS 2 message, perform the pre-processing steps dictated by the model, and then convert the data into an Isaac ROS Tensor List message. For example, a `pre-processor` node could resize an image, normalize it, and then convert it into a Tensor List.
-
-A `post-processor` node should be used to convert the Isaac ROS Tensor List output of the model inference into a useful ROS 2 message. For example, a `post-processor` node may perform argmax to identify the class label from a classification problem.
-
-
-
-
-
-
-
-## Further Reading
-
-For more documentation on Triton, see [here](https://developer.nvidia.com/nvidia-triton-inference-server).
-
-For more documentation on TensorRT, see [here](https://developer.nvidia.com/tensorrt).
diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md
deleted file mode 100644
index 559ce63..0000000
--- a/docs/troubleshooting.md
+++ /dev/null
@@ -1,49 +0,0 @@
-# DNN Inference Troubleshooting
-
-## Seeing operation failed followed by the process dying
-
-One cause of this issue is when the GPU being used does not have enough memory to run the model. For example, DOPE may require up to 6GB of VRAM to operate, depending on the application.
-
-### Symptom
-
-```log
-[component_container_mt-1] 2022-06-27 08:35:37.518 ERROR extensions/tensor_ops/Reshape.cpp@71: reshape tensor failed.
-[component_container_mt-1] 2022-06-27 08:35:37.518 ERROR extensions/tensor_ops/TensorOperator.cpp@151: operation failed.
-[component_container_mt-1] 2022-06-27 08:35:37.518 ERROR gxf/std/entity_executor.cpp@200: Entity with 102 not found!
-[component_container_mt-1] INFO: infer_simple_runtime.cpp:69 TrtISBackend id:164 initialized model: Ketchup
-[component_container_mt-1] 2022-06-27 08:35:37.518 WARN gxf/std/greedy_scheduler.cpp@221: Error while executing entity 87 named 'VERAGYEWGZ_reshaper': GXF_FAILURE
-[component_container_mt-1] [ERROR] [1656318937.518424053] [dope_encoder]: [NitrosPublisher] Vault ("vault/vault", eid=102) was stopped. The graph may have been terminated due to an error.
-[component_container_mt-1] terminate called after throwing an instance of 'std::runtime_error'
-[component_container_mt-1] what(): [NitrosPublisher] Vault ("vault/vault", eid=102) was stopped. The graph may have been terminated due to an error.
-[ERROR] [component_container_mt-1]: process has died [pid 13378, exit code -6, cmd '/opt/ros/humble/install/lib/rclcpp_components/component_container_mt --ros-args -r __node:=dope_container -r __ns:=/'].
-```
-
-### Solution
-
-Try using the Isaac ROS TensorRT node or the Isaac ROS Triton node with the TensorRT backend instead. Otherwise, a discrete GPU with more VRAM may be required.
-
-## Triton fails to create the TensorRT engine and load a model
-
-### Symptom
-
-```log
-1: [component_container_mt-1] I0331 05:56:07.479791 11359 tensorrt.cc:5591] TRITONBACKEND_ModelInitialize: detectnet (version 1)
-1: [component_container_mt-1] I0331 05:56:07.483989 11359 tensorrt.cc:5640] TRITONBACKEND_ModelInstanceInitialize: detectnet (GPU device 0)
-1: [component_container_mt-1] I0331 05:56:08.169240 11359 logging.cc:49] Loaded engine size: 21 MiB
-1: [component_container_mt-1] E0331 05:56:08.209208 11359 logging.cc:43] 1: [runtime.cpp::parsePlan::314] Error Code 1: Serialization (Serialization assertion plan->header.magicTag == rt::kPLAN_MAGIC_TAG failed.)
-1: [component_container_mt-1] I0331 05:56:08.213483 11359 tensorrt.cc:5678] TRITONBACKEND_ModelInstanceFinalize: delete instance state
-1: [component_container_mt-1] I0331 05:56:08.213525 11359 tensorrt.cc:5617] TRITONBACKEND_ModelFinalize: delete model state
-1: [component_container_mt-1] E0331 05:56:08.214059 11359 model_lifecycle.cc:596] failed to load 'detectnet' version 1: Internal: unable to create TensorRT engine
-1: [component_container_mt-1] ERROR: infer_trtis_server.cpp:1057 Triton: failed to load model detectnet, triton_err_str:Invalid argument, err_msg:load failed for model 'detectnet': version 1 is at UNAVAILABLE state: Internal: unable to create TensorRT engine;
-1: [component_container_mt-1]
-1: [component_container_mt-1] ERROR: infer_trtis_backend.cpp:54 failed to load model: detectnet, nvinfer error:NVDSINFER_TRITON_ERROR
-1: [component_container_mt-1] ERROR: infer_simple_runtime.cpp:33 failed to initialize backend while ensuring model:detectnet ready, nvinfer error:NVDSINFER_TRITON_ERROR
-1: [component_container_mt-1] ERROR: Error in createNNBackend() [UID = 16]: failed to initialize triton simple runtime for model:detectnet, nvinfer error:NVDSINFER_TRITON_ERROR
-1: [component_container_mt-1] ERROR: Error in initialize() [UID = 16]: create nn-backend failed, check config file settings, nvinfer error:NVDSINFER_TRITON_ERROR
-```
-
-### Solution
-
-This error can occur when TensorRT attempts to load an incompatible `model.plan` file. The incompatibility may arise due to a versioning or platform mismatch between the time of plan generation and the time of plan execution.
-
-Delete the `model.plan` file that is being passed in as an argument to the Triton node's `model_repository_paths` parameter, and then use the source package's instructions to regenerate the `model.plan` file from the original weights file (often a `.etlt` or `.onnx` file).
diff --git a/isaac_ros_dnn_encoders/CMakeLists.txt b/isaac_ros_dnn_image_encoder/CMakeLists.txt
similarity index 81%
rename from isaac_ros_dnn_encoders/CMakeLists.txt
rename to isaac_ros_dnn_image_encoder/CMakeLists.txt
index 0d2de17..77dc326 100644
--- a/isaac_ros_dnn_encoders/CMakeLists.txt
+++ b/isaac_ros_dnn_image_encoder/CMakeLists.txt
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -15,8 +15,8 @@
#
# SPDX-License-Identifier: Apache-2.0
-cmake_minimum_required(VERSION 3.23.2)
-project(isaac_ros_dnn_encoders LANGUAGES C CXX)
+cmake_minimum_required(VERSION 3.22.1)
+project(isaac_ros_dnn_image_encoder LANGUAGES C CXX)
if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
add_compile_options(-Wall -Wextra -Wpedantic)
@@ -34,10 +34,16 @@ if(BUILD_TESTING)
find_package(ament_lint_auto REQUIRED)
ament_lint_auto_find_test_dependencies()
+
+ # The FindPythonInterp and FindPythonLibs modules are removed
+ if(POLICY CMP0148)
+ cmake_policy(SET CMP0148 OLD)
+ endif()
+
find_package(launch_testing_ament_cmake REQUIRED)
add_launch_test(test/isaac_ros_dnn_image_encoder_test.py)
+ add_launch_test(test/isaac_ros_dnn_image_encoder_image_crop_test.py)
add_launch_test(test/isaac_ros_dnn_image_encoder_image_norm_test.py)
- add_launch_test(test/isaac_ros_dnn_image_encoder_image_resize_test.py)
endif()
ament_auto_package(INSTALL_TO_SHARE config)
diff --git a/isaac_ros_dnn_encoders/config/dnn_image_encoder_node.yaml b/isaac_ros_dnn_image_encoder/config/dnn_image_encoder_node.yaml
similarity index 85%
rename from isaac_ros_dnn_encoders/config/dnn_image_encoder_node.yaml
rename to isaac_ros_dnn_image_encoder/config/dnn_image_encoder_node.yaml
index a89bfda..9f598c6 100644
--- a/isaac_ros_dnn_encoders/config/dnn_image_encoder_node.yaml
+++ b/isaac_ros_dnn_image_encoder/config/dnn_image_encoder_node.yaml
@@ -75,16 +75,76 @@ components:
parameters:
transmitter: data_transmitter_resizer
min_size: 1
-- name: data_transmitter_compositor
+---
+name: resizer
+components:
+- name: data_receiver
+ type: nvidia::gxf::DoubleBufferReceiver
+ parameters:
+ capacity: 12
+ policy: 0
+- type: nvidia::gxf::MessageAvailableSchedulingTerm
+ parameters:
+ receiver: data_receiver
+ min_size: 1
+- name: data_transmitter
type: nvidia::gxf::DoubleBufferTransmitter
parameters:
- capacity: 1
+ capacity: 12
- type: nvidia::gxf::DownstreamReceptiveSchedulingTerm
parameters:
- transmitter: data_transmitter_compositor
+ transmitter: data_transmitter
min_size: 1
+- name: allocator
+ type: nvidia::gxf::BlockMemoryPool
+ parameters:
+ storage_type: 1
+ block_size: 1566720
+ num_blocks: 40
+- name: resize_operator
+ type: nvidia::isaac::tensor_ops::Resize
+ parameters:
+ output_width: 0
+ output_height: 0
+ interp_type: "linear"
+ border_type: "zero"
+ keep_aspect_ratio: true
+ receiver: data_receiver
+ transmitter: data_transmitter
+ pool: allocator
+ input_adapter: global/adapter_video_buffer
+ output_adapter: global/adapter_video_buffer
+ output_name: "image"
---
-name: resizer
+name: bbox
+components:
+- name: data_transmitter
+ type: nvidia::gxf::DoubleBufferTransmitter
+ parameters:
+ capacity: 12
+- type: nvidia::gxf::DownstreamReceptiveSchedulingTerm
+ parameters:
+ transmitter: data_transmitter
+ min_size: 1
+- name: allocator
+ type: nvidia::gxf::BlockMemoryPool
+ parameters:
+ storage_type: 0
+ block_size: 32
+ num_blocks: 40
+- type: nvidia::isaac::tensor_ops::BBoxGenerator
+ parameters:
+ image_width: 0
+ image_height: 0
+ bbox_width: 0
+ bbox_height: 0
+ bbox_loc_x: 0
+ bbox_loc_y: 0
+ bbox_count: 1
+ pool: allocator
+ transmitter: data_transmitter
+---
+name: crop_and_resizer
components:
- name: data_receiver
type: nvidia::gxf::DoubleBufferReceiver
@@ -95,6 +155,15 @@ components:
parameters:
receiver: data_receiver
min_size: 1
+- name: bbox_receiver
+ type: nvidia::gxf::DoubleBufferReceiver
+ parameters:
+ capacity: 12
+ policy: 0
+- type: nvidia::gxf::MessageAvailableSchedulingTerm
+ parameters:
+ receiver: bbox_receiver
+ min_size: 1
- name: data_transmitter
type: nvidia::gxf::DoubleBufferTransmitter
parameters:
@@ -109,15 +178,15 @@ components:
storage_type: 1
block_size: 1566720
num_blocks: 40
-- name: resize_operator
- type: nvidia::isaac::tensor_ops::Resize
+- name: crop_and_resize_operator
+ type: nvidia::isaac::tensor_ops::CropAndResize
parameters:
output_width: 0
output_height: 0
interp_type: "linear"
- border_type: "zero"
keep_aspect_ratio: true
receiver: data_receiver
+ receiver_bbox: bbox_receiver
transmitter: data_transmitter
pool: allocator
input_adapter: global/adapter_video_buffer
@@ -258,6 +327,10 @@ components:
storage_type: 1
block_size: 6266880
num_blocks: 40
+- type: nvidia::gxf::MemoryAvailableSchedulingTerm
+ parameters:
+ allocator: allocator
+ min_blocks: 1
- name: reshape_operator
type: nvidia::isaac::tensor_ops::Reshape
parameters:
@@ -269,47 +342,12 @@ components:
output_name: "input_tensor"
output_shape: []
---
-name: compositor
-components:
-- name: data_receiver_interleaved_to_planar
- type: nvidia::gxf::DoubleBufferReceiver
- parameters:
- capacity: 1
- policy: 0
-- type: nvidia::gxf::MessageAvailableSchedulingTerm
- parameters:
- receiver: data_receiver_interleaved_to_planar
- min_size: 1
-- name: data_receiver_timestamp
- type: nvidia::gxf::DoubleBufferReceiver
- parameters:
- capacity: 1
- policy: 0
-- type: nvidia::gxf::MessageAvailableSchedulingTerm
- parameters:
- receiver: data_receiver_timestamp
- min_size: 1
-- name: data_transmitter
- type: nvidia::gxf::DoubleBufferTransmitter
- parameters:
- capacity: 1
-- type: nvidia::gxf::DownstreamReceptiveSchedulingTerm
- parameters:
- transmitter: data_transmitter
- min_size: 1
-- name: compositor
- type: nvidia::isaac_ros::MessageCompositor
- parameters:
- image_rx: data_receiver_interleaved_to_planar
- camera_model_rx: data_receiver_timestamp
- tx: data_transmitter
----
name: sink
components:
- name: signal
type: nvidia::gxf::DoubleBufferReceiver
parameters:
- capacity: 1
+ capacity: 5
policy: 0
- type: nvidia::gxf::MessageAvailableSchedulingTerm
parameters:
@@ -321,66 +359,67 @@ components:
source: signal
---
components:
-- name: edge0
+- name: edge1
type: nvidia::gxf::Connection
parameters:
source: broadcaster/data_transmitter_resizer
target: resizer/data_receiver
---
components:
-- name: edge1
- type: nvidia::gxf::Connection
- parameters:
- source: resizer/data_transmitter
- target: color_space_converter/data_receiver
----
-components:
- name: edge2
type: nvidia::gxf::Connection
parameters:
- source: color_space_converter/data_transmitter
- target: normalizer/data_receiver
+ source: bbox/data_transmitter
+ target: crop_and_resizer/bbox_receiver
---
components:
- name: edge3
type: nvidia::gxf::Connection
parameters:
- source: normalizer/data_transmitter
- target: interleaved_to_planar/data_receiver
+ source: resizer/data_transmitter
+ target: crop_and_resizer/data_receiver
---
components:
- name: edge4
type: nvidia::gxf::Connection
parameters:
- source: interleaved_to_planar/data_transmitter
- target: reshaper/data_receiver
+ source: crop_and_resizer/data_transmitter
+ target: color_space_converter/data_receiver
---
components:
- name: edge5
type: nvidia::gxf::Connection
parameters:
- source: reshaper/data_transmitter
- target: compositor/data_receiver_interleaved_to_planar
+ source: color_space_converter/data_transmitter
+ target: normalizer/data_receiver
---
components:
- name: edge6
type: nvidia::gxf::Connection
parameters:
- source: broadcaster/data_transmitter_compositor
- target: compositor/data_receiver_timestamp
+ source: normalizer/data_transmitter
+ target: interleaved_to_planar/data_receiver
---
components:
- name: edge7
type: nvidia::gxf::Connection
parameters:
- source: compositor/data_transmitter
+ source: interleaved_to_planar/data_transmitter
+ target: reshaper/data_receiver
+---
+components:
+- name: edge8
+ type: nvidia::gxf::Connection
+ parameters:
+ source: reshaper/data_transmitter
target: sink/signal
---
components:
-- type: nvidia::gxf::GreedyScheduler
+- type: nvidia::gxf::MultiThreadScheduler
parameters:
clock: clock
stop_on_deadlock: false
- check_recession_period_us: 100
+ check_recession_period_ms: 0.05
+ worker_thread_number: 2
- name: clock
type: nvidia::gxf::RealtimeClock
diff --git a/isaac_ros_dnn_encoders/config/namespace_injector_rule.yaml b/isaac_ros_dnn_image_encoder/config/namespace_injector_rule.yaml
similarity index 92%
rename from isaac_ros_dnn_encoders/config/namespace_injector_rule.yaml
rename to isaac_ros_dnn_image_encoder/config/namespace_injector_rule.yaml
index dd699e6..74a7311 100644
--- a/isaac_ros_dnn_encoders/config/namespace_injector_rule.yaml
+++ b/isaac_ros_dnn_image_encoder/config/namespace_injector_rule.yaml
@@ -20,6 +20,8 @@ name: DNN Image Encoder Namespace Injector Rule
operation: namespace_injector
body:
components:
+ - type: nvidia::isaac::tensor_ops::CropAndResize
+ path_parameter_keys: [input_adapter, output_adapter]
- type: nvidia::isaac::tensor_ops::ConvertColorFormat
path_parameter_keys: [input_adapter, output_adapter]
- type: nvidia::isaac::tensor_ops::Resize
diff --git a/isaac_ros_dnn_encoders/include/isaac_ros_dnn_encoders/dnn_image_encoder_node.hpp b/isaac_ros_dnn_image_encoder/include/isaac_ros_dnn_image_encoder/dnn_image_encoder_node.hpp
similarity index 74%
rename from isaac_ros_dnn_encoders/include/isaac_ros_dnn_encoders/dnn_image_encoder_node.hpp
rename to isaac_ros_dnn_image_encoder/include/isaac_ros_dnn_image_encoder/dnn_image_encoder_node.hpp
index 47fbc73..7a3b5bc 100644
--- a/isaac_ros_dnn_encoders/include/isaac_ros_dnn_encoders/dnn_image_encoder_node.hpp
+++ b/isaac_ros_dnn_image_encoder/include/isaac_ros_dnn_image_encoder/dnn_image_encoder_node.hpp
@@ -1,5 +1,5 @@
// SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-// Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -15,8 +15,8 @@
//
// SPDX-License-Identifier: Apache-2.0
-#ifndef ISAAC_ROS_DNN_ENCODERS__DNN_IMAGE_ENCODER_NODE_HPP_
-#define ISAAC_ROS_DNN_ENCODERS__DNN_IMAGE_ENCODER_NODE_HPP_
+#ifndef ISAAC_ROS_DNN_IMAGE_ENCODER__DNN_IMAGE_ENCODER_NODE_HPP_
+#define ISAAC_ROS_DNN_IMAGE_ENCODER__DNN_IMAGE_ENCODER_NODE_HPP_
#include
#include
@@ -31,13 +31,6 @@ namespace isaac_ros
namespace dnn_inference
{
-enum class ResizeMode
-{
- kDistort = 0,
- kPad = 1,
- kCrop = 2
-};
-
class DnnImageEncoderNode : public nitros::NitrosNode
{
public:
@@ -49,17 +42,23 @@ class DnnImageEncoderNode : public nitros::NitrosNode
void postLoadGraphCallback() override;
private:
+ void CalculateResizeAndCropParams();
+ uint16_t GetResizeScalar();
// Desired properties of the image
+ const uint16_t input_image_width_;
+ const uint16_t input_image_height_;
const uint16_t network_image_width_;
const uint16_t network_image_height_;
+ const bool enable_padding_;
const std::vector image_mean_;
const std::vector image_stddev_;
int64_t num_blocks_;
- const ResizeMode resize_mode_;
+ uint16_t resize_out_img_width_;
+ uint16_t resize_out_img_height_;
};
} // namespace dnn_inference
} // namespace isaac_ros
} // namespace nvidia
-#endif // ISAAC_ROS_DNN_ENCODERS__DNN_IMAGE_ENCODER_NODE_HPP_
+#endif // ISAAC_ROS_DNN_IMAGE_ENCODER__DNN_IMAGE_ENCODER_NODE_HPP_
diff --git a/isaac_ros_dnn_encoders/package.xml b/isaac_ros_dnn_image_encoder/package.xml
similarity index 88%
rename from isaac_ros_dnn_encoders/package.xml
rename to isaac_ros_dnn_image_encoder/package.xml
index d8f42f4..43598bc 100644
--- a/isaac_ros_dnn_encoders/package.xml
+++ b/isaac_ros_dnn_image_encoder/package.xml
@@ -20,9 +20,9 @@ SPDX-License-Identifier: Apache-2.0
- isaac_ros_dnn_encoders
- 0.31.0
- Encoders for preprocessing before running deep learning inference
+ isaac_ros_dnn_image_encoder
+ 2.0.0
+ Encoder for preprocessing images into tensors for deep learning inference
Hemal Shah
Apache-2.0
https://developer.nvidia.com/isaac-ros-gems/
@@ -38,6 +38,7 @@ SPDX-License-Identifier: Apache-2.0
isaac_ros_nitros_tensor_list_type
isaac_ros_common
+ isaac_ros_gxf
ament_lint_auto
ament_lint_common
diff --git a/isaac_ros_dnn_encoders/src/dnn_image_encoder_node.cpp b/isaac_ros_dnn_image_encoder/src/dnn_image_encoder_node.cpp
similarity index 72%
rename from isaac_ros_dnn_encoders/src/dnn_image_encoder_node.cpp
rename to isaac_ros_dnn_image_encoder/src/dnn_image_encoder_node.cpp
index e5fb3d3..274b665 100644
--- a/isaac_ros_dnn_encoders/src/dnn_image_encoder_node.cpp
+++ b/isaac_ros_dnn_image_encoder/src/dnn_image_encoder_node.cpp
@@ -15,7 +15,7 @@
//
// SPDX-License-Identifier: Apache-2.0
-#include "isaac_ros_dnn_encoders/dnn_image_encoder_node.hpp"
+#include "isaac_ros_dnn_image_encoder/dnn_image_encoder_node.hpp"
#include
#include
@@ -56,7 +56,7 @@ constexpr char OUTPUT_DEFAULT_TENSOR_FORMAT[] = "nitros_tensor_list_nchw_rgb_f32
constexpr char OUTPUT_TOPIC_NAME[] = "encoded_tensor";
constexpr char APP_YAML_FILENAME[] = "config/dnn_image_encoder_node.yaml";
-constexpr char PACKAGE_NAME[] = "isaac_ros_dnn_encoders";
+constexpr char PACKAGE_NAME[] = "isaac_ros_dnn_image_encoder";
const std::vector> EXTENSIONS = {
{"isaac_ros_gxf", "gxf/lib/std/libgxf_std.so"},
@@ -105,14 +105,21 @@ DnnImageEncoderNode::DnnImageEncoderNode(const rclcpp::NodeOptions options)
GENERATOR_RULE_FILENAMES,
EXTENSIONS,
PACKAGE_NAME),
+ input_image_width_(declare_parameter("input_image_width", 0)),
+ input_image_height_(declare_parameter("input_image_height", 0)),
network_image_width_(declare_parameter("network_image_width", 0)),
network_image_height_(declare_parameter("network_image_height", 0)),
+ enable_padding_(declare_parameter("enable_padding", true)),
image_mean_(declare_parameter>("image_mean", {0.5, 0.5, 0.5})),
image_stddev_(declare_parameter>("image_stddev", {0.5, 0.5, 0.5})),
- num_blocks_(declare_parameter("num_blocks", 40)),
- resize_mode_(static_cast(
- declare_parameter("resize_mode", static_cast(ResizeMode::kDistort))))
+ num_blocks_(declare_parameter("num_blocks", 40))
{
+ if (input_image_width_ == 0) {
+ throw std::invalid_argument("[Dnn Image Encoder] Invalid input_image_width");
+ }
+ if (input_image_height_ == 0) {
+ throw std::invalid_argument("[Dnn Image Encoder] Invalid input_image_height");
+ }
if (network_image_width_ == 0) {
throw std::invalid_argument(
"[Dnn Image Encoder] Invalid network_image_width, "
@@ -120,7 +127,7 @@ DnnImageEncoderNode::DnnImageEncoderNode(const rclcpp::NodeOptions options)
}
if (network_image_height_ == 0) {
throw std::invalid_argument(
- "[Dnn Image Encoder] Invalid network_image_height_, "
+ "[Dnn Image Encoder] Invalid network_image_height, "
"this needs to be set per the model input requirements.");
}
@@ -162,18 +169,55 @@ void DnnImageEncoderNode::preLoadGraphCallback()
offsets);
}
+uint16_t DnnImageEncoderNode::GetResizeScalar()
+{
+ // If both the network dims are same scalar integer of image dims that means all pixels can be
+ // be preserved while keeping the aspect ratio hence 1 is returned else min
+ uint16_t width_scalar = input_image_width_ / network_image_width_;
+ uint16_t height_scalar = input_image_height_ / network_image_height_;
+ return (width_scalar == height_scalar) ? 1 : std::min(width_scalar, height_scalar);
+}
+
+void DnnImageEncoderNode::CalculateResizeAndCropParams()
+{
+ const uint16_t resize_factor = enable_padding_ ? 1 : GetResizeScalar();
+ resize_out_img_width_ = network_image_width_ * resize_factor;
+ resize_out_img_height_ = network_image_height_ * resize_factor;
+}
+
void DnnImageEncoderNode::postLoadGraphCallback()
{
RCLCPP_INFO(get_logger(), "In DNN Image Encoder Node postLoadGraphCallback().");
+ CalculateResizeAndCropParams();
+
+ getNitrosContext().setParameterUInt64(
+ "resizer", "nvidia::isaac::tensor_ops::Resize", "output_width", resize_out_img_width_);
+ getNitrosContext().setParameterUInt64(
+ "resizer", "nvidia::isaac::tensor_ops::Resize", "output_height", resize_out_img_height_);
+
+ getNitrosContext().setParameterUInt64(
+ "crop_and_resizer", "nvidia::isaac::tensor_ops::CropAndResize", "output_width",
+ network_image_width_);
+ getNitrosContext().setParameterUInt64(
+ "crop_and_resizer", "nvidia::isaac::tensor_ops::CropAndResize", "output_height",
+ network_image_height_);
+
+ getNitrosContext().setParameterUInt64(
+ "bbox", "nvidia::isaac::tensor_ops::BBoxGenerator", "image_width", resize_out_img_width_);
+ getNitrosContext().setParameterUInt64(
+ "bbox", "nvidia::isaac::tensor_ops::BBoxGenerator", "image_height", resize_out_img_height_);
getNitrosContext().setParameterUInt64(
- "resizer", "nvidia::isaac::tensor_ops::Resize", "output_width", network_image_width_);
+ "bbox", "nvidia::isaac::tensor_ops::BBoxGenerator", "bbox_width", network_image_width_);
getNitrosContext().setParameterUInt64(
- "resizer", "nvidia::isaac::tensor_ops::Resize", "output_height", network_image_height_);
+ "bbox", "nvidia::isaac::tensor_ops::BBoxGenerator", "bbox_height", network_image_height_);
+ getNitrosContext().setParameterUInt64(
+ "bbox", "nvidia::isaac::tensor_ops::BBoxGenerator", "bbox_loc_x",
+ (resize_out_img_width_ - network_image_width_) / 2);
+ getNitrosContext().setParameterUInt64(
+ "bbox", "nvidia::isaac::tensor_ops::BBoxGenerator", "bbox_loc_y",
+ (resize_out_img_height_ - network_image_height_) / 2);
- getNitrosContext().setParameterBool(
- "resizer", "nvidia::isaac::tensor_ops::Resize", "keep_aspect_ratio",
- resize_mode_ != ResizeMode::kDistort);
const gxf::optimizer::ComponentInfo output_comp_info = {
"nvidia::isaac_ros::MessageRelay", // component_type_name
@@ -189,14 +233,20 @@ void DnnImageEncoderNode::postLoadGraphCallback()
throw std::runtime_error("Unsupported NITROS tensor type.");
} else {
uint64_t block_size = calculate_image_size(
+ image_type->second, resize_out_img_width_, resize_out_img_height_);
+
+ getNitrosContext().setParameterUInt64(
+ "resizer", "nvidia::gxf::BlockMemoryPool", "block_size", block_size);
+
+ RCLCPP_DEBUG(get_logger(), "postLoadGraphCallback() resizer block_size = %ld.", block_size);
+
+ block_size = calculate_image_size(
image_type->second, network_image_width_, network_image_height_);
- RCLCPP_DEBUG(
- get_logger(), "postLoadGraphCallback() block_size = %ld.",
- block_size);
+ RCLCPP_DEBUG(get_logger(), "postLoadGraphCallback() block_size = %ld.", block_size);
getNitrosContext().setParameterUInt64(
- "resizer", "nvidia::gxf::BlockMemoryPool", "block_size", block_size);
+ "crop_and_resizer", "nvidia::gxf::BlockMemoryPool", "block_size", block_size);
getNitrosContext().setParameterUInt64(
"color_space_converter", "nvidia::gxf::BlockMemoryPool", "block_size", block_size);
getNitrosContext().setParameterUInt64(
@@ -210,6 +260,8 @@ void DnnImageEncoderNode::postLoadGraphCallback()
uint64_t num_blocks = std::max(static_cast(num_blocks_), 40);
getNitrosContext().setParameterUInt64(
"resizer", "nvidia::gxf::BlockMemoryPool", "num_blocks", num_blocks);
+ getNitrosContext().setParameterUInt64(
+ "crop_and_resizer", "nvidia::gxf::BlockMemoryPool", "num_blocks", num_blocks);
getNitrosContext().setParameterUInt64(
"color_space_converter", "nvidia::gxf::BlockMemoryPool", "num_blocks", num_blocks);
getNitrosContext().setParameterUInt64(
diff --git a/isaac_ros_dnn_encoders/test/isaac_ros_dnn_image_encoder_image_resize_test.py b/isaac_ros_dnn_image_encoder/test/isaac_ros_dnn_image_encoder_image_crop_test.py
similarity index 60%
rename from isaac_ros_dnn_encoders/test/isaac_ros_dnn_image_encoder_image_resize_test.py
rename to isaac_ros_dnn_image_encoder/test/isaac_ros_dnn_image_encoder_image_crop_test.py
index f9d542e..18b14ed 100644
--- a/isaac_ros_dnn_encoders/test/isaac_ros_dnn_image_encoder_image_resize_test.py
+++ b/isaac_ros_dnn_image_encoder/test/isaac_ros_dnn_image_encoder_image_crop_test.py
@@ -39,23 +39,25 @@
NETWORK_IMAGE_WIDTH = 512
NETWORK_IMAGE_HEIGHT = 512
-IMAGE_MEAN = np.array([0.5, 0.6, 0.25])
-IMAGE_STDDEV = np.array([0.25, 0.8, 0.5])
+IMAGE_MEAN = np.array([0.5, 0.5, 0.5])
+IMAGE_STDDEV = np.array([0.5, 0.5, 0.5])
@pytest.mark.rostest
def generate_test_description():
encoder_node = ComposableNode(
name='encoder',
- package='isaac_ros_dnn_encoders',
+ package='isaac_ros_dnn_image_encoder',
plugin='nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode',
namespace=IsaacROSDnnImageEncoderImageResizeNodeTest.generate_namespace(),
parameters=[{
+ 'input_image_width': INPUT_IMAGE_WIDTH,
+ 'input_image_height': INPUT_IMAGE_HEIGHT,
'network_image_width': NETWORK_IMAGE_WIDTH,
'network_image_height': NETWORK_IMAGE_HEIGHT,
'image_mean': list(IMAGE_MEAN),
'image_stddev': list(IMAGE_STDDEV),
- 'resize_mode': 1 # Pad mode
+ 'enable_padding': True
}],
remappings=[('encoded_tensor', 'tensors')])
@@ -87,25 +89,28 @@ def test_image_resize(self):
Image, self.namespaces['image'], self.DEFAULT_QOS)
subs = self.create_logging_subscribers(
- [('tensors', TensorList)], received_messages)
+ [('tensors', TensorList)], received_messages, accept_multiple_messages=False)
try:
- # Create gray image with colored pixels
- cv_image = np.ones((INPUT_IMAGE_HEIGHT, INPUT_IMAGE_WIDTH, 3), np.uint8) * 127
+ # Create image with colored pixels
+ cv_image = np.ones((INPUT_IMAGE_HEIGHT, INPUT_IMAGE_WIDTH, 3), np.uint8) * 255
# What fraction of each dimension should be colored for tracing
- TRACER_PATCH_SIZE_FRACTION = 0.05
+ TRACER_PATCH_SIZE_FRACTION = 0.5
# Patch guaranteed to be at least 1 pixel large
TRACER_PATCH_HEIGHT = ceil(TRACER_PATCH_SIZE_FRACTION * INPUT_IMAGE_HEIGHT)
TRACER_PATCH_WIDTH = ceil(TRACER_PATCH_SIZE_FRACTION * INPUT_IMAGE_WIDTH)
# Input image layout:
- # -------------------
- # | R G |
- # | |
- # | |
- # | B |
+ # --------------------
+ # | R R R R G G G G |
+ # | R R R R G G G G |
+ # | R R R R G G G G |
+ # | |
+ # | B B B B W W W W |
+ # | B B B B W W W W |
+ # | B B B B W W W W |
# -------------------
# Red pixels in top left corner
@@ -160,45 +165,76 @@ def extract_pixel(data, x, y):
)
)
- red_pixel, green_pixel, blue_pixel = None, None, None
-
# Compute expected values corresponding to R, G, B after normalization
RED_EXPECTED_VAL, GREEN_EXPECTED_VAL, BLUE_EXPECTED_VAL = (
- 1 - IMAGE_MEAN) / IMAGE_STDDEV
-
+ 1 - IMAGE_MEAN) / IMAGE_STDDEV
COLOR_MATCH_TOLERANCE = 0.05
+
+ red_pixel_count = 0
+ green_pixel_count = 0
+ blue_pixel_count = 0
+ white_pixel_count = 0
+ black_pixel_count = 0
+
+ # Expected o/p tensor layout:
+ # --------------------
+ # | Black Black Black|
+ # | R R R R G G G G |
+ # | R R R R G G G G |
+ # | R R R R G G G G |
+ # | R R R R G G G G |
+ # | |
+ # | B B B B W W W W |
+ # | B B B B W W W W |
+ # | B B B B W W W W |
+ # | B B B B W W W W |
+ # | Black Black Black|
+ # --------------------
+
+ EXPECTED_NUM_RED_PIXELS = (NETWORK_IMAGE_WIDTH *
+ (float(INPUT_IMAGE_HEIGHT) / float(INPUT_IMAGE_WIDTH))) *\
+ TRACER_PATCH_SIZE_FRACTION * (NETWORK_IMAGE_WIDTH / 2)
+
+ EXPECTED_NUM_GREEN_PIXELS = EXPECTED_NUM_RED_PIXELS
+ EXPECTED_NUM_BLUE_PIXELS = EXPECTED_NUM_RED_PIXELS
+ EXPECTED_NUM_WHITE_PIXELS = EXPECTED_NUM_RED_PIXELS
+
+ EXPECTED_NUM_BLACK_PIXELS = (NETWORK_IMAGE_WIDTH - NETWORK_IMAGE_WIDTH *
+ (float(INPUT_IMAGE_HEIGHT) / float(INPUT_IMAGE_WIDTH))) *\
+ TRACER_PATCH_SIZE_FRACTION * (NETWORK_IMAGE_WIDTH) * 2
+
for y in range(NETWORK_IMAGE_HEIGHT):
for x in range(NETWORK_IMAGE_WIDTH):
# Extract 3 float values corresponding to the
r, g, b = extract_pixel(tensor.data, x, y)
- # Match pixel based on color channels
- # Only record the first matching pixel
- if red_pixel is None and abs(r - RED_EXPECTED_VAL) < COLOR_MATCH_TOLERANCE:
- red_pixel = (x, y)
- if green_pixel is None and abs(g - GREEN_EXPECTED_VAL) < COLOR_MATCH_TOLERANCE:
- green_pixel = (x, y)
- if blue_pixel is None and abs(b - BLUE_EXPECTED_VAL) < COLOR_MATCH_TOLERANCE:
- blue_pixel = (x, y)
-
- self.assertIsNotNone(
- red_pixel, f'Failed to find any red pixels with r={RED_EXPECTED_VAL}')
- self.assertIsNotNone(
- green_pixel, f'Failed to find any green pixels with g={GREEN_EXPECTED_VAL}')
- self.assertIsNotNone(
- blue_pixel, f'Failed to find any blue pixels with b={BLUE_EXPECTED_VAL}')
-
- # Calculate distances between tracer pixels
- output_width = green_pixel[0] - red_pixel[0] + 1 # Top right - Top left
- output_height = blue_pixel[1] - red_pixel[1] + 1 # Bottom left - Top left
-
- # Ensure aspect ratio was preserved
- self.assertAlmostEquals(
- output_width / output_height,
- INPUT_IMAGE_WIDTH / INPUT_IMAGE_HEIGHT,
- places=2,
- msg='Aspect ratio was not preserved!'
- )
+ if(abs(r - RED_EXPECTED_VAL) < COLOR_MATCH_TOLERANCE
+ and g < COLOR_MATCH_TOLERANCE and b < COLOR_MATCH_TOLERANCE):
+ red_pixel_count += 1
+ if(abs(g - GREEN_EXPECTED_VAL) < COLOR_MATCH_TOLERANCE
+ and r < COLOR_MATCH_TOLERANCE and b < COLOR_MATCH_TOLERANCE):
+ green_pixel_count += 1
+ if(abs(b - BLUE_EXPECTED_VAL) < COLOR_MATCH_TOLERANCE
+ and r < COLOR_MATCH_TOLERANCE and g < COLOR_MATCH_TOLERANCE):
+ blue_pixel_count += 1
+ if(abs(r - RED_EXPECTED_VAL) < COLOR_MATCH_TOLERANCE and
+ abs(g - GREEN_EXPECTED_VAL) < COLOR_MATCH_TOLERANCE and
+ abs(b - BLUE_EXPECTED_VAL) < COLOR_MATCH_TOLERANCE):
+ white_pixel_count += 1
+ if(r < COLOR_MATCH_TOLERANCE and g < COLOR_MATCH_TOLERANCE and
+ b < COLOR_MATCH_TOLERANCE):
+ black_pixel_count += 1
+
+ self.assertEqual(red_pixel_count, EXPECTED_NUM_RED_PIXELS,
+ msg='Count of red pixles do not match')
+ self.assertEqual(green_pixel_count, EXPECTED_NUM_GREEN_PIXELS,
+ msg='Count of green pixles do not match')
+ self.assertEqual(blue_pixel_count, EXPECTED_NUM_BLUE_PIXELS,
+ msg='Count of blue pixles do not match')
+ self.assertEqual(white_pixel_count, EXPECTED_NUM_WHITE_PIXELS,
+ msg='Count of white pixles do not match')
+ self.assertEqual(black_pixel_count, EXPECTED_NUM_BLACK_PIXELS,
+ msg='Count of black pixles do not match')
finally:
self.node.destroy_subscription(subs)
diff --git a/isaac_ros_dnn_encoders/test/isaac_ros_dnn_image_encoder_image_norm_test.py b/isaac_ros_dnn_image_encoder/test/isaac_ros_dnn_image_encoder_image_norm_test.py
similarity index 96%
rename from isaac_ros_dnn_encoders/test/isaac_ros_dnn_image_encoder_image_norm_test.py
rename to isaac_ros_dnn_image_encoder/test/isaac_ros_dnn_image_encoder_image_norm_test.py
index f1fa95f..9490431 100644
--- a/isaac_ros_dnn_encoders/test/isaac_ros_dnn_image_encoder_image_norm_test.py
+++ b/isaac_ros_dnn_image_encoder/test/isaac_ros_dnn_image_encoder_image_norm_test.py
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -42,10 +42,12 @@
def generate_test_description():
encoder_node = ComposableNode(
name='encoder',
- package='isaac_ros_dnn_encoders',
+ package='isaac_ros_dnn_image_encoder',
plugin='nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode',
namespace=IsaacROSDnnImageEncoderImageNormNodeTest.generate_namespace(),
parameters=[{
+ 'input_image_width': DIMENSION_WIDTH,
+ 'input_image_height': DIMENSION_HEIGHT,
'network_image_width': DIMENSION_WIDTH,
'network_image_height': DIMENSION_HEIGHT,
'image_mean': [0.5, 0.6, 0.25],
diff --git a/isaac_ros_dnn_encoders/test/isaac_ros_dnn_image_encoder_test.py b/isaac_ros_dnn_image_encoder/test/isaac_ros_dnn_image_encoder_test.py
similarity index 87%
rename from isaac_ros_dnn_encoders/test/isaac_ros_dnn_image_encoder_test.py
rename to isaac_ros_dnn_image_encoder/test/isaac_ros_dnn_image_encoder_test.py
index 0d124c3..d60129f 100644
--- a/isaac_ros_dnn_encoders/test/isaac_ros_dnn_image_encoder_test.py
+++ b/isaac_ros_dnn_image_encoder/test/isaac_ros_dnn_image_encoder_test.py
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -36,12 +36,15 @@
def generate_test_description():
encoder_node = ComposableNode(
name='encoder',
- package='isaac_ros_dnn_encoders',
+ package='isaac_ros_dnn_image_encoder',
plugin='nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode',
namespace=IsaacROSDnnImageEncoderNodeTest.generate_namespace(),
parameters=[{
+ 'input_image_width': 1920,
+ 'input_image_height': 1080,
'network_image_width': 512,
- 'network_image_height': 512
+ 'network_image_height': 512,
+ 'enable_padding': True
}],
remappings=[('encoded_tensor', 'tensors')])
@@ -85,6 +88,8 @@ def test_maintain_aspect_ratio(self):
try:
json_file = self.filepath / 'test_cases/pose_estimation_0/image.json'
image = JSONConversion.load_image_from_json(json_file)
+ timestamp = self.node.get_clock().now().to_msg()
+ image.header.stamp = timestamp
end_time = time.time() + TIMEOUT
done = False
@@ -96,7 +101,11 @@ def test_maintain_aspect_ratio(self):
done = True
break
self.assertTrue(done, 'Appropriate output not received')
- tensor = received_messages['tensors'].tensors[0]
+ tensor_list = received_messages['tensors']
+ tensor = tensor_list.tensors[0]
+
+ self.assertEqual(str(timestamp), str(tensor_list.header.stamp),
+ 'Timestamps do not match.')
cv_image = CvBridge().imgmsg_to_cv2(image, desired_encoding='rgb8')
diff --git a/isaac_ros_dnn_encoders/test/test_cases/pose_estimation_0/image.jpg b/isaac_ros_dnn_image_encoder/test/test_cases/pose_estimation_0/image.jpg
similarity index 100%
rename from isaac_ros_dnn_encoders/test/test_cases/pose_estimation_0/image.jpg
rename to isaac_ros_dnn_image_encoder/test/test_cases/pose_estimation_0/image.jpg
diff --git a/isaac_ros_dnn_encoders/test/test_cases/pose_estimation_0/image.json b/isaac_ros_dnn_image_encoder/test/test_cases/pose_estimation_0/image.json
similarity index 100%
rename from isaac_ros_dnn_encoders/test/test_cases/pose_estimation_0/image.json
rename to isaac_ros_dnn_image_encoder/test/test_cases/pose_estimation_0/image.json
diff --git a/isaac_ros_dnn_inference_test/CMakeLists.txt b/isaac_ros_dnn_inference_test/CMakeLists.txt
deleted file mode 100644
index 14a83e0..0000000
--- a/isaac_ros_dnn_inference_test/CMakeLists.txt
+++ /dev/null
@@ -1,42 +0,0 @@
-# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# SPDX-License-Identifier: Apache-2.0
-
-cmake_minimum_required(VERSION 3.23.2)
-project(isaac_ros_dnn_inference_test LANGUAGES C CXX)
-
-if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
- add_compile_options(-Wall -Wextra -Wpedantic)
-endif()
-
-find_package(ament_cmake_auto REQUIRED)
-ament_auto_find_build_dependencies()
-
-# test_tensor_publisher_node
-ament_auto_add_library(test_tensor_publisher_node SHARED src/test_tensor_publisher_node.cpp)
-rclcpp_components_register_nodes(test_tensor_publisher_node "nvidia::isaac_ros::dnn_inference::TestTensorPublisherNode")
-set(node_plugins "${node_plugins}nvidia::isaac_ros::dnn_inference::TestTensorPublisherNode;$\n")
-
-# run test tensor publisher executable
-ament_auto_add_executable(run_test_publisher src/test_tensor_publisher_main.cpp)
-target_link_libraries(run_test_publisher test_tensor_publisher_node)
-
-if(BUILD_TESTING)
- find_package(ament_lint_auto REQUIRED)
- ament_lint_auto_find_test_dependencies()
-endif()
-
-ament_auto_package()
diff --git a/isaac_ros_dnn_inference_test/include/isaac_ros_dnn_inference_test/test_tensor_publisher_node.hpp b/isaac_ros_dnn_inference_test/include/isaac_ros_dnn_inference_test/test_tensor_publisher_node.hpp
deleted file mode 100644
index d4d6483..0000000
--- a/isaac_ros_dnn_inference_test/include/isaac_ros_dnn_inference_test/test_tensor_publisher_node.hpp
+++ /dev/null
@@ -1,85 +0,0 @@
-// SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-// Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// SPDX-License-Identifier: Apache-2.0
-
-#ifndef ISAAC_ROS_DNN_INFERENCE_TEST__TEST_TENSOR_PUBLISHER_NODE_HPP_
-#define ISAAC_ROS_DNN_INFERENCE_TEST__TEST_TENSOR_PUBLISHER_NODE_HPP_
-
-#include
-#include
-#include
-
-#include "rclcpp/rclcpp.hpp"
-#include "isaac_ros_tensor_list_interfaces/msg/tensor.hpp"
-#include "isaac_ros_tensor_list_interfaces/msg/tensor_list.hpp"
-
-
-namespace nvidia
-{
-namespace isaac_ros
-{
-namespace dnn_inference
-{
-
-// Sends test ROS tensors of specified dimension to specified channel
-// By default this node will send input tensors compatible with mobilenetv2-1.0
-// Tensors being sent in are of the dimension 1x3x224x224 as specified by mobilenet
-class TestTensorPublisherNode : public rclcpp::Node
-{
-public:
- TestTensorPublisherNode(
- const rclcpp::NodeOptions & options);
-
- ~TestTensorPublisherNode();
-
- TestTensorPublisherNode(const TestTensorPublisherNode &) = delete;
-
- TestTensorPublisherNode & operator=(const TestTensorPublisherNode &) = delete;
-
-private:
- // Dimensions of tensor (eg. {3, 255, 255} for RGB Image of 255x255)
- std::vector dimensions_;
- // Name of tensor
- const std::string tensor_name_;
- // Enum of type of data (check Tensor.msg for mappings)
- const int data_type_;
- // Length of tensor when flattened to 1D
- const int length_;
- // Rank of tensor
- const int rank_;
- // Publisher channel
- std::string pub_channel_;
-
- rclcpp::TimerBase::SharedPtr timer_;
- rclcpp::Publisher::SharedPtr pub_;
-
- // Sets the tensor data based on size and type (supports only: float32)
- template
- void setTensorData(isaac_ros_tensor_list_interfaces::msg::Tensor & tensor, int length)
- {
- std::vector test_data(length);
- tensor.data.resize(test_data.size() * sizeof(T));
- memcpy(tensor.data.data(), test_data.data(), test_data.size() * sizeof(T));
- }
-
- void timer_callback();
-};
-
-} // namespace dnn_inference
-} // namespace isaac_ros
-} // namespace nvidia
-
-#endif // ISAAC_ROS_DNN_INFERENCE_TEST__TEST_TENSOR_PUBLISHER_NODE_HPP_
diff --git a/isaac_ros_dnn_inference_test/package.xml b/isaac_ros_dnn_inference_test/package.xml
deleted file mode 100644
index 1b83a01..0000000
--- a/isaac_ros_dnn_inference_test/package.xml
+++ /dev/null
@@ -1,47 +0,0 @@
-
-
-
-
-
-
- isaac_ros_dnn_inference_test
- 0.31.0
- DNN Inference support for Isaac ROS
-
- Hemal Shah
- Apache-2.0
- https://developer.nvidia.com/isaac-ros-gems/
- Isaac Chang
- Ethan Yu
- Herón Ordóñez Guillén
- Hemal Shah
-
- ament_cmake_auto
-
- rclcpp
- rclcpp_components
- isaac_ros_tensor_list_interfaces
-
- ament_lint_auto
- ament_lint_common
-
-
- ament_cmake
-
-
diff --git a/isaac_ros_dnn_inference_test/src/test_tensor_publisher_main.cpp b/isaac_ros_dnn_inference_test/src/test_tensor_publisher_main.cpp
deleted file mode 100644
index 874859f..0000000
--- a/isaac_ros_dnn_inference_test/src/test_tensor_publisher_main.cpp
+++ /dev/null
@@ -1,43 +0,0 @@
-// SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-// Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// SPDX-License-Identifier: Apache-2.0
-
-#include
-#include
-
-#include "rclcpp/rclcpp.hpp"
-
-#include "isaac_ros_dnn_inference_test/test_tensor_publisher_node.hpp"
-
-int main(int argc, char * argv[])
-{
- rclcpp::init(argc, argv);
-
- rclcpp::executors::SingleThreadedExecutor exec;
-
- rclcpp::NodeOptions node_options;
-
- auto pub_node = std::make_shared(
- node_options);
-
- exec.add_node(pub_node);
-
- // Spin with all the components loaded
- exec.spin();
-
- rclcpp::shutdown();
- return 0;
-}
diff --git a/isaac_ros_dnn_inference_test/src/test_tensor_publisher_node.cpp b/isaac_ros_dnn_inference_test/src/test_tensor_publisher_node.cpp
deleted file mode 100644
index e1197a9..0000000
--- a/isaac_ros_dnn_inference_test/src/test_tensor_publisher_node.cpp
+++ /dev/null
@@ -1,101 +0,0 @@
-// SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-// Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// SPDX-License-Identifier: Apache-2.0
-
-#include "isaac_ros_dnn_inference_test/test_tensor_publisher_node.hpp"
-
-#include
-#include
-#include
-#include
-#include
-
-#include "isaac_ros_tensor_list_interfaces/msg/tensor_shape.hpp"
-
-namespace nvidia
-{
-namespace isaac_ros
-{
-namespace dnn_inference
-{
-
-namespace
-{
-int calculate_length(const std::vector dimensions)
-{
- int length{1};
- for (int dim : dimensions) {
- length = length * dim;
- }
- return length;
-}
-} // namespace
-
-using namespace std::chrono_literals;
-
-TestTensorPublisherNode::TestTensorPublisherNode(
- const rclcpp::NodeOptions & options)
-: Node("test_tensor_publisher", options),
- dimensions_(declare_parameter>("dimensions", {1, 3, 224, 224})),
- tensor_name_(declare_parameter("tensor_name", "input_tensor")),
- data_type_(declare_parameter("data_type", 9)),
- length_(declare_parameter("length", calculate_length(dimensions_))),
- rank_(declare_parameter("rank", 4)),
- pub_channel_(declare_parameter("pub_channel", "tensor_pub"))
-{
- pub_ =
- this->create_publisher(pub_channel_, 10);
- timer_ = this->create_wall_timer(
- 1000ms, std::bind(&TestTensorPublisherNode::timer_callback, this));
-
- // Don't let negative dimensions be passed
- for (const auto & dim : dimensions_) {
- if (dim <= 0) {
- throw std::invalid_argument("Error: received negative or zero dimension!");
- }
- }
-}
-
-void TestTensorPublisherNode::timer_callback()
-{
- auto tensor = isaac_ros_tensor_list_interfaces::msg::Tensor();
- tensor.name = tensor_name_;
- tensor.data_type = data_type_;
- tensor.strides = {}; // Not used in GXF so leave empty
-
- switch (data_type_) {
- case 9:
- setTensorData(tensor, length_);
- break;
- }
-
- auto tensor_shape = isaac_ros_tensor_list_interfaces::msg::TensorShape();
- tensor_shape.rank = rank_;
- for (const int64_t dim : dimensions_) {
- tensor_shape.dims.push_back(static_cast(dim));
- }
- tensor.shape = tensor_shape;
-
- auto tensor_list = isaac_ros_tensor_list_interfaces::msg::TensorList();
- tensor_list.tensors = {tensor};
- pub_->publish(tensor_list);
-}
-
-TestTensorPublisherNode::~TestTensorPublisherNode() {}
-
-} // namespace dnn_inference
-} // namespace isaac_ros
-} // namespace nvidia
diff --git a/isaac_ros_tensor_rt/CMakeLists.txt b/isaac_ros_tensor_rt/CMakeLists.txt
index 67eff32..79716dc 100644
--- a/isaac_ros_tensor_rt/CMakeLists.txt
+++ b/isaac_ros_tensor_rt/CMakeLists.txt
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -15,7 +15,7 @@
#
# SPDX-License-Identifier: Apache-2.0
-cmake_minimum_required(VERSION 3.23.2)
+cmake_minimum_required(VERSION 3.22.1)
project(isaac_ros_tensor_rt LANGUAGES C CXX)
if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
@@ -27,6 +27,7 @@ ament_auto_find_build_dependencies()
# TensorRTNode
ament_auto_add_library(tensor_rt_node SHARED src/tensor_rt_node.cpp)
+target_link_libraries(tensor_rt_node TENSORRT::nvonnxparser)
rclcpp_components_register_nodes(tensor_rt_node "nvidia::isaac_ros::dnn_inference::TensorRTNode")
set(node_plugins "${node_plugins}nvidia::isaac_ros::dnn_inference::TensorRTNode;$\n")
@@ -42,6 +43,12 @@ if(BUILD_TESTING)
find_package(ament_lint_auto REQUIRED)
ament_lint_auto_find_test_dependencies()
+
+ # The FindPythonInterp and FindPythonLibs modules are removed
+ if(POLICY CMP0148)
+ cmake_policy(SET CMP0148 OLD)
+ endif()
+
find_package(launch_testing_ament_cmake REQUIRED)
add_launch_test(test/isaac_ros_tensor_rt_test.py TIMEOUT "300")
endif()
diff --git a/isaac_ros_tensor_rt/config/tensor_rt_inference.yaml b/isaac_ros_tensor_rt/config/tensor_rt_inference.yaml
index 74551e2..a0d27ca 100644
--- a/isaac_ros_tensor_rt/config/tensor_rt_inference.yaml
+++ b/isaac_ros_tensor_rt/config/tensor_rt_inference.yaml
@@ -37,7 +37,11 @@ components:
capacity: 12
policy: 0
- name: pool
- type: nvidia::gxf::UnboundedAllocator
+ type: nvidia::gxf::BlockMemoryPool
+ parameters:
+ storage_type: 1
+ block_size: 67108864
+ num_blocks: 40
- type: nvidia::gxf::TensorRtInference
parameters:
model_file_path: /tmp/model_file.onnx
@@ -64,7 +68,7 @@ components:
- name: rx
type: nvidia::gxf::DoubleBufferReceiver
parameters:
- capacity: 1
+ capacity: 2
policy: 0
- name: tx
type: nvidia::gxf::DoubleBufferTransmitter
@@ -115,11 +119,15 @@ name: utils
components:
- name: clock
type: nvidia::gxf::RealtimeClock
-- type: nvidia::gxf::GreedyScheduler
+- type: nvidia::gxf::MultiThreadScheduler
parameters:
clock: clock
stop_on_deadlock: false
- check_recession_period_us: 100
+ check_recession_period_ms: 1
+ worker_thread_number: 1
+- type: nvidia::gxf::JobStatistics
+ parameters:
+ clock: clock
- name: stream
type: nvidia::gxf::CudaStreamPool
parameters:
diff --git a/isaac_ros_tensor_rt/gxf/tensor_rt/CMakeLists.txt b/isaac_ros_tensor_rt/gxf/tensor_rt/CMakeLists.txt
index 79e5505..23edf30 100644
--- a/isaac_ros_tensor_rt/gxf/tensor_rt/CMakeLists.txt
+++ b/isaac_ros_tensor_rt/gxf/tensor_rt/CMakeLists.txt
@@ -30,7 +30,7 @@ find_package(GXF ${ISAAC_ROS_GXF_VERSION} MODULE REQUIRED
std
)
find_package(TENSORRT 8 MODULE REQUIRED)
-include(YamlCpp)
+find_package(yaml-cpp)
# TensorRT extension
add_library(gxf_tensor_rt SHARED
diff --git a/isaac_ros_tensor_rt/include/isaac_ros_tensor_rt/tensor_rt_node.hpp b/isaac_ros_tensor_rt/include/isaac_ros_tensor_rt/tensor_rt_node.hpp
index e05b197..aaf2ec7 100644
--- a/isaac_ros_tensor_rt/include/isaac_ros_tensor_rt/tensor_rt_node.hpp
+++ b/isaac_ros_tensor_rt/include/isaac_ros_tensor_rt/tensor_rt_node.hpp
@@ -1,5 +1,5 @@
// SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-// Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -23,11 +23,16 @@
#include
#include
+#include "NvInferPlugin.h"
+#include "NvOnnxConfig.h"
+#include "NvOnnxParser.h"
+
#include "rclcpp/rclcpp.hpp"
#include "isaac_ros_nitros/nitros_node.hpp"
using StringList = std::vector;
+
namespace nvidia
{
namespace isaac_ros
@@ -71,6 +76,9 @@ class TensorRTNode : public nitros::NitrosNode
const int32_t max_batch_size_;
const bool enable_fp16_;
const bool relaxed_dimension_check_;
+ const int64_t num_blocks_;
+
+ size_t determineMaxTensorBlockSize();
};
} // namespace dnn_inference
diff --git a/isaac_ros_tensor_rt/launch/isaac_ros_tensor_rt.launch.py b/isaac_ros_tensor_rt/launch/isaac_ros_tensor_rt.launch.py
index 7e33aa2..dc29473 100644
--- a/isaac_ros_tensor_rt/launch/isaac_ros_tensor_rt.launch.py
+++ b/isaac_ros_tensor_rt/launch/isaac_ros_tensor_rt.launch.py
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
diff --git a/isaac_ros_tensor_rt/package.xml b/isaac_ros_tensor_rt/package.xml
index daa5996..86f63b9 100644
--- a/isaac_ros_tensor_rt/package.xml
+++ b/isaac_ros_tensor_rt/package.xml
@@ -21,7 +21,7 @@ SPDX-License-Identifier: Apache-2.0
isaac_ros_tensor_rt
- 0.31.0
+ 2.0.0
DNN Inference support for Isaac ROS
CY Chen
@@ -45,7 +45,6 @@ SPDX-License-Identifier: Apache-2.0
ament_lint_auto
ament_lint_common
isaac_ros_test
- isaac_ros_dnn_inference_test
ament_cmake
diff --git a/isaac_ros_tensor_rt/src/tensor_rt_node.cpp b/isaac_ros_tensor_rt/src/tensor_rt_node.cpp
index 44e4bc9..22843a0 100644
--- a/isaac_ros_tensor_rt/src/tensor_rt_node.cpp
+++ b/isaac_ros_tensor_rt/src/tensor_rt_node.cpp
@@ -21,6 +21,8 @@
#include
#include
+#include "NvInferPluginUtils.h"
+
#include "isaac_ros_nitros_tensor_list_type/nitros_tensor_list.hpp"
#include "rclcpp/rclcpp.hpp"
@@ -87,9 +89,94 @@ const nitros::NitrosPublisherSubscriberConfigMap CONFIG_MAP = {
};
#pragma GCC diagnostic pop
+namespace
+{
constexpr int64_t default_max_workspace_size = 67108864l;
constexpr int64_t default_dla_core = -1;
+class TensorRT_Logger : public nvinfer1::ILogger
+{
+ void log(Severity severity, const char * msg) noexcept override
+ {
+ if (severity == Severity::kERROR) {
+ RCLCPP_ERROR(rclcpp::get_logger("TRT"), "TRT ERROR: %s", msg);
+ }
+ }
+} tensor_rt_logger;
+
+bool readTensorShapesFromEngine(
+ const std::string & engine_file_path, const std::vector & binding_names,
+ std::vector & tensor_shapes,
+ std::vector & tensor_data_types)
+{
+ // Try to load TensorRT engine and query model output dimension
+ std::vector plan;
+ // Open the file in binary mode and seek to the end
+ std::ifstream file(engine_file_path, std::ios::binary | std::ios::ate);
+ if (!file) {
+ return false;
+ }
+
+ // Get the size of the file and seek back to the beginning
+ const size_t size = file.tellg();
+ file.seekg(0);
+ // Reserve enough space in the output buffer and read the file contents into it
+ plan.resize(size);
+ const bool ret = static_cast(file.read(plan.data(), size));
+ file.close();
+
+ if (!ret) {
+ return false;
+ }
+
+ // Add plugins from TRT
+ if (!initLibNvInferPlugins(&tensor_rt_logger, "")) {
+ return false;
+ }
+
+ std::unique_ptr infer_runtime(
+ nvinfer1::createInferRuntime(tensor_rt_logger));
+ std::unique_ptr cuda_engine(
+ infer_runtime->deserializeCudaEngine(plan.data(), plan.size()));
+
+ for (uint64_t i = 0; i < binding_names.size(); i++) {
+ const std::string & binding_name = binding_names[i];
+ tensor_shapes.push_back(cuda_engine->getTensorShape(binding_name.c_str()));
+ tensor_data_types.push_back(cuda_engine->getTensorDataType(binding_name.c_str()));
+ }
+
+ return true;
+}
+
+bool readTensorShapesFromOnnx(
+ const std::string & onnx_file_path, const size_t output_count,
+ std::vector & tensor_shapes,
+ std::vector & tensor_data_types)
+{
+ std::unique_ptr builder(nvinfer1::createInferBuilder(tensor_rt_logger));
+ std::unique_ptr builderConfig(builder->createBuilderConfig());
+ std::unique_ptr network(builder->createNetworkV2(
+ 1U << static_cast(nvinfer1::NetworkDefinitionCreationFlag::kEXPLICIT_BATCH)));
+ std::unique_ptr onnx_parser(
+ nvonnxparser::createParser(*network, tensor_rt_logger));
+ if (!onnx_parser->parseFromFile(
+ onnx_file_path.c_str(),
+ static_cast(nvinfer1::ILogger::Severity::kWARNING)))
+ {
+ return false;
+ }
+
+ for (uint64_t i = 0; i < output_count; i++) {
+ auto * bind_tensor = network->getOutput(i);
+ tensor_shapes.push_back(bind_tensor->getDimensions());
+ tensor_data_types.push_back(bind_tensor->getType());
+ }
+
+ return true;
+}
+
+} // namespace
+
TensorRTNode::TensorRTNode(const rclcpp::NodeOptions & options)
: nitros::NitrosNode(options,
APP_YAML_FILENAME,
@@ -114,7 +201,8 @@ TensorRTNode::TensorRTNode(const rclcpp::NodeOptions & options)
dla_core_(declare_parameter("dla_core", default_dla_core)),
max_batch_size_(declare_parameter("max_batch_size", 1)),
enable_fp16_(declare_parameter("enable_fp16", true)),
- relaxed_dimension_check_(declare_parameter("relaxed_dimension_check", true))
+ relaxed_dimension_check_(declare_parameter("relaxed_dimension_check", true)),
+ num_blocks_(declare_parameter("num_blocks", 40))
{
RCLCPP_DEBUG(get_logger(), "[TensorRTNode] In TensorRTNode's constructor");
@@ -169,11 +257,81 @@ TensorRTNode::TensorRTNode(const rclcpp::NodeOptions & options)
startNitrosNode();
}
+size_t TensorRTNode::determineMaxTensorBlockSize()
+{
+ std::vector shapes;
+ std::vector data_types;
+
+ const auto & tensor_names = output_binding_names_;
+
+ // Read tensor information from engine or ONNX file.
+ if (readTensorShapesFromEngine(engine_file_path_, tensor_names, shapes, data_types)) {
+ RCLCPP_INFO(
+ get_logger(), "Read tensor shape information from TRT Model Engine: %s",
+ engine_file_path_.c_str());
+ } else if (readTensorShapesFromOnnx(model_file_path_, tensor_names.size(), shapes, data_types)) {
+ RCLCPP_INFO(
+ get_logger(), "Read tensor shape information from ONNX file: %s",
+ model_file_path_.c_str());
+ } else {
+ RCLCPP_ERROR(
+ get_logger(),
+ "Unable to read tensor shape info from TRT Model Engine or from ONNX file.");
+ return 0;
+ }
+
+ // Calculate maximum number of bytes needed for any single output tensor.
+ size_t max_tensor_size_bytes = 1;
+ for (uint64_t i = 0; i < shapes.size(); i++) {
+ const auto & shape = shapes[i];
+ const auto data_type = data_types[i];
+
+ uint64_t tensor_element_count = 1;
+ for (int j = 0; j < shape.nbDims; j++) {
+ tensor_element_count *= std::max(shape.d[j], 1);
+ }
+
+ uint64_t bytes_per_element;
+ switch (data_type) {
+ case nvinfer1::DataType::kINT8: bytes_per_element = sizeof(int8_t); break;
+ case nvinfer1::DataType::kFLOAT: bytes_per_element = sizeof(float_t); break;
+ case nvinfer1::DataType::kINT32: bytes_per_element = sizeof(int32_t); break;
+
+ // Fallback to max size of 8 bytes (64 bits) for other types
+ default: bytes_per_element = sizeof(size_t);
+ }
+
+ const size_t tensor_size_bytes = tensor_element_count * bytes_per_element;
+ max_tensor_size_bytes = std::max(max_tensor_size_bytes, tensor_size_bytes);
+ }
+
+ return max_tensor_size_bytes;
+}
+
+
void TensorRTNode::postLoadGraphCallback()
{
RCLCPP_INFO(get_logger(), "In TensorRTNode postLoadGraphCallback().");
- // Forward TensorRT inference Parameters
+ uint64_t block_size = determineMaxTensorBlockSize();
+ if (!block_size) {
+ block_size = default_max_workspace_size;
+ RCLCPP_WARN(
+ get_logger(), "Failed to get block size from model, set to the default size: %ld.",
+ default_max_workspace_size);
+ }
+ getNitrosContext().setParameterUInt64(
+ TENSOR_RT_ENTITY_NAME, "nvidia::gxf::BlockMemoryPool", "block_size", block_size);
+
+ const uint64_t output_num_blocks = output_tensor_names_.size() * num_blocks_;
+ getNitrosContext().setParameterUInt64(
+ TENSOR_RT_ENTITY_NAME, "nvidia::gxf::BlockMemoryPool", "num_blocks", output_num_blocks);
+
+ RCLCPP_INFO(
+ get_logger(), "Tensors %ld bytes, num outputs %ld x tensors per output %ld = %ld blocks",
+ block_size, num_blocks_, output_tensor_names_.size(), output_num_blocks);
+
+
if (!force_engine_update_ && model_file_path_.empty()) {
getNitrosContext().setParameterStr(
TENSOR_RT_ENTITY_NAME, TENSOR_RT_COMPONENT_TYPE, "model_file_path",
diff --git a/isaac_ros_tensor_rt/test/isaac_ros_tensor_rt_test.py b/isaac_ros_tensor_rt/test/isaac_ros_tensor_rt_test.py
index 52df228..b64081a 100644
--- a/isaac_ros_tensor_rt/test/isaac_ros_tensor_rt_test.py
+++ b/isaac_ros_tensor_rt/test/isaac_ros_tensor_rt_test.py
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -40,7 +40,7 @@ def generate_test_description():
# Remove the default trt_engine.plan before starting the node if it exists
try:
os.remove('/tmp/trt_engine.plan')
- print('Deleted exisiting /tmp/trt_engine.plan')
+ print('Deleted existing /tmp/trt_engine.plan')
except OSError as e:
if e.errno != errno.ENOENT:
print('File exists but error deleting /tmp/trt_engine.plan ')
@@ -102,13 +102,28 @@ class IsaacROSTensorRTNodeTest(IsaacROSBaseTest):
MODEL_PATH = '/tmp/trt_engine.plan'
def test_tensor_rt_node(self) -> None:
+ self.node._logger.info(
+ f'Generating model (timeout={self.MODEL_GENERATION_TIMEOUT_SEC}s)')
start_time = time.time()
+ wait_cycles = 1
while not os.path.isfile(self.MODEL_PATH):
- if (time.time() - start_time) > self.MODEL_GENERATION_TIMEOUT_SEC:
+ time_diff = time.time() - start_time
+ if time_diff > self.MODEL_GENERATION_TIMEOUT_SEC:
self.fail('Model generation timed out')
+ if time_diff > wait_cycles*10:
+ self.node._logger.info(
+ f'Waiting for model generation to finish... ({int(time_diff)}s passed)')
+ wait_cycles += 1
time.sleep(1)
+
+ self.node._logger.info(
+ f'Model generation was finished (took {(time.time() - start_time)}s)')
+ self.node._logger.info(
+ f'Waiting {self.GXF_WAIT_SEC}s for the engine to be initialized')
+
# Wait for TensorRT Engine to be initialized
time.sleep(self.GXF_WAIT_SEC)
+
self.node._logger.info('Starting Isaac ROS TensorRT Node POL Test')
received_messages = {}
@@ -148,6 +163,8 @@ def test_tensor_rt_node(self) -> None:
pub_tensor_list.tensors = [pub_tensor]
+ self.node._logger.info(
+ f'Publishing test tensors for {self.PYTHON_SUBSCRIBER_WAIT_SEC}s')
end_time = time.time() + self.PYTHON_SUBSCRIBER_WAIT_SEC
while time.time() < end_time:
tensor_pub.publish(pub_tensor_list)
diff --git a/isaac_ros_triton/CMakeLists.txt b/isaac_ros_triton/CMakeLists.txt
index 8cb89e8..b9369ef 100644
--- a/isaac_ros_triton/CMakeLists.txt
+++ b/isaac_ros_triton/CMakeLists.txt
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -15,7 +15,7 @@
#
# SPDX-License-Identifier: Apache-2.0
-cmake_minimum_required(VERSION 3.23.2)
+cmake_minimum_required(VERSION 3.22.1)
project(isaac_ros_triton LANGUAGES C CXX)
if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
@@ -50,6 +50,12 @@ if(BUILD_TESTING)
find_package(ament_lint_auto REQUIRED)
ament_lint_auto_find_test_dependencies()
+
+ # The FindPythonInterp and FindPythonLibs modules are removed
+ if(POLICY CMP0148)
+ cmake_policy(SET CMP0148 OLD)
+ endif()
+
find_package(launch_testing_ament_cmake REQUIRED)
add_launch_test(test/isaac_ros_triton_test_onnx.py TIMEOUT "300")
add_launch_test(test/isaac_ros_triton_test_tf.py TIMEOUT "300")
diff --git a/isaac_ros_triton/gxf/triton/CMakeLists.txt b/isaac_ros_triton/gxf/triton/CMakeLists.txt
index 3a56073..f5dfa94 100644
--- a/isaac_ros_triton/gxf/triton/CMakeLists.txt
+++ b/isaac_ros_triton/gxf/triton/CMakeLists.txt
@@ -21,9 +21,13 @@ if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
add_compile_options(-fPIC -w)
endif()
+set(CMAKE_WARN_DEPRECATED OFF CACHE BOOL "" FORCE)
+set(LAST_CMAKE_MESSAGE_LOG_LEVEL ${CMAKE_MESSAGE_LOG_LEVEL})
+set(CMAKE_MESSAGE_LOG_LEVEL ERROR)
+
# Dependencies
include(FetchContent)
-include(YamlCpp)
+find_package(yaml-cpp)
find_package(GXF ${ISAAC_ROS_GXF_VERSION} MODULE REQUIRED
COMPONENTS
core
@@ -36,6 +40,7 @@ set(protobuf_BUILD_EXPORT OFF)
set(protobuf_MSVC_STATIC_RUNTIME OFF)
set(Protobuf_USE_STATIC_LIBS ON)
set(Protobuf_BUILD_SHARED_LIBS OFF)
+
fetchcontent_declare(
protobuf
GIT_REPOSITORY https://github.com/protocolbuffers/protobuf.git
@@ -44,6 +49,7 @@ fetchcontent_declare(
)
fetchcontent_makeavailable(protobuf)
find_package(Protobuf QUIET)
+set(CMAKE_MESSAGE_LOG_LEVEL ${LAST_CMAKE_MESSAGE_LOG_LEVEL})
# Compile protocol buffers
file(GLOB ProtoFiles "${CMAKE_CURRENT_SOURCE_DIR}/nvds/include/*.proto")
diff --git a/isaac_ros_triton/gxf/triton/nvds/include/nvdsinfer.h b/isaac_ros_triton/gxf/triton/nvds/include/nvdsinfer.h
index b06bd3d..c8cdd94 100644
--- a/isaac_ros_triton/gxf/triton/nvds/include/nvdsinfer.h
+++ b/isaac_ros_triton/gxf/triton/nvds/include/nvdsinfer.h
@@ -287,11 +287,14 @@ typedef enum
/**
* Holds full dimensions (including batch size) for a layer.
*/
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wmissing-field-initializers"
typedef struct
{
int batchSize = 0;
NvDsInferDims dims = {0};
} NvDsInferBatchDims;
+#pragma GCC diagnostic pop
/**
* Extended structure for bound layer information which additionally includes
diff --git a/isaac_ros_triton/include/isaac_ros_triton_node/triton_node.hpp b/isaac_ros_triton/include/isaac_ros_triton_node/triton_node.hpp
index 83f50da..7606a48 100644
--- a/isaac_ros_triton/include/isaac_ros_triton_node/triton_node.hpp
+++ b/isaac_ros_triton/include/isaac_ros_triton_node/triton_node.hpp
@@ -1,5 +1,5 @@
// SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-// Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
diff --git a/isaac_ros_triton/launch/isaac_ros_triton.launch.py b/isaac_ros_triton/launch/isaac_ros_triton.launch.py
index 4f00cc6..d8f3296 100644
--- a/isaac_ros_triton/launch/isaac_ros_triton.launch.py
+++ b/isaac_ros_triton/launch/isaac_ros_triton.launch.py
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
diff --git a/isaac_ros_triton/package.xml b/isaac_ros_triton/package.xml
index 33c2c6e..28faf35 100644
--- a/isaac_ros_triton/package.xml
+++ b/isaac_ros_triton/package.xml
@@ -21,7 +21,7 @@
isaac_ros_triton
- 0.31.0
+ 2.0.0
DNN Inference support for Isaac ROS
CY Chen
@@ -42,7 +42,6 @@
ament_lint_auto
ament_lint_common
isaac_ros_test
- isaac_ros_dnn_inference_test
ament_cmake
diff --git a/isaac_ros_triton/test/isaac_ros_triton_test_onnx.py b/isaac_ros_triton/test/isaac_ros_triton_test_onnx.py
index e5dfdca..1ce059c 100644
--- a/isaac_ros_triton/test/isaac_ros_triton_test_onnx.py
+++ b/isaac_ros_triton/test/isaac_ros_triton_test_onnx.py
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -36,6 +36,9 @@ def generate_test_description():
dir_path = os.path.dirname(os.path.realpath(__file__))
model_dir = dir_path + '/../../test/models'
+ os.environ['CUDA_DEVICE_ORDER'] = 'PCI_BUS_ID'
+ os.environ['CUDA_VISIBLE_DEVICES'] = '0'
+
triton_node = ComposableNode(
package='isaac_ros_triton',
name='triton',
diff --git a/isaac_ros_triton/test/isaac_ros_triton_test_tf.py b/isaac_ros_triton/test/isaac_ros_triton_test_tf.py
index 648a259..0018ddc 100644
--- a/isaac_ros_triton/test/isaac_ros_triton_test_tf.py
+++ b/isaac_ros_triton/test/isaac_ros_triton_test_tf.py
@@ -1,5 +1,5 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
-# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
diff --git a/resources/0002_rgb.jpg b/resources/0002_rgb.jpg
deleted file mode 100644
index d292c8d..0000000
--- a/resources/0002_rgb.jpg
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:9581e73a9d1c4239646be25f5a4fb28b7f35204458987f2cfb07b81e45d2da78
-size 361994
diff --git a/resources/dope_rviz2.png b/resources/dope_rviz2.png
deleted file mode 100644
index 22cd655..0000000
--- a/resources/dope_rviz2.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:06066bf99bfe6029f08146f2f82a7d16494aca755ec925c30501af2526f4ec5b
-size 150849
diff --git a/resources/graph.png b/resources/graph.png
deleted file mode 100644
index f1234ad..0000000
--- a/resources/graph.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f42c002d79329a3c09a240efded5abaa1de1e46a23dd9d336493b89d1562450d
-size 19462
diff --git a/resources/isaac_ros_dnn_inference_nodegraph.png b/resources/isaac_ros_dnn_inference_nodegraph.png
deleted file mode 100644
index c5b3c7e..0000000
--- a/resources/isaac_ros_dnn_inference_nodegraph.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:7bee2d49ba0e9273bf7f537da16c9d9aefb859ba50d7ebc8d5b7d4b0daeab514
-size 27461
diff --git a/resources/isaac_ros_dnn_inference_peoplesemsegnet.jpg b/resources/isaac_ros_dnn_inference_peoplesemsegnet.jpg
deleted file mode 100644
index 1c100ca..0000000
--- a/resources/isaac_ros_dnn_inference_peoplesemsegnet.jpg
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f98cae81768f159d5be32c2fcef740752d50d9f4c8b07cad85b6b4145b5a796e
-size 183818
diff --git a/resources/isaac_ros_dnn_peoplenet.jpg b/resources/isaac_ros_dnn_peoplenet.jpg
deleted file mode 100644
index 44ba004..0000000
--- a/resources/isaac_ros_dnn_peoplenet.jpg
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:79f5751685d79660fe755a18e9e785dfb0e97785c3532336a5c03784f7a6e68c
-size 505228
diff --git a/resources/peoplesemsegnet_image.jpg b/resources/peoplesemsegnet_image.jpg
deleted file mode 100644
index e004101..0000000
--- a/resources/peoplesemsegnet_image.jpg
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f31f860475aa74fa8e2b4ad5f710484196fe20fee17c2dee19c500090e217467
-size 507577
diff --git a/resources/peoplesemsegnet_rviz2.png b/resources/peoplesemsegnet_rviz2.png
deleted file mode 100644
index 6526627..0000000
--- a/resources/peoplesemsegnet_rviz2.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:0c0d662b0e42d26c872c0d73c2f2bd7033b8f342f4df968b5294e2d383933947
-size 260071
diff --git a/resources/peoplesemsegnet_segimage.png b/resources/peoplesemsegnet_segimage.png
deleted file mode 100644
index 4e66e1a..0000000
--- a/resources/peoplesemsegnet_segimage.png
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:bd9bd0ead5b7a492deab5aa1044d884589f197abe14318886200f967406a5334
-size 8371
+
