Feature/point pillars

[k0suke-murakami]

Status

DEVELOPMENT

Description

Detection results are visualized through a nice "KITTI Viewer Web" from @traveller59 repo.

Implemented new feature PointPillars. paper
This is a neural network algorithm for 3D bounding box detection from pointcloud data.
Good accuracy is proved in KITTI object detection benchmark.KITTI link

Plus, the fastest algorithm in the KITTI benchmark.
According to the paper, it runs in 16ms.
But by writing CUDA code for both preprocess and postprocess, it achieves 12~15ms runtime.

Tested on Shinjuku video and Nisshin video

Related issue

autowarefoundation/autoware_ai#534

Todos

• Tests
• Documentation

Interface

Input: /points_raw [sensor_msg::PointCloud2]
output: /detection/lidar_detector/objects [autoware_msgs::DetectedObjectArray]
output: /detection/lidar_detector/objects_markers [visualization_msgs::MarkerArray]

How to test

catkin_make run_tests_lidar_point_pillars_gtest

The reason for the small amount of test code is because some CUDA codes result in the different output for the same input.

How to launch

Dependency

• CUDA 9.0 or 10.0
• TensorRT: Tested with 5.0.2
• Pretrained model can be downloaded from this repo
• Also, please check the README.md for more details.

Launch

roslaunch lidar_point_pillars lidar_point_pillars.launch pfe_onnx_file:=/PATH/TO/FILE.onnx rpn_onnx_file:=/PATH/TO/FILE.onnx

Or, you can launch through runtime manager in Computing tab.

You will see visualized output by subscribing to /detection/lidar_detector/objects_markers

[yukkysaito]

DeepLearning based detection packages manage setups individually too. Can you prepare the scripts directory and prepare the setup script? (I would like to manage it with ansible etc. in the future)

[amc-nu]

terrific

[dejanpan]

@gbiggs @esteve @kfunaoka this package uses a pre-trained network that is not for commercial purposes.

I think that we need to find a way to clearly mark such packages, otherwise it will be difficult to find them later and it will also cause confusion with Autoware users trying to commercialize such packages.

[gbiggs]

Yes, I agree. I'll add it to my notes for the documentation template.

I think we also need a way for a user to easily select between using a non-commercial pre-trained network that we provide and a network they provide themselves (as well as a good method and documentation for how to train a network). This would improve the usefulness of Autoware without placing a burden on us to provide trained networks. I'll add this to my increasing pile of notes on what Autoware should be. :)

[kargarisaac]

Hi,
What is the trained model for? cars or people or what? In the paper, they mentioned that they trained 2 models for cars and pedestrians/cyclists?
If you use one model for all classes, what is the parameter values? such as thresholds, etc?
Thank you

[kargarisaac]

@k0suke-murakami
I tested this using sample_moriyama_150324.bag file and docker but got nothing. Do I need to install tensorrt for docker? It would be great to write down a pipeline to run it in docker or out of that. I tried to install the point_pillars package in my ros workspace, but not successful yet.