Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(design/autoware-architecture/sensing): add radar sensing document #518

Merged
merged 18 commits into from
Feb 21, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
41eff06
add radar sensing design document
scepter914 Feb 13, 2024
15b5c49
style(pre-commit): autofix
pre-commit-ci[bot] Feb 13, 2024
f4e2c0b
update document
scepter914 Feb 13, 2024
22ae4fe
fix typo
scepter914 Feb 19, 2024
03ba9be
fix from supported-function to reference-implementation
scepter914 Feb 19, 2024
fab964c
fix from supported-function to reference-implementation
scepter914 Feb 19, 2024
e3c7061
Merge branch 'main' into feat/radar_sensing
scepter914 Feb 19, 2024
59a2296
Update docs/design/autoware-architecture/sensing/data-types/radar-dat…
scepter914 Feb 20, 2024
5ed9a17
Update docs/design/autoware-architecture/sensing/data-types/radar-dat…
scepter914 Feb 20, 2024
f9da4fd
Update docs/design/autoware-architecture/sensing/data-types/radar-dat…
scepter914 Feb 20, 2024
966ba2d
Update docs/design/autoware-architecture/sensing/data-types/radar-dat…
scepter914 Feb 20, 2024
738860a
Update docs/design/autoware-architecture/sensing/data-types/radar-dat…
scepter914 Feb 20, 2024
edd574f
Update docs/design/autoware-architecture/sensing/data-types/radar-dat…
scepter914 Feb 20, 2024
27cc117
Update docs/design/autoware-architecture/sensing/data-types/radar-dat…
scepter914 Feb 20, 2024
4a3421c
Update docs/design/autoware-architecture/sensing/data-types/radar-dat…
scepter914 Feb 20, 2024
78db2a2
Update docs/design/autoware-architecture/sensing/data-types/radar-dat…
scepter914 Feb 20, 2024
99cd70c
Update docs/design/autoware-architecture/sensing/data-types/radar-dat…
scepter914 Feb 20, 2024
df7efc6
fix documents
scepter914 Feb 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

This file was deleted.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Radar objects data pre-processing design

## Overview

### Pipeline

This diagram describes the pre-process pipeline for radar objects.

![radar-objects-sensing](image/radar-objects-sensing.drawio.svg)

### Interface

- Input
- Radar data from device
- Twist information of ego vehicle motion
- Output
- Merged radar objects information

### Note

- The radar pre-process package filter noise through the `ros-perception/radar_msgs/msg/RadarTrack.msg` message type with sensor coordinate.
scepter914 marked this conversation as resolved.
Show resolved Hide resolved
- It is recommended to change the coordinate system from each sensor to base_link with message converter.
- If there are multiple radar objects, the object merger package concatenate these objects.
scepter914 marked this conversation as resolved.
Show resolved Hide resolved

## Input

Autoware uses radar objects data type as [radar_msgs/msg/RadarTracks.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarTracks.msg).
In detail, please see [Data message for radars](supported-functions/data-message.md).
scepter914 marked this conversation as resolved.
Show resolved Hide resolved

## Supported functions
miursh marked this conversation as resolved.
Show resolved Hide resolved

### Device driver for radars

Autoware support `ros-perception/radar_msgs/msg/RadarScan.msg` and `autoware_auto_perception_msgs/msg/TrackedObjects.msg` for Radar drivers.
scepter914 marked this conversation as resolved.
Show resolved Hide resolved

In detail, please see [Device driver for radars](supported-functions/device-driver.md).
scepter914 marked this conversation as resolved.
Show resolved Hide resolved

### Noise filter

- [radar_tracks_noise_filter](https://github.com/autowarefoundation/autoware.universe/tree/main/sensing/radar_tracks_noise_filter)

Radar can detect x-axis velocity as doppler velocity, but cannot detect y-axis velocity. Some radar can estimate y-axis velocity inside the device, but it sometimes lack precision. This package treats these objects as noise by y-axis threshold filter.

### Message converter

- [radar_tracks_msgs_converter](https://github.com/autowarefoundation/autoware.universe/tree/main/perception/radar_tracks_msgs_converter)

This package converts from `radar_msgs/msg/RadarTracks` into `autoware_auto_perception_msgs/msg/DetectedObject` with ego vehicle motion compensation and coordinate transform.

### Object merger

- [object_merger](https://github.com/autowarefoundation/autoware.universe/tree/main/perception/object_merger)

This package can merge 2 topics of `autoware_auto_perception_msgs/msg/DetectedObject`.

- [simple_object_merger](https://github.com/autowarefoundation/autoware.universe/tree/main/perception/simple_object_merger)

This package can merge simply multiple topics of `autoware_auto_perception_msgs/msg/DetectedObject`.
Different from [object_merger](https://github.com/autowarefoundation/autoware.universe/tree/main/perception/object_merger), this package doesn't use association algorithm and can merge with low calculation cost.

- [topic_tools](https://github.com/ros-tooling/topic_tools)

If a vehicle has 1 radar, the topic relay tool in `topic_tools` can be used.
Example is as below.

```xml
<launch>
<group>
<push-ros-namespace namespace="radar"/>
<group>
<push-ros-namespace namespace="front_center"/>
<include file="$(find-pkg-share example_launch)/launch/ars408.launch.xml">
<arg name="interface" value="can0" />
</include>
</group>
<node pkg="topic_tools" exec="relay" name="radar_relay" output="log" args="front_center/detected_objects detected_objects"/>
</group>
</launch>
```

## Appendix

### Discussion

Radar architecture design is discussed as below.

- [Discussion 2531](https://github.com/orgs/autowarefoundation/discussions/2531)
- [Discussion 2532](https://github.com/orgs/autowarefoundation/discussions/2532).
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Radar pointcloud data pre-processing design

## Overview

### Pipeline

This diagram describes the pre-process pipeline for radar pointcloud.

![radar-pointcloud-sensing](image/radar-pointcloud-sensing.drawio.svg)

### Interface

- Input
- Radar data from device
- Twist information of ego vehicle motion
- Output
- Dynamic radar pointcloud (`ros-perception/radar_msgs/msg/RadarScan.msg`)
Shin-kyoto marked this conversation as resolved.
Show resolved Hide resolved
- Noise filtered radar pointcloud (`sensor_msgs/msg/Pointcloud2.msg`)

### Note

- In the sensing layer, the radar pre-process packages filter noise through the `ros-perception/radar_msgs/msg/RadarScan.msg` message type with sensor coordinate.
- For use of radar pointcloud data by LiDAR packages, we would like to propose a converter for creating `sensor_msgs/msg/Pointcloud2.msg` from `ros-perception/radar_msgs/msg/RadarScan.msg`.

## Supported functions

### Data message for radars

Autoware uses radar objects data type as [radar_msgs/msg/RadarScan.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarScan.msg).
In detail, please see [Data message for radars](supported-functions/data-message.md).

### Device driver for radars

Autoware support `ros-perception/radar_msgs/msg/RadarScan.msg` and `autoware_auto_perception_msgs/msg/TrackedObjects.msg` for Radar drivers.

In detail, please see [Device driver for radars](supported-functions/device-driver.md).

### Basic noise filter

- [radar_threshold_filter](https://github.com/autowarefoundation/autoware.universe/tree/main/sensing/radar_threshold_filter)

This package removes pointcloud noise with low amplitude, edge angle, too near pointcloud by threshold.
Shin-kyoto marked this conversation as resolved.
Show resolved Hide resolved

### Filter to static/dynamic pointcloud

- [radar_static_pointcloud_filter](https://github.com/autowarefoundation/autoware.universe/tree/main/sensing/radar_static_pointcloud_filter)

This package extract static/dynamic radar pointcloud by using doppler velocity and ego motion.
scepter914 marked this conversation as resolved.
Show resolved Hide resolved
The static radar pointcloud use for localization like NDT scan matching, and the dynamic radar pointclou use for dynamic object detection.

Check warning on line 49 in docs/design/autoware-architecture/sensing/data-types/radar-data/radar-pointcloud-data.md

View workflow job for this annotation

GitHub Actions / spell-check-differential

Unknown word (pointclou)

### Message converter from RadarScan to Pointcloud2

- [radar_scan_to_pointcloud2](https://github.com/autowarefoundation/autoware.universe/tree/main/sensing/radar_scan_to_pointcloud2)

For convenient use of radar pointcloud within existing LiDAR packages, we suggest a `radar_scan_to_pointcloud2_convertor` package for conversion from `ros-perception/radar_msgs/msg/RadarScan.msg` to `sensor_msgs/msg/Pointcloud2.msg`.

| | LiDAR package | Radar package |
| :--------: | :-------------------------------: | :-------------------------------------------: |
| message | `sensor_msgs/msg/Pointcloud2.msg` | `ros-perception/radar_msgs/msg/RadarScan.msg` |
| coordinate | (x, y, z) | (r, θ, φ) |
| value | intensity | amplitude, doppler velocity |

For considered use cases,

- Use [pointcloud_preprocessor](https://github.com/autowarefoundation/autoware.universe/tree/main/sensing/pointcloud_preprocessor) for radar scan.
- Apply obstacle segmentation like [ground segmentation](https://github.com/autowarefoundation/autoware.universe/tree/main/perception/ground_segmentation) to radar points for LiDAR-less (camera + radar) systems.

## Appendix

### Discussion

Radar architecture design is discussed as below.

- [Discussion 2531](https://github.com/orgs/autowarefoundation/discussions/2531)
- [Discussion 2532](https://github.com/orgs/autowarefoundation/discussions/2532).
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Data message for radars

## Summary

To sum up, Autoware uses radar data type as below.

- [radar_msgs/msg/RadarScan.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarScan.msg) for radar pointcloud
- [radar_msgs/msg/RadarTracks.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarTracks.msg) for radar objects.

## Radar data message for pointcloud

### Message definition

- <https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarScan.msg>

```sh
std_msgs/Header header
radar_msgs/RadarReturn[] returns
```

- <https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarReturn.msg>

```sh
# All variables below are relative to the radar's frame of reference.
# This message is not meant to be used alone but as part of a stamped or array message.

float32 range # Distance (m) from the sensor to the detected return.
float32 azimuth # Angle (in radians) in the azimuth plane between the sensor and the detected return.
# Positive angles are anticlockwise from the sensor and negative angles clockwise from the sensor as per REP-0103.
float32 elevation # Angle (in radians) in the elevation plane between the sensor and the detected return.
# Negative angles are below the sensor. For 2D radar, this will be 0.
float32 doppler_velocity # The doppler speeds (m/s) of the return.
float32 amplitude # The amplitude of the of the return (dB)
```

## Radar data message for tracked objects

### Message definition

- [radar_msgs/msg/RadarTrack.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarTrack.msg)

```sh
# Object classifications (Additional vendor-specific classifications are permitted starting from 32000 eg. Car)
uint16 NO_CLASSIFICATION=0
uint16 STATIC=1
uint16 DYNAMIC=2

unique_identifier_msgs/UUID uuid # A unique ID of the object generated by the radar.
# Note: The z component of these fields is ignored for 2D tracking.
geometry_msgs/Point position # x, y, z coordinates of the centroid of the object being tracked.
geometry_msgs/Vector3 velocity # The velocity of the object in each spatial dimension.
geometry_msgs/Vector3 acceleration # The acceleration of the object in each spatial dimension.
geometry_msgs/Vector3 size # The object size as represented by the radar sensor eg. length, width, height OR the diameter of an ellipsoid in the x, y, z, dimensions
# and is from the sensor frame's view.
uint16 classification # An optional classification of the object (see above)
float32[6] position_covariance # Upper-triangle covariance about the x, y, z axes
float32[6] velocity_covariance # Upper-triangle covariance about the x, y, z axes
float32[6] acceleration_covariance # Upper-triangle covariance about the x, y, z axes
float32[6] size_covariance # Upper-triangle covariance about the x, y, z axes
```

### Message usage for RadarTracks

- Object classifications

For object classifications [radar_msgs/msg/RadarTrack.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarTrack.msg), additional vendor-specific classifications are permitted starting from 32000.
In addition to this, considering [existing label definition](https://gitlab.com/autowarefoundation/autoware.auto/autoware_auto_msgs/-/blob/master/autoware_auto_perception_msgs/msg/ObjectClassification.idl), Autoware define object classifications as below.

```sh
uint16 UNKNOWN = 32000;
uint16 CAR = 32001;
uint16 TRUCK = 32002;
uint16 BUS = 32003;
uint16 TRAILER = 32004;
uint16 MOTORCYCLE = 32005;
uint16 BICYCLE = 32006;
uint16 PEDESTRIAN = 32007;
```

For detail implementation, please see [radar_tracks_msgs_converter](https://github.com/autowarefoundation/autoware.universe/tree/main/perception/radar_tracks_msgs_converter).

## Note

### Survey for radar message

Depending on the sensor manufacturer and its purpose, each sensor might exchange raw, post-processed data. This section introduces a survey about the previously developed messaging systems in the open-source community. Although there are many kinds of outputs, radar mainly adopt two types as outputs, pointcloud and objects. Related discussion for message definition in ros-perception are [PR #1](https://github.com/ros-perception/radar_msgs/pull/1), [PR #2](https://github.com/ros-perception/radar_msgs/pull/2), and [PR #3](https://github.com/ros-perception/radar_msgs/pull/3). Existing open source softwares for radar are summarized in these PR.
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Device driver for radars

## Interface for radar devices

The radar driver converts the communication data from radars to ROS2 topics.

Check warning on line 5 in docs/design/autoware-architecture/sensing/data-types/radar-data/supported-functions/device-driver.md

View workflow job for this annotation

GitHub Actions / spell-check-differential

Unknown word (ROS2)
The following communication types are considered:

- CAN
- CAN-FD
- Ethernet

![draw.io figure](image/radar-communication.drawio.svg)

## Software Interface

Autoware support `ros-perception/radar_msgs/msg/RadarScan.msg` and `autoware_auto_perception_msgs/msg/TrackedObjects.msg` for Radar drivers.

![draw.io figure](image/radar-driver.drawio.svg)

ROS driver receive the data from radar devices.

- Scan (pointcloud)
- Tracked objects
- Diagnostics results

The ROS drivers receives the following data from the radar device.

- Time synchronization
- Ego vehicle status, which often need for tracked objects calculation

## Radar driver example

- [ARS408 driver](https://github.com/tier4/ars408_driver)
Loading
Loading