Skip to content

Commit

Permalink
docs: update document for CNN model customization
Browse files Browse the repository at this point in the history
Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>
  • Loading branch information
ktro2828 committed Mar 9, 2023
1 parent 7f064da commit 38eedbc
Showing 1 changed file with 64 additions and 0 deletions.
64 changes: 64 additions & 0 deletions perception/traffic_light_classifier/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,70 @@ These colors and shapes are assigned to the message as follows:
| `red_max_s` | int | the maximum saturation of red color |
| `red_max_v` | int | the maximum value (brightness) of red color |

## Customization of CNN model

Currently, in Autoware, [MobileNetV2](https://arxiv.org/abs/1801.04381v3) is used as CNN classifier by default. The corresponding onnx file is [data/traffic_light_classifier_mobilenetv2.onnx](./data/traffic_light_classifier_mobilenetv2.onnx).
Also, you can apply the following models shown as below, for example.

- [EfficientNet](https://arxiv.org/abs/1905.11946v5)
- [ResNet](https://openaccess.thecvf.com/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html)
- [MobileNetV3](https://arxiv.org/abs/1905.02244)
...

In order to train models and export onnx model, we recommend [open-mmlab/mmclassification](https://github.com/open-mmlab/mmclassification.git).
Please follow the [official document](https://mmclassification.readthedocs.io/en/latest/) to install and experiment with mmclassification. If you get into troubles, [FAQ page](https://mmclassification.readthedocs.io/en/latest/faq.html) would help you.

The following steps are example of a quick-start.

### step 0. Install [MMCV](https://github.com/open-mmlab/mmcv.git) and [MIM](https://github.com/open-mmlab/mim.git)

_NOTE_ : First of all, install [PyTorch](https://pytorch.org/) suitable for your CUDA version (CUDA11.6 is supported in Autoware).

In order to install mmcv suitable for your CUDA version, install it specifying a url.

```shell
# Install mim
$ pip install -U openmim

# Install mmcv on a machine with CUDA11.6 and PyTorch1.13.0
$ pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu116/torch1.13/index.html
```

### step 1. Install MMClassification

You can install MMClassification as a Python package or from source.

```shell
# As a Python package
$ pip install mmcls

# From source
$ git clone https://github.com/open-mmlab/mmclassification.git
$ cd mmclassification
$ pip install -v -e .
```

### step 2. Train your model

Train model with your experiment configuration file. For the details of config file, see [here](https://mmclassification.readthedocs.io/en/latest/tutorials/config.html).

```shell
# [] is optional, you can start training from pre-trained checkpoint
$ mim train mmcls YOUR_CONFIG.py [--resume-from YOUR_CHECKPOINT.pth]
```

### step 3. Export onnx model

In exporting onnx, use `mmclassificatoin/tools/deployment/pytorch2onnx.py` or [open-mmlab/mmdeploy](https://github.com/open-mmlab/mmdeploy.git).

```shell
cd ~/mmclassification/tools/deployment
python3 pytorch2onnx.py YOUR_CONFIG.py ...
```

After obtaining your onnx model, update parameters defined in the launch file (e.g. `model_file_path`, `label_file_path`, `input_h`, `input_w`...).
Note that, we only support labels defined in [autoware_auto_perception_msgs::msg::TrafficLight](https://github.com/tier4/autoware_auto_msgs/blob/tier4/main/autoware_auto_perception_msgs/msg/TrafficLight.idl).

## Assumptions / Known limits

<!-- Write assumptions and limitations of your implementation.
Expand Down

0 comments on commit 38eedbc

Please sign in to comment.