diff --git a/perception/traffic_light_classifier/README.md b/perception/traffic_light_classifier/README.md index 24928c9ee4de0..b598d877d5293 100644 --- a/perception/traffic_light_classifier/README.md +++ b/perception/traffic_light_classifier/README.md @@ -87,6 +87,70 @@ These colors and shapes are assigned to the message as follows: | `red_max_s` | int | the maximum saturation of red color | | `red_max_v` | int | the maximum value (brightness) of red color | +## Customization of CNN model + +Currently, in Autoware, [MobileNetV2](https://arxiv.org/abs/1801.04381v3) is used as CNN classifier by default. The corresponding onnx file is [data/traffic_light_classifier_mobilenetv2.onnx](./data/traffic_light_classifier_mobilenetv2.onnx). +Also, you can apply the following models shown as below, for example. + +- [EfficientNet](https://arxiv.org/abs/1905.11946v5) +- [ResNet](https://openaccess.thecvf.com/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html) +- [MobileNetV3](https://arxiv.org/abs/1905.02244) + ... + +In order to train models and export onnx model, we recommend [open-mmlab/mmclassification](https://github.com/open-mmlab/mmclassification.git). +Please follow the [official document](https://mmclassification.readthedocs.io/en/latest/) to install and experiment with mmclassification. If you get into troubles, [FAQ page](https://mmclassification.readthedocs.io/en/latest/faq.html) would help you. + +The following steps are example of a quick-start. + +### step 0. Install [MMCV](https://github.com/open-mmlab/mmcv.git) and [MIM](https://github.com/open-mmlab/mim.git) + +_NOTE_ : First of all, install [PyTorch](https://pytorch.org/) suitable for your CUDA version (CUDA11.6 is supported in Autoware). + +In order to install mmcv suitable for your CUDA version, install it specifying a url. + +```shell +# Install mim +$ pip install -U openmim + +# Install mmcv on a machine with CUDA11.6 and PyTorch1.13.0 +$ pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu116/torch1.13/index.html +``` + +### step 1. Install MMClassification + +You can install MMClassification as a Python package or from source. + +```shell +# As a Python package +$ pip install mmcls + +# From source +$ git clone https://github.com/open-mmlab/mmclassification.git +$ cd mmclassification +$ pip install -v -e . +``` + +### step 2. Train your model + +Train model with your experiment configuration file. For the details of config file, see [here](https://mmclassification.readthedocs.io/en/latest/tutorials/config.html). + +```shell +# [] is optional, you can start training from pre-trained checkpoint +$ mim train mmcls YOUR_CONFIG.py [--resume-from YOUR_CHECKPOINT.pth] +``` + +### step 3. Export onnx model + +In exporting onnx, use `mmclassificatoin/tools/deployment/pytorch2onnx.py` or [open-mmlab/mmdeploy](https://github.com/open-mmlab/mmdeploy.git). + +```shell +cd ~/mmclassification/tools/deployment +python3 pytorch2onnx.py YOUR_CONFIG.py ... +``` + +After obtaining your onnx model, update parameters defined in the launch file (e.g. `model_file_path`, `label_file_path`, `input_h`, `input_w`...). +Note that, we only support labels defined in [autoware_auto_perception_msgs::msg::TrafficLight](https://github.com/tier4/autoware_auto_msgs/blob/tier4/main/autoware_auto_perception_msgs/msg/TrafficLight.idl). + ## Assumptions / Known limits