diff --git a/README.md b/README.md index f59c83fb..66aba9de 100644 --- a/README.md +++ b/README.md @@ -37,10 +37,9 @@ English | [简体中文](README_cn.md) This is the official implementation of the paper "[DETRs Beat YOLOs on Real-time Object Detection](https://arxiv.org/abs/2304.08069)". - ## Updates!!! ---- +- \[2023.11.07\] Add pytorch *rtdetr_r34vd* for requests [#107](https://github.com/lyuwenyu/RT-DETR/issues/107), [#114](https://github.com/lyuwenyu/RT-DETR/issues/114) - \[2023.11.05\] upgrade the logic of `remap_mscoco_category` to facilitate training of custom datasets, see detils in [*Train custom data*](./rtdetr_pytorch/) part. [#81](https://github.com/lyuwenyu/RT-DETR/commit/95fc522fd7cf26c64ffd2ad0c622c392d29a9ebf) - \[2023.10.23\] Add [*discussion for deployments*](https://github.com/lyuwenyu/RT-DETR/issues/95), supported onnxruntime, TensorRT, openVINO - \[2023.10.12\] Add tuning code for pytorch version, now you can tuning rtdetr based on pretrained weights @@ -56,7 +55,7 @@ This is the official implementation of the paper "[DETRs Beat YOLOs on Real-time ## Implementations - rtdetr paddle: [code](./rtdetr_paddle), [weights](./rtdetr_paddle) -- rtdetr pytorch: [code](./rtdetr_pytorch), [weights](https://github.com/lyuwenyu/RT-DETR/issues/42) +- rtdetr pytorch: [code](./rtdetr_pytorch), [weights](./rtdetr_pytorch) | Model | Epoch | Input shape | Dataset | $AP^{val}$ | $AP^{val}_{50}$| Params(M) | FLOPs(G) | T4 TensorRT FP16(FPS) diff --git a/rtdetr_pytorch/README.md b/rtdetr_pytorch/README.md index cae34f10..f8a2e7ad 100644 --- a/rtdetr_pytorch/README.md +++ b/rtdetr_pytorch/README.md @@ -1,11 +1,35 @@ ## TODO +
+ see details + - [x] Training - [x] Evaluation - [x] Export onnx - [x] Upload source code -- [x] Upload weight convert from paddle, see [links](https://github.com/lyuwenyu/RT-DETR/issues/42) -- [x] Align training details with the [paddle version](../rtdetr_paddle/) -- [x] Tuning rtdetr based on [pretrained weights](https://github.com/lyuwenyu/RT-DETR/issues/42) +- [x] Upload weight convert from paddle, see [*links*](https://github.com/lyuwenyu/RT-DETR/issues/42) +- [x] Align training details with the [*paddle version*](../rtdetr_paddle/) +- [x] Tuning rtdetr based on [*pretrained weights*](https://github.com/lyuwenyu/RT-DETR/issues/42) + +
+ + +## Model Zoo + +| Model | Dataset | Input Size | APval | AP50val | #Params(M) | FPS | checkpoint | +| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | +rtdetr_r18vd | COCO | 640 | 46.4 | 63.7 | 20 | 217 | [url*](https://github.com/lyuwenyu/storage/releases/download/v0.1/rtdetr_r18vd_dec3_6x_coco_from_paddle.pth) +rtdetr_r34vd | COCO | 640 | 48.9 | 66.8 | 31 | 161 | [url*](https://github.com/lyuwenyu/storage/releases/download/v0.1/rtdetr_r34vd_dec4_6x_coco_from_paddle.pth) +rtdetr_r50vd_m | COCO | 640 | 51.3 | 69.5 | 36 | 145 | [url*](https://github.com/lyuwenyu/storage/releases/download/v0.1/rtdetr_r50vd_m_6x_coco_from_paddle.pth) +rtdetr_r50vd | COCO | 640 | 53.1 | 71.2| 42 | 108 | [url*](https://github.com/lyuwenyu/storage/releases/download/v0.1/rtdetr_r50vd_6x_coco_from_paddle.pth) +rtdetr_r101vd | COCO | 640 | 54.3 | 72.8 | 76 | 74 | [url*](https://github.com/lyuwenyu/storage/releases/download/v0.1/rtdetr_r101vd_6x_coco_from_paddle.pth) +rtdetr_18vd | COCO+Objects365 | 640 | 49.0 | 66.5 | 20 | 217 | [url*](https://github.com/lyuwenyu/storage/releases/download/v0.1/rtdetr_r18vd_5x_coco_objects365_from_paddle.pth) +rtdetr_r50vd | COCO+Objects365 | 640 | 55.2 | 73.4 | 42 | 108 | [url*](https://github.com/lyuwenyu/storage/releases/download/v0.1/rtdetr_r50vd_2x_coco_objects365_from_paddle.pth) +rtdetr_r101vd | COCO+Objects365 | 640 | 56.2 | 74.5 | 76 | 74 | [url*](https://github.com/lyuwenyu/storage/releases/download/v0.1/rtdetr_r101vd_2x_coco_objects365_from_paddle.pth) + +Notes +- `COCO + Objects365` in the table means finetuned model on `COCO` using pretrained weights trained on `Objects365`. +- `url``*` is the url of pretrained weights convert from paddle model for save energy. *It may have slight differences between this table and paper* + ## Quick start @@ -79,7 +103,7 @@ python tools/export_onnx.py -c configs/rtdetr/rtdetr_r18vd_6x_coco.yml -r path/t
Train custom data -1. set `remap_mscoco_category: False`. This variable only works for ms-coco dataset. If you want to use `remap_mscoco_category` logic on your dataset, please modify variable [`mscoco_category2name`](https://github.com/lyuwenyu/RT-DETR/blob/main/rtdetr_pytorch/src/data/coco/coco_dataset.py) based on your dataset. +1. set `remap_mscoco_category: False`. This variable only works for ms-coco dataset. If you want to use `remap_mscoco_category` logic on your dataset, please modify variable [`mscoco_category2name`](https://github.com/lyuwenyu/RT-DETR/blob/main/rtdetr_pytorch/src/data/coco/coco_dataset.py#L154) based on your dataset. 2. add `-t path/to/checkpoint` (optinal) to tuning rtdetr based on pretrained checkpoint. see [training script details](./tools/README.md).
diff --git a/rtdetr_pytorch/configs/rtdetr/rtdetr_r34vd_6x_coco.yml b/rtdetr_pytorch/configs/rtdetr/rtdetr_r34vd_6x_coco.yml new file mode 100644 index 00000000..e7779a32 --- /dev/null +++ b/rtdetr_pytorch/configs/rtdetr/rtdetr_r34vd_6x_coco.yml @@ -0,0 +1,48 @@ + +__include__: [ + '../dataset/coco_detection.yml', + '../runtime.yml', + './include/dataloader.yml', + './include/optimizer.yml', + './include/rtdetr_r50vd.yml', +] + + +output_dir: ./output/rtdetr_r34vd_6x_coco + + +PResNet: + depth: 34 + freeze_at: -1 + freeze_norm: False + pretrained: True + + +HybridEncoder: + in_channels: [128, 256, 512] + hidden_dim: 256 + expansion: 0.5 + + +RTDETRTransformer: + num_decoder_layers: 4 + + + +optimizer: + type: AdamW + params: + - + params: '^(?=.*backbone)(?=.*norm|bn).*$' + weight_decay: 0. + lr: 0.00001 + - + params: '^(?=.*backbone)(?!.*norm|bn).*$' + lr: 0.00001 + - + params: '^(?=.*(?:encoder|decoder))(?=.*(?:norm|bn|bias)).*$' + weight_decay: 0. + + lr: 0.0001 + betas: [0.9, 0.999] + weight_decay: 0.0001 diff --git a/rtdetr_pytorch/src/optim/ema.py b/rtdetr_pytorch/src/optim/ema.py index 18465968..bf962b3a 100644 --- a/rtdetr_pytorch/src/optim/ema.py +++ b/rtdetr_pytorch/src/optim/ema.py @@ -83,8 +83,8 @@ def state_dict(self, ): def load_state_dict(self, state): self.module.load_state_dict(state['module']) - self.updates = state['updates'] - self.warmups = state['warmups'] + if 'updates' in state: + self.updates = state['updates'] def forwad(self, ): raise RuntimeError('ema...')