[Feature] Support the results of Tracktor on MOT15, MOT16 and MOT20 (#…

…217)

* support mot15 16 20

* [Feature] Support MOT15 MOT16 and MOT20

* add mot15 and mot16

* add mot20

* add mot20

* rename ckpt path

* delete some useless configs

* fix a bug

* fix a bug

* add fps

* fix a bug

* rebase configs

* fix some bugs

* fix a bug

* fix a bug

* fix some bug

* benchmark

* modify readme

* modify model path

* metafile

* fix a bug

* fix some typos

configs/det/faster-rcnn_r50_fpn_4e_mot15-half.py

@@ -0,0 +1,14 @@
+USE_MMDET = True
+_base_ = ['./faster-rcnn_r50_fpn_4e_mot17-half.py']
+# data
+data_root = 'data/MOT15/'
+data = dict(
+    train=dict(
+        ann_file=data_root + 'annotations/half-train_cocoformat.json',
+        img_prefix=data_root + 'train'),
+    val=dict(
+        ann_file=data_root + 'annotations/half-val_cocoformat.json',
+        img_prefix=data_root + 'train'),
+    test=dict(
+        ann_file=data_root + 'annotations/half-val_cocoformat.json',
+        img_prefix=data_root + 'train'))

configs/det/faster-rcnn_r50_fpn_4e_mot16-half.py

@@ -0,0 +1,14 @@
+USE_MMDET = True
+_base_ = ['./faster-rcnn_r50_fpn_4e_mot17-half.py']
+# data
+data_root = 'data/MOT16/'
+data = dict(
+    train=dict(
+        ann_file=data_root + 'annotations/half-train_cocoformat.json',
+        img_prefix=data_root + 'train'),
+    val=dict(
+        ann_file=data_root + 'annotations/half-val_cocoformat.json',
+        img_prefix=data_root + 'train'),
+    test=dict(
+        ann_file=data_root + 'annotations/half-val_cocoformat.json',
+        img_prefix=data_root + 'train'))

configs/det/faster-rcnn_r50_fpn_4e_mot17.py

@@ -1,5 +1,6 @@
 USE_MMDET = True
 _base_ = ['./faster-rcnn_r50_fpn_4e_mot17-half.py']
+# data
 data_root = 'data/MOT17/'
 data = dict(
     train=dict(ann_file=data_root + 'annotations/train_cocoformat.json'),

configs/det/faster-rcnn_r50_fpn_8e_mot20-half.py

@@ -0,0 +1,28 @@
+USE_MMDET = True
+_base_ = ['./faster-rcnn_r50_fpn_4e_mot17-half.py']
+model = dict(
+    detector=dict(
+        rpn_head=dict(bbox_coder=dict(clip_border=True)),
+        roi_head=dict(
+            bbox_head=dict(bbox_coder=dict(clip_border=True), num_classes=1))))
+# data
+data_root = 'data/MOT20/'
+data = dict(
+    train=dict(
+        ann_file=data_root + 'annotations/half-train_cocoformat.json',
+        img_prefix=data_root + 'train'),
+    val=dict(
+        ann_file=data_root + 'annotations/half-val_cocoformat.json',
+        img_prefix=data_root + 'train'),
+    test=dict(
+        ann_file=data_root + 'annotations/half-val_cocoformat.json',
+        img_prefix=data_root + 'train'))
+# learning policy
+lr_config = dict(
+    policy='step',
+    warmup='linear',
+    warmup_iters=100,
+    warmup_ratio=1.0 / 100,
+    step=[6])
+# runtime settings
+total_epochs = 8

configs/det/faster-rcnn_r50_fpn_8e_mot20.py

@@ -0,0 +1,28 @@
+USE_MMDET = True
+_base_ = ['./faster-rcnn_r50_fpn_4e_mot17-half.py']
+model = dict(
+    detector=dict(
+        rpn_head=dict(bbox_coder=dict(clip_border=True)),
+        roi_head=dict(
+            bbox_head=dict(bbox_coder=dict(clip_border=True), num_classes=1))))
+# data
+data_root = 'data/MOT20/'
+data = dict(
+    train=dict(
+        ann_file=data_root + 'annotations/train_cocoformat.json',
+        img_prefix=data_root + 'train'),
+    val=dict(
+        ann_file=data_root + 'annotations/train_cocoformat.json',
+        img_prefix=data_root + 'train'),
+    test=dict(
+        ann_file=data_root + 'annotations/train_cocoformat.json',
+        img_prefix=data_root + 'train'))
+# learning policy
+lr_config = dict(
+    policy='step',
+    warmup='linear',
+    warmup_iters=100,
+    warmup_ratio=1.0 / 100,
+    step=[6])
+# runtime settings
+total_epochs = 8

configs/mot/deepsort/README.md

@@ -39,3 +39,5 @@ We directly use the ReID model from [Tracktor](https://github.com/phil-bergmann/
 | R50-FasterRCNN-FPN | R50 | half-train | half-val | N     | 13.8  |  63.8 | 69.6 | 15060 | 40326 | 3183 | [config](deepsort_faster-rcnn_fpn_4e_mot17-private-half.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot17-half-64ee2ed4.pth) [reid](https://download.openmmlab.com/mmtracking/mot/reid/tracktor_reid_r50_iter25245-a452f51f.pth) |
 | R50-FasterRCNN-FPN | - | train | train | Y               | 28.3  | 50.9 | 44.5 | 1108 | 153950 | 10522 | [config](sort_faster-rcnn_fpn_4e_mot17-public.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot17-ffa52ae7.pth) |
 | R50-FasterRCNN-FPN | - | train | train | N               | 18.6  | 80.8 | 61.3 | 21537 | 29280 | 13947 | [config](sort_faster-rcnn_fpn_4e_mot17-private.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot17-ffa52ae7.pth) |
+
+Note: When running `demo_mot.py`, we suggest you use the config containing `private`, since `private` means the MOT method doesn't need external detections.

configs/mot/tracktor/README.md

@@ -14,11 +14,25 @@
 }
 ```
 
-## Results and models on MOT17
-
 We implement Tracktor with independent detector and ReID models. To train a model by yourself, you need to train a detector following [here](../../det/) and also train a ReID model following [here](../../reid/).
 The configs in this folder are basiclly for inference.
 
+## Results and models on MOT15
+
+|    Detector     |  ReID  | Train Set | Test Set | Public | Inf time (fps) | MOTA | IDF1 | FP | FN | IDSw. | Config | Download |
+| :-------------: | :----: | :-------: | :------: | :----: | :------------: | :--: | :--: |:--:|:--:| :---: | :----: | :------: |
+| R50-FasterRCNN-FPN | R50 | half-train | half-val | Y     |  -   | 61.8 | 64.9 | 1235 | 6877 | 116 | [config](tracktor_faster-rcnn_r50_fpn_4e_mot15-public-half.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot15-half_20210804_001040-ae733d0c.pth) [reid](https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot15_20210803_192157-65b5e2d7.pth) |
+| R50-FasterRCNN-FPN | R50 | half-train | half-val | N     |  -   | 66.8 | 68.4 | 3049 | 3922 | 179 | [config](tracktor_faster-rcnn_r50_fpn_4e_mot15-private-half.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot15-half_20210804_001040-ae733d0c.pth) [reid](https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot15_20210803_192157-65b5e2d7.pth) |
+
+## Results and models on MOT16
+
+|    Detector     |  ReID  | Train Set | Test Set | Public | Inf time (fps) | MOTA | IDF1 | FP | FN | IDSw. | Config | Download |
+| :-------------: | :----: | :-------: | :------: | :----: | :------------: | :--: | :--: |:--:|:--:| :---: | :----: | :------: |
+| R50-FasterRCNN-FPN | R50 | half-train | half-val | Y     |  -   | 54.1 | 61.5 | 425 | 23894 | 182 | [config](tracktor_faster-rcnn_r50_fpn_4e_mot16-public-half.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot16-half_20210804_001054-73477869.pth) [reid](https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot16_20210803_204826-1b3e3cfd.pth) |
+| R50-FasterRCNN-FPN | R50 | half-train | half-val | N     |  -   | 63.4 | 66.2 | 4175 | 14911 | 444 | [config](tracktor_faster-rcnn_r50_fpn_4e_mot16-private-half.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot16-half_20210804_001054-73477869.pth) [reid](https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot16_20210803_204826-1b3e3cfd.pth) |
+
+## Results and models on MOT17
+
 The implementations of Tracktor follow the offical practices.
 In the table below, the result marked with * (the last line) is the offical one.
 Our implementation outperform it by 4.9 points on MOTA and 3.3 points on IDF1.
@@ -27,7 +41,20 @@ Our implementation outperform it by 4.9 points on MOTA and 3.3 points on IDF1.
 | :-------------: | :----: | :-------: | :------: | :----: | :------------: | :--: | :--: |:--:|:--:| :---: | :----: | :------: |
 | R50-FasterRCNN-FPN | R50 | half-train | half-val | Y     | 3.2  | 57.3 | 63.4 | 1254 | 67091 | 614 | [config](tracktor_faster-rcnn_r50_fpn_4e_mot17-public-half.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot17-half-64ee2ed4.pth) [reid](https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot17-4bf6b63d.pth) |
 | R50-FasterRCNN-FPN | R50 | half-train | half-val | N     | 3.1  | 64.1 | 66.9 | 11088 | 45762 | 1233 | [config](tracktor_faster-rcnn_r50_fpn_4e_mot17-private-half.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot17-half-64ee2ed4.pth) [reid](https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot17-4bf6b63d.pth) |
-| R50-FasterRCNN-FPN | R50 | train      | train    | Y     | 3.2  | 69.3 | 69.4 | 4010 | 97918 | 1540 |  [config](tracktor_faster-rcnn_r50_fpn_4e_mot17-public.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot17-ffa52ae7.pth) [reid](https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot17-4bf6b63d.pth) |
-| R50-FasterRCNN-FPN | R50 | train      | train    | N     | 3.1  | 82.1 | 73.2 | 12795 | 44637 | 3033 | [config](tracktor_faster-rcnn_r50_fpn_4e_mot17-private.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot17-ffa52ae7.pth) [reid](https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot17-4bf6b63d.pth) |
 | R50-FasterRCNN-FPN | R50 | train      | test     | Y     | 3.2  | 61.2 | 58.4 | 8609 | 207627 | 2634 | [config](tracktor_faster-rcnn_r50_fpn_4e_mot17-public.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot17-ffa52ae7.pth) [reid](https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot17-4bf6b63d.pth) |
 | R50-FasterRCNN-FPN* | R50 | train     | test     | Y     | -    | 56.3 | 55.1 | 8866 | 235449 | 1987 | -    | -     |
+
+## Results and models on MOT20
+
+The implementations of Tracktor follow the offical practices.
+In the table below, the result marked with * (the last line) is the offical one.
+Our implementation outperform it by 5.3 points on MOTA and 2.1 points on IDF1.
+
+|    Detector     |  ReID  | Train Set | Test Set | Public | Inf time (fps) | MOTA | IDF1 | FP | FN | IDSw. | Config | Download |
+| :-------------: | :----: | :-------: | :------: | :----: | :------------: | :--: | :--: |:--:|:--:| :---: | :----: | :------: |
+| R50-FasterRCNN-FPN | R50 | half-train | half-val | Y     |  -   | 70.6 | 65.4 | 3652 | 175955 | 1441 | [config](tracktor_faster-rcnn_r50_fpn_8e_mot20-public-half.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_8e_mot20-half_20210805_001244-2c323fd1.pth) [reid](https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot20_20210803_212426-c83b1c01.pth) |
+| R50-FasterRCNN-FPN | R50 | half-train | half-val | N     |  -   | 70.9 | 64.1 | 5539 | 171653 | 1619 | [config](tracktor_faster-rcnn_r50_fpn_8e_mot20-private-half.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_8e_mot20-half_20210805_001244-2c323fd1.pth) [reid](https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot20_20210803_212426-c83b1c01.pth) |
+| R50-FasterRCNN-FPN | R50 | train      | test     | Y     |  -   | 57.9 | 54.8 | 16203 | 199485 | 2299 |  [config](tracktor_faster-rcnn_r50_fpn_8e_mot20-public.py) | [detector](https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_8e_mot20_20210804_162232-7fde5e8d.pth) [reid](https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot20_20210803_212426-c83b1c01.pth) |
+| R50-FasterRCNN-FPN* | R50 | train     | test     | Y     | -    | 52.6 | 52.7 | 6930 | 236680 | 1648 | -    | -     |
+
+Note: When running `demo_mot.py`, we suggest you use the config containing `private`, since `private` means the MOT method doesn't need external detections.

configs/mot/tracktor/metafile.yml

@@ -11,6 +11,66 @@ Collections:
     README: configs/mot/tracktor/README.md
 
 Models:
+  - Name: tracktor_faster-rcnn_r50_fpn_4e_mot15-public-half
+    In Collection: Tracktor
+    Config: configs/mot/tracktor/tracktor_faster-rcnn_r50_fpn_4e_mot15-public-half.py
+    Metadata:
+      Training Data: MOT15-half-train
+    Results:
+      - Task: Multiple Object Tracking
+        Dataset: MOT15-half-val
+        Metrics:
+          MOTA: 61.8
+          IDF1: 64.9
+    Weights:
+      - https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot15-half_20210804_001040-ae733d0c.pth
+      - https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot15_20210803_192157-65b5e2d7.pth
+
+  - Name: tracktor_faster-rcnn_r50_fpn_4e_mot15-private-half
+    In Collection: Tracktor
+    Config: configs/mot/tracktor/tracktor_faster-rcnn_r50_fpn_4e_mot15-private-half.py
+    Metadata:
+      Training Data: MOT15-half-train
+    Results:
+      - Task: Multiple Object Tracking
+        Dataset: MOT15-half-val
+        Metrics:
+          MOTA: 66.8
+          IDF1: 68.4
+    Weights:
+      - https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot15-half_20210804_001040-ae733d0c.pth
+      - https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot15_20210803_192157-65b5e2d7.pth
+
+  - Name: tracktor_faster-rcnn_r50_fpn_4e_mot16-public-half
+    In Collection: Tracktor
+    Config: configs/mot/tracktor/tracktor_faster-rcnn_r50_fpn_4e_mot16-public-half.py
+    Metadata:
+      Training Data: MOT16-half-train
+    Results:
+      - Task: Multiple Object Tracking
+        Dataset: MOT16-half-val
+        Metrics:
+          MOTA: 54.1
+          IDF1: 61.5
+    Weights:
+      - https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot16-half_20210804_001054-73477869.pth
+      - https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot16_20210803_204826-1b3e3cfd.pth
+
+  - Name: tracktor_faster-rcnn_r50_fpn_4e_mot16-private-half
+    In Collection: Tracktor
+    Config: configs/mot/tracktor/tracktor_faster-rcnn_r50_fpn_4e_mot16-private-half.py
+    Metadata:
+      Training Data: MOT16-half-train
+    Results:
+      - Task: Multiple Object Tracking
+        Dataset: MOT16-half-val
+        Metrics:
+          MOTA: 63.4
+          IDF1: 66.2
+    Weights:
+      - https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot16-half_20210804_001054-73477869.pth
+      - https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot16_20210803_204826-1b3e3cfd.pth
+
   - Name: tracktor_faster-rcnn_r50_fpn_4e_mot17-public-half
     In Collection: Tracktor
     Config: configs/mot/tracktor/tracktor_faster-rcnn_r50_fpn_4e_mot17-public-half.py
@@ -35,7 +95,7 @@ Models:
 
   - Name: tracktor_faster-rcnn_r50_fpn_4e_mot17-private-half
     In Collection: Tracktor
-    Config: configs/mot/tracktor/tracktor_faster-rcnn_r50_fpn_4e_mot17-private.py
+    Config: configs/mot/tracktor/tracktor_faster-rcnn_r50_fpn_4e_mot17-private-half.py
     Metadata:
       Training Data: MOT17-half-train
       inference time (ms/im):
@@ -77,24 +137,47 @@ Models:
       - https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot17-ffa52ae7.pth
       - https://download.openmmlab.com/mmtracking/mot/reid/tracktor_reid_r50_iter25245-a452f51f.pth
 
-  - Name: tracktor_faster-rcnn_r50_fpn_4e_mot17-private
+  - Name: tracktor_faster-rcnn_r50_fpn_8e_mot20-public-half
     In Collection: Tracktor
-    Config: configs/mot/tracktor/tracktor_faster-rcnn_r50_fpn_4e_mot17-private.py
+    Config: configs/mot/tracktor/tracktor_faster-rcnn_r50_fpn_8e_mot20-public-half.py
     Metadata:
-      Training Data: MOT17-train
-      inference time (ms/im):
-        - value: 322.6
-          hardware: V100
-          backend: PyTorch
-          batch size: 1
-          mode: FP32
-          resolution: (640, 1088)
+      Training Data: MOT20-half-train
     Results:
       - Task: Multiple Object Tracking
-        Dataset: MOT17-train
+        Dataset: MOT20-half-val
         Metrics:
-          MOTA: 82.1
-          IDF1: 73.4
+          MOTA: 70.6
+          IDF1: 65.4
     Weights:
-      - https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot17-ffa52ae7.pth
-      - https://download.openmmlab.com/mmtracking/mot/reid/tracktor_reid_r50_iter25245-a452f51f.pth
+      - https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_8e_mot20-half_20210805_001244-2c323fd1.pth
+      - https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot20_20210803_212426-c83b1c01.pth
+
+  - Name: tracktor_faster-rcnn_r50_fpn_8e_mot20-private-half
+    In Collection: Tracktor
+    Config: configs/mot/tracktor/tracktor_faster-rcnn_r50_fpn_8e_mot20-private-half.py
+    Metadata:
+      Training Data: MOT20-half-train
+    Results:
+      - Task: Multiple Object Tracking
+        Dataset: MOT20-half-val
+        Metrics:
+          MOTA: 70.9
+          IDF1: 64.1
+    Weights:
+      - https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_8e_mot20-half_20210805_001244-2c323fd1.pth
+      - https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot20_20210803_212426-c83b1c01.pth
+
+  - Name: tracktor_faster-rcnn_r50_fpn_8e_mot20-public
+    In Collection: Tracktor
+    Config: configs/mot/tracktor/tracktor_faster-rcnn_r50_fpn_8e_mot20-public.py
+    Metadata:
+      Training Data: MOT20-train
+    Results:
+      - Task: Multiple Object Tracking
+        Dataset: MOT20-test
+        Metrics:
+          MOTA: 57.9
+          IDF1: 54.8
+    Weights:
+      - https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_8e_mot20_20210804_162232-7fde5e8d.pth
+      - https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot20_20210803_212426-c83b1c01.pth

configs/mot/tracktor/tracktor_faster-rcnn_r50_fpn_4e_mot15-private-half.py

@@ -0,0 +1,21 @@
+_base_ = ['./tracktor_faster-rcnn_r50_fpn_4e_mot17-private-half.py']
+model = dict(
+    pretrains=dict(
+        detector=  # noqa: E251
+        'https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot15-half_20210804_001040-ae733d0c.pth',  # noqa: E501
+        reid=  # noqa: E251
+        'https://download.openmmlab.com/mmtracking/mot/reid/reid_r50_6e_mot15_20210803_192157-65b5e2d7.pth'  # noqa: E501
+    ),
+    reid=dict(head=dict(num_classes=375)))
+# data
+data_root = 'data/MOT15/'
+data = dict(
+    train=dict(
+        ann_file=data_root + 'annotations/half-train_cocoformat.json',
+        img_prefix=data_root + 'train'),
+    val=dict(
+        ann_file=data_root + 'annotations/half-val_cocoformat.json',
+        img_prefix=data_root + 'train'),
+    test=dict(
+        ann_file=data_root + 'annotations/half-val_cocoformat.json',
+        img_prefix=data_root + 'train'))