An investigation for RGBT tracking. Hopefully, it can help other researchers become familiar with multi-modal tracking as soon as possible. This repository is started on 02/07/2023, and will keep on updating.
This repository will give a detail investigation of the RGBT tracking community, including the Datasets, Results, and the Methods.
- Datasets
- Results
- Methods
- Competitions
- ...
Real Data
| Dataset | Publish | GitHub | Introduction |
|---|---|---|---|
| GTOT | TIP'2016 | GTOT | 50 Video pairs, about 1.5W frames |
| RGBT210 | ACM MM'2017 | RGBT210 | 210 Video pairs |
| RGBT234 | PR'2018 | RGBT234 | 234 Video pairs, the extension of RGBT210 |
| VOT-RGBT19 | VOT Community：2019 | VOT-RGBT2019 CODE:TZYD | 60 Video pairs, about 2W frame pairs, a sub-set of RGBT 234 |
| VOT-RGBT20 | VOT Community： 2020 | VOT-RGBT2020 CODE:TZYD | 60 Video pairs, its data is the same with VOT-RGBT2019, but with different evaluation mechanism |
| LasHeR | TIP'2021 | LasHeR | train/test split, imaged in the head-up view, above 70W image pairs in total; 245 videos in the test set |
| VTUAV | CVPR'2022 | VTUAV | imaged in the top-down view by uav, long-term;176 videos in the test-st, which has around 3500 frames in average |
Generated Data
| Dataset | Publish | GitHub | Introduction |
|---|---|---|---|
| LSS-Dataset(from RGB) | TCSVT'2021 | LSS-Dataset ,code(Ye5Q) | Generated from VID |
| LSS-Dataset(from TIR) | TCSVT'2021 | LSS-Dataset,code(IHws) | Generated from TIR |
| GTOT | RGBT210 | RGBT234 | LasHeR | VTUAV | VOT-RGBT2019 | VOT-RGBT2020 | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Methods | Code | Venue | Speed | PR | SR | PR | SR | PR | SR | PR | NPR | SR | PR | NPR | SR | R | A | EAO | R | A | EAO |
| fusionnet | - | NeuroComputing'2018 | - | 0.852 | 0.626 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| mfDiMP | pubished | ICCVW'2019 | 28/GTXTitan | - | - | 0.786 | 0.555 | 0.785 | 0.559 | 0.448 | 0.395 | 0.343 | 0.673 | - | 0.554 | 0.602 | 0.804 | 0.388 | 0.638 | 0.793 | 0.380 |
| MANet | pubished | ICCVW'2019 | - | 0.894 | 0.724 | - | - | 0.777 | 0.539 | 0.457 | - | 0.330 | 0.455 | 0.383 | 0.326 | 0.582 | 0.701 | 0.346 | - | - | - |
| DAPNet | - | ACM MM'2019 | - | 0.882 | 0.707 | - | - | 0.766 | 0.537 | 0.431 | - | 0.314 | 0.431 | 0.383 | 0.314 | - | - | - | - | - | - |
| DAFNet | pubished | ICCVW'2019 | - | 0.891 | 0.712 | - | - | 0.796 | 0.544 | 0.448 | 0.390 | 0.311 | 0.620 | - | 0.458 | - | - | - | - | - | - |
| - | - | ICIP'2019 | - | 0.843 | 0.677 | - | - | 0.787 | 0.545 | - | - | - | - | - | - | - | - | - | - | - | - |
| - | - | FUSION'2019 | - | - | - | - | - | 0.610 | 0.428 | - | - | - | - | - | - | - | - | - | - | - | - |
| FSRPN | published | 2019 | - | - | - | - | - | - | - | - | - | - | 0.653 | - | 0.544 | - | - | 0.355 | - | - | - |
| SiamFT | - | IEEE Access'2019 | 30/RTX1080Ti | 0.826 | 0.700 | - | - | 0.688 | 0.486 | - | - | - | - | - | - | 0.630 | 0.639 | 0.310 | - | - | - |
| MaCNet | pubished | Sensors'2020 | - | 0.880 | 0.714 | - | - | 0.790 | 0.554 | 0.482 | 0.420 | 0.350 | - | - | - | 0.545 | 0.591 | 0.305 | - | - | - |
| CMPP | - | CVPR'2020 | 1.3/RTX2080Ti | 0.926 | 0.738 | - | - | 0.823 | 0.575 | - | - | - | - | - | - | - | - | - | - | - | - |
| CAT | - | ECCV'2020 | - | 0.889 | 0.717 | 0.792 | 0.533 | 0.804 | 0.561 | 0.450 | 0.395 | 0.314 | - | - | - | - | - | - | - | - | - |
| DSiamMFT | - | 2020 | 14.7/RTX1080Ti | - | - | 0.642 | 0.432 | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| JMMAC | pubished | TIP'2021 | 4/RTX2080Ti | 0.902 | 0.732 | - | - | 0.790 | 0.573 | - | - | - | - | - | - | 0.665 | 0.821 | 0.483 | 0.662 | 0.818 | 0.420 |
| MANet++ | - | TIP'2021 | - | 0.901 | 0.723 | - | - | 0.800 | 0.554 | 0.467 | 0.404 | 0.314 | - | - | - | 0.509 | 0.538 | 0.272 | - | - | - |
| CBPNet | - | TMM'2021 | - | 0.885 | 0.716 | - | - | 0.794 | 0.541 | - | - | - | - | - | - | - | - | - | - | - | - |
| CEDiMP | - | Sensors'2021 | - | 0.886 | 0.731 | - | - | 0.808 | 0.561 | - | - | - | - | - | - | - | - | - | - | - | - |
| TFNet | - | TCSVT'2021 | 20/RTX2080Ti | 0.886 | 0.729 | 0.777 | 0.529 | 0.806 | 0.560 | - | - | - | - | - | - | 0.462 | 0.594 | 0.288 | - | - | - |
| FANet | - | TIV'2021 | 19/RTX2080Ti | 0.891 | 0.728 | - | - | 0.787 | 0.553 | 0.441 | 0.384 | 0.309 | - | - | - | 0.472 | 0.508 | 0.247 | - | - | - |
| ADRNet | pubished | IJCV'2021 | 25/RTX2080Ti | 0.904 | 0.739 | - | - | 0.809 | 0.571 | - | - | - | 0.622 | - | 0.466 | 0.622 | 0.766 | 0.396 | - | - | - |
| M^5L | - | TIP'2021 | - | 0.896 | 0.710 | - | - | 0.795 | 0.542 | - | - | - | - | - | - | - | - | - | - | - | - |
| SiamCDA | published | TCSVT'2021 | - | 0.877 | 0.732 | - | - | 0.760 | 0.569 | - | - | - | - | - | - | 0.682 | 0.757 | 0.424 | - | - | - |
| DuSiamRT | - | 2021 | - | 0.766 | 0.628 | - | - | 0.567 | 0.384 | - | - | - | - | - | - | - | - | - | - | - | - |
| Here | - | NeuralComputing'2021 | 35 | 0.869 | 0.711 | - | - | 0.751 | 0.491 | - | - | - | - | - | - | 0.682 | 0.757 | 0.424 | - | - | - |
| EStaple | - | JCSC'2021 | 44/AMD3950 | - | - | - | 0.648 | 0.470 | - | - | - | - | - | - | - | - | - | - | - | - | |
| APFNet | published | AAAI'2022 | - | 0.905 | 0.739 | - | - | 0.827 | 0.579 | 0.500 | - | 0.362 | - | - | - | - | - | - | - | - | - |
| TAAT | arxiv | arxiv'2022 | 39/RTX2080Ti | 0.858 | 0.696 | 0.710 | 0.486 | - | - | - | - | - | 0.696 | - | 0.560 | 0.643 | 0.735 | 0.416 | - | - | - |
| DCNet | - | ICPR'2022 | - | 0.903 | 0.736 | - | - | 0.839 | 0.591 | - | - | - | - | - | - | - | - | - | - | - | - |
| DMCNet | - | TNNLS'2022 | 2.38/RTX2080Ti | 0.909 | 0.733 | 0.797 | 0.555 | 0.839 | 0.593 | 0.490 | 0.431 | 0.355 | - | - | - | - | - | - | - | - | - |
| HMFT | published | CVPR'2022 | 30/GTXTitan | 0.912 | 0.749 | 0.786 | 0.535 | 0.788 | 0.568 | 0.460 | 0.413 | 0.326 | 0.758 | - | 0.627 | 0.553 | 0.760 | 0.347 | - | - | - |
| Here | - | SC IF'2022 | 10/i5-7500k | 0.864 | 0.707 | - | - | 0.666 | 0.484 | - | - | - | - | - | - | - | - | - | - | - | - |
| ProTrack | - | ACMMM'2022 | - | - | - | - | 0.786 | 0.587 | 0.509 | - | 0.421 | - | - | - | - | - | - | - | - | - | - |
| DFAT | published | INFFUS'2023 | 22/RTX2080Ti | 0.893 | 0.723 | - | - | 0.758 | 0.552 | - | - | 0.336 | - | - | - | 0.670 | 0.756 | 0.413 | 0.672 | 0.779 | 0.390 |
| ViPT | published | CVPR'2023 | - | - | - | - | - | 0.835 | 0.617 | 0.651 | - | 0.525 | - | - | - | - | - | - | - | - | - |
| HFRD | - | CVPR'2023 | 30/RTX1080Ti | 0.892 | 0.734 | - | - | 0.824 | 0.584 | 0.590 | 0.546 | 0.464 | - | - | - | - | - | - | - | - | - |
| TBSI | published | CVPR'2023 | 36/RTX3080Ti | - | - | 0.853 | 0.625 | 0.871 | 0.637 | 0.692 | 0.657 | 0.556 | - | - | - | - | - | - | - | - | - |
| BD2Track | - | SPL'2023 | - | - | - | - | - | - | - | 0.560 | - | 0.432 | - | - | - | - | - | - | - | - | - |
| SiamTDR | - | TICPS'2023 | 127/RTX3090 | 88.5 | 71.4 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| SiamAFTS | - | ScienticReport'2023 | - | 84.9 | 77.7 | 87.3 | 56.4 | 89.0 | 60.2 | - | - | - | - | - | - | - | - | - | - | - | - |
| paper | - | Applied Intelligence'2023 | 15/GTX1080Ti | 91.6 | 74.4 | 80.2 | 56.8 | 83.6 | 60.4 | 63.6 | 59.4 | 50.0 | 69.2 | - | 57.2 | - | - | - | - | - | - |
| UMT | - | AIEA'2023 | 3/GTX1080Ti | 89.2 | 73.0 | 84.2 | 60.3 | 49.3 | 43.0 | 36.9 | - | - | - | - | - | - | - | ||||
| RFSTrack | - | SPL'2023 | 83/RTX3090 | 92.1 | 75.3 | - | - | 86.3 | 62.2 | 65.9 | - | 52.6 | - | - | - | - | - | - | - | - | - |
| BAT | - | AAAI'2024 | - | - | - | 86.8 | 64.1 | 70.2 | 56.3 | - | - | - | - | - | - | - | |||||
| GMMT | - | AAAI'2024 | 18/RTX3090Ti | 93.6 | 78.5 | - | - | 87.9 | 64.7 | 70.7 | 67.0 | 56.6 | - | - | - | - | - | - | - | - | - |
- AntiUAV 1st The first AntiUAV challenge is a multi-modal challenge. It was held in 2020. The website for the 1st AntiUAV is covered by the newest one. The winner is team xiaobaibai. One of its member is Tianyang Xu
- VOT-RGBT2019 The first competition in the RGBT tracking community. It was held in 2019. The winner is SiamDW.
- [VOT-RGBT2020] A competition in the RGBT tracking community. It was held in 2020. The winner is DFAT. One of its member is Zhangyong Tang
Questions
If you have any questions, please contact zhangyong_tang_jnu@163.com, and wechat: Tzy18861871359 is also welcomed.