fix fcos gt_areas calculation

[simonJJJ]

I fix the gt_areas calculation in the FCOS model.

[facebook-github-bot]

Hi @simonJJJ!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

[facebook-github-bot]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

[simonJJJ]

Hi @datumbox , can you take a look at this?

[datumbox]

@simonJJJ LGTM, thanks a lot for patching this. This is definitely a typo which remained undetected.

@xiaohu2015 @zhiqwang @jdsgomes I assume this ended up having little effect on the overall accuracy. Do we want to retrain and confirm?

[github-actions]

Hey @datumbox!

You merged this PR, but no labels were added. The list of valid labels is available at https://github.com/pytorch/vision/blob/main/.github/process_commit.py

[xiaohu2015]

@simonJJJ LGTM, thanks a lot for patching this. This is definitely a typo which remained undetected.

@xiaohu2015 @zhiqwang @jdsgomes I assume this ended up having little effect on the overall accuracy. Do we want to retrain and confirm?

yes, I think it maybe has little effect on the overall accuracy, becuase here we don't have many situations which one position match multiple GTs (it rarely happens with the scale range constraint). But I think we should retrain the model if possible.

[xiaohu2015]

I fix the gt_areas calculation in the FCOS model.

Thanks a lot

[jdsgomes]

Thanks for the fix @simonJJJ . I am running the training just to make sure the impact is not significant.

[jdsgomes]

Thanks for the fix @simonJJJ . I am running the training just to make sure the impact is not significant.

I have re-run the training job and the performance difference does not seem significant.

With the fix:

IoU metric: bbox
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.390
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.578
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.418
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.218
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.423
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.513
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.326
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.530
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.565
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.358
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.602
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.731
Training time 5:50:18
submitit INFO (2022-04-21 01:28:12,411) - Job completed successfully

Original results:

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.392
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.582
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.420
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.222
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.423
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.513
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.324
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.529
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.562
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.350
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.609
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.728

So I don't think there is a need to update the pre-trained weights or take any further action.

In any case thank you @simonJJJ again for making the fix!

[datumbox]

Agreed. I don't think it's worth releasing a new pair of weights for 0.2 mAP.

Soon we can try to retraining FCOS after #5410 is completed to really push its accuracy similarly to what we did with FasterRCNN and RetinaNet. This should give us a material push on the accuracy of several mAP points.