You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the COCO metrics are a bit different than what we get with the official cocoapi. While the matching algorithm and other calculations should be the same, there are some differences in our implementation:
We do not do anything special with "iscrowd":1 ground truths
We do not use the area attribute from COCO json ground truths (which is obtained from the segmentation mask): instead, we multiply the width and height of the bounding box to obtain the area
I do not use maxDets - that is, I do not limit the maximum number of detections per image. This limiting is usually done during inference (usually "fused" with NMS), or I might additionally add it to IO constructors for Detections - but not as part of the evaluation
However, even if remove these three things (as is done in our integration example), the results still differ. So the best thing would be to narrow the differences down:
take only 1 size range, one IoU threshold and one class, and make sure we still have the difference.
Then, try to get the difference down to one image (it should be possible)
For this image, evaluate what went wrong - manually inspect the matches that were made
Hopefully this reveals some un-noticed difference in our matching algorithm, as compared to COCO, and then this issue will be fixed
The text was updated successfully, but these errors were encountered:
Currently, the COCO metrics are a bit different than what we get with the official
cocoapi
. While the matching algorithm and other calculations should be the same, there are some differences in our implementation:"iscrowd":1
ground truthsarea
attribute from COCO json ground truths (which is obtained from the segmentation mask): instead, we multiply the width and height of the bounding box to obtain the areamaxDets
- that is, I do not limit the maximum number of detections per image. This limiting is usually done during inference (usually "fused" with NMS), or I might additionally add it to IO constructors forDetections
- but not as part of the evaluationHowever, even if remove these three things (as is done in our integration example), the results still differ. So the best thing would be to narrow the differences down:
Hopefully this reveals some un-noticed difference in our matching algorithm, as compared to COCO, and then this issue will be fixed
The text was updated successfully, but these errors were encountered: