Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix tile bug & add more tests #3012

Merged

Conversation

eugene123tw
Copy link
Contributor

@eugene123tw eugene123tw commented Feb 29, 2024

Summary

Good news, OV model maintains its accuracy without too much drop (1-2% on inst-seg). Interestingly, it even outperforms PyTorch model in terms of accuracy on detection models. Probably will look into it in another PR, but I think the risk for Geti is low at the moment.

OV Model dataset Image Size Tile Size Version Torch Test/mAP50 OV Test/mAP_50 Inference Duration(s)
MaskRCNN-Eff Vitens Aeromonas 3500 x 3500 1663 x 1663 2.x (PR) 0.9168 0.9061  59.016
MaskRCNN-R50 Vitens Aeromonas 3500 x 3500 1663 x 1663 2.x (PR) 0.8979 0.9052  72.322
MaskRCNN-Eff Vitens Coliform 3500 x 3500 1663 x 1663 2.x (PR) 0.5311 0.5134 48.878
MaskRCNN-R50 Vitens Coliform 3500 x 3500 1663 x 1663 2.x (PR) 0.4854 0.5054  70.749
yolox_s Vitens Aeromonas 3500 x 3500 1663 x 1663 2.x (PR) 0.6198 0.7948 37.497
yolox_s Vitens Coliform 3500 x 3500 1663 x 1663 2.x (PR) 0.3721 0.42 23.833
yolox_tiny Vitens Aeromonas 3500 x 3500 1663 x 1663 2.x (PR) 0.9354 0.9529 34.481
yolox_tiny Vitens Coliform 3500 x 3500 1663 x 1663 2.x (PR) 0.5171 0.5943 21.262
yolox_l Vitens Aeromonas 3500 x 3500 1663 x 1663 2.x (PR) 0.7258 0.7989 71.366
yolox_l Vitens Coliform 3500 x 3500 1663 x 1663 2.x (PR) 0.43689 0.4206 51.628

How to test

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added e2e tests for validation.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

@github-actions github-actions bot added TEST Any changes in tests OTX 2.0 labels Feb 29, 2024
@eugene123tw eugene123tw changed the title [Draft] Fix tile bug & add more tests Fix tile bug & add more tests Mar 1, 2024
@eugene123tw eugene123tw marked this pull request as ready for review March 1, 2024 15:47
@vinnamkim vinnamkim merged commit 2ad3653 into openvinotoolkit:v2 Mar 5, 2024
10 checks passed
@eugene123tw eugene123tw deleted the eugene/CVS-132685-fix-tile-bug-tests branch March 5, 2024 08:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
TEST Any changes in tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants