Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Accuracy drop on Post-Training Quantization (PTQ) optimized model for maskrcnn-swimt #3593

Closed
yunchu opened this issue Jun 10, 2024 · 4 comments
Closed

Accuracy drop on Post-Training Quantization (PTQ) optimized model for maskrcnn-swimt #3593

yunchu opened this issue Jun 10, 2024 · 4 comments
Assignees
@sovrasov
Milestone
2.2.0

Comments

@yunchu
Copy link
Contributor

yunchu commented Jun 10, 2024

Post-Training Quantization (PTQ) optimization applied to maskrcnn_swimt in the instance segmentation task may result in significantly reduced accuracy.

How to run the ins_seg benchmark tests for OTX 2.0.0

pytest -vs tests/perf/test_instance_segmentation.py --output-root benchmark_otx_2.0.0_result --model-category all --data-root /path/to/val-data/v2 --eval-upto optimize --num-repeat 5

image-2024-06-05-16-29-29-929
@yunchu yunchu added this to the 2.1.0 milestone Jun 10, 2024
@chuneuny-emily
Copy link
Contributor

cc'ed @eugene123tw

@eugene123tw
Copy link
Contributor

Fixed in #3685

@chuneuny-emily
Copy link
Contributor

This issue is expected to be addressed with an upgrade to OpenVINO and NNCF in a future release.

@sovrasov sovrasov modified the milestones: 2.1.0, 2.2.0 Jul 24, 2024
@sovrasov
Copy link
Contributor

Fixed in #3900

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sovrasov sovrasov

Labels
None yet
Projects
None yet
Milestone
2.2.0
Development

No branches or pull requests
5 participants
@yunchu @sovrasov @eugene123tw @harimkang @chuneuny-emily