Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add configs for export and custom SegNext head #2787

Merged
merged 9 commits into from
Jan 12, 2024
Merged

Add configs for export and custom SegNext head #2787

merged 9 commits into from
Jan 12, 2024

Conversation

kprokofi
Copy link
Collaborator

Summary

How to test

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added e2e tests for validation.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

@kprokofi kprokofi requested a review from sovrasov January 12, 2024 08:31
@kprokofi kprokofi changed the title Kp/v2 export Add segmentation configs for export and custom SegNext head Jan 12, 2024
@github-actions github-actions bot added DEPENDENCY Any changes in any dependencies (new dep or its version) should be produced via Change Request on PM BUILD labels Jan 12, 2024
@kprokofi kprokofi changed the title Add segmentation configs for export and custom SegNext head Add configs for export and custom SegNext head Jan 12, 2024
@kprokofi kprokofi marked this pull request as ready for review January 12, 2024 13:45
@sovrasov sovrasov merged commit 4f3448a into vs/v2_export Jan 12, 2024
1 check passed
@sovrasov sovrasov deleted the kp/v2-export branch January 12, 2024 14:14
kprokofi added a commit that referenced this pull request Jan 31, 2024
@sovrasov @kprokofi
@sungmanc @samet-akcay @yunchu @eunwoosh
Add export entry point (#2774)
4ddd9f8 
* Add export entry point

* Update onnx export

* Fix missing export config

* Add base metadata

* Add classification metadata

* Add configs for export and custom SegNext head (#2787)

* added proper configs

* minor

* fix config

* fix head, updated configs

* fix pre-commit. Part 1

* fix pre-commit. Part 2

* fix pre-commit. Part3

* added default numbers for HW, mean and scale

* Add OV export via ONNX

* Fix linters

* Update openvino

* Fix unit tests

* Skip CLI unit for old python

* Fix linters

* add raising an error when metric is None

* Fix dinov2 configs

* Add export tests

* Add smoke tests for export

* Add smoke tests for ov infer

* Update export cli tests

* Refactored ONNX export, updated configs for segmentation (#2797)

* refactored export

* change export to onnx

* fix pre-commit. part 1

* Update exceptions for tests

* Fix multilabel configs

* Remove old export function

* Restore classification export

* Restore multiclass export

* Restore export of multilabel classificaiton

* Restore cli tests

* Add metadata for segmentation

* Add common export for segmentation

* Fix unit tests

* Add multilabel OV inference

* Fix snapshot loading in engine

* Set RGB for OV models

* Added segmentation configuration for export to align new interface (#2808)

* added segmentation configuration. Added image_size

* fix pre-commit

* remove getting resolution from mmconfig

* remove resolution in models constructor

* remove resolution in models constructor2

* remove resoluion in export params

* fix some pre-commit

* fix pre-commit

* added Hlabel class

* small refactoring

* added multilabel

* Extract exporter to a class

* Minor fixes

* Fix linters in exporters

* Update cli test description

* pre-commit fix

* added collate_unstacked_fn for classification

* fixed pre-commit tests

* Fix missing attributes in cls models

* added multilabel inference (#2814)

* Swith to installation of MAPI from master

* Add ov inference integration tests

* Update export test output

* Add returning the test metrics

* Fix a type

* Add a workaround for OVC single-output problem

* Set MAPI version

* Return checkpoint path from export

* Fix mapi version

* Update inference test

* change logic

* Update src/otx/core/model/entity/classification.py

Co-authored-by: Sungman Cho <sungman.cho@intel.com>

* added doc string

* Skip visual_prompting export tests

* Update src/otx/core/data/dataset/base.py

Co-authored-by: Samet Akcay <samet.akcay@intel.com>

* fix pre-commit

* modified configs

* fix integration tests

* fix unit test

* fix integration for dinov2

* Remove obsolette entry points

* Adapt engine export to new CLI

* Fix missing label names in model

* Fix cli tests

* Disable labels updating in export

* Disable failing cli tests

* Update docstrings in engine

* Del obsolete configs

* return inheritance from OTXModel back

* return CLI changes back

* Update cli tests

* Fix dataset type in cls ov configs

* Update export configuration for hrnet

* Update device in cli ov inferance

* Fix multilabel inferece

* Restore accuracy smoke tests

* Fix docs

* Disable tests for dino v2

* Updatee the name of the inference test case

* Fix ssewg task name

* Fix pre-commit

* Deterministic train in sccuracy tests

* Update deterministic mode in tests

* Fix mapi configuration update in multilabel cls

* Fix linters

* Load label infor from checkpoint

* Fix minor comments

* Fix mode comments

* Update comment regarding missing label names

* Add cli usage to docs

* Extract precision to a standalone file

* Add model base name to model.export params

* fix segnext head

* fix pre-commit

* Fix linters

* Refactor configuration

* Fix linters

* Fix pre-commit

* Update inference tests

* Extract common fixture in CLI tests

* Fix missing mean/scale metadata in native exporter

* Workaround OV failure on ONNX models with metadata

* Simplify checkpoint loading

* Cleanup

* Fix cli tests

* Fix litehrnet export config

* Fix import of custom ham seg head

* Del unused import

* Revert "added CUBLAS_WORKSPACE_CONFIG for testing deterministic option"

This reverts commit 4c41650.

* Fix typos in docs

* added CUBLAS_WORKSPACE_CONFIG for testing deterministic option

* add CUBLAS_WORKSPACE_CONFIG=:4096:8 for integration test

---------

Co-authored-by: Prokofiev Kirill <kirill.prokofiev@intel.com>
Co-authored-by: Sungman Cho <sungman.cho@intel.com>
Co-authored-by: Samet Akcay <samet.akcay@intel.com>
Co-authored-by: Yunchu Lee <yunchu.lee@intel.com>
Co-authored-by: eunwoosh <eunwoo.shin@intel.com>
sovrasov added a commit that referenced this pull request Feb 2, 2024
@sovrasov @kprokofi
@sungmanc @samet-akcay
Add PTQ optimization (#2825)
17e50cd 
* Add export entry point

* Update onnx export

* Fix missing export config

* Add base metadata

* Add classification metadata

* Add configs for export and custom SegNext head (#2787)

* added proper configs

* minor

* fix config

* fix head, updated configs

* fix pre-commit. Part 1

* fix pre-commit. Part 2

* fix pre-commit. Part3

* added default numbers for HW, mean and scale

* Add OV export via ONNX

* Fix linters

* Update openvino

* Fix unit tests

* Skip CLI unit for old python

* Fix linters

* add raising an error when metric is None

* Fix dinov2 configs

* Add export tests

* Add smoke tests for export

* Add smoke tests for ov infer

* Update export cli tests

* Refactored ONNX export, updated configs for segmentation (#2797)

* refactored export

* change export to onnx

* fix pre-commit. part 1

* Update exceptions for tests

* Fix multilabel configs

* Remove old export function

* Restore classification export

* Restore multiclass export

* Restore export of multilabel classificaiton

* Restore cli tests

* Add metadata for segmentation

* Add common export for segmentation

* Fix unit tests

* Add multilabel OV inference

* Fix snapshot loading in engine

* Set RGB for OV models

* Added segmentation configuration for export to align new interface (#2808)

* added segmentation configuration. Added image_size

* fix pre-commit

* remove getting resolution from mmconfig

* remove resolution in models constructor

* remove resolution in models constructor2

* remove resoluion in export params

* fix some pre-commit

* fix pre-commit

* added Hlabel class

* small refactoring

* added multilabel

* Extract exporter to a class

* Minor fixes

* Fix linters in exporters

* Update cli test description

* pre-commit fix

* added collate_unstacked_fn for classification

* fixed pre-commit tests

* Fix missing attributes in cls models

* added multilabel inference (#2814)

* Swith to installation of MAPI from master

* Add ov inference integration tests

* Update export test output

* Add returning the test metrics

* Fix a type

* Add a workaround for OVC single-output problem

* Set MAPI version

* Return checkpoint path from export

* Fix mapi version

* Update inference test

* change logic

* Update src/otx/core/model/entity/classification.py

Co-authored-by: Sungman Cho <sungman.cho@intel.com>

* added doc string

* Skip visual_prompting export tests

* Update src/otx/core/data/dataset/base.py

Co-authored-by: Samet Akcay <samet.akcay@intel.com>

* fix pre-commit

* modified configs

* fix integration tests

* fix unit test

* fix integration for dinov2

* Remove obsolette entry points

* Adapt engine export to new CLI

* Fix missing label names in model

* Fix cli tests

* Disable labels updating in export

* Disable failing cli tests

* Update docstrings in engine

* Del obsolete configs

* return inheritance from OTXModel back

* return CLI changes back

* Update cli tests

* Fix dataset type in cls ov configs

* Update export configuration for hrnet

* Update device in cli ov inferance

* Fix multilabel inferece

* Restore accuracy smoke tests

* Fix docs

* Disable tests for dino v2

* Updatee the name of the inference test case

* Fix ssewg task name

* Fix pre-commit

* Deterministic train in sccuracy tests

* Update deterministic mode in tests

* Fix mapi configuration update in multilabel cls

* Fix linters

* Load label infor from checkpoint

* Add nncf to list of deps

* Add optimize entry point

* Add initial version of ptq optimization

* Add PTQ entry point in new CLI

* Upgrade OV deps

* Cleanup

* Fix linters

* Add ptq test

* Resitict nncf dataset size

* Fix minor comments

* Fix mode comments

* Update comment regarding missing label names

* Add cli usage to docs

* Extract precision to a standalone file

* Add model base name to model.export params

* fix segnext head

* fix pre-commit

* Minor changes in the interface

* Fix linters

* Refactor configuration

* Fix linters

* Fix pre-commit

* Update inference tests

* Extract common fixture in CLI tests

* Fix missing mean/scale metadata in native exporter

* Workaround OV failure on ONNX models with metadata

* Simplify checkpoint loading

* Update PTQ tests

* Cleanup

* Fix merge artifacts

* Fix cli tests

* Fix litehrnet export config

* Fix import of custom ham seg head

* Del unused import

* Del merge artifacts

* Improve docs

* Simplify ptq dataset size limiting

* Add PTQ optimization configs via embedding it into IR (#2859)

* Fix SegNext head (#2835)

* fix segnext head

* fix pre-commit

* poc ptq

* resolve merge problems part 1

* pre-commit fix

* fix merge conflicts

* updated ignore scope and fix litehrnet

* remove aggregator

* minor

* minor

* remove checking accuracy for ptq

* Refactor ptq configuration

* Remove duplicated size limit

* Add a small note on subset size parameter

---------

Co-authored-by: Prokofiev Kirill <kirill.prokofiev@intel.com>
Co-authored-by: Sungman Cho <sungman.cho@intel.com>
Co-authored-by: Samet Akcay <samet.akcay@intel.com>
sungmanc added a commit that referenced this pull request Feb 5, 2024
@kprokofi @sovrasov
@sungmanc @samet-akcay
Add exportable code + demo (#2824)
d2491d6 
* Add export entry point

* Update onnx export

* Fix missing export config

* Add base metadata

* Add classification metadata

* Add configs for export and custom SegNext head (#2787)

* added proper configs

* minor

* fix config

* fix head, updated configs

* fix pre-commit. Part 1

* fix pre-commit. Part 2

* fix pre-commit. Part3

* added default numbers for HW, mean and scale

* Add OV export via ONNX

* Fix linters

* Update openvino

* Fix unit tests

* Skip CLI unit for old python

* Fix linters

* add raising an error when metric is None

* Fix dinov2 configs

* Add export tests

* Add smoke tests for export

* Add smoke tests for ov infer

* Update export cli tests

* Refactored ONNX export, updated configs for segmentation (#2797)

* refactored export

* change export to onnx

* fix pre-commit. part 1

* Update exceptions for tests

* Fix multilabel configs

* Remove old export function

* Restore classification export

* Restore multiclass export

* Restore export of multilabel classificaiton

* Restore cli tests

* Add metadata for segmentation

* Add common export for segmentation

* Fix unit tests

* Add multilabel OV inference

* added segmentation configuration. Added image_size

* fix pre-commit

* remove getting resolution from mmconfig

* remove resolution in models constructor

* remove resolution in models constructor2

* remove resoluion in export params

* fix some pre-commit

* fix pre-commit

* Fix snapshot loading in engine

* Set RGB for OV models

* Added segmentation configuration for export to align new interface (#2808)

* added segmentation configuration. Added image_size

* fix pre-commit

* remove getting resolution from mmconfig

* remove resolution in models constructor

* remove resolution in models constructor2

* remove resoluion in export params

* fix some pre-commit

* fix pre-commit

* added Hlabel class

* small refactoring

* added multilabel

* Extract exporter to a class

* Minor fixes

* Fix linters in exporters

* Update cli test description

* pre-commit fix

* added collate_unstacked_fn for classification

* fixed pre-commit tests

* Fix missing attributes in cls models

* added multilabel inference (#2814)

* Swith to installation of MAPI from master

* Add ov inference integration tests

* Update export test output

* Add returning the test metrics

* Fix a type

* Add a workaround for OVC single-output problem

* Set MAPI version

* Return checkpoint path from export

* Fix mapi version

* Update inference test

* change logic

* start debugging

* Update src/otx/core/model/entity/classification.py

Co-authored-by: Sungman Cho <sungman.cho@intel.com>

* added doc string

* added demo

* Skip visual_prompting export tests

* Update src/otx/core/data/dataset/base.py

Co-authored-by: Samet Akcay <samet.akcay@intel.com>

* fix pre-commit

* modified configs

* fix integration tests

* fix unit test

* fix integration for dinov2

* Remove obsolette entry points

* fixed issues with demo pipeline

* added drawer for classification

* Adapt engine export to new CLI

* Fix missing label names in model

* Fix cli tests

* Disable labels updating in export

* Disable failing cli tests

* Update docstrings in engine

* Del obsolete configs

* return inheritance from OTXModel back

* return CLI changes back

* added per task visualizators. Not debugged. Not working

* fix segnext head

* added segmentation visualizer

* Fix SegNext head (#2835)

* fix segnext head

* fix pre-commit

* Update cli tests

* Fix dataset type in cls ov configs

* Update export configuration for hrnet

* Update device in cli ov inferance

* Fix multilabel inferece

* Restore accuracy smoke tests

* Fix docs

* Disable tests for dino v2

* Updatee the name of the inference test case

* Fix ssewg task name

* Fix pre-commit

* Deterministic train in sccuracy tests

* update OD demo

* Update deterministic mode in tests

* Fix mapi configuration update in multilabel cls

* Fix linters

* Load label infor from checkpoint

* debugged all tasks. Debugged Async

* Fix minor comments

* Fix mode comments

* Update comment regarding missing label names

* Add cli usage to docs

* Extract precision to a standalone file

* Add model base name to model.export params

* fix segnext head

* fix pre-commit

* fix ruff. Phase 1

* fix mypy

* fix linter las part

* fix pre-commit. Move init to base.py

* few fixes. Added setup.py

* minor

* apply comments

* fix pre-commit

* updated license

* aligned license for  other files

* added raise Runtime Error, when get_results is None

---------

Co-authored-by: Vladislav Sovrasov <vladislav.sovrasov@intel.com>
Co-authored-by: Sungman Cho <sungman.cho@intel.com>
Co-authored-by: Samet Akcay <samet.akcay@intel.com>
eunwoosh added a commit that referenced this pull request Feb 6, 2024
@eunwoosh @sovrasov
@kprokofi @sungmanc @samet-akcay @yunchu
Add mmdeploy export (#2806)
e3cc66f 
* Add export entry point

* Update onnx export

* Fix missing export config

* Add base metadata

* Add classification metadata

* Add configs for export and custom SegNext head (#2787)

* added proper configs

* minor

* fix config

* fix head, updated configs

* fix pre-commit. Part 1

* fix pre-commit. Part 2

* fix pre-commit. Part3

* added default numbers for HW, mean and scale

* Add OV export via ONNX

* Fix linters

* Update openvino

* Fix unit tests

* Skip CLI unit for old python

* Fix linters

* add raising an error when metric is None

* Fix dinov2 configs

* Add export tests

* Add smoke tests for export

* Add smoke tests for ov infer

* Update export cli tests

* Refactored ONNX export, updated configs for segmentation (#2797)

* refactored export

* change export to onnx

* fix pre-commit. part 1

* Update exceptions for tests

* Fix multilabel configs

* Remove old export function

* Restore classification export

* Restore multiclass export

* Restore export of multilabel classificaiton

* Restore cli tests

* Add metadata for segmentation

* Add common export for segmentation

* Fix unit tests

* Add multilabel OV inference

* draft commit

* support mmdeploy export to detection model

* support mmdeploy export to instnace segmentation model

* add comment

* make code clean a little bit

* remove debugging code

* move _get_export_parameters to each model class

* Fix snapshot loading in engine

* Set RGB for OV models

* Added segmentation configuration for export to align new interface (#2808)

* added segmentation configuration. Added image_size

* fix pre-commit

* remove getting resolution from mmconfig

* remove resolution in models constructor

* remove resolution in models constructor2

* remove resoluion in export params

* fix some pre-commit

* fix pre-commit

* added Hlabel class

* small refactoring

* added multilabel

* Extract exporter to a class

* Minor fixes

* Fix linters in exporters

* Update cli test description

* pass model weight to mmdeploy

* bugfix

* pre-commit fix

* added collate_unstacked_fn for classification

* fixed pre-commit tests

* Fix missing attributes in cls models

* added multilabel inference (#2814)

* Swith to installation of MAPI from master

* Add ov inference integration tests

* Update export test output

* Add returning the test metrics

* Fix a type

* get export params when necessary in ins seg

* add empty label for ins-set while preparing export

* Add a workaround for OVC single-output problem

* Set MAPI version

* Return checkpoint path from export

* Fix mapi version

* Update inference test

* change logic

* draft implementation for new design

* Update src/otx/core/model/entity/classification.py

Co-authored-by: Sungman Cho <sungman.cho@intel.com>

* added doc string

* mmdeploy export w/ det and i-seg

* apply change to cls and seg

* fill in missing part

* Skip visual_prompting export tests

* add docstring

* make mmdeploy use mmdet pipline module

* add docstring and remove unused code

* add missing docstring

* refactor mmdeploy files

* onnx_config -> ir_config

* remove onnx file after OV export is done

* align with pre-commit

* align with pre-commit

* test det models in export integration test

* fix typo

* Update src/otx/core/data/dataset/base.py

Co-authored-by: Samet Akcay <samet.akcay@intel.com>

* fix pre-commit

* modified configs

* fix integration tests

* fix unit test

* fix integration for dinov2

* Remove obsolette entry points

* Adapt engine export to new CLI

* Fix missing label names in model

* Fix cli tests

* Disable labels updating in export

* Disable failing cli tests

* Update docstrings in engine

* revert mm registry default scope after mmdeploy is executed

* revert export intg test

* Del obsolete configs

* support export for atss_r50_fpn and rtmdet_tiny

* merge related model calss into the single file

* return inheritance from OTXModel back

* return CLI changes back

* Fix SegNext head (#2835)

* fix segnext head

* fix pre-commit

* Update cli tests

* Fix dataset type in cls ov configs

* Update export configuration for hrnet

* Update device in cli ov inferance

* Fix multilabel inferece

* Restore accuracy smoke tests

* Fix docs

* Disable tests for dino v2

* Updatee the name of the inference test case

* Fix ssewg task name

* Fix pre-commit

* Deterministic train in sccuracy tests

* MMdeployExporter argument test_pipeline is native python obj

* align with pre-commit

* fix minor bug

* align export_param implementation

* Update deterministic mode in tests

* Fix mapi configuration update in multilabel cls

* Fix linters

* Load label infor from checkpoint

* Fix minor comments

* Fix mode comments

* Update comment regarding missing label names

* Add cli usage to docs

* Extract precision to a standalone file

* Add model base name to model.export params

* fix segnext head

* fix pre-commit

* Fix linters

* Refactor configuration

* Fix linters

* Fix pre-commit

* Update inference tests

* Extract common fixture in CLI tests

* Fix missing mean/scale metadata in native exporter

* Workaround OV failure on ONNX models with metadata

* Simplify checkpoint loading

* Cleanup

* Fix cli tests

* Fix litehrnet export config

* Fix import of custom ham seg head

* Del unused import

* Revert "added CUBLAS_WORKSPACE_CONFIG for testing deterministic option"

This reverts commit 4c41650.

* Fix typos in docs

* added CUBLAS_WORKSPACE_CONFIG for testing deterministic option

* add CUBLAS_WORKSPACE_CONFIG=:4096:8 for integration test

* align with new export deisgn

* refactor code

* fix minor bugs

* fix typo

* align with pre-commit

* align with torch model test pipeline

* give more epoch to some models during intg test

* remove commented code

* add ins-seg into export intg test

* set 10eph to atss_resnext101 in intg test

* exclude i-seg models from comparing scores in intg test

* move mmdeploy config files

* make test_pipeline in _export_parameters

* align with pre-commit

* revert left thing

* change fit_to_window to resize

* exclude unstable model

* revert integration test

* add integration test

* update docstring

* add reference in mmdeploy config

---------

Co-authored-by: Vladislav Sovrasov <vladislav.sovrasov@intel.com>
Co-authored-by: Prokofiev Kirill <kirill.prokofiev@intel.com>
Co-authored-by: Sungman Cho <sungman.cho@intel.com>
Co-authored-by: Samet Akcay <samet.akcay@intel.com>
Co-authored-by: Yunchu Lee <yunchu.lee@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sovrasov sovrasov sovrasov approved these changes

@samet-akcay samet-akcay Awaiting requested review from samet-akcay

@harimkang harimkang Awaiting requested review from harimkang

@vinnamkim vinnamkim Awaiting requested review from vinnamkim

@jaegukhyun jaegukhyun Awaiting requested review from jaegukhyun

@sungmanc sungmanc Awaiting requested review from sungmanc

@eugene123tw eugene123tw Awaiting requested review from eugene123tw

@chuneuny-emily chuneuny-emily Awaiting requested review from chuneuny-emily

Assignees
No one assigned
Labels
BUILD DEPENDENCY Any changes in any dependencies (new dep or its version) should be produced via Change Request on PM
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kprokofi @sovrasov