Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update instance-segmentation tutorial documentation #2073

Closed
wants to merge 21 commits into from
Closed

Update instance-segmentation tutorial documentation #2073

wants to merge 21 commits into from

Conversation

harimkang
Copy link
Contributor

Summary

Rewrote the Instance-segmentation tutorial, which was using the toy dataset, by modifying it to use a WGISD dataset like detection.
(car_tree_bug -> WGISD dataset)

How to test

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added e2e tests for validation.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

harimkang and others added 17 commits April 14, 2023 09:05
@harimkang
HOT-FIX: Revert segmentation model's ignore mode in CLI (Develop) (#2012
c4c0e05 
)

Revert segmentation ignore=True
@goodsong81
Bump up version to 1.3.0rc0 (#2016)
6cbac46
@goodsong81 @harimkang
@eugene123tw @sungmanc @yunchu @jaegukhyun @cih9088
Merge back 1.2.0rc1 (#2026)
09c9dac 
* HOT-FIX: Revert segmentation model's ignore mode in CLI (#2011)

* Improve tiling preprocess (#2013)

* Fix reg tests (#2008)

* Fix openvino import error due to Tiler init import (#2015)

* Bump up version to 1.2.0 (#2017)

* Set the python version to "3.10" for code-scan workflow

* Add missing __init__.py (#2019)

* Fix issue that str2bool not being applied in certain cases (#2023)

* Fix default dict (#2025)

---------

Co-authored-by: Harim Kang <harim.kang@intel.com>
Co-authored-by: Eugene Liu <eugene.liu@intel.com>
Co-authored-by: Sungman Cho <sungman.cho@intel.com>
Co-authored-by: Yunchu Lee <yunchu.lee@intel.com>
Co-authored-by: Jaeguk Hyun <jaeguk.hyun@intel.com>
Co-authored-by: Inhyuk Cho <andy.inhyuk.jo@intel.com>
@goodsong81
HOTFIX: change doc version to 1.3.0
09a29f5
@sovrasov
Remove unused LabelGraph class (#2014)
068464a 
remove unused LabelGraph class
@goodsong81
Upgrade mmdeploy==0.14.0 from official PyPI (#2044)
37ae52c 
Signed-off-by: Songki Choi <songki.choi@intel.com>
@goodsong81 @harimkang
@eugene123tw @sungmanc @yunchu @jaegukhyun @cih9088 @eunwoosh @supersoob @GalyaZalesskaya @vinnamkim
Merge back 1.2.0rc2 (#2045)
d80a8bd 
* HOT-FIX: Revert segmentation model's ignore mode in CLI (#2011)

Revert segmentation ignore=True

* Improve tiling preprocess (#2013)

* prevent timeout during init phase

* Fix reg tests (#2008)

* Edit regression tests

* Change the dataset root

* Miss typo

* Fix pre-commit

* Fix openvino import error due to Tiler init import (#2015)

Remove init import for Tiler to prevent OpenVINO import

* Bump up version to 1.2.0 (#2017)

* Set the python version to "3.10" for code-scan workflow

* Add missing __init__.py (#2019)

* Add missing __init__.py

* Change license

* Release 1.2.0rc1

* Fix issue that str2bool not being applied in certain cases (#2023)

* Add workaround solution

* Fix minor

* Remove str int

* Fix default dict (#2025)

fix: change default to configdict

Signed-off-by: Inhyuk Andy Cho <andy.inhyuk.jo@intel.com>

* Convert dummy datasets to toy datasets (#1988)

* Update cls, det datsets

* Remove useless files

* Change action datasets

* Edit action dataset

* change dir

* Add xml files

* Remove useless

* Edite tets

* Fix tests

* Fix tests

* Remove ptc

* Remove

* Fix precommit

* Update dataset, fix cls bug

* Remove useless dataset

* Edit drop_last

* Fix missed part

* Change threshold values to unifying

* bugfix: squeezing to 1 dimenetion

* Change threshold for deployment

* Fix multi-gpu issue, e2e tests

* Decrease num_workers for tiling test and tiling processes

* Revert num_workers for tests

* Fix datsets

---------

Co-authored-by: eunwoosh <eunwoo.shin@intel.com>

* Fix E2E tests (#2032)

* Optimize data preprocessing time and enhance overall performance in semantic segmentation (#2020)

* HOTFIX: change doc version to 1.2.0

* Add storage cache in Apache Arrow format using Datumaro (#2009)

* feat: change label entity to dictionay

* feat: add datumaro arrow cache

* refacor: move to proper directory

* fix: align to the latest

* fix: align data to otx

* fix: align new version

* refactor: disable storage cache for action tasks

* test: fix

* fix: version back

* docs: add to changelog

* fix: keep __height, __width

* docs: add description

* test: revert tests

* fix: revert back to list

* style: ruff

* HOT-FIX: Revert segmentation model's ignore mode in CLI (Develop) (#2012)

Revert segmentation ignore=True

* fix: make force verbose

* test: add storage cache test

* feat: datumaro 1.2.0

* test: test path exists

* test: do deepcopy

* style: make black happy

---------

Signed-off-by: Inhyuk Andy Cho <andy.inhyuk.jo@intel.com>
Co-authored-by: Harim Kang <harim.kang@intel.com>

* Fix typo in prediction_to_annotation_converter.py (#2028)

* HOT-FIX: Revert segmentation model's ignore mode in CLI (Develop) (#2012)

Revert segmentation ignore=True

* Bump up version to 1.3.0rc0 (#2016)

* bug fix

* del

* revert

* revert changlog

---------

Co-authored-by: Harim Kang <harim.kang@intel.com>
Co-authored-by: Songki Choi <songki.choi@intel.com>

* Fix tiling config loading bug (#2030)

* fix tiling loading bug

* fix mypy

* Make multi gpu child processes done right after evaluation (#2033)

* Fixes in exportable code (#2031)

* Create Actions domain and task type despite flag

* Move import torch to the function

* Fix str in dump_features

* Move dump_frames to otx/api/utils

* Remove __pycache__ from dunped exportable code

* Add tests for demo --output option

* Update sha for exportable code requirements

* Add flag to task_type_to_label_domain

* Roll back tests creation to add it in separate PR

* Add FEATURE_FLAGS_OTX_ACTION_TASKS init in demo.py

* Remove extra comments

* Fix linter

* Add documentation for the noisy label detection feature (#2034)

* Add documentation for noisy label detection feature

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>

* Update CHANGELOG.md

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>

* Add documentation

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>

* Fix typo

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>

* Fix small typo

---------

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
Co-authored-by: Songki Choi <songki.choi@intel.com>

* Remove skip tests, fix regression tests (#2036)

* Remove skip tests, fix regression tests

* Fix precommit

* Hide internal options from external GUI (#2037)

Signed-off-by: Songki Choi <songki.choi@intel.com>

* Add unit test for classification task and configurer (#2035)

* Reduce the depth of aumix process (#2038)

Enable light augmix

* Change samples_per_gpu in _infer_model(Detection) (#2041)

Change samples_per_gpu in _infer_model

* Decrease batch size if CUDA OOM occurs (#2022)

* implement adpating bs

* refine impl

* implement adaptive bs also in cls, seg task

* refine adapt bs algo to consider gpu util

* refactor code

* write comment and docstring

* implement decreasig bs on action task

* update learning rate after decreasing batch size

* implement test code of mmcv automatic_bs file

* remove meta modification

* remove unused improt

* implement test code of torch automatic_bs file

* align with pre commit

* add line to tell not supporting anomaly

* update CHANGELOG

* update docs

* change argument help

* change file name

* apply pr comment

* add auto_decrease_bs in learning parameters

* align with pre commit

* fix typo

* add integration test

* bugfix

* update test code

* not execute algo in nncf

* suppor nncf

* apply comment

* align with pre commit

* change method to set value

* refine warning comment

* remove breakpoint

* make hpo not use auto decrease batch size

* refine warning & typo fix

* align with pre commit

* Update CHANGELOG.md

Co-authored-by: Sungman Cho <sungman.cho@intel.com>

* update unit test

* update integration test

* bufix

* Release 1.2.0rc2

Signed-off-by: Songki Choi <songki.choi@intel.com>

* Update OTX commit hash for exportable code requiements

---------

Signed-off-by: Inhyuk Andy Cho <andy.inhyuk.jo@intel.com>
Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
Signed-off-by: Songki Choi <songki.choi@intel.com>
Co-authored-by: Harim Kang <harim.kang@intel.com>
Co-authored-by: Eugene Liu <eugene.liu@intel.com>
Co-authored-by: Sungman Cho <sungman.cho@intel.com>
Co-authored-by: Yunchu Lee <yunchu.lee@intel.com>
Co-authored-by: Jaeguk Hyun <jaeguk.hyun@intel.com>
Co-authored-by: Inhyuk Cho <andy.inhyuk.jo@intel.com>
Co-authored-by: eunwoosh <eunwoo.shin@intel.com>
Co-authored-by: Soobee Lee <soobee.lee@intel.com>
Co-authored-by: Galina Zalesskaya <galina.zalesskaya@intel.com>
Co-authored-by: Vinnam Kim <vinnam.kim@intel.com>
@eunwoosh
Bugfix: workspace is made by all of processes when multi GPU training…
6253762 
… with torchrun (#2048)

bugfix
@jaegukhyun
Change the way to get subset data in datumaro adapter (#2049)
c99811d 
* Change the way to get subset data in datumaro adapter

* Modify for unit, cli test

* Remove unnecessary print
@eunwoosh
Bug fix: value of validation variable is changed after auto decrease …
ff5be9f 
…batch size (#2054)

fix validate overriding
@GalyaZalesskaya
Add tests for --output parameter for demo (CLI and exportable) (#2043)
9858139 
* Add tests for --output parameter for demo

* Update OTX==1.2 commit in exp code
@sovrasov
Add configuration to segmentation IR (#2029)
2a05810 
* Add configuration to segmentation ir

* Add tests for ir metadata

* update license

* Fix linters

* pd changelog

* Fix segmentation unit test
@jaegukhyun
Modify e2e tests properly (#2056)
5742c5f 
* Modify e2e tests properly

* Revert openvino eval threshold for detection

* Remove unnecessary skip in tiling detection
@supersoob @harimkang
@eugene123tw @sungmanc @goodsong81 @yunchu @jaegukhyun @cih9088 @eunwoosh
Integrate new ignored loss in semantic segmentation (#2055)
919ac9e 
* HOT-FIX: Revert segmentation model's ignore mode in CLI (#2011)

Revert segmentation ignore=True

* Improve tiling preprocess (#2013)

* prevent timeout during init phase

* Fix reg tests (#2008)

* Edit regression tests

* Change the dataset root

* Miss typo

* Fix pre-commit

* Fix openvino import error due to Tiler init import (#2015)

Remove init import for Tiler to prevent OpenVINO import

* Bump up version to 1.2.0 (#2017)

* Set the python version to "3.10" for code-scan workflow

* Add missing __init__.py (#2019)

* Add missing __init__.py

* Change license

* Release 1.2.0rc1

* Fix issue that str2bool not being applied in certain cases (#2023)

* Add workaround solution

* Fix minor

* Remove str int

* Fix default dict (#2025)

fix: change default to configdict

Signed-off-by: Inhyuk Andy Cho <andy.inhyuk.jo@intel.com>

* Convert dummy datasets to toy datasets (#1988)

* Update cls, det datsets

* Remove useless files

* Change action datasets

* Edit action dataset

* change dir

* Add xml files

* Remove useless

* Edite tets

* Fix tests

* Fix tests

* Remove ptc

* Remove

* Fix precommit

* Update dataset, fix cls bug

* Remove useless dataset

* Edit drop_last

* Fix missed part

* Change threshold values to unifying

* bugfix: squeezing to 1 dimenetion

* Change threshold for deployment

* Fix multi-gpu issue, e2e tests

* Decrease num_workers for tiling test and tiling processes

* Revert num_workers for tests

* Fix datsets

---------

Co-authored-by: eunwoosh <eunwoo.shin@intel.com>

* Fix E2E tests (#2032)

* Optimize data preprocessing time and enhance overall performance in semantic segmentation (#2020)

* add unit test

* exp

* intg agg

* implement ignore label

* no update in model.py

* ignore in label

* only updated ignored loss

* final

* final

* exp

* refactor

* refactor ignore loss

* revert

* default mode is ignore false

* unit test revised

* model update

* revise detcon

* fix error in intg test

* test case revised

* make run in semisl

* add type hints

* update docs

* test case added

---------

Signed-off-by: Inhyuk Andy Cho <andy.inhyuk.jo@intel.com>
Co-authored-by: Harim Kang <harim.kang@intel.com>
Co-authored-by: Eugene Liu <eugene.liu@intel.com>
Co-authored-by: Sungman Cho <sungman.cho@intel.com>
Co-authored-by: Songki Choi <songki.choi@intel.com>
Co-authored-by: Yunchu Lee <yunchu.lee@intel.com>
Co-authored-by: Jaeguk Hyun <jaeguk.hyun@intel.com>
Co-authored-by: Inhyuk Cho <andy.inhyuk.jo@intel.com>
Co-authored-by: eunwoosh <eunwoo.shin@intel.com>
Co-authored-by: Lee, Soobee <soobeele@intel.com>
@eunwoosh
Action task supports multi GPU training (#2057)
3c0233d 
* draft multi GPU enablement

* align with pre commit

* revert logger

* add integration test

* update changelog

* add e2e test

* add type hint

* add pr link
@supersoob
Remove unused modules in semantic segmentation (#2067)
c91930a
@harimkang
Update instance-segmentation doc tutorial
7658ce4
@harimkang harimkang added the DOC Improvements or additions to documentation label Apr 26, 2023
@harimkang harimkang added this to the 1.3.0 milestone Apr 26, 2023
@harimkang harimkang requested a review from a team as a code owner April 26, 2023 04:30
eunwoosh
eunwoosh reviewed Apr 26, 2023
View reviewed changes
Copy link
Contributor

@eunwoosh eunwoosh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for update :) I left some comments. please take a look.

docs/source/guide/tutorials/base/how_to_train/instance_segmentation.rst Show resolved Hide resolved
docs/source/guide/tutorials/base/how_to_train/instance_segmentation.rst Outdated Show resolved Hide resolved
docs/source/guide/tutorials/base/how_to_train/instance_segmentation.rst Outdated Show resolved Hide resolved
harimkang and others added 2 commits April 26, 2023 13:56
@harimkang @eunwoosh
Update docs/source/guide/tutorials/base/how_to_train/instance_segment…
c349c1d 
…ation.rst

Co-authored-by: Eunwoo Shin <eunwoo.shin@intel.com>
@harimkang @eunwoosh
Update docs/source/guide/tutorials/base/how_to_train/instance_segment…
2cf110e 
…ation.rst

Co-authored-by: Eunwoo Shin <eunwoo.shin@intel.com>
eunwoosh
eunwoosh previously approved these changes Apr 26, 2023
View reviewed changes
@sungmanc
Copy link
Contributor

@goodsong81 , @harimkang , @eunwoosh I think this PR could be applied to release/1.2.1 since it could bring more useful examples. What do you think?

@eunwoosh
Copy link
Contributor

@goodsong81 , @harimkang , @eunwoosh I think this PR could be applied to release/1.2.1 since it could bring more useful examples. What do you think?

I agreed. or just merging it to releases/1.2.1 and merge back into develop branch is also possible if it's not urgent.

sungmanc
sungmanc requested changes Apr 26, 2023
View reviewed changes
Copy link
Contributor

@sungmanc sungmanc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to decide the target of this PR

@goodsong81
Copy link
Contributor

OK. Let's retarget to 1.2.1.

goodsong81
goodsong81 previously approved these changes Apr 26, 2023
View reviewed changes
@goodsong81 goodsong81 changed the base branch from develop to releases/1.2.1 April 26, 2023 08:57
@goodsong81 goodsong81 dismissed stale reviews from eunwoosh and themself April 26, 2023 08:57

The base branch was changed.

@goodsong81 goodsong81 modified the milestones: 1.3.0, 1.2.0 Apr 26, 2023
@harimkang harimkang mentioned this pull request Apr 27, 2023
Merged
8 tasks
@harimkang
Copy link
Contributor Author

Reopen the PR here (#2082) by retargeting to releases/1.2.1.

@harimkang harimkang closed this Apr 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sungmanc sungmanc sungmanc approved these changes

@eunwoosh eunwoosh eunwoosh left review comments

@goodsong81 goodsong81 goodsong81 left review comments

Assignees
No one assigned
Labels
DOC Improvements or additions to documentation
Projects
None yet
Milestone
1.2.0
Development

Successfully merging this pull request may close these issues.
8 participants
@harimkang @sungmanc @eunwoosh @goodsong81 @sovrasov @jaegukhyun @GalyaZalesskaya @supersoob