Add Configuration Converter in otx.tools

[harimkang]

Summary

https://jira.devtools.intel.com/browse/CVS-134576

• Add the Converter to otx.tools.
• Modify the functions in Converter to be static.
• Improve readability of some parameter update code.
• Add an instantiate function to Converter that creates a dict with engine and train arguments based on a configuration of type dict.

Converter is now available as shown below.

from otx.tools.converter import ConfigConverter

geti_det_config = "tests/assets/geti-configs/det.json"
data_root = "tests/assets/car_tree_bug"
work_dir = "otx-workspace-geti"


config = ConfigConverter.convert(geti_det_config)
engine, train_kwargs = ConfigConverter.instantiate(
    config=config,
    data_root=data_root,
    work_dir=work_dir,
)

engine.train(**train_kwargs)

How to test

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have added e2e tests for validation.
• I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
• I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
• I have linked related issues.

License

• I submit my code changes under the same Apache License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below).

# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

[codecov]

Codecov Report

Attention: Patch coverage is 92.20779% with 12 lines in your changes are missing coverage. Please review.

Project coverage is 78.52%. Comparing base (7df78c1) to head (ae6f407).

Files	Patch %	Lines
src/otx/tools/converter.py	92.76%	11 Missing ⚠️
src/otx/engine/engine.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #3254      +/-   ##
===========================================
+ Coverage    78.39%   78.52%   +0.13%     
===========================================
  Files          179      180       +1     
  Lines        15169    15323     +154     
===========================================
+ Hits         11891    12033     +142     
- Misses        3278     3290      +12     

Flag	Coverage Δ
py310	78.52% <92.20%> (+0.13%)	⬆️
py311	78.52% <92.20%> (+0.13%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[harimkang]

@vinnamkim @jaegukhyun
I found some issue with multi-label, h-label classification,

In OTX 1.X (1.6), The template is used without distinction between multi class classification, multi label classification, and h label classification (They share one template.), and the model is determined at runtime via the functions below.

training_extensions/src/otx/algorithms/classification/task.py

Lines 134 to 157 in 620dfd1

	def _is_multi_label(self, label_groups: List[LabelGroup], all_labels: List[LabelEntity]):
	"""Check whether the current training mode is multi-label or not."""
	# NOTE: In the current Geti, multi-label should have `___` symbol for all group names.
	find_multilabel_symbol = ["___" in getattr(i, "name", "") for i in label_groups]
	return (
	(len(label_groups) > 1) and (len(label_groups) == len(all_labels)) and (False not in find_multilabel_symbol)
	)
	def _set_train_mode(self):
	label_groups = self._task_environment.label_schema.get_groups(include_empty=False)
	all_labels = self._task_environment.label_schema.get_labels(include_empty=False)
	self._multilabel = self._is_multi_label(label_groups, all_labels)
	if self._multilabel:
	logger.info("Classification mode: multilabel")
	elif len(label_groups) > 1:
	logger.info("Classification mode: hierarchical")
	self._hierarchical = True
	self._hierarchical_info = get_hierarchical_info(self._task_environment.label_schema)
	if not self._multilabel and not self._hierarchical:
	logger.info("Classification mode: multiclass")
	if self._hyperparams.algo_backend.train_type == TrainType.Selfsupervised:
	self._selfsl = True

training_extensions/src/otx/algorithms/classification/adapters/mmcls/task.py

Lines 92 to 106 in 620dfd1

	def _init_task(self): # noqa
	"""Initialize task."""
	if self._multilabel:
	cfg_path = os.path.join(self._model_dir, "model_multilabel.py")
	elif self._hierarchical:
	cfg_path = os.path.join(self._model_dir, "model_hierarchical.py")
	else:
	cfg_path = os.path.join(self._model_dir, "model.py")
	self._recipe_cfg = OTXConfig.fromfile(cfg_path)
	self._recipe_cfg.domain = self._task_type.domain
	self._recipe_cfg.model.multilabel = self._multilabel
	self._recipe_cfg.model.hierarchical = self._hierarchical
	if self._hierarchical:
	self._recipe_cfg.model.head.hierarchical_info = self._hierarchical_info

We need to discuss how to handle this in Geti. I think we should get the relevant task information from Geti or Dataset.

[vinnamkim]

We need to discuss how to handle this in Geti. I think we should get the relevant task information from Geti or Dataset.

See https://github.com/intel-innersource/frameworks.ai.interactive-ai-workflow.sonoma-creek/blob/433e1c46f123033b559935739552a9e1c6c9fc4a/src/workload_configurations/otx2/scripts/definitions.py#L99-L105
You can assume that task_type will be given

[harimkang]

We need to discuss how to handle this in Geti. I think we should get the relevant task information from Geti or Dataset.

See https://github.com/intel-innersource/frameworks.ai.interactive-ai-workflow.sonoma-creek/blob/433e1c46f123033b559935739552a9e1c6c9fc4a/src/workload_configurations/otx2/scripts/definitions.py#L99-L105 You can assume that task_type will be given

Thanks for letting me know.