Update reqs

.github/workflows/pre_merge.yml

@@ -35,7 +35,7 @@ jobs:
       - name: Install dependencies
         run: python -m pip install -r requirements/dev.txt
       - name: Code quality checks
-        run: tox -vv -e pre-commit-all-py310
+        run: tox -vv -e pre-commit-all-py310-pt1
   Unit-Test:
     runs-on: [self-hosted, linux, x64, dev]
     needs: Code-Quality-Checks
@@ -71,7 +71,7 @@ jobs:
       - name: Install dependencies
         run: python -m pip install -r requirements/dev.txt
       - name: Run unit test
-        run: tox -vv -e unittest-all-${{ matrix.tox-env }}
+        run: tox -vv -e unittest-all-${{ matrix.tox-env }}-pt1
       - name: Upload coverage artifact
         if: ${{ matrix.python-version == '3.8' }}
         uses: actions/upload-artifact@v3

.github/workflows/run_tests_in_tox.yml

@@ -52,7 +52,7 @@ jobs:
       - name: Install dependencies
         run: python -m pip install -r requirements/dev.txt
       - name: E2E Tests
-        run: tox -vv -e tests-${{ inputs.toxenv-task }}-${{ inputs.toxenv-pyver }} -- ${{ inputs.tests-dir }}
+        run: tox -vv -e tests-${{ inputs.toxenv-task }}-${{ inputs.toxenv-pyver }}-pt1 -- ${{ inputs.tests-dir }}
       - name: Upload test results
         uses: actions/upload-artifact@v3
         with:

CHANGELOG.md

@@ -15,21 +15,40 @@ All notable changes to this project will be documented in this file.
 - Add a new object detector Lite-DINO(<https://github.com/openvinotoolkit/training_extensions/pull/2457>)
 - Add Semi-SL Mean Teacher algorithm for Instance Segmentation task(<https://github.com/openvinotoolkit/training_extensions/pull/2444>)
 - Official supports for YOLOX-X, YOLOX-L, YOLOX-S, ResNeXt101-ATSS (<https://github.com/openvinotoolkit/training_extensions/pull/2485>)
-- Add new argument to track resource usage in train command(<https://github.com/openvinotoolkit/training_extensions/pull/2500>)
+- Add new argument to track resource usage in train command (<https://github.com/openvinotoolkit/training_extensions/pull/2500>)
+- Add Self-SL for semantic segmentation of SegNext families (<https://github.com/openvinotoolkit/training_extensions/pull/2215>)
+- Adapt input size automatically based on dataset statistics (<https://github.com/openvinotoolkit/training_extensions/pull/2499>)
 
 ### Enhancements
 
 - Refine input data in-memory caching (<https://github.com/openvinotoolkit/training_extensions/pull/2416>)
 - Adapt timeout value of initialization for distributed training (<https://github.com/openvinotoolkit/training_extensions/pull/2422>)
 - Optimize data loading by merging load & resize operations w/ caching support for cls/det/iseg/sseg (<https://github.com/openvinotoolkit/training_extensions/pull/2438>, <https://github.com/openvinotoolkit/training_extensions/pull/2453>, <https://github.com/openvinotoolkit/training_extensions/pull/2460>)
+- Support torch==2.0.1 (<https://github.com/openvinotoolkit/training_extensions/pull/2465>)
 
 ### Bug fixes
 
 - Fix F1 auto-threshold to choose best largest confidence (<https://github.com/openvinotoolkit/training_extensions/pull/2371>)
 
 ### Known issues
 
-- OpenVINO(==2023.0) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch==1.13.1
+- OpenVINO(==2023.0) IR inference is not working well on 2-stage models (e.g. Mask-RCNN) exported from torch>=1.13.1
+- NNCF QAT optimization is disabled for MaskRCNN models due to CUDA runtime error in ROIAlign kernel on torch==2.0.1
+
+## \[v1.4.2\]
+
+### Enhancements
+
+- Bump datumaro version to 1.5.0rc0 (<https://github.com/openvinotoolkit/training_extensions/pull/2470>)
+- Set tox version constraint (<https://github.com/openvinotoolkit/training_extensions/pull/2472>)
+- Add model category attributes to model template (<https://github.com/openvinotoolkit/training_extensions/pull/2439>)
+
+### Bug fixes
+
+- Bug fix for albumentations (<https://github.com/openvinotoolkit/training_extensions/pull/2467>)
+- Add workaround for the incorrect meta info M-RCNN (used for XAI) (<https://github.com/openvinotoolkit/training_extensions/pull/2437>)
+- Fix label list order for h-label classification (<https://github.com/openvinotoolkit/training_extensions/pull/2440>)
+- Modified fq numbers for lite HRNET e2e tests (<https://github.com/openvinotoolkit/training_extensions/pull/2445>)
 
 ## \[v1.4.1\]
 

README.md

@@ -76,7 +76,7 @@ OpenVINO™ Training Extensions will provide the following features in coming re
 
 Please refer to the [installation guide](https://openvinotoolkit.github.io/training_extensions/latest/guide/get_started/installation.html).
 
-Note: Python 3.8 and 3.9 were tested, along with Ubuntu 18.04 and 20.04.
+Note: Python 3.8, 3.9 and 3.10 were tested, along with Ubuntu 18.04, 20.04 and 22.04.
 
 ### OpenVINO™ Training Extensions CLI Commands
 
@@ -137,6 +137,15 @@ Please use [Issues](https://github.com/openvinotoolkit/training_extensions/issue
 
 ## Known limitations
 
-[misc](https://github.com/openvinotoolkit/training_extensions/tree/misc) branch contains training, evaluation, and export scripts for models based on TensorFlow and PyTorch. These scripts are not ready for production. They are exploratory and have not been validated.
+[misc](https://github.com/openvinotoolkit/training_extensions/tree/misc) branch contains training, evaluation, and export scripts for models based on TensorFlow and PyTorch.
+These scripts are not ready for production. They are exploratory and have not been validated.
+
+---
+
+## Disclaimer
+
+Intel is committed to respecting human rights and avoiding complicity in human rights abuses.
+See Intel's [Global Human Rights Principles](https://www.intel.com/content/www/us/en/policy/policy-human-rights.html).
+Intel's products and software are intended only to be used in applications that do not cause or contribute to a violation of an internationally recognized human right.
 
 ---

docs/source/conf.py

@@ -11,14 +11,21 @@
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
 
+import os
+import sys
+
+sys.path.insert(0, os.path.abspath("../../src"))
+
+from otx import __version__
+
 # ruff: noqa
 
 # -- Project information ----------------------------------------------------- #
 
 project = 'OpenVINO™ Training Extensions'
 copyright = '2023, OpenVINO™ Training Extensions Contributors'
 author = 'OpenVINO™ Training Extensions Contributors'
-release = '1.5.0'
+release = __version__
 
 # -- General configuration --------------------------------------------------- #
 

docs/source/guide/explanation/additional_features/auto_configuration.rst

@@ -120,3 +120,10 @@ OpenVINO™ Training Extensions will automatically recognize these types of task
 
 .. note::
     To use auto template configuration with Self-SL training type `--task` option is required since it is impossible to recognize task type by folder with only images.
+
+Auto-adapt input size
+---------------------
+
+"Auto" input size feature tries to automatically select the right model input size
+based on given dataset statictics.
+See :ref:`adaptive-input-size`.

docs/source/guide/explanation/additional_features/config_input_size.rst

@@ -21,10 +21,50 @@ The available input sizes are currently as follows:
 
 - 64x64 (only for classification)
 - 128x128 (only for classification)
+- 224x224 (only for classification)
 - 256x256
 - 384x384
 - 512x512
+- 768x768
 - 1024x1024
+- Default (per-model default input size)
+- Auto (adaptive to dataset statistics)
+
+.. _adaptive-input-size:
+
+Adaptive Input Size
+-------------------
+
+"Auto" mode tries to automatically select the right size
+based on given dataset statictics.
+
+1. OTX analyzes the input dataset to get robust statistics.
+
+2. Input size is initially set to typical large image size.
+
+.. code-block::
+
+    input_size = large_image_size
+
+3. (Optionally) Input size is adjusted by object sizes in the dataset, if any.
+   The input size from image size is rescaled accoridng to the ratio of
+   minimum recongnizable object size of models, which is typically 16x16 ~ 32x32,
+   and the typical small object size in the dataset.
+   In short, if objects are 64x64 in general in 512x512 image,
+   it will be down-scaled to 256x256 as 32x32 objects are enough to be detected.
+
+.. code-block::
+
+    input_size = input_size * MIN_RECOGNIZABLE_OBJECT_SIZE / small_object_size
+
+4. Select the closest size from standard preset sizes
+
+5. Restrict scale-up
+
+.. code-block::
+
+    input_size = min(input_size, default_model_input_size)
+
 
 .. Note::
     Using smaller input size with datasets having lower image resolutions or larger objects can yield a speed advantage with minimal impact on model performance.

docs/source/guide/get_started/installation.rst

@@ -46,15 +46,15 @@ Refer to the `official installation guide <https://pytorch.org/get-started/previ
 
 .. note::
 
-    Currently, only torch==1.13.1 was fully validated, torch==2.x will be supported in the future (Previous versions are not supported due to security issues).
+    Currently, only torch==1.13.1 ~ 2.0.1 was fully validated. (older versions are not supported due to security issues).
 
 .. code-block::
 
-    # install command for torch==1.13.1 for CUDA 11.7:
-    pip install torch==1.13.1 torchvision==0.14.1 --extra-index-url https://download.pytorch.org/whl/cu117
+    # Install command for torch==2.0.1 for CUDA 11.7:
+    pip install torch==2.0.1 torchvision==0.15.2 --extra-index-url https://download.pytorch.org/whl/cu117
 
-    # or install command for torch==1.13.1 for CUDA 11.1:
-    pip install torch==1.13.1 torchvision==0.14.1 --extra-index-url https://download.pytorch.org/whl/cu111
+    # Or, install command for torch==1.13.1 for CUDA 11.7:
+    pip install torch==1.13.1 torchvision==0.14.1 --extra-index-url https://download.pytorch.org/whl/cu117
 
 4. Install OpenVINO™ Training Extensions package from either:
 

requirements/action.txt

@@ -4,4 +4,4 @@ mmcv-full==1.7.0
 mmaction2==0.24.1
 mmdet==2.28.1
 mmdeploy==0.14.0
-yapf<0.40.0  # it should be removed after https://github.com/google/yapf/issues/1118 is solved 
+yapf<0.40.0  # it should be removed after https://github.com/google/yapf/issues/1118 is solved

requirements/anomaly.txt

@@ -1,6 +1,4 @@
 # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
 # Anomaly Requirements.
-torchvision<0.15.1
-torchtext<0.15.1
 anomalib==0.5.1
 kornia==0.6.9

requirements/classification.txt

@@ -5,4 +5,4 @@ mmcls==0.25.0
 timm==0.6.12
 mmdeploy==0.14.0
 pytorchcv
-yapf<0.40.0  # it should be removed after https://github.com/google/yapf/issues/1118 is solved 
+yapf<0.40.0  # it should be removed after https://github.com/google/yapf/issues/1118 is solved

requirements/detection.txt

@@ -8,4 +8,4 @@ timm==0.6.12
 mmdeploy==0.14.0
 mmengine==0.7.4
 scikit-image
-yapf<0.40.0  # it should be removed after https://github.com/google/yapf/issues/1118 is solved 
+yapf<0.40.0  # it should be removed after https://github.com/google/yapf/issues/1118 is solved

requirements/segmentation.txt

@@ -7,4 +7,4 @@ mmdeploy==0.14.0
 timm==0.6.12
 pytorchcv
 einops==0.6.1
-yapf<0.40.0  # it should be removed after https://github.com/google/yapf/issues/1118 is solved 
+yapf<0.40.0  # it should be removed after https://github.com/google/yapf/issues/1118 is solved

src/otx/algorithms/anomaly/tasks/openvino.py

@@ -242,7 +242,8 @@ def _populate_metadata_legacy(self, model: ModelEntity) -> Dict[str, Any]:
             to_tensor=True,
         )
         metadata = {
-            "transform": transform.to_dict(),
+            # TODO: Replace with transform.to_dict() when OTX supports albumentations 1.3.0
+            "transform": {"transform": transform._to_dict()},
             "image_threshold": image_threshold,
             "pixel_threshold": pixel_threshold,
             "min": min_value,

src/otx/algorithms/classification/adapters/mmcls/configurer.py

@@ -19,7 +19,6 @@
 from otx.algorithms.common.adapters.mmcv.semisl_mixin import SemiSLConfigurerMixin
 from otx.algorithms.common.adapters.mmcv.utils.config_utils import (
     InputSizeManager,
-    get_configured_input_size,
     recursively_update_cfg,
     update_or_add_custom_hook,
 )
@@ -166,11 +165,17 @@ def configure_input_size(
         cfg, input_size_config: InputSizePreset = InputSizePreset.DEFAULT, model_ckpt_path: Optional[str] = None
     ):
         """Change input size if necessary."""
-        input_size = get_configured_input_size(input_size_config, model_ckpt_path)
-        if input_size is None:
+        manager = InputSizeManager(cfg)
+        input_size = manager.get_configured_input_size(input_size_config, model_ckpt_path)
+        if input_size is None:  # InputSizePreset.DEFAULT
             return
 
-        InputSizeManager(cfg.data).set_input_size(input_size)
+        if input_size == (0, 0):  # InputSizePreset.AUTO
+            input_size = BaseConfigurer.adapt_input_size_to_dataset(cfg, manager)
+            if input_size is None:
+                return
+
+        manager.set_input_size(input_size)
         logger.info("Input size is changed to {}".format(input_size))
 
 

src/otx/algorithms/classification/adapters/mmcls/task.py

@@ -635,7 +635,7 @@ def patch_input_preprocessing(deploy_cfg):
                 mo_options.flags = list(set(mo_options.flags))
 
             def patch_input_shape(deploy_cfg):
-                input_size_manager = InputSizeManager(cfg.data)
+                input_size_manager = InputSizeManager(cfg)
                 size = input_size_manager.get_input_size_from_cfg("test")
                 assert all(isinstance(i, int) and i > 0 for i in size)
                 # default is static shape to prevent an unexpected error

src/otx/algorithms/classification/adapters/openvino/task.py

@@ -36,6 +36,7 @@
 from otx.algorithms.classification.configs import ClassificationConfig
 from otx.algorithms.classification.utils import (
     get_cls_deploy_config,
+    get_cls_inferencer_configuration,
     get_hierarchical_label_list,
 )
 from otx.algorithms.common.utils import OTXOpenVinoDataLoader
@@ -113,7 +114,19 @@ def __init__(
             max_num_requests=num_requests,
             plugin_config={"PERFORMANCE_HINT": "THROUGHPUT"},
         )
-        self.model = Model.create_model(model_adapter, "Classification", {}, preload=True)
+        self.configuration = get_cls_inferencer_configuration(self.label_schema)
+
+        # create a dummy hierarchical config for backward compatibility, which is not actually used
+        if self.configuration["hierarchical"]:
+            try:
+                model_adapter.get_rt_info(["model_info", "hierarchical_config"])
+            except RuntimeError:
+                self.configuration["hierarchical_config"] = json.dumps(
+                    {"cls_heads_info": {"label_to_idx": [], "all_groups": []}, "label_tree_edges": []}
+                )
+
+        self.model = Model.create_model(model_adapter, "Classification", self.configuration, preload=True)
+
         self.converter = ClassificationToAnnotationConverter(self.label_schema)
         self.callback_exceptions: List[Exception] = []
         self.model.inference_adapter.set_callback(self._async_callback)

src/otx/algorithms/classification/configs/configuration.yaml

@@ -287,8 +287,10 @@ learning_parameters:
     header: Configure model input size.
     options:
       DEFAULT: "Default"
+      AUTO: "Auto"
       _64x64: "64x64"
       _128x128: "128x128"
+      _224x224: "224x224"
       _256x256: "256x256"
       _384x384: "384x384"
       _512x512: "512x512"
@@ -438,7 +440,7 @@ algo_backend:
     warning: null
   mem_cache_size:
     affects_outcome_of: TRAINING
-    default_value: 0
+    default_value: 1000000000
     description: Size of memory pool for caching decoded data to load data faster (bytes).
     editable: true
     header: Size of memory pool

src/otx/algorithms/common/adapters/mmcv/configurer.py

@@ -3,8 +3,9 @@
 # SPDX-License-Identifier: Apache-2.0
 #
 
+import json
 import os
-from typing import Any, Dict, List, Optional
+from typing import Any, Dict, List, Optional, Tuple
 
 import numpy as np
 import torch
@@ -18,6 +19,7 @@
     patch_persistent_workers,
 )
 from otx.algorithms.common.adapters.mmcv.utils.config_utils import (
+    InputSizeManager,
     patch_color_conversion,
     patch_from_hyperparams,
     recursively_update_cfg,
@@ -26,6 +28,7 @@
 from otx.algorithms.common.configs.configuration_enums import InputSizePreset
 from otx.algorithms.common.tasks.base_task import OnHookInitialized
 from otx.algorithms.common.utils import UncopiableDefaultDict, append_dist_rank_suffix
+from otx.algorithms.common.utils.data import compute_robust_dataset_statistics
 from otx.algorithms.common.utils.logger import get_logger
 from otx.api.usecases.reporting.time_monitor_callback import TimeMonitorCallback
 from otx.core.data import caching
@@ -492,3 +495,41 @@ def get_data_cfg(cfg, subset):
                 dataset = dataset.dataset
             return dataset
         return cfg.data[subset]
+
+    @staticmethod
+    def adapt_input_size_to_dataset(
+        cfg, input_size_manager: InputSizeManager, downscale_only: bool = True, use_annotations: bool = False
+    ) -> Optional[Tuple[int, int]]:
+        """Compute appropriate model input size w.r.t. dataset statistics.
+
+        Args:
+            cfg (Dict): Global configuration.
+            input_size_manager: (InputSizeManager): Pre-configured input size manager
+            downscale_only (bool) : Whether to allow only smaller size than default setting. Defaults to True.
+            use_annotations (bool): Whether to consider annotation shapes to compute input size. Defaults to False.
+
+        Returns:
+            Tuple[int, int]: (width, height) or None
+        """
+
+        data_cfg = BaseConfigurer.get_data_cfg(cfg, "train")
+        dataset = data_cfg.get("otx_dataset", None)
+        if dataset is None:
+            return None
+
+        stat = compute_robust_dataset_statistics(dataset, use_annotations)
+        if not stat:
+            return None
+        logger.info(f"Dataset stat: {json.dumps(stat, indent=4)}")
+
+        # Fit to typical large image size (conservative)
+        # -> "avg" size might be preferrable for efficiency
+        image_size = stat["image"]["robust_max"]
+        object_size = None
+        if use_annotations and stat["annotation"]:
+            # Refine using annotation shape size stat
+            # Fit to typical small object size (conservative)
+            # -> "avg" size might be preferrable for efficiency
+            object_size = stat["annotation"].get("size_of_shape", {}).get("robust_min", None)
+
+        return input_size_manager.adapt_input_size_to_dataset(image_size, object_size, downscale_only)