Skip to content

Commit

Permalink
Merge pull request #3765 from sungchul2/doc-data-config-override
Browse files Browse the repository at this point in the history
Update `tutorials/advanced/configuration.rst`
  • Loading branch information
sungchul2 authored Jul 25, 2024
2 parents 8e1d5c3 + d0948a4 commit e2566b8
Showing 1 changed file with 144 additions and 21 deletions.
165 changes: 144 additions & 21 deletions docs/source/guide/tutorials/advanced/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -51,39 +51,93 @@ We can use the ``~.yaml`` with the above values configured.
The basic configuration is the same as the configuration configuration format for jsonargparse.
`Jsonargparse Documentation <https://jsonargparse.readthedocs.io/en/v4.27.4/#configuration-files>_`

### Configuration overrides

***********************
Configuration Overrides
***********************

Here we provide a feature called ``overrides``.
This feature allows you to override the values need from the default configuration.
Currently, you can overrides ``data``, ``callbacks``, ``logger``, and other single value configurations.
Also, you can use ``reset`` to reset the configurations from the default values to the new values.

To update single value configurations, just set them in the overrides.

.. code-block:: yaml
...
overrides:
max_epochs: 10 # update to 10
data:
image_color_channel: BGR # update to BGR
...
If you want to add new configuration which isn't set before, just set them in the overrides and it will be appended.

.. code-block:: yaml
...
overrides:
data:
config:
train_subset:
transforms:
- type: LoadImageFromFile
- backend: cv2
scale: 224
type: RandomResizedCrop
- direction: horizontal
prob: 0.5
type: RandomFlip
- type: PackInputs
new_configuration: 1 # add new_configuration with 1
...
This feature allows you to override the values need from the default configuration.
You can see the final configuration with the command below.
.. note::

You can see the final configuration with the command below.

.. code-block:: shell
$ otx train --config <config-file-path> --print_config
--------------
Data overrides
--------------

``data`` can currently be provided as a list of different transforms.
The way to override this is as follows.

Let's try to change the size of Resize and the prob of RandomFlip which are already set in `base data configuration of instance segmentation <https://github.com/openvinotoolkit/training_extensions/blob/develop/src/otx/recipe/_base_/data/instance_segmentation.yaml>`_.
To change them, you can just set the values in the overrides.

.. code-block:: yaml
.. code-block:: shell
...
overrides:
data:
train_subset:
transforms:
- class_path: otx.core.data.transform_libs.torchvision.Resize
init_args:
size: # update `size` from 1024 to 512
- 512
- 512
# Pad is used as is because it is not set here
- class_path: otx.core.data.transform_libs.torchvision.RandomFlip
init_args:
prob: 0 # update `prob` from 0.5 to 0
# ToDtype and Normalize are used as is because they are not set here
...
Like single value configurations, when adding new transforms in overrides it will be appended.

.. code-block:: yaml
...
overrides:
data:
train_subset:
transforms:
- class_path: new_transform
...
$ otx train --config <config-file-path> --print_config
### Callbacks & Logger overrides
----------------------------
Callbacks & Logger overrides
----------------------------

``callbacks`` and ``logger`` can currently be provided as a list of different callbacks and loggers. The way to override this is as follows.
``callbacks`` and ``logger`` can currently be provided as a list of different callbacks and loggers.
The way to override this is as follows.

For example, if you want to change the patience of EarlyStopping, you can configure the overrides like this

Expand All @@ -92,6 +146,75 @@ For example, if you want to change the patience of EarlyStopping, you can config
overrides:
...
callbacks:
- class_path: ligthning.pytorch.callbacks.EarlyStopping
- class_path: ligthning.pytorch.callbacks.EarlyStopping
init_args:
patience: 3
patience: 3
---------------
Reset overrides
---------------

If you want to **reset** the configurations to the default values, especially ``data``, ``callbacks``, or ``logger`` that are difficult to be reset, you can use the ``reset`` keyword.
The way to override this is as follows.

Let's try to reset all transforms which are already set in `base data configuration of instance segmentation <https://github.com/openvinotoolkit/training_extensions/blob/develop/src/otx/recipe/_base_/data/instance_segmentation.yaml>`_.
To reset them, you can just add the keys in ``reset`` in the overrides.
``reset`` also supports both types, string and list.
If you want to reset single one, string or list can be used.
But if you want to reset multiple ones, list should be used.

.. tab-set::

.. tab-item:: single one

.. code-block:: yaml
...
overrides:
reset:
- data.train_subset.transforms
# or
# reset: data.train_subset.transforms
data:
train_subset:
transforms:
# previous ones are not used anymore
- class_path: new_transform_1
- class_path: new_transform_2
...
.. tab-item:: multiple ones

.. code-block:: yaml
...
overrides:
reset:
- data.train_subset.transforms
- data.val_subset.transforms
- data.test_subset.transforms
- callbacks
# reset: data.train_subset.transforms cannot be used because there are multiple resets
data:
train_subset:
transforms:
# previous ones are not used anymore
- class_path: new_transform_1
- class_path: new_transform_2
val_subset:
transforms:
# previous ones are not used anymore
- class_path: new_transform_1
- class_path: new_transform_2
test_subset:
transforms:
# previous ones are not used anymore
- class_path: new_transform_1
- class_path: new_transform_2
...
callbacks:
# previous ones are not used anymore
- class_path: new_callback_1
- class_path: new_callback_2
...

0 comments on commit e2566b8

Please sign in to comment.