Refactor configuration & CLI

[harimkang]

Summary

I wanted to break up the PR as much as possible, but the CLI and configuration are still intertwined, so please excuse the large size of the PR. (Most files are YAML files.)

• Hydra -> Jsonargparse
• config + recipe -> recipe
• Configuration structure refactoring
• Add Jsonargparse based CLI
• LightnigModule Argument update (optimizer & scheduler argument update): https://lightning.ai/docs/pytorch/stable/cli/lightning_cli_advanced_3.html

Next PR

• Auto-configuration
• Engine.from_config

How to test

(otx) otx train --config <Config-Path> --data_root <Data-Path> --model.num_classes 2

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have added e2e tests for validation.
• I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
• I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
• I have linked related issues.

License

• I submit my code changes under the same Apache License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below).

# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

[samet-akcay]

i've reviewed until cli.py. Will continue tomorrow

[samet-akcay]

thanks @harimkang. I think I'm happy with the changes. Great effort!

[harimkang]

the process doesn't end as shown below with integration test.

It is only reproduced in the Visual Prompting model (GPU train & test) + openvino_model (CPU test only) test case.
so I disable visual_prompting SAM model in integration test case temporarily.
I have reported this to Vinnam and will revisit this at a later date.

[vinnamkim]

Sorry for your inconvenience, but since the time zone is different, please resolve the conflict and merge after this PR #2764.

[harimkang]

Sorry for your inconvenience, but since the time zone is different, please resolve the conflict and merge after this PR #2764.

Yes, of course. If that PR passes all the tests and then gets merged, I'll fix it here.

[vinnamkim]

Since the integration tests of #2764 were failed and seem hard to fix this time, let merge this first.