Make the CLI hierarchical (#8787)

Currently, the CLI has one level of subcommands, and all subcommands
work on tasks. This leaves no room for subcommands that work on other
CVAT resources.

This change redesigns the CLI interface by adding another level of
subcommand hierarchy. Instead of running `cvat-cli <action>`, the user
will now run `cvat-cli <resource> <action>`. Previously available
commands are left available as deprecated aliases.

As a proof of concept, this PR adds some basic project actions.

I have also used this opportunity to correct some of the task action
names, specifically `export`, `import`, `dump` and `upload`. These names
don't correspond to either SDK function names, API endpoints, or UI
labels corresponding to these actions. In the new subcommand hierarchy,
I renamed those commands to `backup`, `create-from-backup`,
`export-dataset` and `import-dataset`, which are more consistent with
how other CVAT components call these actions.

I rewrote the introduction and usage sections of the cli README and
reference in order to reduce clutter and remove the need to
resynchronize the help output after every interface change.

changelog.d/20241206_184906_roman_cli_hierarchy.md

@@ -0,0 +1,20 @@
+### Added
+
+- \[CLI\] Added new commands: `project create`, `project delete`, `project ls`
+  (<https://github.com/cvat-ai/cvat/pull/8787>)
+
+- \[SDK\] You can now use `client.projects.remove_by_ids` to remove multiple
+  projects
+  (<https://github.com/cvat-ai/cvat/pull/8787>)
+
+### Changed
+
+- \[CLI\] Switched to a new subcommand hierarchy; now CLI subcommands
+  have the form `cvat-cli <resource> <action>`
+  (<https://github.com/cvat-ai/cvat/pull/8787>)
+
+### Deprecated
+
+- \[CLI\] All existing CLI commands of the form `cvat-cli <action>`
+  are now deprecated. Use `cvat-cli task <action>` instead
+  (<https://github.com/cvat-ai/cvat/pull/8787>)

cvat-cli/README.md

@@ -1,57 +1,60 @@
 # Command-line client for CVAT
 
-A simple command line interface for working with CVAT tasks. At the moment it
+A simple command line interface for working with CVAT. At the moment it
 implements a basic feature set but may serve as the starting point for a more
 comprehensive CVAT administration tool in the future.
 
-Overview of functionality:
+The following subcommands are supported:
 
-- Create a new task (supports name, bug tracker, project, labels JSON, local/share/remote files)
-- Delete tasks (supports deleting a list of task IDs)
-- List all tasks (supports basic CSV or JSON output)
-- Download JPEG frames (supports a list of frame IDs)
-- Dump annotations (supports all formats via format string)
-- Upload annotations for a task in the specified format (e.g. 'YOLO 1.1')
-- Export and download a whole task
-- Import a task
+- Projects:
+  - `create` - create a new project
+  - `delete` - delete projects
+  - `ls` - list all projects
+
+- Tasks:
+  - `create` - create a new task
+  - `create-from-backup` - create a task from a backup file
+  - `delete` - delete tasks
+  - `ls` - list all tasks
+  - `frames` - download frames from a task
+  - `export-dataset` - export a task as a dataset
+  - `import-dataset` - import annotations into a task from a dataset
+  - `backup` - back up a task
+  - `auto-annotate` - automatically annotate a task using a local function
 
 ## Installation
 
 `pip install cvat-cli`
 
 ## Usage
 
-```bash
-$ cvat-cli --help
-
-usage: cvat-cli [-h] [--version] [--auth USER:[PASS]]
-  [--server-host SERVER_HOST] [--server-port SERVER_PORT] [--debug]
-  {create,delete,ls,frames,dump,upload,export,import} ...
-
-Perform common operations related to CVAT tasks.
-
-positional arguments:
-  {create,delete,ls,frames,dump,upload,export,import}
-
-optional arguments:
-  -h, --help            show this help message and exit
-  --version             show program's version number and exit
-  --auth USER:[PASS]    defaults to the current user and supports the PASS
-                        environment variable or password prompt
-                        (default: current user)
-  --server-host SERVER_HOST
-                        host (default: localhost)
-  --server-port SERVER_PORT
-                        port (default: 8080)
-  --debug               show debug output
+The general form of a CLI command is:
+
+```console
+$ cvat-cli <common options> <resource> <action> <options>
+```
+
+where:
+
+- `<common options>` are options shared between all subcommands;
+- `<resource>` is a CVAT resource, such as `task`;
+- `<action>` is the action to do with the resource, such as `create`;
+- `<options>` is any options specific to a particular resource and action.
+
+You can list available subcommands and options using the `--help` option:
+
+```
+$ cvat-cli --help # get help on available common options and resources
+$ cvat-cli <resource> --help # get help on actions for the given resource
+$ cvat-cli <resource> <action> --help # get help on action-specific options
 ```
 
 ## Examples
 
 Create a task with local images:
 
 ```bash
-cvat-cli --auth user create
+cvat-cli --auth user task create
     --labels '[{"name": "car"}, {"name": "person"}]'
     "test_task"
     "local"
@@ -63,5 +66,5 @@ List tasks on a custom server with auth:
 ```bash
 cvat-cli --auth admin:password \
     --server-host cvat.my.server.com --server-port 30123 \
-    ls
+    task ls
 ```

cvat-cli/src/cvat_cli/__main__.py

@@ -10,7 +10,7 @@
 import urllib3.exceptions
 from cvat_sdk import exceptions
 
-from ._internal.commands import COMMANDS
+from ._internal.commands_all import COMMANDS
 from ._internal.common import build_client, configure_common_arguments, configure_logger
 from ._internal.utils import popattr
 

cvat-cli/src/cvat_cli/_internal/command_base.py

@@ -3,10 +3,15 @@
 # SPDX-License-Identifier: MIT
 
 import argparse
+import json
+import textwrap
 import types
-from collections.abc import Mapping
+from abc import ABCMeta, abstractmethod
+from collections.abc import Mapping, Sequence
 from typing import Callable, Protocol
 
+from cvat_sdk import Client
+
 
 class Command(Protocol):
     @property
@@ -51,3 +56,71 @@ def execute(self) -> None:
         # It should be impossible for a command group to be executed,
         # because configure_parser requires that a subcommand is specified.
         assert False, "unreachable code"
+
+
+class DeprecatedAlias:
+    def __init__(self, command: Command, replacement: str) -> None:
+        self._command = command
+        self._replacement = replacement
+
+    @property
+    def description(self) -> str:
+        return textwrap.dedent(
+            f"""\
+            {self._command.description}
+            (Deprecated; use "{self._replacement}" instead.)
+            """
+        )
+
+    def configure_parser(self, parser: argparse.ArgumentParser) -> None:
+        self._command.configure_parser(parser)
+
+    def execute(self, client: Client, **kwargs) -> None:
+        client.logger.warning('This command is deprecated. Use "%s" instead.', self._replacement)
+        self._command.execute(client, **kwargs)
+
+
+class GenericCommand(metaclass=ABCMeta):
+    @abstractmethod
+    def repo(self, client: Client): ...
+
+    @property
+    @abstractmethod
+    def resource_type_str(self) -> str: ...
+
+
+class GenericListCommand(GenericCommand):
+    @property
+    def description(self) -> str:
+        return f"List all CVAT {self.resource_type_str}s in either basic or JSON format."
+
+    def configure_parser(self, parser: argparse.ArgumentParser) -> None:
+        parser.add_argument(
+            "--json",
+            dest="use_json_output",
+            default=False,
+            action="store_true",
+            help="output JSON data",
+        )
+
+    def execute(self, client: Client, *, use_json_output: bool = False):
+        results = self.repo(client).list(return_json=use_json_output)
+        if use_json_output:
+            print(json.dumps(json.loads(results), indent=2))
+        else:
+            for r in results:
+                print(r.id)
+
+
+class GenericDeleteCommand(GenericCommand):
+    @property
+    def description(self):
+        return f"Delete a list of {self.resource_type_str}s, ignoring those which don't exist."
+
+    def configure_parser(self, parser: argparse.ArgumentParser) -> None:
+        parser.add_argument(
+            "ids", type=int, help=f"list of {self.resource_type_str} IDs", nargs="+"
+        )
+
+    def execute(self, client: Client, *, ids: Sequence[int]) -> None:
+        self.repo(client).remove_by_ids(ids)

cvat-cli/src/cvat_cli/_internal/commands_all.py

@@ -0,0 +1,27 @@
+# Copyright (C) 2024 CVAT.ai Corporation
+#
+# SPDX-License-Identifier: MIT
+
+from .command_base import CommandGroup, DeprecatedAlias
+from .commands_projects import COMMANDS as COMMANDS_PROJECTS
+from .commands_tasks import COMMANDS as COMMANDS_TASKS
+
+COMMANDS = CommandGroup(description="Perform operations on CVAT resources.")
+
+COMMANDS.add_command("project", COMMANDS_PROJECTS)
+COMMANDS.add_command("task", COMMANDS_TASKS)
+
+_legacy_mapping = {
+    "create": "create",
+    "ls": "ls",
+    "delete": "delete",
+    "frames": "frames",
+    "dump": "export-dataset",
+    "upload": "import-dataset",
+    "export": "backup",
+    "import": "create-from-backup",
+    "auto-annotate": "auto-annotate",
+}
+
+for _legacy, _new in _legacy_mapping.items():
+    COMMANDS.add_command(_legacy, DeprecatedAlias(COMMANDS_TASKS.commands[_new], f"task {_new}"))

cvat-cli/src/cvat_cli/_internal/commands_projects.py

@@ -0,0 +1,91 @@
+# Copyright (C) 2024 CVAT.ai Corporation
+#
+# SPDX-License-Identifier: MIT
+
+import argparse
+import textwrap
+
+from cvat_sdk import Client, models
+
+from .command_base import CommandGroup, GenericCommand, GenericDeleteCommand, GenericListCommand
+from .parsers import parse_label_arg
+
+COMMANDS = CommandGroup(description="Perform operations on CVAT projects.")
+
+
+class GenericProjectCommand(GenericCommand):
+    resource_type_str = "project"
+
+    def repo(self, client: Client):
+        return client.projects
+
+
+@COMMANDS.command_class("ls")
+class ProjectList(GenericListCommand, GenericProjectCommand):
+    pass
+
+
+@COMMANDS.command_class("create")
+class ProjectCreate:
+    description = textwrap.dedent(
+        """\
+        Create a new CVAT project, optionally importing a dataset.
+        """
+    )
+
+    def configure_parser(self, parser: argparse.ArgumentParser) -> None:
+        parser.add_argument("name", type=str, help="name of the project")
+        parser.add_argument(
+            "--bug_tracker", "--bug", default=argparse.SUPPRESS, type=str, help="bug tracker URL"
+        )
+        parser.add_argument(
+            "--labels",
+            default=[],
+            type=parse_label_arg,
+            help="string or file containing JSON labels specification (default: %(default)s)",
+        )
+        parser.add_argument(
+            "--dataset_path",
+            default="",
+            type=str,
+            help="path to the dataset file to import",
+        )
+        parser.add_argument(
+            "--dataset_format",
+            default="CVAT 1.1",
+            type=str,
+            help="format of the dataset file being uploaded"
+            " (only applies when --dataset_path is specified; default: %(default)s)",
+        )
+        parser.add_argument(
+            "--completion_verification_period",
+            dest="status_check_period",
+            default=2,
+            type=float,
+            help="period between status checks"
+            " (only applies when --dataset_path is specified; default: %(default)s)",
+        )
+
+    def execute(
+        self,
+        client: Client,
+        *,
+        name: str,
+        labels: dict,
+        dataset_path: str,
+        dataset_format: str,
+        status_check_period: int,
+        **kwargs,
+    ) -> None:
+        project = client.projects.create_from_dataset(
+            spec=models.ProjectWriteRequest(name=name, labels=labels, **kwargs),
+            dataset_path=dataset_path,
+            dataset_format=dataset_format,
+            status_check_period=status_check_period,
+        )
+        print(f"Created project ID {project.id}")
+
+
+@COMMANDS.command_class("delete")
+class ProjectDelete(GenericDeleteCommand, GenericProjectCommand):
+    pass