Kedro modelify does not accept --pip-requirements

[Lasica]

Description

The modelify command does not accept env context.

Context

I was packaging a kedro pipeline for serving with modelify and I did not provide any context info (--conda-env or --pip-requirements options), and it generated it with some (defaults?) incorrect conda-env specifications. I run it locally with Pipfile and local python virtualenv.

Steps to Reproduce

Call modelify on inference pipeline and try to specify requirements.

Example command:
kedro mlflow modelify --pipeline=inference --input-name=raw_data --infer-signature --infer-input-example --pip-requirements ./src/requirements.txt

Expected Result

Pass the requirements to mlflow.pyfunc.log_model and log model.

Actual Result

Exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/user/projects/monitoring/housing-with-evidently/.venv/bin/kedro:8 in <module>          │
│                                                                                                  │
│ /home/user/projects/monitoring/housing-with-evidently/.venv/lib/python3.10/site-packages/ked │
│ ro/framework/cli/cli.py:211 in main                                                              │
│                                                                                                  │
│   208 │   """                                                                                    │
│   209 │   _init_plugins()                                                                        │
│   210 │   cli_collection = KedroCLI(project_path=Path.cwd())                                     │
│ ❱ 211 │   cli_collection()                                                                       │
│   212                                                                                            │
│                                                                                                  │
│ /home/user/projects/monitoring/housing-with-evidently/.venv/lib/python3.10/site-packages/cli │
│ ck/core.py:1130 in __call__                                                                      │
│                                                                                                  │
│ /home/user/projects/monitoring/housing-with-evidently/.venv/lib/python3.10/site-packages/ked │
│ ro/framework/cli/cli.py:139 in main                                                              │
│                                                                                                  │
│   136 │   │   )                                                                                  │
│   137 │   │                                                                                      │
│   138 │   │   try:                                                                               │
│ ❱ 139 │   │   │   super().main(                                                                  │
│   140 │   │   │   │   args=args,                                                                 │
│   141 │   │   │   │   prog_name=prog_name,                                                       │
│   142 │   │   │   │   complete_var=complete_var,                                                 │
│                                                                                                  │
│ /home/user/projects/monitoring/housing-with-evidently/.venv/lib/python3.10/site-packages/cli │
│ ck/core.py:1055 in main                                                                          │
│                                                                                                  │
│ /home/user/projects/monitoring/housing-with-evidently/.venv/lib/python3.10/site-packages/cli │
│ ck/core.py:1657 in invoke                                                                        │
│                                                                                                  │
│ /home/user/projects/monitoring/housing-with-evidently/.venv/lib/python3.10/site-packages/cli │
│ ck/core.py:1657 in invoke                                                                        │
│                                                                                                  │
│ /home/user/projects/monitoring/housing-with-evidently/.venv/lib/python3.10/site-packages/cli │
│ ck/core.py:1404 in invoke                                                                        │
│                                                                                                  │
│ /home/user/projects/monitoring/housing-with-evidently/.venv/lib/python3.10/site-packages/cli │
│ ck/core.py:760 in invoke                                                                         │
│                                                                                                  │
│ /home/user/projects/monitoring/housing-with-evidently/.venv/lib/python3.10/site-packages/ked │
│ ro_mlflow/framework/cli/cli.py:365 in modelify                                                   │
│                                                                                                  │
│   362 │   │   │   │   log_model_kwargs["extra_pip_requirements"] = extra_pip_requirements        │
│   363 │   │   │                                                                                  │
│   364 │   │   │   with mlflow.start_run(run_id=run_id):                                          │
│ ❱ 365 │   │   │   │   mlflow.pyfunc.log_model(**log_model_kwargs)                                │
│   366 │   │   │   │   run_id = mlflow.active_run().info.run_id                                   │
│   367 │   │   │   │   LOGGER.info(f"Model successfully logged in run '{run_id}'")                │
│   368                                                                                            │
│                                                                                                  │
│ /home/user/projects/monitoring/housing-with-evidently/.venv/lib/python3.10/site-packages/mlf │
│ low/pyfunc/__init__.py:1459 in log_model                                                         │
│                                                                                                  │
│   1456 │   :return: A :py:class:`ModelInfo <mlflow.models.model.ModelInfo>` instance that conta  │
│   1457 │   │   │    metadata of the logged model.                                                │
│   1458 │   """                                                                                   │
│ ❱ 1459 │   return Model.log(                                                                     │
│   1460 │   │   artifact_path=artifact_path,                                                      │
│   1461 │   │   flavor=mlflow.pyfunc,                                                             │
│   1462 │   │   loader_module=loader_module,                                                      │
│                                                                                                  │
│ /home/user/projects/monitoring/housing-with-evidently/.venv/lib/python3.10/site-packages/mlf │
│ low/models/model.py:373 in log                                                                   │
│                                                                                                  │
│   370 │   │   │   local_path = tmp.path("model")                                                 │
│   371 │   │   │   run_id = mlflow.tracking.fluent._get_or_start_run().info.run_id                │
│   372 │   │   │   mlflow_model = cls(artifact_path=artifact_path, run_id=run_id)                 │
│ ❱ 373 │   │   │   flavor.save_model(path=local_path, mlflow_model=mlflow_model, **kwargs)        │
│   374 │   │   │   mlflow.tracking.fluent.log_artifacts(local_path, mlflow_model.artifact_path)   │
│   375 │   │   │   try:                                                                           │
│   376 │   │   │   │   mlflow.tracking.fluent._record_logged_model(mlflow_model)                  │
│                                                                                                  │
│ /home/user/projects/monitoring/housing-with-evidently/.venv/lib/python3.10/site-packages/mlf │
│ low/pyfunc/__init__.py:1284 in save_model                                                        │
│                                                                                                  │
│   1281 │   :param pip_requirements: {{ pip_requirements }}                                       │
│   1282 │   :param extra_pip_requirements: {{ extra_pip_requirements }}                           │
│   1283 │   """                                                                                   │
│ ❱ 1284 │   _validate_env_arguments(conda_env, pip_requirements, extra_pip_requirements)          │
│   1285 │                                                                                         │
│   1286 │   mlflow_model = kwargs.pop("model", mlflow_model)                                      │
│   1287 │   if len(kwargs) > 0:                                                                   │
│                                                                                                  │
│ /home/user/projects/monitoring/housing-with-evidently/.venv/lib/python3.10/site-packages/mlf │
│ low/utils/environment.py:399 in _validate_env_arguments                                          │
│                                                                                                  │
│   396 │   ]                                                                                      │
│   397 │   specified = [arg for arg in args if arg is not None]                                   │
│   398 │   if len(specified) > 1:                                                                 │
│ ❱ 399 │   │   raise ValueError(                                                                  │
│   400 │   │   │   "Only one of `conda_env`, `pip_requirements`, and "                            │
│   401 │   │   │   "`extra_pip_requirements` can be specified"                                    │
│   402 │   │   )                                                                                  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: Only one of `conda_env`, `pip_requirements`, and `extra_pip_requirements` can be specified

Your Environment

Include as many relevant details about the environment in which you experienced the bug:

• kedro and kedro-mlflow version used (pip show kedro and pip show kedro-mlflow): 0.18.4, 0.11.7
• Python version used (python -V): 3.10.9
• Operating system and version: archlinux 64 bit

Does the bug also happen with the last version on master?

Yes.

[Galileo-Galilei]

Thank you for the bug report. It appears that they are several problems:

• if no conda_env is passed, there is an attempt to create one with obvious wrong default:

  kedro-mlflow/kedro_mlflow/framework/cli/cli.py

  Lines 346 to 347 in 4888b0e

  	if conda_env is None:
  	conda_env = {"python": "3.7.0", "dependencies": ["kedro==0.16.5"]}

• even if pip requirements are passed, this conda environment is created and this raises a conflict.

I'll try to fix this soon, but feel free to open a PR !

[Lasica]

Thanks for the quick response, happy to help