[BUG] Tests breaking due to a error in protobuf dependency

[miguelgfierro]

Description

Protobuf made a new release https://pypi.org/project/protobuf/#history
on Nov 27, 2024. It's breaking the tests.

    File "/home/runner/work/recommenders/recommenders/tests/ci/azureml_tests/post_pytest.py", line 75, in <module>
      runs = mlflow.search_runs(
    File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/mlflow/tracking/fluent.py", line 2069, in search_runs
      experiment_by_name = get_experiment_by_name(n)
    File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/mlflow/tracking/fluent.py", line 1671, in get_experiment_by_name
      return MlflowClient().get_experiment_by_name(name)
    File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/mlflow/tracking/client.py", line 134, in __init__
      self._tracking_client = TrackingServiceClient(final_tracking_uri)
    File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py", line 83, in __init__
      self.store
    File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py", line 87, in store
      return utils._get_store(self.tracking_uri)
    File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/utils.py", line 208, in _get_store
      return _tracking_store_registry.get_store(store_uri, artifact_uri)
    File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/registry.py", line 45, in get_store
      return self._get_store_with_resolved_uri(resolved_store_uri, artifact_uri)
    File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/registry.py", line 56, in _get_store_with_resolved_uri
      return builder(store_uri=resolved_store_uri, artifact_uri=artifact_uri)
    File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/azureml/mlflow/entry_point_loaders.py", line 33, in azureml_store_builder
      from azureml.mlflow._store.tracking.store import AzureMLRestStore
    File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/azureml/mlflow/_store/tracking/store.py", line 17, in <module>
      from azureml.mlflow._protos.aml_service_pb2 import (
    File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/azureml/mlflow/_protos/aml_service_pb2.py", line 10, in <module>
      from google.protobuf import service as _service
    File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/google/protobuf/service.py", line 78
      raise NotImplementedError
                               ^
  IndentationError: unindent does not match any outer indentation level
  Error: Process completed with exit code 1.

In which platform does it happen?

How do we replicate the issue?

https://github.com/recommenders-team/recommenders/actions/runs/12100479486/job/33739131547

Expected behavior (i.e. solution)

I expect Google to test their libraries before making a release.

Willingness to contribute

• Yes, I can contribute for this issue independently.
• Yes, I can contribute for this issue with guidance from Recommenders community.
• No, I cannot contribute at this time.

Other Comments

[miguelgfierro]

Confirmed other people got the same error: protocolbuffers/protobuf#19430

[miguelgfierro]

Update: @imatiach-msft and I talked with the AzureML dev team and they are going to remove the code that caused the error. Not sure how long they are going to take. In the meantime, if we want a quick fix would be to restrict protobuf<5.29.0.

FYI @SimonYansenZhao @anargyri

[miguelgfierro]

Protobuf 5.29.0 was finally yanked: https://pypi.org/project/protobuf/5.29.0/

[miguelgfierro]

The yank of the version fixed the problem. See https://github.com/recommenders-team/recommenders/actions/runs/12168238375

The AzureML team is expected to update the deprecated dependency.
Closing this.