Failed to resolve requirements for tensorflow-macos==2.13.0

[zuzana-valesova-emplifi]

Describe the bug
I have tensorflow==2.13 in my python dependencies. I generated a lockfile succesfully and when I try to do pants export to create a virtual environment I am getting following error:

Failed to find a compatible PEX_PYTHON=/Users/zuzanajelinkova/.pyenv/versions/3.9.15/bin/python3.9.

Examined the following interpreters:
1.) /Users/zuzanajelinkova/.pyenv/versions/3.9.15/bin/python3.9 CPython==3.9.15

No interpreter compatible with the requested constraints was found:

  Failed to resolve requirements from PEX environment @ /Users/zuzanajelinkova/.cache/pants/named_caches/pex_root/unzipped_pexes/72e2ad14362780be5563a6042d3fffc91f1ee1c5.
  Needed cp39-cp39-macosx_13_0_arm64 compatible dependencies for:
   1: tensorflow-macos==2.13.0; platform_system == "Darwin" and platform_machine == "arm64"
      Required by:
        tensorflow 2.13.0
      But this pex had no ProjectName(raw='tensorflow-macos', normalized='tensorflow-macos') distributions.

Pants version
2.17.0

OS
MacOS , Ventura 13.1

Additional info

• Installing tensorflow==2.13 using pip is without issue

The only similar issue I've found here is #18936 . If I understand correctly it was an issue of metadata inconsistency of pytorch library.

Thanks in advance for your help!

[jsirois]

Looks similar to me @jelinkovaz. I downloaded the 4 CPython 3.11 wheels for the 2.13.0 release and find:

$ find . -name "tensorflow*.whl" | while read whl; do echo "Wheel: $whl" && unzip -qc $whl tensorflow-2.13.0.dist-info/METADATA | grep "Requires-" && echo; done
Wheel: ./tensorflow-2.13.0-cp311-cp311-win_amd64.whl
Requires-Python: >=3.8
Requires-Dist: tensorflow-macos (==2.13.0) ; platform_system == "Darwin" and platform_machine == "arm64"
Requires-Dist: tensorflow-cpu-aws (==2.13.0) ; platform_system == "Linux" and (platform_machine == "arm64" or platform_machine == "aarch64")
Requires-Dist: tensorflow-intel (==2.13.0) ; platform_system == "Windows"

Wheel: ./tensorflow-2.13.0-cp311-cp311-macosx_12_0_arm64.whl
Requires-Python: >=3.8
Requires-Dist: tensorflow-macos (==2.13.0) ; platform_system == "Darwin" and platform_machine == "arm64"
Requires-Dist: tensorflow-cpu-aws (==2.13.0) ; platform_system == "Linux" and (platform_machine == "arm64" or platform_machine == "aarch64")
Requires-Dist: tensorflow-intel (==2.13.0) ; platform_system == "Windows"

Wheel: ./tensorflow-2.13.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Requires-Python: >=3.8
Requires-Dist: absl-py (>=1.0.0)
Requires-Dist: astunparse (>=1.6.0)
Requires-Dist: flatbuffers (>=23.1.21)
Requires-Dist: gast (<=0.4.0,>=0.2.1)
Requires-Dist: google-pasta (>=0.1.1)
Requires-Dist: grpcio (<2.0,>=1.24.3)
Requires-Dist: h5py (>=2.9.0)
Requires-Dist: keras (<2.14,>=2.13.1)
Requires-Dist: libclang (>=13.0.0)
Requires-Dist: numpy (<=1.24.3,>=1.22)
Requires-Dist: opt-einsum (>=2.3.2)
Requires-Dist: packaging
Requires-Dist: protobuf (!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3)
Requires-Dist: setuptools
Requires-Dist: six (>=1.12.0)
Requires-Dist: tensorboard (<2.14,>=2.13)
Requires-Dist: tensorflow-estimator (<2.14,>=2.13.0)
Requires-Dist: termcolor (>=1.1.0)
Requires-Dist: typing-extensions (<4.6.0,>=3.6.6)
Requires-Dist: wrapt (>=1.11.0)
Requires-Dist: tensorflow-io-gcs-filesystem (>=0.23.1) ; platform_machine != "arm64" or platform_system != "Darwin"
Requires-Dist: tensorflow-cpu-aws (==2.13.0) ; platform_system == "Linux" and (platform_machine == "arm64" or platform_machine == "aarch64")
Requires-Dist: tensorflow-intel (==2.13.0) ; platform_system == "Windows"

Wheel: ./tensorflow-2.13.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Requires-Python: >=3.8
Requires-Dist: tensorflow-macos (==2.13.0) ; platform_system == "Darwin" and platform_machine == "arm64"
Requires-Dist: tensorflow-cpu-aws (==2.13.0) ; platform_system == "Linux" and (platform_machine == "arm64" or platform_machine == "aarch64")
Requires-Dist: tensorflow-intel (==2.13.0) ; platform_system == "Windows"

IOW, just like pytorch, tensorflow suffers from inconsistent METADATA across different distributions of the same release.

I hate to close this since there is no nice solution, but hopefully that answers your question. Please speak up either way.

[jsirois]

@jelinkovaz I'll close this issue as an answered question by end of day. If that's inappropriate and you have more questions, please speak up.

[zuzana-valesova-emplifi]

@jsirois Thank you very much for the reply! I've opened the issue to add the metadata in the tensorflow project tensorflow/tensorflow#62346

[jsirois]

Ok, great. Thank you. I commented on that issue to spell the situation out a bit more. I'm not sure how compelling they'll view the use-case as, but worth a try.