loosening google-cloud-* package constraints for TFX 1.13

[jeongukjae]

• Have I specified the code to reproduce the issue (Yes, No):
• Environment in which the code is executed (e.g., Local(Linux/MacOS/Windows),
  Interactive Notebook, Google Cloud, etc):
• TensorFlow version: 2.12
• TFX Version: 1.13
• Python version: 3.9
• Python dependencies (from pip freeze output):

Describe the current behavior

https://github.com/tensorflow/tensorflow/blob/r2.12/tensorflow/tools/pip_package/setup.py#L100-L107

TF 2.12 can accept protobuf>=3.20.3,

but the TFX comment says

https://github.com/tensorflow/tfx/blob/r1.13.0/tfx/dependencies.py#L83-L90

So, it seems reasonable to loosen google-cloud-* and google-api-core package constraints to use more updated versions.

[singhniraj08]

@jeongukjae,

Can you try to update TFX to 1.14.0 release? google-cloud-* package constraints have been updated to support updated package versions. Thank you!

[jeongukjae]

@singhniraj08 Sadly, I'm using TFX with Vertex AI, so cannot update it. 1.13 release is the latest TFX version that Vertex AI supports.

https://cloud.google.com/vertex-ai/docs/supported-frameworks-list#supported-tfx-sdk

[singhniraj08]

@jeongukjae,

Updating google-cloud-* package constraints were planned to be done after TF 2.12 release but not sure why it couldn't make to TFX 1.13 release. I tried to checkout the 1.13 release in my local and bumped up the google-cloud-* package constraints but could install it because of dependency errors.

I guess you have to wait till TFX 1.14 support is introduced in Vertex AI to use the updated google-cloud packages. Or can you raise a feature request for TFX 1.14 SDK on Vertex AI side? Meanwhile I will try to check internally, if this is something which can be done on our end.

[Writtic]

@jeongukjae @singhniraj08 Hi, I raised a feature request with GCP support case for below context.

As you can see in r1.13.0/tfx/dependencies.py#L89, the google-cloud-aiplatform>=1.6.2,<1.18 constraint makes it difficult to utilize the latest features of aiplatform in TFX 1.13.0.

In TFX 1.14.0, this constraint is loosened to google-cloud-aiplatform>=1.6.2,<2, but currently Vertex AI supports TFX versions up to 1.13.0.

I found this #6375 issue. As mentioned in the issue, there are conflicting version requirements for TFX and TF, so I think it can be solved together if TFX 1.14.0 is supported.

[jeongukjae]

@singhniraj08 It looks like it's possible to loosen them, and I tested it. Successfully installed them, so I wrote the PR for this issue: #6384

Can you review it and consider the TFX 1.13.1 release with that?

[singhniraj08]

@jeongukjae, Let me check internally if this can make a TFX 1.13.1 release. Thank you for the contributions.

[lego0901]

Sorry for my late response. But looks like the docsite itself is just outdated. Does the VAI genuinely not support TFX 1.14.0? I tried the tutorial, and looks like it works with the latest version TFX v1.14.0.

I appreciate if you confirm from your side again. If it is okay, then I will update the doc accordingly.

Thanks for your consideration.

[Writtic]

@lego0901 I asked it to GCP Support Team. I heard they are going to support 1.14.0 officially soon and internal test is ongoing for now.

[lego0901]

Sorry for my too-late response.. Just updating the branch does not impact to the SDK release or docker images, so I just recompiled and reuploaded the newly-built docker image (from v1.14.0) to the dockerhub with "latest" tag. (i.e., tensorflow/tfx:latest) Thank you.

[github-actions]

Are you satisfied with the resolution of your issue?
Yes
No