Upgrade to Pydantic 2 #33956
Merged
Upgrade to Pydantic 2 #33956
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Checking if a) it will bring pydantic 2 b) whether it is really needed for the AWS tests. |
70f4fe8 to
6e25123
Compare
9cd16da to
72af244
Compare
mhenc
approved these changes
Aug 31, 2023
|
Seems that we do not need cnf-lint - just openapi_schema_validator and jsonschema ... And we can upgrade to Pydantic 2 NOW. It also needed to fix our test case to replace removed |
ephraimbuddy
approved these changes
Aug 31, 2023
72af244 to
c189fb9
Compare
vincbeck
approved these changes
Aug 31, 2023
The only blocking factor to migrate to Pydantic 2 was the aws-sam-translator which was transitive dependency to `moto[cloudformation]` via `cfn-lint` and we do not really need everything in that extra - used only for testing. While aws-sam-translator is already preparing to release Pydantic 2 compatible version, we do not want to wait - instead we replace the cloudformation extra with openapi_spec_validator and jsonschema needed by the cloudformation tests.
c189fb9 to
ad07b96
Compare
|
I had to also fix Pydantic serialization detection |
potiuk
added a commit
to potiuk/airflow
that referenced
this pull request
Aug 31, 2023
With latest change enabling Pydantic apache#33956 some old dependencies (aws-sam-translator) remained in the CI image from cached installation and they are breaking `pip check` when refreshing the image cache. This PR bumps EPOCH numbers so that the dependencies are not installed from cache and we have a clean, new image with just those depencies we need.
potiuk
added a commit
that referenced
this pull request
Aug 31, 2023
With latest change enabling Pydantic #33956 some old dependencies (aws-sam-translator) remained in the CI image from cached installation and they are breaking `pip check` when refreshing the image cache. This PR bumps EPOCH numbers so that the dependencies are not installed from cache and we have a clean, new image with just those depencies we need.
ephraimbuddy
pushed a commit
that referenced
this pull request
Sep 1, 2023
With latest change enabling Pydantic #33956 some old dependencies (aws-sam-translator) remained in the CI image from cached installation and they are breaking `pip check` when refreshing the image cache. This PR bumps EPOCH numbers so that the dependencies are not installed from cache and we have a clean, new image with just those depencies we need. (cherry picked from commit dd7cc87)
ephraimbuddy
pushed a commit
that referenced
this pull request
Sep 1, 2023
The only blocking factor to migrate to Pydantic 2 was the aws-sam-translator which was transitive dependency to `moto[cloudformation]` via `cfn-lint` and we do not really need everything in that extra - used only for testing. While aws-sam-translator is already preparing to release Pydantic 2 compatible version, we do not want to wait - instead we replace the cloudformation extra with openapi_spec_validator and jsonschema needed by the cloudformation tests. (cherry picked from commit 1cda0c3)
97 tasks
potiuk
added a commit
to potiuk/airflow
that referenced
this pull request
Sep 4, 2023
The apache#33956 and apache#33998 updated limits for Pydantic to Pydantic 2 only and removed Pydantic 1 compatibility. However it turns out that some of important 3rd-party libraries have not yet upgraded and it will make it impossible to install them on Airflow 2.7.1+ if we keep the limit. For now we bring back Pydantic 1 compatibility, we remove the limit and filter the warnings that made us remove the compatibility code.
potiuk
added a commit
that referenced
this pull request
Sep 4, 2023
The #33956 and #33998 updated limits for Pydantic to Pydantic 2 only and removed Pydantic 1 compatibility. However it turns out that some of important 3rd-party libraries have not yet upgraded and it will make it impossible to install them on Airflow 2.7.1+ if we keep the limit. For now we bring back Pydantic 1 compatibility, we remove the limit and filter the warnings that made us remove the compatibility code.
ephraimbuddy
pushed a commit
that referenced
this pull request
Sep 4, 2023
The #33956 and #33998 updated limits for Pydantic to Pydantic 2 only and removed Pydantic 1 compatibility. However it turns out that some of important 3rd-party libraries have not yet upgraded and it will make it impossible to install them on Airflow 2.7.1+ if we keep the limit. For now we bring back Pydantic 1 compatibility, we remove the limit and filter the warnings that made us remove the compatibility code. (cherry picked from commit 30ddfc5)
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The only blocking factor to migrate to Pydantic 2 was the
aws-sam-translator which was transitive dependency to
moto[cloudformation]viacfn-lintand we do not really needeverything in that extra - used only for testing.
While aws-sam-translator is already preparing to release Pydantic 2
compatible version, we do not want to wait - instead we replace the
cloudformation extra with openapi_spec_validator and jsonschema
needed by the cloudformation tests.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.