Convert remaining providers to the new structure #46045
Comments
|
I'll work on Zendesk, ydb |
|
I'm working on Trino, Apprise |
|
Lets please make sure we merge open PRs for the providers (if we can) before the conversion. |
|
PR for Weaviate. |
Yeah before attempting to convert a particular provider It would be great to check if there are outsanding PRs. I am going through the PRs from the last few weeks to see if they apply - but if anyone will want to take on a provider - just checking if there are no related PRs before will do the job :) |
|
Three to go:
|
|
Two to go ! |
|
I 'll work on Amazon |
|
@o-nikolas already works on Amazon |
|
For anyone coming in new, |
|
@potiuk I didn't think it would actually let me do that, but I edited your last comment with the names of the person working on the last two so people don't keep pinging you that they can do them. |
…apache#46139) As part of apache#46045 we are moving all providers to the new structure, each with it's own pyproject.toml. However we noticed that constraint generation is somewhat broken during the move: * providers moved to the new structure disappeared from pypi constraints * http provider in SOME PRs started to appear in "source constraints" Both should not happen: * pypi constraints should contain all providers (latest versions in PyPI) * source providers should contain no providers - only their dependencies There are two reasons for those two issues, and this PR fixes both: 1) The PyPI providers were removed because `uv pip install .` without editable flag is currently - pretty unexpectedly - installing local workspaced providers rather than install them from PyPI and there are no controls to disable it. Workaround for that is that during constraint generation we remove temporarily all providers from workspace definition in the pyproject.toml - this way providers are installed from PyPI. Feature request to handle it has been raised in the astral-sh/uv#10991 2) The http provider was added to the constraint in PRs where the http provider is a required dependency (for example in discord provider). Since the http provider is not yet migrated, discord installation in workspace will pull it from PyPI rather than from the local workspace. This has been fixed by simply migrating the http provider to the new structure,
cool |
|
Just Amazon left (and almost there: @o-nikolas ) :) |
|
DONE ! 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 |
|
Fantastic! Now the cleanup can start! |
Oh. Absolutely :) |
Fixes: apache#46045 Co-authored-by: Niko Oliveira <onikolas@amazon.com>
|
Awesome! Great work everyone! 😃 |
|
Wooww!! Thanks everyone! It was a great team effort. 😄
…On Mon, 10 Feb 2025 at 23:10, Niko Oliveira ***@***.***> wrote:
Awesome! Great work everyone! 😃
—
Reply to this email directly, view it on GitHub
<#46045 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ALPQ3RRQ5A4WIMMCYBZVUKL2PDQBPAVCNFSM6AAAAABV3Y7SMWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNBYG44DCOJWGQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Great work! Amazing team effort! This kind of big chunk of work always feels better 😄 Thanks all! |
We have now a number of providers converted to the new structure. We have almost 90 of them to be converted. And we have automated scripts that do bulk of the work. This was part of #44511
Process of the conversion of each of the provider is easy - and described in https://github.com/apache/airflow/blob/main/dev/moving_providers/README.md
The devlist announcement: https://lists.apache.org/thread/dzbj5yx5kwpbwyr5yscp4wnlsp6p9v8l
The list below keeps track of the conversion. The list is generated via:
The text was updated successfully, but these errors were encountered: