Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove reparse on DAG depending on asset alias #44866

Merged
merged 1 commit into from
Dec 16, 2024

Conversation

uranusjr
Copy link
Member

I suspect we no longer need this since alias resolution now happens very late in the scheduling process. A DAG should be able to automatically understand the alias is resolved without a reparse after refactorings introduced in cccc933.

Let’s see what CI thinks.

@Lee-W Lee-W self-requested a review December 12, 2024 09:07
I suspect we no longer need this since alias resolution now happens very
late in the scheduling process. A DAG *should* be able to automatically
understand the alias is resolved without a reparse after refactorings
introduced in cccc933.
@uranusjr uranusjr marked this pull request as ready for review December 12, 2024 12:00
@Lee-W
Copy link
Member

Lee-W commented Dec 13, 2024

@uranusjr would like to confirm whether it works on your local. I tried with commit 7bdf1877 (there's task_sdk issue currently I think) but it seems to break the scheduler

@uranusjr
Copy link
Member Author

Still some problems. Don’t merge this yet.

@uranusjr uranusjr marked this pull request as draft December 13, 2024 04:01
@uranusjr
Copy link
Member Author

I can’t come up with a good way to test this (our test setup does not lend well toward real scheduler isolation). I’m going to just cross fingers and merge this if @Lee-W does not reject.

@uranusjr uranusjr marked this pull request as ready for review December 16, 2024 07:17
@uranusjr uranusjr merged commit aaf29ee into apache:main Dec 16, 2024
66 checks passed
@uranusjr uranusjr deleted the try-no-reparse branch December 16, 2024 07:20
LefterisXefteris pushed a commit to LefterisXefteris/airflow that referenced this pull request Jan 5, 2025
I suspect we no longer need this since alias resolution now happens very
late in the scheduling process. A DAG *should* be able to automatically
understand the alias is resolved without a reparse after refactorings
introduced in cccc933.
got686-yandex pushed a commit to got686-yandex/airflow that referenced this pull request Jan 30, 2025
I suspect we no longer need this since alias resolution now happens very
late in the scheduling process. A DAG *should* be able to automatically
understand the alias is resolved without a reparse after refactorings
introduced in cccc933.
jedcunningham added a commit to astronomer/airflow that referenced this pull request Mar 13, 2025
This functionality was removed in apache#44866, but the methods weren't
removed. The reparse functionality is being refactored, so removing them
is easier than repairing them.
kaxil pushed a commit that referenced this pull request Mar 13, 2025
This functionality was removed in #44866, but the methods weren't
removed. The reparse functionality is being refactored, so removing them
is easier than repairing them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants