Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modify DagBag/Parsing loop to be DAG bundle backend aware #42289

Open
1 task done
jedcunningham opened this issue Sep 18, 2024 · 0 comments
Open
1 task done

Modify DagBag/Parsing loop to be DAG bundle backend aware #42289

jedcunningham opened this issue Sep 18, 2024 · 0 comments
Assignees
@jedcunningham
Labels
AIP-66: DAG Bundle/Manifest area:dag-parsing kind:meta High-level information important to the community

Comments

@jedcunningham
Copy link
Member

Body

The DagBag and parsing loop, in particular when configured not to use the DB, needs to be backend bundle aware. TBD what this really means though.

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.
@jedcunningham jedcunningham added the kind:meta High-level information important to the community label Sep 18, 2024
@jedcunningham jedcunningham self-assigned this Dec 2, 2024
@jedcunningham jedcunningham mentioned this issue Jan 2, 2025
Open
1 task
jedcunningham added a commit to astronomer/airflow that referenced this issue Jan 2, 2025
@jedcunningham
Temporarily remove support for dags reserialize command
7567f56 
As we work toward getting bundle parsing support in main (apache#42289), it'll be
easier for reviewers to review the new `dags reserialize` command
separate from the rest of the parsing changes (that PR will be rather
large already). But in the meantime, simply using DagBag on a single
directory won't continue working.

Tracking issue to add it back: apache#45336
@jedcunningham jedcunningham mentioned this issue Jan 2, 2025
Merged
potiuk pushed a commit that referenced this issue Jan 2, 2025
@jedcunningham
Temporarily remove support for dags reserialize command (#45337)
a0e5861 
As we work toward getting bundle parsing support in main (#42289), it'll be
easier for reviewers to review the new `dags reserialize` command
separate from the rest of the parsing changes (that PR will be rather
large already). But in the meantime, simply using DagBag on a single
directory won't continue working.

Tracking issue to add it back: #45336
LefterisXefteris pushed a commit to LefterisXefteris/airflow that referenced this issue Jan 5, 2025
@jedcunningham
Temporarily remove support for dags reserialize command (apache#45337)
e60bfa2 
As we work toward getting bundle parsing support in main (apache#42289), it'll be
easier for reviewers to review the new `dags reserialize` command
separate from the rest of the parsing changes (that PR will be rather
large already). But in the meantime, simply using DagBag on a single
directory won't continue working.

Tracking issue to add it back: apache#45336
agupta01 pushed a commit to agupta01/airflow that referenced this issue Jan 6, 2025
@jedcunningham @agupta01
Temporarily remove support for dags reserialize command (apache#45337)
bdf991e 
As we work toward getting bundle parsing support in main (apache#42289), it'll be
easier for reviewers to review the new `dags reserialize` command
separate from the rest of the parsing changes (that PR will be rather
large already). But in the meantime, simply using DagBag on a single
directory won't continue working.

Tracking issue to add it back: apache#45336
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jedcunningham jedcunningham

Labels
AIP-66: DAG Bundle/Manifest area:dag-parsing kind:meta High-level information important to the community
Projects
AIP-66: DAG Bundles & Parsing
Status: In Progress
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jedcunningham