support dag level db cleanup on airflow cli

[davidkl97]

Description

While full db cleanup is useful to run in rare occasions, deleting data regarding to specific dag ids provides modular maintenance method to clean specific "intensive" dags and provide clear context and responsivness in the web ui and interfaces in cluster.

an initial idea was gathered to add optional dag_id key in _TableConfig at db_cleanup.py and append condition in _build_query(after adding the argument of course). Those changes could be provided also in the helm chart as list of cleanup jobs.

would love to hear you inputs about the use case and the implementation relating to the current effort(e.g #24590).
Thanks.

Use case/motivation

• cleanup of specific "intensive" dags.
• modular and configurable context and metadata in the web server and the different interfaces.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[potiuk]

Feel free to implement it. the #24590 is going to be removed - we had recently a discussion in #24591 that led us to rethink if and how we are going to implement click, so there is no impact from that on your work.

It would be a nice feature to add filter by dag-id

[potiuk]

Assigned you

[jkbngl]

Any update on this one? We would also appreciate this feature.

[potiuk]

Any update on this one? We would also appreciate this feature.

FYI. This issue is > 2 years old if you have not noticed, so it's very likely there are no updates. Contributions are always welcome - and actually contributing something is the fastest way to have something implemented.

So if your "any updates on this ?" means - "I would like to contribute that", then it's cool. Otherwise it does not move a needle - someone has to commit to it and implement it.

Shall I assign you maybe? Or maybe you can find someone who will commit and implement it @jkbngl ?