Rescue if a DagRun's DAG was removed from db

[uranusjr]

Fix #17442.

The exception happens when a DAG is removed from the database (via web UI or something else), but there are still unfinished runs associated to it. This catches the scenario and use the existing fallback setting max_active_runs to zero.

[github-actions]

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

[jedcunningham]

Should we add a test for this as well?

[uranusjr]

Good point, I'll add one (and fix the typo in the commit message 🤦‍♂️)

[uranusjr]

Writing a proper test turns out to be a long journey figuring out a right combination to trigger this bug. So to trigger this bug, a DAG should not be in the DagBag, but in the DagModel table, but not in the SerializedDagModel table.

@georborodin Does the DAG you deleted from the web UI have subdags? This is the only situation I’ve managed to set up a proper environment for the test. (Although I wouldn’t be surprised if there’s more I couldn’t locate.)

[uranusjr]

For future reference, the complete steps to reproduce this from the web UI is:

1. Create a DAG with a subDAG that runs for a long-ish time. Wait for them to show up in the web UI.
2. Run the DAG (thus trigging the subDAG).
3. delete the parent DAG from the web UI. (This does not delete the subDAG.)
4. Try to block the subDAG’s unfinished run from step 2. (Should fail with SerializedDagNotFound before this patch.)