Add support to TestBehavior.BUILD

[tatiana]

By default, Cosmos uses TestBehavior.AFTER_EACH, creating an Airflow TaskGroup that contains two tasks:

• one to run the model, seed or snapshot
• another to run the tests related to that dbt resource

While many users desire and expect this behaviour, it can also mean additional overhead, especially in dbt projects with more than 500 models. Each time the dbt command is executed, there is an overhead, even when using optimisations such as partial parsing and dbtRunner. There is also an overhead on splitting a task into multiple Airflow workers.

Illustrating some numbers with data shared by an Astronomer customer regarding the dbt command execution (between the logs "running dbt with arguments" and "Done."):

• Running dbt build for a particular model + its tests: 46s
• Running dbt run + dbt test individually: 2min15s

This PR introduces a new behaviour, TestBehavior.BUILD, where Cosmos can run both the model/seed/snapshot and the associated tests using a single command (dbt build). For documentation on the dbt build, check https://docs.getdbt.com/reference/commands/build.

This is an example of how the DAG will render when using this test behaviour when running:

 airflow dags test example_cosmos_dbt_build 

And this is an example of the output, showing both the model is being run and also the tests, using the build command:

[2024-12-10 15:19:23,667] {local.py:405} INFO - Trying to run dbtRunner with:
 ['build', '--models', 'customers', '--full-refresh', '--project-dir', '/var/folders/td/522y78v91d1f5wgh67mj3p0m0000gn/T/tmpghz8naek', '--profiles-dir', '/tmp/profile/ac4e9cde9bc05d574c157e795dcbcc6b60246a73ca1d92d4fc669e90a1e494e0', '--profile', 'default', '--target', 'dev']
 in /var/folders/td/522y78v91d1f5wgh67mj3p0m0000gn/T/tmpghz8naek
[2024-12-10T15:19:23.667+0000] {local.py:405} INFO - Trying to run dbtRunner with:
 ['build', '--models', 'customers', '--full-refresh', '--project-dir', '/var/folders/td/522y78v91d1f5wgh67mj3p0m0000gn/T/tmpghz8naek', '--profiles-dir', '/tmp/profile/ac4e9cde9bc05d574c157e795dcbcc6b60246a73ca1d92d4fc669e90a1e494e0', '--profile', 'default', '--target', 'dev']
 in /var/folders/td/522y78v91d1f5wgh67mj3p0m0000gn/T/tmpghz8naek
15:19:23  Running with dbt=1.8.0
15:19:23  Registered adapter: postgres=1.8.0
15:19:23  Found 5 models, 3 seeds, 20 data tests, 528 macros
15:19:23  
15:19:23  Concurrency: 1 threads (target='dev')
15:19:23  
15:19:23  1 of 4 START sql table model public.customers .................................. [RUN]
15:19:23  1 of 4 OK created sql table model public.customers ............................. [SELECT 100 in 0.04s]
15:19:23  2 of 4 START test not_null_customers_customer_id ............................... [RUN]
15:19:23  2 of 4 PASS not_null_customers_customer_id ..................................... [PASS in 0.02s]
15:19:23  3 of 4 START test relationships_orders_customer_id__customer_id__ref_customers_  [RUN]
15:19:23  3 of 4 PASS relationships_orders_customer_id__customer_id__ref_customers_ ...... [PASS in 0.02s]
15:19:23  4 of 4 START test unique_customers_customer_id ................................. [RUN]
15:19:23  4 of 4 PASS unique_customers_customer_id ....................................... [PASS in 0.02s]
15:19:23  
15:19:23  Finished running 1 table model, 3 data tests in 0 hours 0 minutes and 0.19 seconds (0.19s).
15:19:24  
15:19:24  Completed successfully
15:19:24  
15:19:24  Done. PASS=4 WARN=0 ERROR=0 SKIP=0 TOTAL=4

Closes: #892

[cloudflare-workers-and-pages]

Deploying astronomer-cosmos with    Cloudflare Pages

Latest commit:	988ae35
Status:	✅  Deploy successful!
Preview URL:	https://3fa1c3fd.astronomer-cosmos.pages.dev
Branch Preview URL:	https://issue-892.astronomer-cosmos.pages.dev

View logs

[dwreeves]

The code makes sense to me, so giving it a LGTM. Thanks for taking this on @tatiana!

---

A thought as I read this, for another day though:

dbt 1.8+ has a new concept called unit tests, which it differentiates from data tests (formerly just "tests"). dbt's preferred way of running things is: unit tests -> build model -> data tests.

For the TestBehavior.BUILD, we comport with how dbt wants to run things 👍 because the order of operations is resolved automatically by dbt build.

We don't currently handle dbt unit tests for TestBehavior.AFTER_ALL and TestBehavior.AFTER_EACH though. 🤔 A natural way to do this would be to have AFTER_ALL also mean "before all" when it comes to unit tests, and similarly AFTER_EACH means "before each" for unit tests.

Alternatively, these could be decoupled; i.e. "before each, unit test" and "after all, data test," similarly "before all, unit test" and "after each, data test." But for TestBehavior.BUILD, decoupling gets weird. 🤷

No actionable item here other than, potentially, to mention in the docs that TestBehavior.BUILD is currently the only way to run dbt unit tests using Cosmos's automatic graph parsing. Just thinking out loud about the future.

[pankajkoti]

LGTM. Amazing support added and nice optimization. Happy to merge once #1374 is merged & this branch is rebased, and the CI passes after.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.24%. Comparing base (2fa5d01) to head (988ae35).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1377   +/-   ##
=======================================
  Coverage   96.23%   96.24%           
=======================================
  Files          67       67           
  Lines        4042     4051    +9     
=======================================
+ Hits         3890     3899    +9     
  Misses        152      152           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[tatiana]

Thanks a lot for the review and feedback, @dwreeves @pankajastro @pankajkoti!

I added a follow-up action to review what @dwreeves mentioned (unit tests x data tests split): #1386 I temptively added this to Cosmos 2.x, but we may want to handle this before.