Skip to content

Commit

Permalink
Add basic docs
Browse files Browse the repository at this point in the history
  • Loading branch information
pankajastro committed Nov 15, 2024
1 parent dcbf538 commit 1b4bde3
Show file tree
Hide file tree
Showing 2 changed files with 75 additions and 0 deletions.
51 changes: 51 additions & 0 deletions dev/dags/example_clone.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
from datetime import datetime

from airflow import DAG

from cosmos import DbtCloneLocalOperator, DbtRunLocalOperator, DbtSeedLocalOperator, ProfileConfig

DBT_PROJ_DIR = "/usr/local/airflow/dbt/jaffle_shop"

profile_config1 = ProfileConfig(
profile_name="bigquery_dev",
target_name="dev",
profiles_yml_filepath="/usr/local/airflow/dbt/jaffle_shop/profiles.yml",
)

profile_config2 = ProfileConfig(
profile_name="bigquery_clone",
target_name="dev",
profiles_yml_filepath="/usr/local/airflow/dbt/jaffle_shop/profiles.yml",
)


with DAG("test-id-1", start_date=datetime(2024, 1, 1), catchup=False) as dag:
seed_operator = DbtSeedLocalOperator(
profile_config=profile_config1,
project_dir=DBT_PROJ_DIR,
task_id="seed",
dbt_cmd_flags=["--select", "raw_customers"],
install_deps=True,
append_env=True,
)
run_operator = DbtRunLocalOperator(
profile_config=profile_config1,
project_dir=DBT_PROJ_DIR,
task_id="run",
dbt_cmd_flags=["--models", "stg_customers"],
install_deps=True,
append_env=True,
)

# [START clone_example]
clone_operator = DbtCloneLocalOperator(
profile_config=profile_config2,
project_dir=DBT_PROJ_DIR,
task_id="clone",
dbt_cmd_flags=["--models", "stg_customers", "--state", "/usr/local/airflow/dbt/jaffle_shop/target"],
install_deps=True,
append_env=True,
)
# [END clone_example]

seed_operator >> run_operator >> clone_operator
24 changes: 24 additions & 0 deletions docs/getting_started/operators.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
.. _operators:

Operators
=========

Cosmos exposes individual operators that correspond to specific dbt commands, which can be used just like traditional
`Apache Airflow® <https://airflow.apache.org/>`_ operators. Cosmos names these operators using the format ``Dbt<dbt-command><execution-mode>Operator``. For example, ``DbtBuildLocalOperator``.

Clone
-----

Requirement

* Cosmos >= 1.8.0
* dbt-core >= 1.6.0

The ``DbtCloneLocalOperator`` implement `dbt clone <https://docs.getdbt.com/reference/commands/clone>`_ command.

Example of how to use

.. literalinclude:: ../../dev/dags/example_clone.py
:language: python
:start-after: [START clone_example]
:end-before: [END clone_example]

0 comments on commit 1b4bde3

Please sign in to comment.