Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make PostgresUserPasswordProfileMapping schema profile argument optional #675

Closed
tatiana opened this issue Nov 15, 2023 · 1 comment · Fixed by #683
Closed

Make PostgresUserPasswordProfileMapping schema profile argument optional #675

tatiana opened this issue Nov 15, 2023 · 1 comment · Fixed by #683
Labels
area:profile Related to ProfileConfig, like Athena, BigQuery, Clickhouse, Spark, Trino, etc bug Something isn't working dbt:run Primarily related to dbt run command or functionality profile:postgres Related to Postgres ProfileConfig
Milestone

Comments

@tatiana
Copy link
Collaborator

tatiana commented Nov 15, 2023

Context

As of Cosmos 1.2.3, when users instantiate a ProfileMapping using PostgresUserPasswordProfileMapping, Cosmos expects users to define a schema:

As illustrated below:

from cosmos.profiles import PostgresUserPasswordProfileMapping
profile_config = ProfileConfig(
   profile_name="my_profile_name",
   target_name="my_target_name",
   profile_mapping=PostgresUserPasswordProfileMapping(
       conn_id="my_postgres_conn_id",
       profile_args={
           "database": "my_postgres_database",
           "schema": "stage",
       },
   ),
)

However, dbt-postgres does not have this property as a mandatory field. This is usually not an issue unless users want to define schemas specific to models. When this happens, dbt considers the schema set at the profile level as the target_schema for applying a custom_schema (macro generate_schema_name): https://docs.getdbt.com/docs/build/custom-schemas.

The consequence is that model-specific schemas are resolved incorrectly if the profile schema="", by having an underscore pre-pre-appended.

This issue was originally raised by tariq.f in the Airflow #ariflow-dbt slack channel:
https://apache-airflow.slack.com/archives/C059CC42E9W/p1700049843620749

Acceptance criteria

  • Make PostgresUserPasswordProfileMapping schema profile argument optional
@tatiana tatiana added bug Something isn't working profile:postgres Related to Postgres ProfileConfig labels Nov 15, 2023
@tatiana tatiana added this to the 1.3.0 milestone Nov 15, 2023
Copy link

dosubot bot commented Nov 15, 2023

🤖

Hello again @tatiana! It's great to see you're still actively contributing to the Cosmos project!

It seems like you've got this under control, if you want help or have specific questions, let me know what I can do for you!

@dosubot dosubot bot added area:profile Related to ProfileConfig, like Athena, BigQuery, Clickhouse, Spark, Trino, etc dbt:run Primarily related to dbt run command or functionality priority:medium Medium priority issues are important issues that may have a workaround and medium impact labels Nov 15, 2023
@tatiana tatiana removed the priority:medium Medium priority issues are important issues that may have a workaround and medium impact label Nov 15, 2023
FouziaTariq added a commit to FouziaTariq/astronomer-cosmos that referenced this issue Nov 16, 2023
@tatiana tatiana modified the milestones: 1.3.0, 1.4.0 Jan 4, 2024
tatiana added a commit that referenced this issue Feb 29, 2024
Made PostgresUserPasswordProfileMapping schema profile argument optional

Closes: #675

Co-authored-by: Tatiana Al-Chueyr <tatiana.alchueyr@gmail.com>
arojasb3 pushed a commit to arojasb3/astronomer-cosmos that referenced this issue Jul 14, 2024
…ronomer#683)

Made PostgresUserPasswordProfileMapping schema profile argument optional

Closes: astronomer#675

Co-authored-by: Tatiana Al-Chueyr <tatiana.alchueyr@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:profile Related to ProfileConfig, like Athena, BigQuery, Clickhouse, Spark, Trino, etc bug Something isn't working dbt:run Primarily related to dbt run command or functionality profile:postgres Related to Postgres ProfileConfig
Projects
None yet
1 participant