Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-1355] Add column type constraints as dbt native configs #6076

Closed
sungchun12 opened this issue Oct 17, 2022 · 4 comments
Closed

[CT-1355] Add column type constraints as dbt native configs #6076

sungchun12 opened this issue Oct 17, 2022 · 4 comments
Assignees

Comments

@sungchun12
Copy link
Contributor

sungchun12 commented Oct 17, 2022

Problem:

  • Data types are created after the fact(assumed by the database) and/or a dbt user has to explicitly cast data types in their SQL. Because of this, there isn't much incentive to assure dbt models have expected data types for consumers to depend on(it's tedious work :/). This may not be a problem for 50 dbt models, but becomes a big problem when you have 2000+ dbt models. There should be an easier way to configure data type constraints for any dbt models and provide options to enable/disable data type enforcement(think: what mypy does for the python static typing experience).

Research:

Considerations:

  • have the configs embedded in dbt-core and then each adapter can take stable configs to adjust table materialization macros
  • Enforce column positions based on schema config
  • dbt-core contains the hub of native configs while adapters own specific implementations as each database has nuances to which enforceable constraints are valid.
  • think about not null and default values for a table
  • Work with Jon Neo from Canva
  • Include a check constraint as pgsql enables it?

Team:

@sungchun12 sungchun12 self-assigned this Oct 17, 2022
@github-actions github-actions bot changed the title Add column type constraints as dbt native configs [CT-1355] Add column type constraints as dbt native configs Oct 17, 2022
@sungchun12
Copy link
Contributor Author

Related discussion: #5244 (reply in thread)

@sungchun12
Copy link
Contributor Author

Example BigQuery Constraints Macro:https://gist.github.com/sungchun12/f7ea081773ae824a83294649530d6e41

@sungchun12
Copy link
Contributor Author

What if constraints could be defined once within a subfolder path in dbt_project.yml and then in automatically creates not null constraints in the DDL?

@sungchun12
Copy link
Contributor Author

Closing in favor of this: #6079

@sungchun12 sungchun12 closed this as not planned Won't fix, can't repro, duplicate, stale Oct 17, 2022
Repository owner moved this from In Progress to Done in dbt Multi-Project Dreams: Building Blocks in Motion Oct 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant