Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add type information to dbt's configs #2268

Closed
drewbanin opened this issue Mar 30, 2020 · 1 comment
Closed

Add type information to dbt's configs #2268

drewbanin opened this issue Mar 30, 2020 · 1 comment
Labels
enhancement New feature or request

Comments

@drewbanin
Copy link
Contributor

Describe the feature

dbt should default to using a jinja native environment for parsing jinja-rendered fields out of .yml files. This includes:

  • schema.yml files
  • dbt_project.yml
  • profiles.yml

Jinja's native environment will help coerce string values to their native types (int, float, boolean, etc). The most obvious immediate use-case is:

models:
  enabled: "{{ true if target.name == 'dev' else false }}"

We want this to return a boolean, not a string, and a jinja native env would help us do that.

This is a big big change, and i think it will make 95% of people really happy and 5% of people sad. We should verify that there's a sane upgrade path here, so if you have some currently working code like:

some_field: "{{ 'True' }}

you still have a recourse for rendering this as a string, rather than a boolean, once this change is live.

Describe alternatives you've considered

Make configs type-aware and coerce stringy input into an expected type.

Additional context

I have not yet completely wrapped my head around the exact nature of the jinja native environment behavior. Here are some things that I want to be true, that I would love to verify before we start working on this:

-- this should return a boolean True value
my-config: "{{ true }}"

-- this should return a string value [??]
my-config: "{{ 'True' }}"

-- this should return a string value
my-config: "{{ 'true' }}"

-- this should return an integer value
my-config: "{{ 123 }}"

-- this should return a string value [??]
my-config: "{{ '123' }}"

The lines marked [??] are places where I am presently unsure of the jinja native env behavior. We should think really really hard about the implications of this change.

As beneficial as I think this change will be, I don't know that we can ship it if there's no ability to return a stringy value like "True" or "123" in this context. As long as there's some sane way to do that, then I'm all for making getting a PR to address this issue merged.

Who will this benefit?

Dynamic models configurers

@drewbanin drewbanin added the enhancement New feature or request label Mar 30, 2020
@drewbanin drewbanin added this to the Octavius Catto milestone Mar 30, 2020
@drewbanin drewbanin changed the title Use native jinja environment for config parsing in .yml files Add type information to dbt's configs Apr 6, 2020
@drewbanin
Copy link
Contributor Author

closed by #2312

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant