Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document intersection of "secret" env vars + partial parsing #1066

Closed
1 task done
jtcohen6 opened this issue Jan 31, 2022 · 1 comment
Closed
1 task done

Document intersection of "secret" env vars + partial parsing #1066

jtcohen6 opened this issue Jan 31, 2022 · 1 comment
Labels
content Improvements or additions to content improvement Use this when an area of the docs needs improvement as it's currently unclear priority: medium Fix or enhancement to existing information that’s creating some requests from customers size: small This change will take 1 to 2 days to address

Comments

@jtcohen6
Copy link
Collaborator

Contributions

  • I have read the contribution docs, and understand what's expected of me.

Link to the page on docs.getdbt.com requiring updates

What part(s) of the page would you like to see updated?

Env vars prefixed DBT_ENV_SECRET_ can be used in profiles.yml + packages.yml. Their values will be scrubbed from logs.

We should make clear that changes to secrets used in profiles.yml do not trigger full re-parses (unlike changes to other env vars used in the active profile), because dbt will not be storing their values (once dbt-labs/dbt-core#4650 is fixed). This can be a limitation or a feature, depending on your point of view.

Additional information

See dbt-labs/dbt-core#4642 (comment)

@jtcohen6 jtcohen6 added content Improvements or additions to content improvement Use this when an area of the docs needs improvement as it's currently unclear labels Jan 31, 2022
@runleonarun runleonarun added size: small This change will take 1 to 2 days to address priority: medium Fix or enhancement to existing information that’s creating some requests from customers labels Jan 31, 2022
@jtcohen6
Copy link
Collaborator Author

In discussing the fix proposed in dbt-labs/dbt-core#9795 with @gshank, we are thinking that while this is an outcome of the current implementation of (secret) env vars + partial parsing, it's not actually the desirable behavior longer-term. The right way to avoid full parses when inputs haven't changed is to more granularly track where those inputs are being used — that is, extending what we've done for env vars to --vars and {{ target }} values.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
content Improvements or additions to content improvement Use this when an area of the docs needs improvement as it's currently unclear priority: medium Fix or enhancement to existing information that’s creating some requests from customers size: small This change will take 1 to 2 days to address
Projects
None yet
Development

No branches or pull requests

2 participants