Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stricter secret env vars: allow in profiles.yml + packages.yml, disallow elsewhere #4310

Closed
jtcohen6 opened this issue Nov 19, 2021 · 0 comments · Fixed by #4311
Closed

Stricter secret env vars: allow in profiles.yml + packages.yml, disallow elsewhere #4310

jtcohen6 opened this issue Nov 19, 2021 · 0 comments · Fixed by #4311
Labels

Comments

@jtcohen6
Copy link
Contributor

In v0.21, we implemented "secret" env vars (prefixed with DBT_ENV_SECRET_) that enabled users to ensure that their configured credentials or access tokens would be scrubbed from logs.

While this is a convenience for many users, it's still possible to use those env vars in all the ways + places that {{ env_var() }} can be used in dbt projects—in arbitrary macros, query results, models that write to the database, etc.

After further discussion and deliberation, we've identified two tightly constrained use cases for "secret" env vars:

  • environment-based configuration of connection profiles (in profiles.yml)
  • git access tokens for private packages (in packages.yml)

As such, we should restrict use of secret-prefixed env vars, enabling them for use in those two files only, and raising an explicit error if they're used elsewhere. This enables project administrators to unblock their colleagues from connecting to the database and accessing dbt packages, without exposing those secrets to misuse.

Questions we're considering now, and will likely answer in the future:

  • Should this tight constraint be the default behavior? Should it be possible to re-enable more "permissive" use?
  • Are there other use cases we aren't capturing above? (As soon as secrets are allowed in dbt_project.yml or macros, they're effectively allowed everywhere)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant