You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the merge_update_columns config for incremental models can be used to define which columns a merge statement should update.
The opposite behavior (to ignore a column) is achieved by omitting a column in this config. I've seen bugs being introduced into models many times when their logic changes and not enough attention is paid to the config block (omission is much harder to work with than an explicit config).
It would be useful to explicitly name columns that should be ignored. This could be implemented with a config called merge_ignore_columns (or similar).
This is useful for auditing columns like etl_created_timestamp. The value of this column should be determined when a record is first inserted, but should never be updated afterwards.
If both merge_update_columns and merge_ignore_columns configs are set and a column is in both, an exception should be raised.
Describe alternatives you've considered
None
Who will this benefit?
This feature would help make the config block more compact (especially in cases where a table has a large amount of columns), as well as add clarity by explicitly calling out the behavior of a column, instead having the behavior be determined by omission.
Are you interested in contributing this feature?
I don't know enough about dbt core to feel comfortable contributing (yet)
Anything else?
No response
The text was updated successfully, but these errors were encountered:
github-actionsbot
changed the title
[Feature] Add a "ignore_merge_columns" config for incremental models
[CT-757] [Feature] Add a "ignore_merge_columns" config for incremental models
Jun 16, 2022
christoph-meier-offerup
changed the title
[CT-757] [Feature] Add a "ignore_merge_columns" config for incremental models
[CT-757] [Feature] Add a "merge_ignore_columns" config for incremental models
Jun 16, 2022
@christoph-meier-offerup Totally! Thanks for opening. I'm going to close this issue only because it's a duplicate of another recent issue, asking for the same functionality: #5260
Is this your first time opening an issue?
Yes
Describe the Feature
Currently, the
merge_update_columns
config for incremental models can be used to define which columns a merge statement should update.The opposite behavior (to ignore a column) is achieved by omitting a column in this config. I've seen bugs being introduced into models many times when their logic changes and not enough attention is paid to the config block (omission is much harder to work with than an explicit config).
It would be useful to explicitly name columns that should be ignored. This could be implemented with a config called
merge_ignore_columns
(or similar).This is useful for auditing columns like
etl_created_timestamp
. The value of this column should be determined when a record is first inserted, but should never be updated afterwards.If both
merge_update_columns
andmerge_ignore_columns
configs are set and a column is in both, an exception should be raised.Describe alternatives you've considered
None
Who will this benefit?
This feature would help make the config block more compact (especially in cases where a table has a large amount of columns), as well as add clarity by explicitly calling out the behavior of a column, instead having the behavior be determined by omission.
Are you interested in contributing this feature?
I don't know enough about dbt core to feel comfortable contributing (yet)
Anything else?
No response
The text was updated successfully, but these errors were encountered: