Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-757] [Feature] Add a "merge_ignore_columns" config for incremental models #5387

Closed
1 task done
christoph-meier-offerup opened this issue Jun 16, 2022 · 1 comment
Closed
1 task done
Labels
duplicate This issue or pull request already exists enhancement New feature or request

Comments

@christoph-meier-offerup
Copy link

christoph-meier-offerup commented Jun 16, 2022

Is this your first time opening an issue?

Yes

Describe the Feature

Currently, the merge_update_columns config for incremental models can be used to define which columns a merge statement should update.

The opposite behavior (to ignore a column) is achieved by omitting a column in this config. I've seen bugs being introduced into models many times when their logic changes and not enough attention is paid to the config block (omission is much harder to work with than an explicit config).

It would be useful to explicitly name columns that should be ignored. This could be implemented with a config called merge_ignore_columns (or similar).

This is useful for auditing columns like etl_created_timestamp. The value of this column should be determined when a record is first inserted, but should never be updated afterwards.

If both merge_update_columns and merge_ignore_columns configs are set and a column is in both, an exception should be raised.

Describe alternatives you've considered

None

Who will this benefit?

This feature would help make the config block more compact (especially in cases where a table has a large amount of columns), as well as add clarity by explicitly calling out the behavior of a column, instead having the behavior be determined by omission.

Are you interested in contributing this feature?

I don't know enough about dbt core to feel comfortable contributing (yet)

Anything else?

No response

@github-actions github-actions bot changed the title [Feature] Add a "ignore_merge_columns" config for incremental models [CT-757] [Feature] Add a "ignore_merge_columns" config for incremental models Jun 16, 2022
@christoph-meier-offerup christoph-meier-offerup changed the title [CT-757] [Feature] Add a "ignore_merge_columns" config for incremental models [CT-757] [Feature] Add a "merge_ignore_columns" config for incremental models Jun 16, 2022
@jtcohen6 jtcohen6 added duplicate This issue or pull request already exists and removed triage labels Jun 17, 2022
@jtcohen6
Copy link
Contributor

@christoph-meier-offerup Totally! Thanks for opening. I'm going to close this issue only because it's a duplicate of another recent issue, asking for the same functionality: #5260

@jtcohen6 jtcohen6 closed this as not planned Won't fix, can't repro, duplicate, stale Jun 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants