Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-1946] [Bug] dbt snapshot fails if the last line is a comment #6781

Closed
2 tasks done
RobbertDM opened this issue Jan 30, 2023 · 5 comments · Fixed by #6780
Closed
2 tasks done

[CT-1946] [Bug] dbt snapshot fails if the last line is a comment #6781

RobbertDM opened this issue Jan 30, 2023 · 5 comments · Fixed by #6780
Labels
bug Something isn't working Team:Adapters Issues designated for the adapter area of the code

Comments

@RobbertDM
Copy link
Contributor

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When making a snapshot and using the check strategy with specified columns (not all), if the last line of your snapshot is a comment, it yields
14:50:11 TrinoUserError(type=USER_ERROR, name=SYNTAX_ERROR, message="line 12:16: mismatched input '<EOF>'. Expecting: ')'", query_id=20230130_145011_02592_b7axv)
because the produced query is:

select * from (
            select col1, col2, col3 from (
select * from iceberg.dbt_robbert.my_table
-- some comment ) subq
        ) as __dbt_sbq
        where false
        limit 0

Check out the -- some comment ) subq line.

Expected Behavior

When making a snapshot and using the check strategy with specified columns (not all), if the last line of your snapshot is a comment, it yields
a working snapshot
because the produced query is:

select * from (
            select col1, col2, col3 from (
select * from iceberg.dbt_robbert.my_table
-- some comment 
) subq
        ) as __dbt_sbq
        where false
        limit 0

Check out the ) subq line.

Steps To Reproduce

  1. Configure a snapshot to use check strategy
  2. Configure the snapshot to check_cols=['col1','col2'], not all
  3. Make sure the last line of the snapshot is a -- comment
  4. Run the snapshot

Relevant log output

No response

Environment

- OS: Ubuntu 22.04
- Python: 3.10.6
- dbt: 1.3.2

Which database adapter are you using with dbt?

other (mention it in "Additional Context")

Additional Context

adapter used: dbt-trino

@RobbertDM RobbertDM added bug Something isn't working triage labels Jan 30, 2023
@github-actions github-actions bot changed the title [Bug] dbt snapshot fails if the last line is a comment [CT-1946] [Bug] dbt snapshot fails if the last line is a comment Jan 30, 2023
@dbeatty10
Copy link
Contributor

Thank you for reporting this (and opening a PR!) @RobbertDM

This is similar to this one for incremental models:

How we should test for this (and similar issues):

@dbeatty10 dbeatty10 added Team:Adapters Issues designated for the adapter area of the code and removed triage labels Jan 30, 2023
@VersusFacit
Copy link
Contributor

Extremely naive question -- why would these models end in comments? What's a use case? I just find it strange that adding whitespace is apparently a fix here. Not used to seeing that

@jtcohen6
Copy link
Contributor

jtcohen6 commented Feb 3, 2023

It's a fair question — but also something we should support. Comments, wherever they are, should work.

We should add a comment to the end of models in integration tests for snapshots + each of our materialization types + singular and generic tests to catch any regressions in the future

During BLG:

  • We should add a test like this. A model with a comment at the end, every materialization (including singular test + snapshot), every incremental_strategy, every adapter, every day of the week.
  • Neat chat with @colin-rogers-dbt about how we might support property-based or "matrix"-style testing. This feels like a good example, and maybe we'd want to spike an experiment with pytest_matrix

@RobbertDM
Copy link
Contributor Author

Extremely naive question -- why would these models end in comments? What's a use case? I just find it strange that adding whitespace is apparently a fix here. Not used to seeing that

  • For me, I had a post_hook that I didn't want to run, but also wanted to keep somewhere. I just moved it to the end of the snapshot and commented it out. That broke things in an unexpected way, and it took me a while to figure out where the problem was. If this is fixed, we prevent people from going through the same struggle in the future.
  • Whitespace indeed is seldom a fix, but in this case, -- is valid for only one line, so adding whitespace transforms ) subq from a comment to actual code.

@VersusFacit
Copy link
Contributor

VersusFacit commented Feb 7, 2023

Makes total sense. I knew I was just lacking imagination and I appreciate you filling me in!

edit: just circle back to say well of course comment should work everywhere. I regret ever being questioning here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Team:Adapters Issues designated for the adapter area of the code
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants