Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix whitespace issue with incremental sql #458

Merged
merged 11 commits into from
Jan 9, 2023
7 changes: 7 additions & 0 deletions .changes/unreleased/Fixes-20230109-105325.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
kind: Fixes
body: stop eliminating trailing whitespace in incremental merge sql
time: 2023-01-09T10:53:25.837837-08:00
custom:
Author: colin-rogers-dbt
Issue: "457"
PR: "458"
3 changes: 1 addition & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -182,11 +182,10 @@ jobs:
uses: actions/setup-python@v4.3.0
with:
python-version: ${{ matrix.python-version }}

- name: Install python dependencies
run: |
python -m pip install --user --upgrade pip
python -m pip install --upgrade wheel
python -m pip install --upgrade wheel setuptools twine check-wheel-contents
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did adding the following turn out to be necessary?

  • setuptools
  • twine
  • check-wheel-contents

If they aren't strictly necessary, I'd advocate for peeling them out.

python -m pip --version
- uses: actions/download-artifact@v2
with:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
{{ wrap_with_time_ingestion_partitioning_sql(build_partition_time_exp(partition_by), sql, True) }}
{%- else -%}
{{sql}}
{%- endif -%}
{%- endif %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTW!

)
{%- endif -%}
{%- endset -%}
Expand Down
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ def _dbt_core_version(plugin_version: str) -> str:
"googleapis-common-protos~=1.6",
"google-cloud-storage~=2.4",
"google-cloud-dataproc~=5.0",
"agate>=1.6.3,<1.7",
],
zip_safe=False,
classifiers=[
Expand Down
44 changes: 44 additions & 0 deletions tests/functional/test_incremental_materialization.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
import pytest
import os
from dbt.tests.util import (
run_dbt
)

# This is a short term hack, we need to go back
# and make adapter implementations of:
# https://github.com/dbt-labs/dbt-core/pull/6330

_INCREMENTAL_MODEL = """
{{
config(
materialized="incremental",
)
}}

{% if not is_incremental() %}

select 10 as id, cast('2020-01-01 01:00:00' as datetime) as date_hour union all
select 30 as id, cast('2020-01-01 02:00:00' as datetime) as date_hour

{% else %}

select 20 as id, cast('2020-01-01 01:00:00' as datetime) as date_hour union all
select 40 as id, cast('2020-01-01 02:00:00' as datetime) as date_hour

{% endif %}
-- Test Comment To Prevent Reccurence of https://github.com/dbt-labs/dbt-core/issues/6485
dbeatty10 marked this conversation as resolved.
Show resolved Hide resolved
"""

class BaseIncrementalModelConfig:
@pytest.fixture(scope="class")
def models(self):
return {
"test_incremental.sql": _INCREMENTAL_MODEL
}

class TestIncrementalModel(BaseIncrementalModelConfig):
def test_incremental_model_succeeds(self, project):
results = run_dbt(["run"])
assert len(results) == 1
results = run_dbt(["run"])
assert len(results) == 1