Skip to content

Commit

Permalink
Port: Not dropping table for incremental full refresh with delta (#124)
Browse files Browse the repository at this point in the history
### Description

Ports the upstream change: Not dropping table for incremental full refresh with delta (dbt-labs/dbt-spark#287)
  • Loading branch information
ueshin authored Jul 5, 2022
1 parent 4796fc0 commit a722a59
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 1 deletion.
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
## dbt-databricks 1.2.0 (Release TBD)

### Fixes
- Incremental materialization updated to not drop table first if full refresh for delta lake format, as it already runs _create or replace table_ ([#286](https://github.com/dbt-labs/dbt-spark/issues/286), [#287](https://github.com/dbt-labs/dbt-spark/pull/287/))

### Contributors
- [@grindheim](https://github.com/grindheim) ([#287](https://github.com/dbt-labs/dbt-spark/pull/287/))

## dbt-databricks 1.1.1 (Release TBD)

### Features
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,14 @@

{{ run_hooks(pre_hooks) }}

{% set is_delta = (file_format == 'delta' and existing_relation.is_delta) %}

{% if existing_relation is none %}
{% set build_sql = create_table_as(False, target_relation, sql) %}
{% elif existing_relation.is_view or full_refresh_mode %}
{% do adapter.drop_relation(existing_relation) %}
{% if not is_delta %} {#-- If Delta, we will `create or replace` below, so no need to drop --#}
{% do adapter.drop_relation(existing_relation) %}
{% endif %}
{% set build_sql = create_table_as(False, target_relation, sql) %}
{% else %}
{% do run_query(create_table_as(True, tmp_relation, sql)) %}
Expand Down

0 comments on commit a722a59

Please sign in to comment.