[CT-2543] Execution <> Adapters: Cloning from production

[stu-k]

Description

One of the initiatives that the execution team is taking on in Q2 is cloning from a production environment. The work here is to core to enable core users to clone a schema using zero-copy clones if it is supported by the DWH, and views if is not.

Jeremy currently has a draft PR up that touches the adapter zone of core.

Talk to @stu-k for additional details.

[ChenyuLInx]

There are 3 functions each adapter need to implement, they are

• can_clone_tables
• get_pointer_sql
• get_clone_table_sql
  Basically these are functions that we have a dispatch macro dbt-core. The default implementation(and some implementation for snowflake) can be found in Jerco's PR linked above.

The work for adapter team would be figure out which functions each adapter need to implement and get them implemented.

[VersusFacit]

I've read through this, and I think the general strategy makes sense. I'm more concerned about the names we use in the design elements -- I found this quite tricky to get to know as an outsider because of the abstractions we've thus far adopted.

Re: naming matters

state_relation

I find this one to be vague/unintuitive, especially in the Python runtime. Here's my reasoning: With everything I understand currently, I think it's a sister term of target_relation, and the like. I feel like something far more descriptive would be stateful_relation or production_relation really anything that captures the essence that the relation referenced is the state being employed indirectly.

pointer

I take total responsibility that this just might be me, and my own arbitrary sensibilities: I find pointer to be very odd word choice. We're not really pointing at a memory address. Instead, I believe we're referencing an existing relation, and we should thusly name it. Maybe referrant_sql or something like that.

=====

I believe Matt is going to do a more thorough check on the code with how it relates to the adapters, but I like the general design. We're very fortunate that this isn't something DBMSs have felt the need to differentiate in (yet).

[aranke]

@VersusFacit Good call on naming conventions, I'm very supportive of whatever the adapters team decides is the right nomenclature here.

[jtcohen6]

@aranke @McKnight-42 Shall we open separate issues to track the implementations / testing in each of:

• dbt-snowflake (I see there's a WIP PR: [Feature] dbt_clone sf implementation dbt-snowflake#664)
• dbt-redshift WIP PR: initial implementation of redshift dbt_clone macros and test design a… dbt-redshift#505
• dbt-bigquery WIP PR: add macros and adapter test for new dbt_clone flag dbt-bigquery#784
• dbt-spark & dbt-databricks (I believe only the Databricks Runtime supports shallow clone, and only with Delta tables) WIP PR: dbt_clone macros, materialization and tests to dbt-spark dbt-spark#816

[jtcohen6]

Merged into all the adapters!