Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aggregate distinct customer ids in email rollup #58

Merged
merged 11 commits into from
May 17, 2023

Conversation

fivetran-jamie
Copy link
Contributor

@fivetran-jamie fivetran-jamie commented May 9, 2023

Are you a current Fivetran customer?

internal

What change(s) does this PR introduce?

  • adds a distinct to the fivetran_utils.string_agg of customer ids in int_shopify__customer_email_rollup
  • adds extra line to README about the shopify holistic reporting package having a dependency on this package
  • adds lower() to some spots we are looking at transaction.kind (i have always seen it as lowercase in the source data but just in case -- i changed this while investigating something and felt it didn't hurt to leave in)
  • remove unused fields from int_shopify__customer_email_rollup

Did you update the CHANGELOG?

  • Yes

Does this PR introduce a breaking change?

  • Yes (please provide breaking change details below.)
  • No (please provide an explanation as to how the change is non-breaking below.)

no columns were changed -- the customer_ids field will just not aggregate duplicate IDs

Did you update the dbt_project.yml files with the version upgrade (please leverage standard semantic versioning)? (In both your main project and integration_tests)

  • Yes

Is this PR in response to a previously created Bug or Feature Request

How did you test the PR changes?

  • Buildkite
  • Local (please provide additional testing details below)

before change:
image

after change:
image

Aggregated metrics tie out as well

Select which warehouse(s) were used to test the PR

  • BigQuery
  • Redshift
  • Snowflake
  • Postgres
  • Databricks
  • Other (provide details below)

Provide an emoji that best describes your current mood

👾

Feedback

We are so excited you decided to contribute to the Fivetran community dbt package! We continue to work to improve the packages and would greatly appreciate your feedback on our existing dbt packages or what you'd like to see next.

@fivetran-jamie fivetran-jamie self-assigned this May 9, 2023
Copy link
Contributor

@fivetran-avinash fivetran-avinash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @fivetran-jamie! So it looks like the customer ids are distinct now, but the aggregation is still not changing and doesn't match with the order counts and amount summations for customer ids with multiple tags. Can you take a look at the issues I saw in int_shopify__customer_email_rollup and see if there's validity to what I'm seeing? Thanks!

fivetran-jamie and others added 4 commits May 16, 2023 11:52
Co-authored-by: Avinash Kunnath <108772760+fivetran-avinash@users.noreply.github.com>
Copy link
Contributor

@fivetran-avinash fivetran-avinash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All's good! Whenever Databricks stops timing out you're good to release.

@fivetran-jamie fivetran-jamie merged commit 2049f8d into main May 17, 2023
@fivetran-jamie fivetran-jamie deleted the bug/aggregate-only-distinct-customer-ids branch May 17, 2023 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants