Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: add property_id #321

Merged
merged 2 commits into from
Jun 5, 2024
Merged

feature: add property_id #321

merged 2 commits into from
Jun 5, 2024

Conversation

portalhacker
Copy link
Contributor

@portalhacker portalhacker commented May 1, 2024

Description & motivation

This PR adds a new property_id column to the base_ga4__events model which is derived from either the name of the imported dataset or the property_ids dbt variable (depending on whether this is a multi-property or single-property project).

Note that the default behavior of dbt is to ignore new columns during incremental runs. This behavior can be overridden with the on_schema_change configuration https://docs.getdbt.com/docs/build/incremental-models#what-if-the-columns-of-my-incremental-model-change

Checklist

  • I have verified that these changes work locally
  • [na] I have updated the README.md (if applicable)
  • [na] I have added tests & descriptions to my models (and macros if applicable)
  • I have run dbt test and python -m pytest . to validate existing tests

@adamribaudo-velir
Copy link
Collaborator

The stream ID is a more robust method of determining where the data came from given that 1 property can have multiple streams. Can you describe a use case where the property ID provides value above and beyond the stream ID?

@portalhacker
Copy link
Contributor Author

The property ID simplifies analyzes by property ID and name (with a mapping seed) regardless of the number of streams for each property.

It also enables to build URLs to the GA4 UI.

@adamribaudo-velir
Copy link
Collaborator

@FloDevelops have you tested this on a property with intraday tables? I get the following error:

13:23:54  Completed with 1 error and 0 warnings:
13:23:54
13:23:54    Database Error in model base_ga4__events (models\staging\base\base_ga4__events.sql)
  Bad int64 value: intraday_20240604237690264
  compiled Code at target\run\ga4\models\staging\base\base_ga4__events.sql

@portalhacker
Copy link
Contributor Author

@adamribaudo-velir Indeed I did not test on intraday tables.
I just fixed that now.

@adamribaudo-velir
Copy link
Collaborator

This looks good. Just be aware that the default behavior when adding a new column to an incremental model is to ignore the change until a full refresh. Users can update their config to append the new column

https://docs.getdbt.com/docs/build/incremental-models

@adamribaudo-velir adamribaudo-velir self-assigned this Jun 5, 2024
@adamribaudo-velir adamribaudo-velir merged commit 3ef51b6 into Velir:main Jun 5, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants