feature/remove-age (#22)

* feature/remove-age * regen docs * Apply suggestions from code review Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com> * update changelog * Apply suggestions from code review Co-authored-by: Jamie Rodriguez <65564846+fivetran-jamie@users.noreply.github.com> * update documentation * Update packages.yml --------- Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com> Co-authored-by: Jamie Rodriguez <65564846+fivetran-jamie@users.noreply.github.com>
fivetran · Dec 10, 2024 · c7bb80a · c7bb80a
1 parent baaebf6
commit c7bb80a
Show file tree

Hide file tree

Showing 9 changed files with 56 additions and 46 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,19 @@
+# dbt_tiktok_ads v0.7.0
+[PR #22](https://github.com/fivetran/dbt_tiktok_ads/pull/22) includes the following updates:
+
+## Breaking Changes
+- **Upstream Breaking Change:** In the [July 2023 TikTok Ads connector update](https://fivetran.com/docs/connectors/applications/tiktok-ads/changelog#july2023) for the `ADGROUP_HISTORY` table, the `age` column was renamed to `age_groups`.
+  - This change primarily affects the upstream `stg_tiktok_ads__ad_group_history` model in `dbt_tiktok_source`. The `age_groups` column was not previously used in any downstream models within `dbt_tiktok_ads`, so this change impacts only the upstream staging model.
+  - Previously, we coalesced the `age` and `age_groups` columns in the `stg_tiktok_ads__ad_group_history` model to accommodate connectors using the old naming convention. However, due to inconsistent data types, this approach is no longer viable.
+  - As a result, the coalesced field has been removed in favor of the `age_groups` column.
+  - If necessary, you can populate historical data in the `age_groups` column by performing a resync of the `ADGROUP_HISTORY` table, as TikTok provides all data regardless of the previous sync state.
+  - For more details, see the [DECISIONLOG entry](https://github.com/fivetran/dbt_tiktok_ads/blob/main/DECISIONLOG.md#age_groups-and-age-columns).
+
+## Documentation  
+- Added `DECISIONLOG` file:  
+  - Detailed the removal of the previously mentioned coalesced `age` and `age_groups` column.  
+  - Clarified why aggregation differences occur across varying grains.  
+
 # dbt_tiktok_ads v0.6.0
 [PR #20](https://github.com/fivetran/dbt_tiktok_ads/pull/20) includes the following **BREAKING CHANGE** updates:
 

diff --git a/DECISIONLOG.md b/DECISIONLOG.md
@@ -0,0 +1,20 @@
+# DECISION LOG
+
+## Why don't metrics add up across different grains (Ex. ad level vs campaign level)?
+Not all ads are served at the ad level. In other words, there are some ads that are served only at the ad group, campaign, etc. levels. The implications are that since not ads are included in the ad-level report, their associated spend, for example, won't be included at that grain. Therefore your spend totals may differ across the ad grain and another grain. 
+
+This is a reason why we have broken out the ad reporting packages into separate hierarchical end models (Ad, Ad Group, Campaign, and more). Because if we only used ad-level reports, we could be missing data.
+
+## `age_groups` and `age` columns
+In the [July 2023 TikTok Ads connector update](https://fivetran.com/docs/connectors/applications/tiktok-ads/changelog#july2023), we renamed the `age` column in the `ADGROUP_HISTORY` table to `age_groups`. To maintain compatibility with connectors using the old column name, our data models previously coalesced the `age` and `age_groups` columns. However, due to inconsistent data types between `age` and `age_groups`, this approach began causing errors.
+
+### Decision
+- The `age` column has been removed from the `stg_tiktok_ads__ad_group_history` model and the `fill` staging macro for the ad group history table.
+- Models now rely exclusively on the `age_groups` column.
+
+### Impact
+- Customers using the `age` column will no longer have access to this field in the models.
+- The change simplifies logic and resolves data inconsistency issues caused by merging fields with different data types.
+
+### Action for Customers Requiring Historical `age` Data
+Customers who still need the historical `age` column data can resync the `ADGROUP_HISTORY` table in their TikTok Ads connector. TikTok provides all historical data in the `age_groups` column, allowing the data to be fully populated.
diff --git a/README.md b/README.md
@@ -1,4 +1,6 @@
-<p align="center">
+# Tiktok Ads Transformation dbt Package ([Docs](https://fivetran.github.io/dbt_tiktok_ads/))
+
+<p align="left">
     <a alt="License"
         href="https://github.com/fivetran/dbt_tiktok_ads/blob/main/LICENSE">
         <img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" /></a>
@@ -10,7 +12,6 @@
         <img src="https://img.shields.io/badge/Contributions-welcome-blueviolet" /></a>
 </p>
 
-# Tiktok Ads Transformation dbt Package ([Docs](https://fivetran.github.io/dbt_tiktok_ads/))
 ## What does this dbt package do?
 - Produces modeled tables that leverage Tiktok Ads data from [Fivetran's connector](https://fivetran.com/docs/applications/tiktok-ads) in the format described by [this ERD](https://fivetran.com/docs/applications/tiktok-ads#schemainformation) and builds off the output of our [Tiktok Ads source package](https://github.com/fivetran/dbt_tiktok_ads_source).
 - Creates reports on daily marketing performance across various levels of granularity.
@@ -54,7 +55,7 @@ Include the following tiktok_ads package version in your `packages.yml` file _if
 ```yaml
 packages:
   - package: fivetran/tiktok_ads
-    version: [">=0.6.0", "<0.7.0"]
+    version: [">=0.7.0", "<0.8.0"]
 
 ```
 Do **NOT** include the `tiktok_ads_source` package in this file. The transformation package itself has a dependency on it and will install the source package as well.
@@ -143,7 +144,7 @@ This dbt package is dependent on the following dbt packages. These dependencies
 ```yml
 packages:
     - package: fivetran/tiktok_ads_source
-      version: [">=0.6.0", "<0.7.0"]
+      version: [">=0.7.0", "<0.8.0"]
 
     - package: fivetran/fivetran_utils
       version: [">=0.4.0", "<0.5.0"]

diff --git a/dbt_project.yml b/dbt_project.yml
@@ -1,5 +1,5 @@
 name: 'tiktok_ads'
-version: '0.6.0'
+version: '0.7.0'
 config-version: 2
 require-dbt-version: [">=1.3.0", "<2.0.0"]
 models:

diff --git a/docs/catalog.json b/docs/catalog.json
diff --git a/docs/index.html b/docs/index.html
diff --git a/docs/manifest.json b/docs/manifest.json
diff --git a/integration_tests/dbt_project.yml b/integration_tests/dbt_project.yml
@@ -1,5 +1,5 @@
 name: 'tiktok_ads_integration_tests'
-version: '0.6.0'
+version: '0.7.0'
 profile: 'integration_tests'
 config-version: 2
 

diff --git a/packages.yml b/packages.yml
@@ -1,3 +1,3 @@
 packages:
 - package: fivetran/tiktok_ads_source
-  version: [">=0.6.0", "<0.7.0"]
+  version: [">=0.7.0", "<0.8.0"]