[CT-1255] Are we thinking too hard about this and we should simply evolve the functionality of dbt exposures?

[sungchun12]

Success Criteria:

• Links to existing files in dbt-core that need to be updated
• Net new files in dbt-core and why it needs new files
• Potential "gotcha" implementations based on the above(think: if we update this file here, we have to update 100 files downstream)
• pseudo-code snippets on initial approach for the above to make this functionality work
• Loom demo video

[matt-winkler]

@sungchun12 I added a new branch for this.

In terms of generating useful outputs at compile time in the manifest.json, this kind of works.

Here is my exposure definition:

version: 2

exposures:
  - name: finance_team
    type: contract
    owner: 
      name: Mateo
      email: me@jaffleshop.com
    description: a thought experiment of what defining producer contracts in an exposure would look like
    depends_on:
      - ref('my_second_dbt_model')
    config:
      version: 0.0.1 # this isn't fully worked out. How do we understand state over time to map to versions, if not forcing a version tag in the project repo?
      required_tests: # opinion that this should be a yes/no
        - unique
        - not_null
      freshness: # We should be testing the target of this job, not it's sources, for data updates and marking them with a timestamp
        warn_after: {count: 6, period: hour}
        error_after: {count: 12, period: hour}
      on_schema_change: 'fail' # means we disallow schema changes, inclusive of data types

And here is the relevant section output in the manifest. The relevant stuff is in the type and config fields:

"exposure.dbt_dev_project.finance_team": {
            "fqn": [
                "dbt_dev_project",
                "example",
                "finance_team"
            ],
            "unique_id": "exposure.dbt_dev_project.finance_team",
            "package_name": "dbt_dev_project",
            "root_path": "/Users/matt.winkler/Projects/dbt/dev_project",
            "path": "example/exposures.yml",
            "original_file_path": "models/example/exposures.yml",
            "name": "finance_team",
            "type": "contract",
            "owner": {
                "email": "me@jaffleshop.com",
                "name": "Mateo"
            },
            "resource_type": "exposure",
            "description": "a thought experiment of what defining producer contracts in an exposure would look like",
            "maturity": null,
            "meta": {},
            "tags": [],
            "config": {
                "enabled": true,
                "version": "0.0.1",
                "required_tests": [
                    "unique",
                    "not_null"
                ],
                "freshness": {
                    "warn_after": {
                        "count": 6,
                        "period": "hour"
                    },
                    "error_after": {
                        "count": 12,
                        "period": "hour"
                    }
                },
                "on_schema_change": "fail"
            },

[sungchun12]

Thanks Matt! I'll figure out a way to unite this with the work Doug and I are doing here: #5945

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

[github-actions]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.