-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(ingest/databricks): ingest hive metastore by default, more docs #9601
feat(ingest/databricks): ingest hive metastore by default, more docs #9601
Conversation
metadata-ingestion/docs/sources/databricks/unity-catalog_post.md
Outdated
Show resolved
Hide resolved
@@ -135,7 +135,7 @@ class UnityCatalogSourceConfig( | |||
) | |||
|
|||
include_metastore: bool = pydantic.Field( | |||
default=True, | |||
default=False, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's call out that this is a breaking change?
It might also be good to show examples of what the hierarchy would look like when this is true/false
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also we should say we recommend keeping this set to false, and that folks should use platform_instance
instead
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll add the section in doc about hierarchy + update description of include_metastore
config field.
As far as I understand, we'll get rid of this config, eventually. So the section would not be relevant - post that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- added in updating-datahub.md as a breaking change
- added the examples in updating-datahub doc, I believe, you meant this only and not to add this in source docs ?
- we already have pydantic validator for warning if this is not set to false, so we are covered there. changed the error message to represent current state.
metadata-ingestion/docs/sources/databricks/unity-catalog_post.md
Outdated
Show resolved
Hide resolved
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
…ue-hive-metastore-urn' into master+ing-467-unique-hive-metastore-urn
Checklist