Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes to open lineage connectors #8370

Merged
merged 2 commits into from
Sep 2, 2024
Merged

Conversation

mandy-chessell
Copy link
Contributor

Description

The Lineage Integrator OMIS has special support for open lineage events which enables events to be passed between connectors registered with a lineage integration context manager. When the open lineage support was written, there was only one lineage integration context manager per integration daemon server. Now there is potential for multiple lineage integration context manager:

  • One when the Lineage Integrator OMIS is explicitly activated in the integration daemon's configuration
  • One for each integration group registered with the integration daemon that also has at least one catalog target that is a Lineage Integration Connector.

This fix ensures that an open lineage event passed through the lineage integrator OMIS's REST API is passed to all active lineage integration context managers.

In addition the base class for the open lineage API publsher and open lineage file publisher connectors was not extracting the endpoint correctly from the catalog target if hte catalog target is an asset.

Related Issue(s)

None

Testing

Tested open lineage events pssed into the REST API are pased to a registered OpenLineageFilePublisher connector and written to a folder attached as a catalog target to this connector.

Release Notes & Documentation

No changed needed

Additional notes

None

Signed-off-by: Mandy Chessell <mandy.e.chessell@gmail.com>
@mandy-chessell mandy-chessell merged commit d8297d8 into odpi:main Sep 2, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant