Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Improve UniForm Hudi support for Lists/Maps and Schema Evolution #2738

Closed
8 tasks
the-other-tim-brown opened this issue Mar 8, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@the-other-tim-brown
Copy link
Contributor

Feature request

Which Delta project/connector is this regarding?

  • Spark
  • Standalone
  • Flink
  • Kernel
  • Other (Unfiorm)

Overview

The UniForm support for Hudi does not properly translate Map & List fields and the testing does not validate that the schema can be evolved in the source Delta Lake table and properly reflected in the Hudi table.

Motivation

UniForm users will benefit from a more complete translation between Delta Lake and Hudi.

Further details

The XTable project has already handled such translations and the approach can be ported over to this repo.

Willingness to contribute

The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?

  • Yes. I can contribute this feature independently.
  • Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
  • No. I cannot contribute this feature at this time.
@anniewang-db
Copy link
Contributor

List/map support has been added in #3320

@MrPowers
Copy link
Contributor

Great, looks like this one has been completed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Development

No branches or pull requests

3 participants