Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[native] Support reading Iceberg V2 tables with Position deletes #21980

Merged

Conversation

imjalpreet
Copy link
Member

Description

This PR adds support for reading Iceberg V2 tables with Position Deletes. An unsupported exception will be thrown if the table contains Equality Deletes

Motivation and Context

facebookincubator/velox#7847

Impact

With this change, users will be able to read a subset of Iceberg V2 Tables, i.e. tables with position deletes.

Test Plan

Tested manually by creating Iceberg V2 tables with multiple Position Deletes.

Contributor checklist

  • Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

Prestissimo (Native Execution) Changes
* Add support to read Iceberg V2 tables with Position Deletes 

@imjalpreet imjalpreet requested a review from yingsu00 February 21, 2024 14:05
@imjalpreet imjalpreet self-assigned this Feb 21, 2024
@imjalpreet imjalpreet requested a review from a team as a code owner February 21, 2024 14:06
@imjalpreet imjalpreet force-pushed the icebergPrestissimoPositionalDelete branch from 10c5d42 to 6d75943 Compare February 21, 2024 14:08
@imjalpreet imjalpreet marked this pull request as draft February 22, 2024 07:46
@imjalpreet imjalpreet force-pushed the icebergPrestissimoPositionalDelete branch 6 times, most recently from f30926b to a5dc684 Compare February 22, 2024 10:26
@imjalpreet imjalpreet marked this pull request as ready for review February 22, 2024 11:50
@yingsu00
Copy link
Contributor

yingsu00 commented Mar 1, 2024

@imjalpreet Have you tried the copy constructors yet? Do they work?

@imjalpreet imjalpreet force-pushed the icebergPrestissimoPositionalDelete branch from a5dc684 to db85a6a Compare March 1, 2024 09:14
@imjalpreet
Copy link
Member Author

imjalpreet commented Mar 1, 2024

Have you tried the copy constructors yet? Do they work?

@yingsu00 sorry I forgot to push my commit, I thought I had already done that. Yes, I had made the changes and those worked.

@yingsu00
Copy link
Contributor

yingsu00 commented Mar 1, 2024

@imjalpreet There are test failures

org.apache.iceberg.exceptions.NotFoundException: Failed to open input stream for file: file:/tmp/PrestoTest1442477645755568799/iceberg_data/catalog/test_metadata_schema/iceberg_table1/metadata/00000-ff582f52-419c-493e-90f2-f1ea493599cb.metadata.json

Can you please take a look? Thanks!

@imjalpreet
Copy link
Member Author

There are test failures

Unrelated failures since we haven't made any changes to the Java code. Most probably intermittent, I have re-run the test suite to verify.

@yingsu00 yingsu00 merged commit e7b2b2c into prestodb:master Mar 1, 2024
59 checks passed
@wanglinsong wanglinsong mentioned this pull request May 1, 2024
48 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants