Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Support getting snapshot by timestamp #2276

Closed
2 of 8 tasks
vkorukanti opened this issue Nov 8, 2023 · 0 comments · Fixed by #2662
Closed
2 of 8 tasks

[Feature Request] Support getting snapshot by timestamp #2276

vkorukanti opened this issue Nov 8, 2023 · 0 comments · Fixed by #2662
Assignees
Labels
enhancement New feature or request good medium issue Good for those with Delta Lake experience kernel
Milestone

Comments

@vkorukanti
Copy link
Collaborator

Feature request

Which Delta project/connector is this regarding?

  • Spark
  • Standalone
  • Flink
  • Kernel
  • Other (fill in here)

Overview

Currently, Kernel only supports reading the latest snapshot in a table (API here) This task is to add support for fetching a snapshot by timestamp.

interface Table
{
  /**
   * Get the latest snapshot that is create at given time or the first snapshot
   * committed after the given timestamp. Given timestamp in in seconds sinc
   * epoch in UTC timezone.
   */
  Snapshot getSnapshotAtOrAfterTimestamp(long secondsSinceEpochUTC)
}

Design Questions

  • Should we even support the getSnapshotAtOrAfterTimestamp?
    • If yes, what should be the input timestamp? Timezone? seconds , micorseconds, milliseconds?
  • How should the FileStatus returned by FileSystemClient.listFrom contain the modification time? Which timezone? Does the filesystem expose the mod time in UTC?

Willingness to contribute

The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?

  • Yes. I can contribute this feature independently.
  • Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
  • No. I cannot contribute this feature at this time.
@vkorukanti vkorukanti added enhancement New feature or request kernel good medium issue Good for those with Delta Lake experience labels Nov 8, 2023
@vkorukanti vkorukanti added this to the 3.1.0 milestone Nov 8, 2023
@vkorukanti vkorukanti removed this from the 3.1.0 milestone Jan 2, 2024
@allisonport-db allisonport-db self-assigned this Feb 1, 2024
@vkorukanti vkorukanti added this to the 3.2.0 milestone Feb 22, 2024
allisonport-db added a commit that referenced this issue Mar 4, 2024
#### Which Delta project/connector is this regarding?

- [ ] Spark
- [ ] Standalone
- [ ] Flink
- [x] Kernel
- [ ] Other (fill in here)

## Description

Resolves #2276

Adds support for reading the snapshot of the table at a specific timestamp using `Table:: getSnapshotAtTimestamp`. 

## How was this patch tested?

Adds unit tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good medium issue Good for those with Delta Lake experience kernel
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants