Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Time32 and Time64 types #2272

Closed
rebasedming opened this issue Mar 9, 2024 · 3 comments
Closed

Support Time32 and Time64 types #2272

rebasedming opened this issue Mar 9, 2024 · 3 comments
Assignees
Labels
enhancement New feature or request on-hold Issues and Pull Requests that are on hold for some reason

Comments

@rebasedming
Copy link
Contributor

Description

The delta-rs PrimitiveType supports timestamp and date, but not time. This means that Arrow Time32 and Time64 types are not supported.

Use Case

I would like to use Time32 and Time64 types in my DataFusion DeltaTables.

Related Issue(s)

@rebasedming rebasedming added the enhancement New feature or request label Mar 9, 2024
@rebasedming
Copy link
Contributor Author

take

@ldacey
Copy link
Contributor

ldacey commented Mar 10, 2024

Are other types possible to add as well, such as Duration? I had a source the other day where most columns were duration format in Polars (HH:MM:SS) but it failed to save to the Delta Table. I converted to an integer for now.

            pl.col("response_sla").dt.total_milliseconds().alias("response_sla_ms"),

@ion-elgreco
Copy link
Collaborator

All these types are not supported by the delta protocol, so we can't add them.

For duration types i actually made an upstream request at the delta io repo, so you could up vote or ping the databricks folks there to make it happen

@ion-elgreco ion-elgreco added the on-hold Issues and Pull Requests that are on hold for some reason label Mar 28, 2024
@ion-elgreco ion-elgreco closed this as not planned Won't fix, can't repro, duplicate, stale Dec 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request on-hold Issues and Pull Requests that are on hold for some reason
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants