You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When writing a RecordBatch via ipc (and then reading the stream from python) dates are not correct.
I have a date column from 2000-02-01 to 2021-11-02 on rust side but once converted to ipc pyarrow table, I see dates from 1998-01-16 to 2019-11-12. Dates are saved in a Date64Array.
Writing the same recordbatch as csv displays the correct dates.
To Reproduce
I can't really share the code but here is the rust->python (using pyo3) part.
@tafia can you possibly provide an example of what is in the batch: &RecordBatch argument? Specifically, what is the exact type (DataType::Timestamp(..))? If so, what is the unit?
Describe the bug
When writing a
RecordBatch
via ipc (and then reading the stream from python) dates are not correct.I have a date column from 2000-02-01 to 2021-11-02 on rust side but once converted to ipc pyarrow table, I see dates from 1998-01-16 to 2019-11-12. Dates are saved in a
Date64Array
.Writing the same recordbatch as csv displays the correct dates.
To Reproduce
I can't really share the code but here is the rust->python (using pyo3) part.
Expected behavior
Same dates
Additional context
I am new to arrow so there is a high chance I'm doing something wrong
The text was updated successfully, but these errors were encountered: