Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: read_pickle and period data #10439

Closed
scarrucciu opened this issue Jun 25, 2015 · 1 comment · Fixed by #10866
Closed

BUG: read_pickle and period data #10439

scarrucciu opened this issue Jun 25, 2015 · 1 comment · Fixed by #10866
Labels
Compat pandas objects compatability with Numpy or Python functions Period Period data type
Milestone

Comments

@scarrucciu
Copy link

I can pickle dataframes with periods but after import Period format becomes corrupted. After pickle Period stride appears to be none instead of 'Q' in the following example.


import pandas as pd

df = pd.DataFrame({'period': [pd.Period('2014Q4')] * 5})
msk = df['period'] == pd.Period('2014Q4')
print msk

df.to_pickle('tmp.pkl')
df = pd.read_pickle('tmp.pkl')

msk = df['period'] == pd.Period('2014Q4')

Traceback (most recent call last):
  File "/Users/scarrucciu/Downloads/period_pickle_test.py", line 11, in 
    msk = df['period'] == pd.Period('2014Q4')
  File "/Users/scarrucciu/anaconda/lib/python2.7/site-packages/pandas/core/ops.py", line 614, in wrapper
    res = na_op(values, other)
  File "/Users/scarrucciu/anaconda/lib/python2.7/site-packages/pandas/core/ops.py", line 562, in na_op
    result = lib.scalar_compare(x, y, op)
  File "pandas/lib.pyx", line 696, in pandas.lib.scalar_compare (pandas/lib.c:12434)
  File "das/src/period.pyx", line 743, in pandas._period.Period.__richcmp__ (pandas/src/period.c:11121)
ValueError: Cannot compare non-conforming periods

INSTALLED VERSIONS

commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Darwin
OS-release: 13.4.0
machine: x86_64
processor: i386
byteorder: little
pandas: 0.16.2

@jreback
Copy link
Contributor

jreback commented Jun 25, 2015

ATM, Period doesn't have ANY pickle support so not really sure what it is doing.

pull-requests are welcome! (basically need to add __reduce__/__setstate__), see for example pandas/tslib.pyx/Timestamp on how to do it.

Also not tested. PeriodIndex has support for this, but I don't think tested either.

@jreback jreback added Period Period data type Compat pandas objects compatability with Numpy or Python functions labels Jun 25, 2015
@jreback jreback added this to the 0.17.0 milestone Jun 25, 2015
jreback pushed a commit to jreback/pandas that referenced this issue Aug 20, 2015
springcoil pushed a commit to springcoil/pandas that referenced this issue Aug 21, 2015
update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue pandas-dev#10828

TST: pandas-dev#10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes pandas-dev#10564

DOC: typo

A few changes in docs
springcoil pushed a commit to springcoil/pandas that referenced this issue Aug 21, 2015
update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue pandas-dev#10828

TST: pandas-dev#10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes pandas-dev#10564

DOC: typo

ENH: pickle support for Period pandas-dev#10439

update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue pandas-dev#10828

TST: pandas-dev#10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes pandas-dev#10564

DOC: typo

A few changes in docs
springcoil pushed a commit to springcoil/pandas that referenced this issue Aug 22, 2015
update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue pandas-dev#10828

TST: pandas-dev#10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes pandas-dev#10564

DOC: typo

A few changes in docs
springcoil pushed a commit to springcoil/pandas that referenced this issue Aug 22, 2015
update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue pandas-dev#10828

TST: pandas-dev#10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes pandas-dev#10564

DOC: typo

ENH: pickle support for Period pandas-dev#10439

update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue pandas-dev#10828

TST: pandas-dev#10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes pandas-dev#10564

DOC: typo

A few changes in docs
springcoil added a commit to springcoil/pandas that referenced this issue Aug 30, 2015
BUG: 10633 - some last errors removed

ENH: pickle support for Period pandas-dev#10439

update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue pandas-dev#10828

TST: pandas-dev#10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes pandas-dev#10564

DOC: typo

A few changes in docs

TST: Changes in test

ENH: pickle support for Period pandas-dev#10439

update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue pandas-dev#10828

TST: pandas-dev#10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes pandas-dev#10564

DOC: typo

ENH: pickle support for Period pandas-dev#10439

update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue pandas-dev#10828

TST: pandas-dev#10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes pandas-dev#10564

DOC: typo

A few changes in docs

ERR: 10720

BUG: 10633 and 10800 fix

merging

ENH: pickle support for Period pandas-dev#10439

update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue pandas-dev#10828

TST: pandas-dev#10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes pandas-dev#10564

DOC: typo

A few changes in docs

TST: Changes in test

Fixing a slight messup

DOC:Updating consistent imports in the merging.rst file pandas-dev#9886

DOC: GH9886 Part V

DOC: GH9886 Part V - some merging issues
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Compat pandas objects compatability with Numpy or Python functions Period Period data type
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants