Fixed Value Error when doing HDFStore.Select of contiguous mixed-data #17670

amolkahat · 2017-09-25T18:50:38Z

Fixed Value Error when doing HDFStore.Select of contiguous mixed-data table ft. VLArray

Closes 17021

Signed-off-by: Amol Kahat akahat@redhat.com

closes BUG: ValueError when doing HDFStore.Select of contiguous mixed-data table ft. VLArray #17021
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

jreback

looks pretty good.

jreback · 2017-09-25T23:09:31Z

doc/source/whatsnew/v0.21.0.txt

@@ -580,6 +580,7 @@ I/O
 - Bug in :func:`read_html` where import check fails when run in multiple threads (:issue:`16928`)
 - Bug in :func:`read_csv` where automatic delimiter detection caused a ``TypeError`` to be thrown when a bad line was encountered rather than the correct error message (:issue:`13374`)
 - Bug in ``DataFrame.to_html()`` with ``notebook=True`` where DataFrames with named indices or non-MultiIndex indices had undesired horizontal or vertical alignment for column or row labels, respectively (:issue:`16792`)
+- Bug in :func:`HDFStore.select` ValueError when doing HDFStore.Select of contiguous mixed-data table ft. VLArray (:issue:`17021`)


you have an extra ValueError here.

Bug in :func:`HDFStore.selct` when reading a contiguous mixed-data .....

jreback · 2017-09-25T23:10:11Z

pandas/tests/io/test_pytables.py

@@ -5678,3 +5678,14 @@ def test_dst_transitions(self):
                store.append('df', df)
                result = store.select('df')
                assert_frame_equal(result, df)
+
+    def test_contiguous_mixed_data_table(self):


can you also add a case that uses start & stop that are not the len

move this near the other selecting tests

pep8speaks · 2017-09-26T12:22:57Z

Hello @amolkahat! Thanks for updating the PR.

Cheers ! There are no PEP8 issues in this Pull Request. 🍻

Comment last updated on September 27, 2017 at 12:09 Hours UTC

jreback · 2017-09-26T12:46:04Z

pandas/tests/io/test_pytables.py

+                        'b': Series(['ab', 'cd', 'ab'])})
+        df.to_hdf('test.h5', 'test_dataset')
+
+        with pd.HDFStore('test.h5') as fd:


pls use the existing style, IOW ensure_clean_store

jreback

looks good as from some style comments.

codecov · 2017-09-27T09:26:02Z

Codecov Report

Merging #17670 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #17670      +/-   ##
==========================================
+ Coverage   91.24%   91.25%   +<.01%     
==========================================
  Files         163      163              
  Lines       49762    49761       -1     
==========================================
+ Hits        45406    45408       +2     
+ Misses       4356     4353       -3

Flag	Coverage Δ
#multiple	`89.04% <100%> (+0.02%)`	⬆️
#single	`40.33% <100%> (-0.07%)`	⬇️

Impacted Files	Coverage Δ
pandas/io/pytables.py	`92.79% <100%> (-0.01%)`	⬇️
pandas/io/gbq.py	`25% <0%> (-58.34%)`	⬇️
pandas/core/frame.py	`97.73% <0%> (-0.1%)`	⬇️
pandas/plotting/_converter.py	`65.2% <0%> (+1.81%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d3be81a...2ed2da9. Read the comment docs.

codecov · 2017-09-27T09:26:21Z

Codecov Report

Merging #17670 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #17670      +/-   ##
==========================================
+ Coverage   91.24%   91.25%   +<.01%     
==========================================
  Files         163      163              
  Lines       49762    49761       -1     
==========================================
+ Hits        45406    45408       +2     
+ Misses       4356     4353       -3

Flag	Coverage Δ
#multiple	`89.04% <100%> (+0.02%)`	⬆️
#single	`40.33% <100%> (-0.07%)`	⬇️

Impacted Files	Coverage Δ
pandas/io/pytables.py	`92.79% <100%> (-0.01%)`	⬇️
pandas/io/gbq.py	`25% <0%> (-58.34%)`	⬇️
pandas/core/frame.py	`97.73% <0%> (-0.1%)`	⬇️
pandas/plotting/_converter.py	`65.2% <0%> (+1.81%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d3be81a...2ed2da9. Read the comment docs.

codecov · 2017-09-27T09:26:42Z

Codecov Report

Merging #17670 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #17670      +/-   ##
==========================================
+ Coverage   91.24%   91.25%   +<.01%     
==========================================
  Files         163      163              
  Lines       49762    49761       -1     
==========================================
+ Hits        45406    45408       +2     
+ Misses       4356     4353       -3

Flag	Coverage Δ
#multiple	`89.04% <100%> (+0.02%)`	⬆️
#single	`40.34% <100%> (-0.07%)`	⬇️

Impacted Files	Coverage Δ
pandas/io/pytables.py	`92.79% <100%> (-0.01%)`	⬇️
pandas/io/gbq.py	`25% <0%> (-58.34%)`	⬇️
pandas/core/frame.py	`97.73% <0%> (-0.1%)`	⬇️
pandas/plotting/_converter.py	`65.2% <0%> (+1.81%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 44747c8...4861f63. Read the comment docs.

jreback · 2017-09-27T11:11:55Z

doc/source/whatsnew/v0.21.0.txt

@@ -580,6 +580,7 @@ I/O
 - Bug in :func:`read_html` where import check fails when run in multiple threads (:issue:`16928`)
 - Bug in :func:`read_csv` where automatic delimiter detection caused a ``TypeError`` to be thrown when a bad line was encountered rather than the correct error message (:issue:`13374`)
 - Bug in ``DataFrame.to_html()`` with ``notebook=True`` where DataFrames with named indices or non-MultiIndex indices had undesired horizontal or vertical alignment for column or row labels, respectively (:issue:`16792`)
+- Bug in :func:`HDFStore.select` when reading a contiguous mixed-data table ft. VLArray (:issue:`17021`)


can you expand ft.

jreback · 2017-09-27T11:13:04Z

pandas/tests/io/test_pytables.py

@@ -4387,6 +4387,24 @@ def test_path_pathlib(self):
            lambda p: pd.read_hdf(p, 'df'))
        tm.assert_frame_equal(df, result)

+    def test_contiguous_mixed_data_table(self):


can parametrize this for start/stop, also include start=None, stop=None

jreback

minor comments, ping on green.

table featuring VLArray Closes 17021 Signed-off-by: Amol Kahat <akahat@redhat.com>

jreback · 2017-09-28T14:12:53Z

thanks!

…pandas-dev#17670)

jreback added Bug IO HDF5 read_hdf, HDFStore labels Sep 25, 2017

jreback requested changes Sep 25, 2017

View reviewed changes

jreback reviewed Sep 26, 2017

View reviewed changes

jreback requested changes Sep 26, 2017

View reviewed changes

jreback reviewed Sep 27, 2017

View reviewed changes

jreback requested changes Sep 27, 2017

View reviewed changes

jreback added this to the 0.21.0 milestone Sep 27, 2017

Fixed Value Error when doing HDFStore.Select of contiguous mixed-data

4861f63

table featuring VLArray Closes 17021 Signed-off-by: Amol Kahat <akahat@redhat.com>

jreback approved these changes Sep 28, 2017

View reviewed changes

jreback merged commit 074b485 into pandas-dev:master Sep 28, 2017

alanbato pushed a commit to alanbato/pandas that referenced this pull request Nov 10, 2017

Fixed Value Error when doing HDFStore.Select of contiguous mixed-data (…

32338e4

…pandas-dev#17670)

No-Stream pushed a commit to No-Stream/pandas that referenced this pull request Nov 28, 2017

Fixed Value Error when doing HDFStore.Select of contiguous mixed-data (…

d53673a

…pandas-dev#17670)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixed Value Error when doing HDFStore.Select of contiguous mixed-data #17670

Fixed Value Error when doing HDFStore.Select of contiguous mixed-data #17670

amolkahat commented Sep 25, 2017

jreback left a comment

jreback Sep 25, 2017

jreback Sep 25, 2017

jreback Sep 25, 2017

pep8speaks commented Sep 26, 2017 •

edited

Loading

jreback Sep 26, 2017

jreback left a comment

codecov bot commented Sep 27, 2017 •

edited

Loading

codecov bot commented Sep 27, 2017 •

edited

Loading

codecov bot commented Sep 27, 2017 •

edited

Loading

jreback Sep 27, 2017

jreback Sep 27, 2017

jreback left a comment

jreback commented Sep 28, 2017

Fixed Value Error when doing HDFStore.Select of contiguous mixed-data #17670

Fixed Value Error when doing HDFStore.Select of contiguous mixed-data #17670

Conversation

amolkahat commented Sep 25, 2017

jreback left a comment

Choose a reason for hiding this comment

jreback Sep 25, 2017

Choose a reason for hiding this comment

jreback Sep 25, 2017

Choose a reason for hiding this comment

jreback Sep 25, 2017

Choose a reason for hiding this comment

pep8speaks commented Sep 26, 2017 • edited Loading

Comment last updated on September 27, 2017 at 12:09 Hours UTC

jreback Sep 26, 2017

Choose a reason for hiding this comment

jreback left a comment

Choose a reason for hiding this comment

codecov bot commented Sep 27, 2017 • edited Loading

Codecov Report

codecov bot commented Sep 27, 2017 • edited Loading

Codecov Report

codecov bot commented Sep 27, 2017 • edited Loading

Codecov Report

jreback Sep 27, 2017

Choose a reason for hiding this comment

jreback Sep 27, 2017

Choose a reason for hiding this comment

jreback left a comment

Choose a reason for hiding this comment

jreback commented Sep 28, 2017

pep8speaks commented Sep 26, 2017 •

edited

Loading

codecov bot commented Sep 27, 2017 •

edited

Loading

codecov bot commented Sep 27, 2017 •

edited

Loading

codecov bot commented Sep 27, 2017 •

edited

Loading