max_rows seems to have max value of 60

[hayd]

Max_rows seems to have a maximum value of 60 (in 0.11 and dev, this works as expected in 0.10.1):

In [1]: %paste
from pandas import *
n = 100
foo = DataFrame(index=range(n))
foo['floats'] = np.random.randn(n)

set_option('display.max_rows', 101)

## -- End pasted text --

In [2]: foo.head(100)
Out[2]:
<class 'pandas.core.frame.DataFrame'>
Int64Index: 100 entries, 0 to 99
Columns: 1 entries, floats to floats
dtypes: float64(1)

In [3]: foo.head(61)
Out[3]:
<class 'pandas.core.frame.DataFrame'>
Int64Index: 61 entries, 0 to 60
Data columns (total 1 columns):
floats    61  non-null values
dtypes: float64(1)

In [4]: foo.head(60)
Out[4]:
      floats
0  -0.631717
1   0.231468
2  -0.465971
3  -1.427025
...

In [5]: %paste
set_option('display.max_rows', 10)

## -- End pasted text --

In [6]: foo.head(10)
Out[6]:
     floats
0 -0.194574
1 -0.130100
2  0.547474
3  0.669904
4  0.776946
5 -2.384084
6 -0.011147
7 -0.272660
8 -0.051326
9  1.757020

In [7]: foo.head(11)
Out[7]:
<class 'pandas.core.frame.DataFrame'>
Int64Index: 11 entries, 0 to 10
Data columns (total 1 columns):
floats    11  non-null values
dtypes: float64(1)

In [8]: version.version # same true for 0.11
Out[8]: '0.12.0.dev-d925966'

Essentially from http://stackoverflow.com/questions/16424493/pandas-setting-no-of-max-rows.

Related: #3395, #3426.

[lodagro]

There are two limits that apply display.heigth and display.max_rows.

[cpcloud]

@hayd do u want max_rows to override height?

[ghost]

The display logic has bells and it has whistles and admittedly it's not very simple anymore.

Basically, max_rows determines whether we need to somehow reformat the output
to look good on the screen, and height dictates the dimensions for which we are formatting.

I understand it's not as intuitive as it one might hope, but aside from better docs, I don't think
this can be altered.

marked as usage+docs, and we should add something to the documentation explaining this.

[hayd]

@cpcloud I think there is a case for that, but provided it is clear in the docs (i.e. is mentioned explicitly in the max_rows part of the set_options docstring) I can live with it. :)

[hayd]

http://stackoverflow.com/questions/16652651/pandas-df-head-error

I think it should overwrite the display.height...

[hayd]

maybe there should be a UserWarning saying if max_rows is less than display_height?

[ghost]

Pretty sure it's the default value of display.width that's at play in that SO question.

@hayd, could you please test out #3657, and ask the SO question owner to do the same?
I'd like to know this solves his problem, and that he knows we care.

[ghost]

pushed 88d892a, can we close this?

[hayd]

@y-p thanks for putting these together, the SO OP said he't test in the morning and seemed happy :)

I'm still a but sulky from the example at the top of this thread, but happy to close this is everyone else is ok with it.

[ghost]

Oh no! a sulky pandas user.

You have a fair point. There are now two options controlling roughly
the same thing, height was supposed to control the formatting of expand_repr
only. Need to rework the display logic again. sigh

[hayd]

Sorry! ...I am concerned about the kittens, honest.

[ghost]

If #3663 sees gets enough testing, I'll merge it for 0.11.1.

[agconti]

Hi guys,

I'm the SO user from the above post. I believe I tested y-p's update correctly, I did so by forking y-p's pandas repo and then physically overwriting the appropriate files with his commits, but it has not solved my problem. I can link my IPython notebook if it helps. Please let me know I did not test out the fix correctly. Thanks for looking into the issue for me!

[ghost]

You shouldn't need to manually overwrite the files.
try:

git clone http://github.com/y-p/pandas
git co GH3541 # checkout the PR branch
sudo python setup.py install
ipython notebook

then, in ipnb, try running:

from pandas.util.testing import makeCustomDataframe as mkdf
mkdf(10,20) # generates a dataframe with 10 rows, and 20 cols

you should see HTML output, probbaly wider then the browser window.

IIUC, that's what you were after.

If you want to roll back, reinstall pandas in whatever way you prefer.
You can just checkout a version into a new branch and install that:

git checkout v0.11.0 -b v0.11.0 # or whatever version you prefer
sudo python setup.py install

[agconti]

Hi y-p,

Thank you for the walk-through on getting the pull request. I ran into a hitch though: I tried your steps several times but ran into this error when running the final code in an ipnb.

Code:

from pandas.util.testing import makeCustomDataframe as mkdf
mkdf(10,20) # generates a dataframe with 10 rows, and 20 cols

Error:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-a622f6364bf3> in <module>()
----> 1 from pandas.util.testing import makeCustomDataframe as mkdf
      2 mkdf(10,20) # generates a dataframe with 10 rows, and 20 cols

C:\Users\Andrew\Documents\Kaggle\pandas_git\pandas\pandas\__init__.py in <module>()
      4 
      5 try:
----> 6     from . import hashtable, tslib, lib
      7 except Exception:  # pragma: no cover
      8     import sys

ImportError: cannot import name hashtable

cannot import name hashtable

I also received this error while running setup.py :

running build_ext
skipping 'pandas\index.c' Cython extension (up-to-date)
building 'pandas.index' extension
error: Unable to find vcvarsall.bat

At any rate, I was not able to verify the html output. Any tips on how I might successfully test your fix?

Thanks for your help and your time.

[ghost]

Oh, you're on windows. that does complicate matters, It can be tricky to setup a
windows build env. I understand now why You tried to simply overwrite the python files.
If you're using a 0.11.0 binray though, the results are unpredictable, since the
python may be out of sync with the compiled portions.

At least one other user besides me reported that the fix work, I plan to merge it for 0.11.1
which should be released soon.

As an interim solution, I think you can work around the problem by setting display.width to a larger value.

[agconti]

Thanks y-p. I'll try playing with display.width until 0.11.1 comes out; thanks for the help in the meantime.

[wesm]

Merged in #3663 so closing this. thanks guys