Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: add environment, e.g. "longtable", to Styler.to_latex #41866

Merged
merged 36 commits into from
Jul 28, 2021

Conversation

attack68
Copy link
Contributor

@attack68 attack68 commented Jun 8, 2021

checks an item off #41649

closes #37443

adds the longtable template, similar to DataFrame.to_latex.

also allows for a custom environment if \begin{table} should be replaced by \begin{figure} etc.

@@ -480,6 +481,10 @@ def to_latex(
the left, centrally, or at the right.
siunitx : bool, default False
Set to ``True`` to structure LaTeX compatible with the {siunitx} package.
environment : str, optional
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure i love this name, maybe template: and this should be 'longtable', 'table' ?

your comment on position_float is hard to interpret here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

environment is the proper LaTeX name for these blocks: LaTeX environments

'longtable' and 'table' are common environments for this but there are others. e.g. see #37443

some arguments are nullified by the use of the 'longtable' environment, such as 'position_float'. can try and rephrase this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe should raise ValueError if invalid combinations of parameters are given

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there was already validation on position_float so this was non-contentious addition.

@jreback jreback added the Code Style Code style, linting, code_checks label Jun 9, 2021
@attack68 attack68 added Styler conditional formatting using DataFrame.style Enhancement IO LaTeX to_latex and removed Code Style Code style, linting, code_checks labels Jun 9, 2021
@jreback
Copy link
Contributor

jreback commented Jun 16, 2021

can you rebase this

attack68 added 2 commits June 16, 2021 09:21
# Conflicts:
#	doc/source/whatsnew/v1.3.0.rst
#	pandas/io/formats/style.py
#	pandas/io/formats/templates/latex.tpl
#	pandas/tests/io/formats/style/test_to_latex.py
@attack68
Copy link
Contributor Author

@jreback rebased and updated.

Copy link
Member

@ivanovmg ivanovmg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To me it seems like a good extension!
Please find my comments below.

pandas/tests/io/formats/style/test_to_latex.py Outdated Show resolved Hide resolved
pandas/tests/io/formats/style/test_to_latex.py Outdated Show resolved Hide resolved
pandas/tests/io/formats/style/test_to_latex.py Outdated Show resolved Hide resolved
pandas/tests/io/formats/style/test_to_latex.py Outdated Show resolved Hide resolved
pandas/tests/io/formats/style/test_to_latex.py Outdated Show resolved Hide resolved
@@ -484,8 +484,160 @@ def test_parse_latex_css_conversion(css, expected):
assert result == expected


@pytest.mark.parametrize("environment", ["tabular", "longtable"])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tabular or table?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"tabular" is correct, this is a quirk of the structure of longtable to parametrize the test.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, but should it work with table as well?

Copy link
Contributor Author

@attack68 attack68 Jun 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no because {tabular} is a sub environment of any other environment including {table}, except {longtable}, i.e. we have:

\begin{table}
\begin{tabular}
...           <<-- CSS conversion only done here
\end{tabular}
\end{table}

this tests the inner environment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case I think it is necessary to add one end-to-end test for table with inner tabular.
For now this particular test function test_parse_latex_css_convert_minimal tests only a portion of the output (end of inner environment).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't necessary. There is already test_comprehensive which performs a full output check.

This test is designed to selectively test only the relevant parts of the template(s) which convert css to latex styles, which only occurs in data-cells. Since there are two templates, where the inner environment is defined as either longtable or tabular environment, the test covers both cases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but in test_comprehensive you do not test environment kwarg.

My point is as follows.
You have a good test for environment='longtable', which compares the full output.
Meanwhile for environment='tabular' there is only a check for the final portion of the output.
How can we ensure that some future changes would not break the upper part of the output when environment='tabular'?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I can see some confusion has arisen here, since environment="tabular" should never be used by a user. It would result in an outer and inner {tabular} environment erroneously. I have used it here as a shortcut to not needing 2 variables. However, I have now amended this test to avoid the confusion.

But with respect to broader testing I think cases are covered through dependencies, for example:

  • test_minimal_latex_tabular: tests just the core {tabular} structure without additions.
  • test_longtable_minimal: tests just the core {longtable} structure without additions.
  • test_comprehensive: tests the core outer {table} and inner {tabular} structure.
  • test_longtable_comprehensive: tests the core {longtable} structure with features.
  • test_latex_environment: checks {table} is properly substituted for another value in the outer structure.

However, reflecting on your comments, I have removed test_latex_enviroment and have, instead, incorporated it now into test_comprehensive as parameters, which I think you will prefer, and also this increased the power of the test.

pandas/tests/io/formats/style/test_to_latex.py Outdated Show resolved Hide resolved
pandas/tests/io/formats/style/test_to_latex.py Outdated Show resolved Hide resolved
@attack68 attack68 requested a review from ivanovmg June 24, 2021 06:41
Copy link
Member

@ivanovmg ivanovmg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comments.
For some reason mypy fails.
I guess it is not because of your changes...

pandas/tests/io/formats/style/test_to_latex.py Outdated Show resolved Hide resolved
pandas/tests/io/formats/style/test_to_latex.py Outdated Show resolved Hide resolved
@@ -484,8 +484,160 @@ def test_parse_latex_css_conversion(css, expected):
assert result == expected


@pytest.mark.parametrize("environment", ["tabular", "longtable"])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, but should it work with table as well?

@attack68 attack68 requested a review from ivanovmg June 30, 2021 07:46
@ivanovmg ivanovmg requested a review from jreback June 30, 2021 09:44
@@ -367,8 +368,8 @@ def test_comprehensive(df):
\\end{tabular}
\\end{table}
"""
)
assert s.format(precision=2).to_latex() == expected
).replace("table", environment if environment else "table")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it not be clearer if you used an f-string here with environment as a placeholder, while

if environment is None:
    environment = "table"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could either:

  • use an f-string and need to replace every curly brace with a double to avoid iterpretation as a var
  • break the dedent into multiple parts with f-string only around {table} to avoid the above
  • use replace at end

I subjectively chose the latter, which I felt was clearest - should not be a blocker.

@attack68
Copy link
Contributor Author

@jreback can this be merged as is? I've addressed almost all comments, with the one subjective suggestion ignored due to preference.

@ivanovmg thanks for review - I think after the iterations the tests are better and clearer.

@attack68 attack68 added this to the 1.4 milestone Jul 27, 2021
@@ -483,6 +486,12 @@ def to_latex(
the left, centrally, or at the right.
siunitx : bool, default False
Set to ``True`` to structure LaTeX compatible with the {siunitx} package.
environment : str, optional
If given, the environment that will replace 'table' in ``\\begin{table}``.
If 'longtable' is specified then a more suitable template is
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small comment here. can you list the valid values? (or provide a link to them). can be a followon.

Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm small comment for followup.

@jreback jreback merged commit 6aee8e7 into pandas-dev:master Jul 28, 2021
@attack68 attack68 deleted the longtable_to_latex branch July 28, 2021 17:32
CGe0516 pushed a commit to CGe0516/pandas that referenced this pull request Jul 29, 2021
feefladder pushed a commit to feefladder/pandas that referenced this pull request Sep 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement IO LaTeX to_latex Styler conditional formatting using DataFrame.style
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ENH: to_latex(): add posibility to define the latex environment (instead of table)
4 participants