Skip to content

Commit

Permalink
[SPARK-50667][PYTHON][TESTS] Make jinja2 optional in PySpark Tests
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?
Make `jinja2` optional in PySpark Tests

### Why are the changes needed?
`jinja2` is an optional dependency of `pandas`

https://pypi.org/pypi/pandas/2.2.0/json

```
'jinja2>=3.1.2; extra == "output-formatting"'
```

It is not a mandatory requirement of pyspark, so PySpark tests should succeed even it is not installed

### Does this PR introduce _any_ user-facing change?
no, test-only

### How was this patch tested?
manually test after uninstalling it

### Was this patch authored or co-authored using generative AI tooling?
no

Closes apache#49288 from zhengruifeng/optional_jinja2.

Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
  • Loading branch information
zhengruifeng authored and HyukjinKwon committed Dec 26, 2024
1 parent 2475b35 commit 5c075c3
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 1 deletion.
2 changes: 1 addition & 1 deletion python/pyspark/pandas/frame.py
Original file line number Diff line number Diff line change
Expand Up @@ -2632,7 +2632,7 @@ def to_latex(
... 'mask': ['red', 'purple'],
... 'weapon': ['sai', 'bo staff']},
... columns=['name', 'mask', 'weapon'])
>>> print(df.to_latex(index=False)) # doctest: +NORMALIZE_WHITESPACE
>>> print(df.to_latex(index=False)) # doctest: +SKIP
\begin{tabular}{lll}
\toprule
name & mask & weapon \\
Expand Down
2 changes: 2 additions & 0 deletions python/pyspark/pandas/tests/io/test_dataframe_conversion.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
from pyspark import pandas as ps
from pyspark.testing.pandasutils import PandasOnSparkTestCase, TestUtils
from pyspark.testing.sqlutils import SQLTestUtils
from pyspark.testing.utils import have_jinja2, jinja2_requirement_message


class DataFrameConversionMixin:
Expand Down Expand Up @@ -199,6 +200,7 @@ def test_to_clipboard(self):
psdf.to_clipboard(sep=";", index=False), pdf.to_clipboard(sep=";", index=False)
)

@unittest.skipIf(not have_jinja2, jinja2_requirement_message)
def test_to_latex(self):
pdf = self.pdf
psdf = self.psdf
Expand Down
3 changes: 3 additions & 0 deletions python/pyspark/testing/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,9 @@ def have_package(name: str) -> bool:
have_flameprof = have_package("flameprof")
flameprof_requirement_message = None if have_flameprof else "No module named 'flameprof'"

have_jinja2 = have_package("jinja2")
jinja2_requirement_message = None if have_jinja2 else "No module named 'jinja2'"

pandas_requirement_message = None
try:
from pyspark.sql.pandas.utils import require_minimum_pandas_version
Expand Down

0 comments on commit 5c075c3

Please sign in to comment.