Skip to content

Commit

Permalink
[MINOR][DOCS] Clarify spark.remote and spark.master in pyspark-connec…
Browse files Browse the repository at this point in the history
…t and pyspark-client at install.rst

### What changes were proposed in this pull request?

This PR fixes the installation page for PySpark to clarify spark.remote and spark.master in pyspark-connect and pyspark-client at install.rst

### Why are the changes needed?

To clarify spark.remote and spark.master in pyspark-connect and pyspark-client

### Does this PR introduce _any_ user-facing change?

No. doc-only change.

### How was this patch tested?

CI build in this PR

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#50054 from HyukjinKwon/minor-doc-change1.

Authored-by: Hyukjin Kwon <gurwls223@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
  • Loading branch information
HyukjinKwon committed Feb 24, 2025
1 parent f1d78dc commit c6097c7
Showing 1 changed file with 2 additions and 4 deletions.
6 changes: 2 additions & 4 deletions python/docs/source/getting_started/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -96,9 +96,7 @@ If you want to make Spark Connect default, you can install and additional librar
It will automatically install ``pyspark`` library as well as dependencies that are necessary for Spark Connect.
If you want to customize ``pyspark``, you need to install ``pyspark`` with the instructions above in advance.

Note that ``pyspark`` command will use ``--master`` option for Spark Connect remote URL instead of ``--remote`` option.

See also `Quickstart: Spark Connect <quickstart_connect.html>`_ for how to use it.
This package supports both ``spark.master`` (``--master``) with a locally running Spark Connect server, and ``spark.remote`` (``--remote``) including local clusters, e.g., ``local[*]`` as well as connection URIs such as ``sc://localhost``. See also `Quickstart: Spark Connect <quickstart_connect.html>`_ for how to use it.


Python Spark Connect Client
Expand All @@ -111,7 +109,7 @@ To install the Python Spark Connect client via `PyPI (pyspark-client) <https://p
pip install pyspark-client
See also `Quickstart: Spark Connect <quickstart_connect.html>`_ for how to use it.
This package only supports `spark.remote` with connection URIs, e.g., ``sc://localhost``. See also `Quickstart: Spark Connect <quickstart_connect.html>`_ for how to use it.


Using Conda
Expand Down

0 comments on commit c6097c7

Please sign in to comment.