Skip to content

Commit

Permalink
[SPARK-49007][CORE] Improve MasterPage to support custom title
Browse files Browse the repository at this point in the history
This PR aims to improve `MasterPage` to support custom title.

When there exists multiple Spark clusters, custom title can be more helpful than the spark master address because it can contain semantics like the role of the clusters. In addition, the URL field in the same page already provides the spark master information even when we use a custom title.

**BEFORE**
```
sbin/start-master.sh
```
![Screenshot 2024-07-25 at 14 01 11](https://github.com/user-attachments/assets/7055d700-4bd6-4785-a535-2f8ce6dba47d)

**AFTER**
```
SPARK_MASTER_OPTS='-Dspark.master.ui.title="Projext X Staging Cluster"' sbin/start-master.sh
```
![Screenshot 2024-07-25 at 14 05 38](https://github.com/user-attachments/assets/f7e45fd6-fa2b-4547-ae39-1403b1e910d9)

Pass the CIs with newly added test case.

No.

Closes apache#47491 from dongjoon-hyun/SPARK-49007.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
  • Loading branch information
dongjoon-hyun committed Jul 26, 2024
1 parent dc926a4 commit 7fab371
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,13 @@ import org.apache.spark.deploy.DeployMessages.{KillDriverResponse, MasterStateRe
import org.apache.spark.deploy.JsonProtocol
import org.apache.spark.deploy.StandaloneResourceUtils._
import org.apache.spark.deploy.master._
import org.apache.spark.internal.config.UI.MASTER_UI_TITLE
import org.apache.spark.ui.{UIUtils, WebUIPage}
import org.apache.spark.util.Utils

private[ui] class MasterPage(parent: MasterWebUI) extends WebUIPage("") {
private val master = parent.masterEndpointRef
private val title = parent.master.conf.get(MASTER_UI_TITLE)
private val jsonFieldPattern = "/json/([a-zA-Z]+).*".r

def getMasterState: MasterStateResponse = {
Expand Down Expand Up @@ -267,7 +269,7 @@ private[ui] class MasterPage(parent: MasterWebUI) extends WebUIPage("") {
}
</div>;

UIUtils.basicSparkPage(request, content, "Spark Master at " + state.uri)
UIUtils.basicSparkPage(request, content, title.getOrElse("Spark Master at " + state.uri))
}

private def workerRow(showResourceColumn: Boolean): WorkerInfo => Seq[Node] = worker => {
Expand Down
7 changes: 7 additions & 0 deletions core/src/main/scala/org/apache/spark/internal/config/UI.scala
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,13 @@ private[spark] object UI {
.checkValues(Set("ALLOW", "LOCAL", "DENY"))
.createWithDefault("LOCAL")

val MASTER_UI_TITLE = ConfigBuilder("spark.master.ui.title")
.version("4.0.0")
.doc("Specifies the title of the Master UI page. If unset, `Spark Master at <MasterURL>` " +
"is used by default.")
.stringConf
.createOptional

val UI_SQL_GROUP_SUB_EXECUTION_ENABLED = ConfigBuilder("spark.ui.groupSQLSubExecutionEnabled")
.doc("Whether to group sub executions together in SQL UI when they belong to the same " +
"root execution")
Expand Down

0 comments on commit 7fab371

Please sign in to comment.