Skip to content

Commit

Permalink
[SPARK-26044][WEBUI] Aggregated Metrics table sort based on executor ID
Browse files Browse the repository at this point in the history
## What changes were proposed in this pull request?
Aggregated Metrics table in the stage page is not sorted based on the executorID properly. Because executorID is string and also the logs of the executors are in the same column.
In this PR, I created a new column for executor logs.

## How was this patch tested?
Before patch:
![screenshot from 2018-11-14 02-05-12](https://user-images.githubusercontent.com/23054875/48441529-caa77580-e7b1-11e8-90ea-b16f63438102.png)

After patch:
![screenshot from 2018-11-14 02-05-29](https://user-images.githubusercontent.com/23054875/48441540-d2671a00-e7b1-11e8-9059-890bfe80c961.png)

Closes apache#23024 from shahidki31/AggSort.

Authored-by: Shahid <shahidki31@gmail.com>
Signed-off-by: Sean Owen <sean.owen@databricks.com>
  • Loading branch information
shahidki31 authored and srowen committed Nov 15, 2018
1 parent 91405b3 commit cae5879
Showing 1 changed file with 7 additions and 10 deletions.
17 changes: 7 additions & 10 deletions core/src/main/scala/org/apache/spark/ui/jobs/ExecutorTable.scala
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ private[ui] class ExecutorTable(stage: StageData, store: AppStatusStore) {
Blacklisted
</span>
</th>
<th>Logs</th>
</thead>
<tbody>
{createExecutorTable(stage)}
Expand All @@ -92,16 +93,7 @@ private[ui] class ExecutorTable(stage: StageData, store: AppStatusStore) {
executorSummary.toSeq.sortBy(_._1).map { case (k, v) =>
val executor = store.asOption(store.executorSummary(k))
<tr>
<td>
<div style="float: left">{k}</div>
<div style="float: right">
{
executor.map(_.executorLogs).getOrElse(Map.empty).map {
case (logName, logUrl) => <div><a href={logUrl}>{logName}</a></div>
}
}
</div>
</td>
<td>{k}</td>
<td>{executor.map { e => e.hostPort }.getOrElse("CANNOT FIND ADDRESS")}</td>
<td sorttable_customkey={v.taskTime.toString}>{UIUtils.formatDuration(v.taskTime)}</td>
<td>{v.failedTasks + v.succeededTasks + v.killedTasks}</td>
Expand Down Expand Up @@ -145,6 +137,11 @@ private[ui] class ExecutorTable(stage: StageData, store: AppStatusStore) {
<td>false</td>
}
}
<td> {executor.map(_.executorLogs).getOrElse(Map.empty).map {
case (logName, logUrl) => <div><a href={logUrl}>{logName}</a></div>
}}
</td>

</tr>
}
}
Expand Down

0 comments on commit cae5879

Please sign in to comment.