[SPARK-15591] [WEBUI] Paginate Stage Table in Stages tab #13708

nblintao · 2016-06-16T17:05:41Z

What changes were proposed in this pull request?

This patch adds pagination support for the Stage Tables in the Stage tab. Pagination is provided for all of the four Job Tables (active, pending, completed, and failed). Besides, the paged stage tables are also used in JobPage (the detail page for one job) and PoolPage.

Interactions (jumping, sorting, and setting page size) for paged tables are also included.

How was this patch tested?

Tested manually by using checking the Web UI after completing and failing hundreds of jobs. Same as the testings for Paginate Job Table in Jobs tab.

This shows the pagination for completed stages:

SparkQA · 2016-06-16T17:08:44Z

Test build #60645 has finished for PR 13708 at commit 4255c09.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-06-16T19:29:03Z

Test build #60646 has finished for PR 13708 at commit 4de6d68.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

ajbozarth · 2016-06-16T23:26:33Z

I haven't been able to look at the code yet but I checked it out and ran it and found a bug. When you change one table (completed or failed) it resets the other table. I'll also check if this happens on your jobs page pr as well

nblintao · 2016-06-17T01:40:19Z

@ajbozarth Thanks for pointing it out. It is true for both the stages page and the job page. It doesn't keep track of the status of other tables when one table is changed.

I will try to fix it recently.

nblintao · 2016-06-21T18:24:13Z

@ajbozarth Could you please try again to see whether or not have my new commit fixed the problem you mentioned above?
Also, regarding the bug you mentioned in #13620, are the stage tables working properly in the history servers?
Thanks!

ajbozarth · 2016-06-21T18:28:01Z

Thanks, I'll check it out when I have a chance today. And I'll double check the history server, but I'm pretty sure it was both.

SparkQA · 2016-06-21T20:32:00Z

Test build #60945 has finished for PR 13708 at commit 246432d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-06-22T16:47:40Z

Test build #61046 has started for PR 13708 at commit 7ccda86.

…ol page

nblintao · 2016-06-22T18:50:29Z

I have fixed some linking bugs in stage tables (including the history server problem reported by @ajbozarth ). I have also prevented displaying empty tables in pool pages.
I have tested the links for the AllStagesPage and the JobPage in both the normal UI and the history UI, and tested the PoolPage in the normal UI.
Please have a review of my code and check if it works properly. I will fix the corresponding bugs in #13620 after this PR is merged.

SparkQA · 2016-06-22T20:25:46Z

Test build #61058 has finished for PR 13708 at commit 4af0630.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

ajbozarth · 2016-06-22T21:14:44Z

I finally had a chance to checkout and test your updates and it looks great on both Web UI and History server (I did't check Pool Page because I've never used it). One small thing though, how are the >> and << arrows supposed to work? Because my assumption would be that they go to the first or last page overall, but you have them implemented to skip 10 pages forward or backward. Just wondering if this was your intent and if so why?

nblintao · 2016-06-23T01:24:22Z

@ajbozarth Thanks. That was implemented in the pageNavigation of the PagedTable by @zsxwing from PR #7399. I didn't change it, and it seems useful to me. It works like a pagination for the page navigator itself, since the navigator displays 10 links at once.

nblintao · 2016-06-23T01:30:51Z

Jenkins, retest this please

ajbozarth · 2016-06-23T02:15:39Z

@nblintao thanks for that PR link, that had a great explanation for why he decided it should work that way.

I haven't looked at the code yet, but from a UI perspective this is great.

SparkQA · 2016-06-23T03:28:58Z

Test build #61086 has finished for PR 13708 at commit 4af0630.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

zsxwing · 2016-07-05T17:32:19Z

core/src/main/scala/org/apache/spark/ui/jobs/AllStagesPage.scala

-          parent.basePath, parent.progressListener, isFairScheduler = parent.isFairScheduler,
-          killEnabled = false)
+        new StageTableBase(request, pendingStages, "pendingStage", parent.basePath, subPath,
+          parent.progressListener, parent.isFairScheduler, false, false)


nit: could you add the parameter names for falses back to improve the readability?

zsxwing · 2016-07-05T18:04:17Z

@nblintao thanks for your contribution. Looks pretty good except some nits. Could you fix them, please?

SparkQA · 2016-07-06T07:13:11Z

Test build #61817 has finished for PR 13708 at commit 3a98c4d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

nblintao · 2016-07-06T07:58:37Z

Thanks, @zsxwing . I have fixed them in the commit above.
P.S.:
My computer's time setting was incorrect when it is committed, which lead to the weird order on the GitHub page of this PR (the commit appear below the test and this message). To be clear, the commit I am saying is 3a98c4d.

zsxwing · 2016-07-06T17:27:30Z

LGTM. @nblintao Thanks. Merging to master.

## What changes were proposed in this pull request? This issue was reported by wangyum. In the AllJobsPage, JobPage and StagePage, the description length was limited before like as follows. ![ui-2 0 0](https://cloud.githubusercontent.com/assets/4736016/21319673/8b225246-c651-11e6-9041-4fcdd04f4dec.gif) But recently, the limitation seems to have been accidentally removed. ![ui-2 1 0](https://cloud.githubusercontent.com/assets/4736016/21319825/104779f6-c652-11e6-8bfa-dfd800396352.gif) The cause is that some tables are no longer `sortable` class although they were, and `sortable` class does not only mark tables as sortable but also limited the width of their child `td` elements. The reason why now some tables are not `sortable` class is because another sortable mechanism was introduced by #13620 and #13708 with pagination feature. To fix this issue, I've introduced new class `table-cell-width-limited` which limits the description cell width and the description is like what it was. <img width="1260" alt="2016-12-20 1 00 34" src="https://cloud.githubusercontent.com/assets/4736016/21320478/89141c7a-c654-11e6-8494-f8f91325980b.png"> ## How was this patch tested? Tested manually with my browser. Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp> Closes #16338 from sarutak/SPARK-18837.

## What changes were proposed in this pull request? This issue was reported by wangyum. In the AllJobsPage, JobPage and StagePage, the description length was limited before like as follows. ![ui-2 0 0](https://cloud.githubusercontent.com/assets/4736016/21319673/8b225246-c651-11e6-9041-4fcdd04f4dec.gif) But recently, the limitation seems to have been accidentally removed. ![ui-2 1 0](https://cloud.githubusercontent.com/assets/4736016/21319825/104779f6-c652-11e6-8bfa-dfd800396352.gif) The cause is that some tables are no longer `sortable` class although they were, and `sortable` class does not only mark tables as sortable but also limited the width of their child `td` elements. The reason why now some tables are not `sortable` class is because another sortable mechanism was introduced by #13620 and #13708 with pagination feature. To fix this issue, I've introduced new class `table-cell-width-limited` which limits the description cell width and the description is like what it was. <img width="1260" alt="2016-12-20 1 00 34" src="https://cloud.githubusercontent.com/assets/4736016/21320478/89141c7a-c654-11e6-8494-f8f91325980b.png"> ## How was this patch tested? Tested manually with my browser. Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp> Closes #16338 from sarutak/SPARK-18837. (cherry picked from commit f2ceb2a) Signed-off-by: Sean Owen <sowen@cloudera.com>

## What changes were proposed in this pull request? This issue was reported by wangyum. In the AllJobsPage, JobPage and StagePage, the description length was limited before like as follows. ![ui-2 0 0](https://cloud.githubusercontent.com/assets/4736016/21319673/8b225246-c651-11e6-9041-4fcdd04f4dec.gif) But recently, the limitation seems to have been accidentally removed. ![ui-2 1 0](https://cloud.githubusercontent.com/assets/4736016/21319825/104779f6-c652-11e6-8bfa-dfd800396352.gif) The cause is that some tables are no longer `sortable` class although they were, and `sortable` class does not only mark tables as sortable but also limited the width of their child `td` elements. The reason why now some tables are not `sortable` class is because another sortable mechanism was introduced by apache#13620 and apache#13708 with pagination feature. To fix this issue, I've introduced new class `table-cell-width-limited` which limits the description cell width and the description is like what it was. <img width="1260" alt="2016-12-20 1 00 34" src="https://cloud.githubusercontent.com/assets/4736016/21320478/89141c7a-c654-11e6-8494-f8f91325980b.png"> ## How was this patch tested? Tested manually with my browser. Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp> Closes apache#16338 from sarutak/SPARK-18837.

### What changes were proposed in this pull request? SPARK-15591(#13708) introduced the `MissingStageTableRowData`, but it is no longer used after SPARK-20648(#19698), so this PR removes it. ### Why are the changes needed? Clean up unused code. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass GitHub Actions ### Was this patch authored or co-authored using generative AI tooling? No Closes #43748 from LuciferYang/SPARK-45875. Authored-by: yangjie01 <yangjie01@baidu.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>

paginate stage table in stages tab

4255c09

add newline at end of AllStagesPage

4de6d68

fix problem of resitting other tables

246432d

fix the links to one-stage pages

7ccda86

fix linking problems in history page & not display empty tables in po…

4af0630

…ol page

nblintao mentioned this pull request Jun 23, 2016

[SPARK-15590] [WEBUI] Paginate Job Table in Jobs tab #13620

Closed

zsxwing reviewed Jul 5, 2016
View reviewed changes

fix nits to improve readability

3a98c4d

asfgit closed this in 478b71d Jul 6, 2016

sarutak mentioned this pull request Dec 19, 2016

[SPARK-18837][WEBUI] Very long stage descriptions do not wrap in the UI #16338

Closed

LuciferYang mentioned this pull request Nov 10, 2023

[SPARK-45875][CORE] Remove MissingStageTableRowData from core module #43748

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SPARK-15591] [WEBUI] Paginate Stage Table in Stages tab #13708

[SPARK-15591] [WEBUI] Paginate Stage Table in Stages tab #13708

nblintao commented Jun 16, 2016

SparkQA commented Jun 16, 2016

SparkQA commented Jun 16, 2016

ajbozarth commented Jun 16, 2016

nblintao commented Jun 17, 2016

nblintao commented Jun 21, 2016

ajbozarth commented Jun 21, 2016

SparkQA commented Jun 21, 2016

SparkQA commented Jun 22, 2016

nblintao commented Jun 22, 2016

SparkQA commented Jun 22, 2016

ajbozarth commented Jun 22, 2016

nblintao commented Jun 23, 2016

nblintao commented Jun 23, 2016

ajbozarth commented Jun 23, 2016

SparkQA commented Jun 23, 2016

zsxwing Jul 5, 2016 •

edited

Loading

zsxwing commented Jul 5, 2016

SparkQA commented Jul 6, 2016

nblintao commented Jul 6, 2016 •

edited

Loading

zsxwing commented Jul 6, 2016

[SPARK-15591] [WEBUI] Paginate Stage Table in Stages tab #13708

[SPARK-15591] [WEBUI] Paginate Stage Table in Stages tab #13708

Conversation

nblintao commented Jun 16, 2016

What changes were proposed in this pull request?

How was this patch tested?

SparkQA commented Jun 16, 2016

SparkQA commented Jun 16, 2016

ajbozarth commented Jun 16, 2016

nblintao commented Jun 17, 2016

nblintao commented Jun 21, 2016

ajbozarth commented Jun 21, 2016

SparkQA commented Jun 21, 2016

SparkQA commented Jun 22, 2016

nblintao commented Jun 22, 2016

SparkQA commented Jun 22, 2016

ajbozarth commented Jun 22, 2016

nblintao commented Jun 23, 2016

nblintao commented Jun 23, 2016

ajbozarth commented Jun 23, 2016

SparkQA commented Jun 23, 2016

zsxwing Jul 5, 2016 • edited Loading

Choose a reason for hiding this comment

zsxwing commented Jul 5, 2016

SparkQA commented Jul 6, 2016

nblintao commented Jul 6, 2016 • edited Loading

zsxwing commented Jul 6, 2016

zsxwing Jul 5, 2016 •

edited

Loading

nblintao commented Jul 6, 2016 •

edited

Loading