[feature](analyze_cmd) add show-tablets-belong stmt for analyzing a batch of tablet-ids #27994

Yulei-Yang · 2023-12-05T03:30:17Z

Some cmds return tablet-id list, such as: show proc '/cluster_health/tablet_health/73421';

but it is hard to identify the tables and show their infos, which these tablets belong to.
this pr add a show-tablets-belong stmt to resolve this problem.

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

…batch of tablet-ids

Yulei-Yang · 2023-12-05T03:31:05Z

run buildall

doris-robot · 2023-12-05T05:44:12Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.39 seconds
stream load tsv: 565 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.2 seconds inserted 10000000 Rows, about 354K ops/s
storage size: 17163775417 Bytes

doris-robot · 2023-12-05T06:14:54Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 7806607ac282a371f2457bee16966e5349794e73, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4898	4744	4691	4691
q2	362	165	162	162
q3	1481	1294	1309	1294
q4	1136	951	957	951
q5	3220	3217	3213	3213
q6	257	127	132	127
q7	1017	510	546	510
q8	2225	2248	2226	2226
q9	6829	6815	6835	6815
q10	3253	3313	3326	3313
q11	339	210	203	203
q12	358	206	210	206
q13	4625	3881	3851	3851
q14	250	218	213	213
q15	608	541	530	530
q16	448	388	399	388
q17	1028	645	631	631
q18	8057	7547	7325	7325
q19	1525	1520	1526	1520
q20	582	300	331	300
q21	3390	2978	2921	2921
q22	380	306	310	306
Total cold run time: 46268 ms
Total hot run time: 41696 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4653	4608	4611	4608
q2	300	196	191	191
q3	3718	3725	3721	3721
q4	2531	2520	2506	2506
q5	6126	6105	6136	6105
q6	240	122	123	122
q7	2572	1948	1959	1948
q8	3720	3704	3742	3704
q9	9394	9362	9365	9362
q10	4013	4105	4123	4105
q11	604	484	482	482
q12	814	648	677	648
q13	4348	3637	3651	3637
q14	273	237	236	236
q15	609	537	542	537
q16	513	483	496	483
q17	1957	1963	1993	1963
q18	9237	8872	9329	8872
q19	1721	1718	1783	1718
q20	2315	1962	1953	1953
q21	7104	6716	6717	6716
q22	651	549	565	549
Total cold run time: 67413 ms
Total hot run time: 64166 ms

xy720

LGTM

github-actions · 2023-12-05T06:22:09Z

PR approved by at least one committer and no changes requested.

github-actions · 2023-12-05T06:22:12Z

PR approved by anyone and no changes requested.

docs/sidebars.json

lide-reed

LGTM

morrySnow

What does it have to do with ANALYZE? ANALYZE should only used for collection statistic infomation. i cannot find any relation bewteen this statement and ANALYZE. If u want show detail info about tablet, please use more appropriate keyword

Yulei-Yang · 2023-12-05T12:57:27Z

run buildall

doris-robot · 2023-12-05T14:51:45Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.17 seconds
stream load tsv: 575 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.0 seconds inserted 10000000 Rows, about 357K ops/s
storage size: 17164312562 Bytes

doris-robot · 2023-12-05T15:01:58Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 91c3121ba00e535aff8618c12f7079075a63448c, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4788	4495	4511	4495
q2	370	145	157	145
q3	1469	1255	1228	1228
q4	1124	947	917	917
q5	3174	3198	3201	3198
q6	255	130	131	130
q7	998	502	485	485
q8	2207	2232	2178	2178
q9	6724	6685	6710	6685
q10	3226	3263	3296	3263
q11	331	212	195	195
q12	370	222	205	205
q13	4621	3883	3777	3777
q14	247	216	216	216
q15	567	531	515	515
q16	437	381	388	381
q17	1011	620	600	600
q18	7477	7182	8444	7182
q19	1527	1433	1388	1388
q20	954	325	295	295
q21	3117	2689	2707	2689
q22	352	294	294	294
Total cold run time: 45346 ms
Total hot run time: 40461 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4449	4457	4409	4409
q2	269	164	178	164
q3	3540	3537	3527	3527
q4	2393	2382	2380	2380
q5	5767	5748	5754	5748
q6	243	119	119	119
q7	2382	1845	1843	1843
q8	3538	3544	3549	3544
q9	9121	9071	9035	9035
q10	3917	3995	4013	3995
q11	511	393	378	378
q12	761	608	582	582
q13	4324	3561	3551	3551
q14	272	255	248	248
q15	568	525	519	519
q16	490	461	460	460
q17	1868	1860	1876	1860
q18	8667	8213	8244	8213
q19	1748	1778	1769	1769
q20	2258	1943	1932	1932
q21	6528	6156	6158	6156
q22	513	405	434	405
Total cold run time: 64127 ms
Total hot run time: 60837 ms

github-actions · 2023-12-06T02:22:48Z

PR approved by at least one committer and no changes requested.

Yulei-Yang · 2023-12-06T02:24:57Z

run feut

Yulei-Yang · 2023-12-06T02:25:10Z

run p0

Yulei-Yang · 2023-12-06T03:58:49Z

run buildall

doris-robot · 2023-12-06T04:37:18Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.17 seconds
stream load tsv: 561 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17162311623 Bytes

doris-robot · 2023-12-06T04:39:03Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit d077f95876a9d4784f89ff431c702f0447199c7c, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4695	4394	4495	4394
q2	382	150	159	150
q3	1464	1253	1247	1247
q4	1127	967	922	922
q5	3183	3174	3223	3174
q6	247	129	127	127
q7	993	502	489	489
q8	2221	2229	2197	2197
q9	6718	6697	6686	6686
q10	3220	3282	3293	3282
q11	327	202	214	202
q12	352	216	214	214
q13	4605	3830	3819	3819
q14	243	217	218	217
q15	570	532	528	528
q16	444	394	403	394
q17	1021	611	658	611
q18	7483	7175	8502	7175
q19	1523	1376	1447	1376
q20	549	411	296	296
q21	3123	2680	2682	2680
q22	351	288	299	288
Total cold run time: 44841 ms
Total hot run time: 40468 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4372	4374	4332	4332
q2	270	163	172	163
q3	3540	3520	3520	3520
q4	2393	2375	2376	2375
q5	5752	5741	5740	5740
q6	242	122	123	122
q7	2371	1860	1824	1824
q8	3532	3537	3530	3530
q9	9067	9006	9049	9006
q10	3921	4008	4010	4008
q11	499	377	388	377
q12	756	634	604	604
q13	4310	3539	3536	3536
q14	281	262	253	253
q15	570	531	527	527
q16	490	462	463	462
q17	1874	1848	1881	1848
q18	8622	8321	8296	8296
q19	1760	1754	1776	1754
q20	2263	1958	1918	1918
q21	6517	6197	6140	6140
q22	497	432	440	432
Total cold run time: 63899 ms
Total hot run time: 60767 ms

lide-reed

LGTM

lide-reed · 2023-12-06T03:54:13Z

fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java

@@ -2704,6 +2709,62 @@ private void handleShowAnalyze() {
        resultSet = new ShowResultSet(showStmt.getMetaData(), resultRows);
    }

+    private void handleShowAnalyzeTablet() {


rename to belongxxx?

…atch of tablet-ids (apache#27994)

This #27994 introduce a new keyword BELONG, need to add it as reserve keyword

…atch of tablet-ids (#27994) (#31255) bp #27994 #31230

This apache#27994 introduce a new keyword BELONG, need to add it as reserve keyword

…atch of tablet-ids (apache#27994) (apache#31255) bp apache#27994 apache#31230

[feature](analyze_cmd) add show-analyze-tablets stmt for analyzing a …

7806607

…batch of tablet-ids

xy720 previously approved these changes Dec 5, 2023

View reviewed changes

github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Dec 5, 2023

lide-reed reviewed Dec 5, 2023

View reviewed changes

docs/sidebars.json Show resolved Hide resolved

lide-reed previously approved these changes Dec 5, 2023

View reviewed changes

morrySnow requested changes Dec 5, 2023

View reviewed changes

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 5, 2023

rename stmt

91c3121

Yulei-Yang dismissed stale reviews from lide-reed and xy720 via 91c3121 December 5, 2023 12:55

Yulei-Yang requested a review from morrySnow December 5, 2023 12:59

morrySnow previously approved these changes Dec 6, 2023

View reviewed changes

Yulei-Yang changed the title ~~[feature](analyze_cmd) add show-analyze-tablets stmt for analyzing a batch of tablet-ids~~ [feature](analyze_cmd) add show-tablets-belong stmt for analyzing a batch of tablet-ids Dec 6, 2023

github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 6, 2023

update

d077f95

Yulei-Yang dismissed morrySnow’s stale review via d077f95 December 6, 2023 03:56

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 6, 2023

xy720 approved these changes Dec 6, 2023

View reviewed changes

lide-reed approved these changes Dec 6, 2023

View reviewed changes

lide-reed merged commit fa5096f into apache:master Dec 6, 2023

Yulei-Yang mentioned this pull request Dec 8, 2023

[Improvement](cmd) add show-tablets-belong stmt for analyzing a batch of tablet-ids #28165

Merged

XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023

[feature](analyze_cmd) add show-tablets-belong stmt for analyzing a b…

255bac9

…atch of tablet-ids (apache#27994)

morningman added the dev/1.2.8-merged label Feb 9, 2024

morningman mentioned this pull request Feb 21, 2024

[fix](keyword) add BELONG as reserve keyword #31230

Merged

morningman pushed a commit to morningman/doris that referenced this pull request Feb 22, 2024

[feature](analyze_cmd) add show-tablets-belong stmt for analyzing a b…

d0ae247

…atch of tablet-ids (apache#27994)

morningman mentioned this pull request Feb 22, 2024

[feature](analyze_cmd) add show-tablets-belong stmt for analyzing a batch of tablet-ids (#27994) #31255

Merged

morningman added the dev/2.0.5 label Feb 22, 2024

morningman pushed a commit to morningman/doris that referenced this pull request Feb 22, 2024

[feature](analyze_cmd) add show-tablets-belong stmt for analyzing a b…

f6858d0

…atch of tablet-ids (apache#27994)

morningman added a commit that referenced this pull request Feb 22, 2024

[fix](keyword) add BELONG as reserve keyword (#31230)

07162c4

This #27994 introduce a new keyword BELONG, need to add it as reserve keyword

yiguolei pushed a commit that referenced this pull request Feb 22, 2024

[fix](keyword) add BELONG as reserve keyword (#31230)

a5ee1b3

This #27994 introduce a new keyword BELONG, need to add it as reserve keyword

morningman added a commit that referenced this pull request Feb 22, 2024

[feature](analyze_cmd) add show-tablets-belong stmt for analyzing a b…

762820d

…atch of tablet-ids (#27994) (#31255) bp #27994 #31230

morningman added dev/2.0.5-merged and removed dev/2.0.5 labels Feb 22, 2024

feiniaofeiafei pushed a commit to feiniaofeiafei/doris that referenced this pull request Feb 23, 2024

[fix](keyword) add BELONG as reserve keyword (apache#31230)

d0667be

This apache#27994 introduce a new keyword BELONG, need to add it as reserve keyword

Yulei-Yang deleted the show_analyze_tablets branch April 3, 2024 05:29

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024

[feature](analyze_cmd) add show-tablets-belong stmt for analyzing a b…

b1f0725

…atch of tablet-ids (apache#27994) (apache#31255) bp apache#27994 apache#31230

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feature](analyze_cmd) add show-tablets-belong stmt for analyzing a batch of tablet-ids #27994

[feature](analyze_cmd) add show-tablets-belong stmt for analyzing a batch of tablet-ids #27994

Yulei-Yang commented Dec 5, 2023 •

edited

Loading

Yulei-Yang commented Dec 5, 2023

doris-robot commented Dec 5, 2023

doris-robot commented Dec 5, 2023

xy720 left a comment

github-actions bot commented Dec 5, 2023

github-actions bot commented Dec 5, 2023

lide-reed left a comment

morrySnow left a comment •

edited

Loading

Yulei-Yang commented Dec 5, 2023

doris-robot commented Dec 5, 2023

doris-robot commented Dec 5, 2023

github-actions bot commented Dec 6, 2023

Yulei-Yang commented Dec 6, 2023

Yulei-Yang commented Dec 6, 2023

Yulei-Yang commented Dec 6, 2023

doris-robot commented Dec 6, 2023

doris-robot commented Dec 6, 2023

lide-reed left a comment

lide-reed Dec 6, 2023

Yulei-Yang Dec 6, 2023

[feature](analyze_cmd) add show-tablets-belong stmt for analyzing a batch of tablet-ids #27994

[feature](analyze_cmd) add show-tablets-belong stmt for analyzing a batch of tablet-ids #27994

Conversation

Yulei-Yang commented Dec 5, 2023 • edited Loading

Proposed changes

Further comments

Yulei-Yang commented Dec 5, 2023

doris-robot commented Dec 5, 2023

doris-robot commented Dec 5, 2023

xy720 left a comment

Choose a reason for hiding this comment

github-actions bot commented Dec 5, 2023

github-actions bot commented Dec 5, 2023

lide-reed left a comment

Choose a reason for hiding this comment

morrySnow left a comment • edited Loading

Choose a reason for hiding this comment

Yulei-Yang commented Dec 5, 2023

doris-robot commented Dec 5, 2023

doris-robot commented Dec 5, 2023

github-actions bot commented Dec 6, 2023

Yulei-Yang commented Dec 6, 2023

Yulei-Yang commented Dec 6, 2023

Yulei-Yang commented Dec 6, 2023

doris-robot commented Dec 6, 2023

doris-robot commented Dec 6, 2023

lide-reed left a comment

Choose a reason for hiding this comment

lide-reed Dec 6, 2023

Choose a reason for hiding this comment

Yulei-Yang Dec 6, 2023

Choose a reason for hiding this comment

Yulei-Yang commented Dec 5, 2023 •

edited

Loading

morrySnow left a comment •

edited

Loading