[fix](session) fix select * from variables system table #34529

nextdreamblue · 2024-05-08T07:50:32Z

Proposed changes

Issue Number: close #xxx

after #23017, information_schema.session_variables and information_schema.global_variables had 2 new columns, but SchemaVariablesScanner just 2 columns. when select * from *_variables,will get 'no match column for this column' error

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

doris-robot · 2024-05-08T07:50:38Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

nextdreamblue · 2024-05-08T07:50:50Z

run buildall

github-actions · 2024-05-08T07:55:44Z

clang-tidy review says "All clean, LGTM! 👍"

zhiqiang-hhhh · 2024-05-08T07:59:49Z

I have tried to do similar fix in #30998, but not merged.

doris-robot · 2024-05-08T09:19:40Z

TeamCity be ut coverage result:
Function Coverage: 35.69% (8983/25170)
Line Coverage: 27.34% (74215/271422)
Region Coverage: 26.58% (38365/144339)
Branch Coverage: 23.40% (19567/83626)
Coverage Report: http://coverage.selectdb-in.cc/coverage/5a1ff28574e17ffd59ec4bcb8749b073768c96f3_5a1ff28574e17ffd59ec4bcb8749b073768c96f3/report/index.html

morningman

need test case

nextdreamblue · 2024-05-08T11:24:03Z

I have tried to do similar fix in #30998, but not merged.

... why not

zhiqiang-hhhh · 2024-05-08T11:59:24Z

I have tried to do similar fix in #30998, but not merged.

... why not

We want to use something like a general table value function to handle all similar situation. For example, it could be used like select * from system_tables("information_schema.session_variables")

Signed-off-by: nextdreamblue <zxw520blue1@163.com>

nextdreamblue · 2024-07-31T07:41:16Z

run buildall

nextdreamblue · 2024-07-31T08:42:34Z

run buildall

Signed-off-by: nextdreamblue <zxw520blue1@163.com>

nextdreamblue · 2024-07-31T09:37:14Z

run buildall

github-actions · 2024-07-31T09:40:01Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-07-31T10:03:48Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-07-31T11:17:17Z

TPC-H: Total hot run time: 42035 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 66e5be136c59cf71ff7f44fa17b0969a4147f952, data reload: false

------ Round 1 ----------------------------------
q1	17700	4367	4073	4073
q2	2031	210	214	210
q3	10544	1302	1358	1302
q4	10203	807	972	807
q5	7621	3083	3009	3009
q6	219	138	138	138
q7	1052	620	614	614
q8	9439	1953	1980	1953
q9	8496	6619	6671	6619
q10	8745	3844	3844	3844
q11	425	260	250	250
q12	412	232	219	219
q13	17756	2940	2984	2940
q14	275	246	244	244
q15	525	483	485	483
q16	489	403	388	388
q17	988	961	940	940
q18	8093	7389	7338	7338
q19	1700	1230	1232	1230
q20	570	342	339	339
q21	5468	4815	4969	4815
q22	354	280	285	280
Total cold run time: 113105 ms
Total hot run time: 42035 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4118	4001	4039	4001
q2	328	227	219	219
q3	3006	3123	3156	3123
q4	2079	2053	1990	1990
q5	5682	5498	5428	5428
q6	222	135	131	131
q7	2122	1825	1832	1825
q8	3359	3405	3393	3393
q9	8727	8708	8859	8708
q10	3941	4090	3939	3939
q11	551	471	468	468
q12	789	642	591	591
q13	16366	3129	3089	3089
q14	295	287	278	278
q15	550	491	478	478
q16	463	416	419	416
q17	1769	1734	1733	1733
q18	8210	7882	7812	7812
q19	1706	1700	1710	1700
q20	2091	1831	1827	1827
q21	5872	5555	5407	5407
q22	521	458	475	458
Total cold run time: 72767 ms
Total hot run time: 57014 ms

doris-robot · 2024-07-31T11:28:03Z

TPC-DS: Total hot run time: 170934 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 66e5be136c59cf71ff7f44fa17b0969a4147f952, data reload: false

query1	912	391	372	372
query2	6491	1815	1724	1724
query3	6673	213	223	213
query4	20059	17518	17289	17289
query5	3669	520	523	520
query6	269	164	173	164
query7	4595	295	296	295
query8	246	190	197	190
query9	8518	2431	2412	2412
query10	422	287	277	277
query11	10547	10207	10076	10076
query12	123	97	90	90
query13	1657	385	373	373
query14	8903	7899	7492	7492
query15	200	163	170	163
query16	6908	479	497	479
query17	944	573	579	573
query18	1935	294	293	293
query19	203	156	157	156
query20	96	88	89	88
query21	212	105	102	102
query22	4087	3930	4182	3930
query23	34070	33700	33548	33548
query24	10275	3087	3117	3087
query25	687	442	421	421
query26	1706	158	162	158
query27	2877	281	289	281
query28	7482	2049	2106	2049
query29	1321	449	430	430
query30	231	157	169	157
query31	935	778	790	778
query32	98	54	56	54
query33	663	306	331	306
query34	905	496	524	496
query35	882	790	756	756
query36	1066	913	894	894
query37	305	92	83	83
query38	3016	2814	2945	2814
query39	893	818	825	818
query40	256	109	113	109
query41	47	45	43	43
query42	132	102	101	101
query43	480	451	464	451
query44	1165	730	742	730
query45	205	181	180	180
query46	1077	800	795	795
query47	1808	1744	1718	1718
query48	361	293	289	289
query49	933	414	414	414
query50	911	443	435	435
query51	6805	6668	6663	6663
query52	102	90	91	90
query53	261	193	186	186
query54	656	459	456	456
query55	76	75	76	75
query56	289	265	252	252
query57	1134	1075	1024	1024
query58	269	273	291	273
query59	2701	2515	2440	2440
query60	297	269	281	269
query61	96	93	96	93
query62	860	657	673	657
query63	219	181	184	181
query64	5602	1925	1917	1917
query65	3157	3090	3142	3090
query66	1306	352	330	330
query67	15243	14972	14873	14873
query68	4333	559	572	559
query69	446	304	300	300
query70	1130	1064	1088	1064
query71	414	280	279	279
query72	7074	2727	2490	2490
query73	770	333	331	331
query74	6027	5663	5626	5626
query75	3407	2782	2744	2744
query76	2336	1340	1393	1340
query77	422	310	307	307
query78	9454	8903	8880	8880
query79	2790	598	543	543
query80	991	503	504	503
query81	574	226	222	222
query82	844	132	127	127
query83	252	169	174	169
query84	269	81	86	81
query85	1295	323	301	301
query86	453	282	299	282
query87	3301	3116	3118	3116
query88	3922	2481	2498	2481
query89	390	287	302	287
query90	1788	195	192	192
query91	125	98	100	98
query92	55	49	50	49
query93	2196	638	641	638
query94	835	305	299	299
query95	381	279	269	269
query96	600	284	283	283
query97	3221	3040	3070	3040
query98	231	199	192	192
query99	1704	1307	1308	1307
Total cold run time: 263559 ms
Total hot run time: 170934 ms

doris-robot · 2024-07-31T11:33:35Z

ClickBench: Total hot run time: 30.76 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 66e5be136c59cf71ff7f44fa17b0969a4147f952, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.03	0.04
query3	0.23	0.05	0.05
query4	1.68	0.08	0.08
query5	0.49	0.47	0.48
query6	1.14	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.57	0.52	0.52
query10	0.57	0.56	0.57
query11	0.16	0.12	0.12
query12	0.16	0.12	0.13
query13	0.61	0.60	0.60
query14	0.78	0.80	0.79
query15	0.89	0.86	0.88
query16	0.35	0.35	0.36
query17	0.98	1.02	1.00
query18	0.22	0.21	0.20
query19	1.85	1.72	1.75
query20	0.01	0.01	0.01
query21	15.39	0.74	0.66
query22	4.04	7.95	1.98
query23	17.99	1.41	1.31
query24	2.25	0.23	0.21
query25	0.18	0.08	0.08
query26	0.31	0.22	0.21
query27	0.46	0.24	0.23
query28	13.16	1.00	0.96
query29	12.64	3.32	3.32
query30	0.26	0.05	0.05
query31	2.89	0.40	0.40
query32	3.23	0.49	0.48
query33	2.89	2.98	2.95
query34	15.43	4.29	4.27
query35	4.30	4.31	4.29
query36	0.68	0.48	0.48
query37	0.19	0.16	0.17
query38	0.16	0.15	0.16
query39	0.04	0.04	0.04
query40	0.16	0.12	0.13
query41	0.10	0.05	0.05
query42	0.05	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 107.72 s
Total hot run time: 30.76 s

nextdreamblue · 2024-08-01T02:03:48Z

run cloud_p1

dataroaring

LGTM

github-actions · 2024-08-01T07:35:26Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-08-01T07:35:28Z

PR approved by anyone and no changes requested.

after #23017, information_schema.session_variables and information_schema.global_variables had 2 new columns, but SchemaVariablesScanner just 2 columns. when select * from *_variables, will get 'no match column for this column' error Signed-off-by: nextdreamblue <zxw520blue1@163.com>

after apache#23017, information_schema.session_variables and information_schema.global_variables had 2 new columns, but SchemaVariablesScanner just 2 columns. when select * from *_variables, will get 'no match column for this column' error Signed-off-by: nextdreamblue <zxw520blue1@163.com>

pick #34529 Signed-off-by: nextdreamblue <zxw520blue1@163.com> Co-authored-by: xueweizhang <zxw520blue1@163.com>

after apache#23017, information_schema.session_variables and information_schema.global_variables had 2 new columns, but SchemaVariablesScanner just 2 columns. when select * from *_variables, will get 'no match column for this column' error Signed-off-by: nextdreamblue <zxw520blue1@163.com>

nextdreamblue marked this pull request as ready for review May 8, 2024 07:50

github-actions bot added the meta-change label May 8, 2024

xiaokang added the dev/2.0.x label May 8, 2024

morningman reviewed May 8, 2024

View reviewed changes

nextdreamblue added 2 commits July 31, 2024 11:05

[fix](session) fix select * from variables system table

8aaff00

Signed-off-by: nextdreamblue <zxw520blue1@163.com>

test

88675d1

Signed-off-by: nextdreamblue <zxw520blue1@163.com>

nextdreamblue force-pushed the fix_variables_table branch from 5a1ff28 to 88675d1 Compare July 31, 2024 07:40

fix

66e5be1

Signed-off-by: nextdreamblue <zxw520blue1@163.com>

dataroaring approved these changes Aug 1, 2024

View reviewed changes

dataroaring added dev/2.1.x dev/3.0.x labels Aug 1, 2024

github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 1, 2024

github-actions bot added the reviewed label Aug 1, 2024

Yulei-Yang approved these changes Aug 2, 2024

View reviewed changes

nextdreamblue merged commit 23e83f2 into apache:master Aug 2, 2024
26 of 30 checks passed

dataroaring added dev/3.0.2-merged and removed dev/3.0.x labels Aug 11, 2024

nextdreamblue mentioned this pull request Aug 26, 2024

[fix](thrift) fix thrift for variables when version compatibility #39904

Open

wangbo mentioned this pull request Feb 8, 2025

[branch-2.1] fix select * from variables system table (#34529) #47660

Merged

yiguolei pushed a commit that referenced this pull request Feb 9, 2025

[branch-2.1] fix select * from variables system table (#34529) (#47660)

f4e92d7

pick #34529 Signed-off-by: nextdreamblue <zxw520blue1@163.com> Co-authored-by: xueweizhang <zxw520blue1@163.com>

yiguolei added dev/2.1.9-merged and removed dev/2.1.x labels Feb 9, 2025

wangbo mentioned this pull request Feb 10, 2025

[branch-2.0] fix select * from variables system table (#34529) #47674

Open

wm1581066 added the usercase Important user case type label label Feb 12, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fix](session) fix select * from variables system table #34529

[fix](session) fix select * from variables system table #34529

nextdreamblue commented May 8, 2024 •

edited

Loading

doris-robot commented May 8, 2024

nextdreamblue commented May 8, 2024

github-actions bot commented May 8, 2024

zhiqiang-hhhh commented May 8, 2024

doris-robot commented May 8, 2024

morningman left a comment

nextdreamblue commented May 8, 2024

zhiqiang-hhhh commented May 8, 2024

nextdreamblue commented Jul 31, 2024

nextdreamblue commented Jul 31, 2024

nextdreamblue commented Jul 31, 2024

github-actions bot commented Jul 31, 2024

github-actions bot commented Jul 31, 2024

doris-robot commented Jul 31, 2024

doris-robot commented Jul 31, 2024

doris-robot commented Jul 31, 2024

nextdreamblue commented Aug 1, 2024

dataroaring left a comment

github-actions bot commented Aug 1, 2024

github-actions bot commented Aug 1, 2024

[fix](session) fix select * from variables system table #34529

[fix](session) fix select * from variables system table #34529

Conversation

nextdreamblue commented May 8, 2024 • edited Loading

Proposed changes

Further comments

doris-robot commented May 8, 2024

nextdreamblue commented May 8, 2024

github-actions bot commented May 8, 2024

zhiqiang-hhhh commented May 8, 2024

doris-robot commented May 8, 2024

morningman left a comment

Choose a reason for hiding this comment

nextdreamblue commented May 8, 2024

zhiqiang-hhhh commented May 8, 2024

nextdreamblue commented Jul 31, 2024

nextdreamblue commented Jul 31, 2024

nextdreamblue commented Jul 31, 2024

github-actions bot commented Jul 31, 2024

github-actions bot commented Jul 31, 2024

doris-robot commented Jul 31, 2024

doris-robot commented Jul 31, 2024

doris-robot commented Jul 31, 2024

nextdreamblue commented Aug 1, 2024

dataroaring left a comment

Choose a reason for hiding this comment

github-actions bot commented Aug 1, 2024

github-actions bot commented Aug 1, 2024

nextdreamblue commented May 8, 2024 •

edited

Loading