Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](mtmv) Fix select literal result wrongly in group by when use materialized view #38958

Merged
merged 6 commits into from
Aug 20, 2024
Merged

[fix](mtmv) Fix select literal result wrongly in group by when use materialized view #38958

merged 6 commits into from
Aug 20, 2024

Conversation

seawinde
Copy link
Contributor

@seawinde seawinde commented Aug 6, 2024

Proposed changes

This is brought by #34274

if mv def is

select o_orderdate from  orders group by o_orderdate;

query is as followiing, the result is wrong. this pr fix the problem.

select 1 from orders group by o_orderdate;

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added the doing label Aug 6, 2024
@seawinde
Copy link
Contributor Author

seawinde commented Aug 6, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41954 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5382b3cafbe8efc678058606997c108cf61a1d73, data reload: false

------ Round 1 ----------------------------------
q1	17652	4172	4092	4092
q2	2033	201	199	199
q3	10584	1304	1382	1304
q4	10272	852	943	852
q5	7672	3028	2985	2985
q6	221	137	140	137
q7	1067	607	620	607
q8	9431	1957	1937	1937
q9	8603	6611	6606	6606
q10	8788	3855	3835	3835
q11	433	256	252	252
q12	422	235	235	235
q13	17759	2949	2933	2933
q14	267	250	242	242
q15	529	488	502	488
q16	522	393	397	393
q17	983	945	936	936
q18	8159	7305	7423	7305
q19	1381	1220	1215	1215
q20	565	323	350	323
q21	5383	4792	4829	4792
q22	357	286	290	286
Total cold run time: 113083 ms
Total hot run time: 41954 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4054	4024	4073	4024
q2	341	220	216	216
q3	2996	3023	3156	3023
q4	2013	2041	1962	1962
q5	5636	5506	5463	5463
q6	218	132	133	132
q7	2099	1809	1817	1809
q8	3316	3366	3486	3366
q9	8660	8712	8701	8701
q10	3966	3958	4050	3958
q11	559	455	472	455
q12	762	610	620	610
q13	16693	3123	3119	3119
q14	308	289	277	277
q15	536	495	478	478
q16	476	427	419	419
q17	1761	1722	1725	1722
q18	8223	7744	7871	7744
q19	1733	1741	1728	1728
q20	2063	1820	1834	1820
q21	5722	5335	5589	5335
q22	556	461	474	461
Total cold run time: 72691 ms
Total hot run time: 56822 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169174 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5382b3cafbe8efc678058606997c108cf61a1d73, data reload: false

query1	912	372	364	364
query2	6445	1678	1658	1658
query3	6682	210	218	210
query4	19985	17521	17294	17294
query5	3661	504	513	504
query6	264	168	154	154
query7	4591	298	292	292
query8	264	197	193	193
query9	8523	2396	2400	2396
query10	437	316	275	275
query11	10546	10020	9983	9983
query12	120	95	87	87
query13	1634	379	380	379
query14	9124	7314	7051	7051
query15	225	169	167	167
query16	6969	465	469	465
query17	954	584	555	555
query18	1920	283	344	283
query19	185	143	138	138
query20	93	84	86	84
query21	207	95	95	95
query22	4288	3946	3948	3946
query23	33501	33698	33494	33494
query24	9415	3123	3086	3086
query25	643	418	388	388
query26	802	157	153	153
query27	2407	297	288	288
query28	6342	2026	2019	2019
query29	951	432	444	432
query30	241	162	157	157
query31	934	760	776	760
query32	104	54	64	54
query33	701	335	327	327
query34	936	515	520	515
query35	907	765	762	762
query36	1062	863	895	863
query37	143	81	84	81
query38	2932	2924	2770	2770
query39	893	827	809	809
query40	235	112	108	108
query41	45	43	43	43
query42	117	99	100	99
query43	462	423	427	423
query44	1198	738	726	726
query45	220	182	181	181
query46	1090	811	780	780
query47	1815	1703	1728	1703
query48	370	291	294	291
query49	864	414	416	414
query50	894	428	432	428
query51	6717	6652	6668	6652
query52	103	84	88	84
query53	256	187	189	187
query54	632	448	448	448
query55	78	75	76	75
query56	278	256	250	250
query57	1123	1049	1037	1037
query58	260	269	271	269
query59	2467	2272	2492	2272
query60	311	272	275	272
query61	98	124	94	94
query62	877	671	665	665
query63	217	185	192	185
query64	4492	1893	1885	1885
query65	3151	3096	3126	3096
query66	838	325	365	325
query67	15398	14588	14903	14588
query68	4411	555	577	555
query69	698	369	314	314
query70	1130	1090	1062	1062
query71	407	280	278	278
query72	7599	2724	2483	2483
query73	774	327	332	327
query74	5953	5678	5582	5582
query75	3379	2752	2733	2733
query76	2838	1213	1274	1213
query77	684	325	337	325
query78	9352	8887	8878	8878
query79	2825	542	530	530
query80	990	510	504	504
query81	557	228	229	228
query82	1184	132	140	132
query83	251	179	174	174
query84	264	80	81	80
query85	1169	334	291	291
query86	462	288	320	288
query87	3264	3101	3066	3066
query88	3893	2411	2459	2411
query89	385	281	282	281
query90	1684	192	197	192
query91	127	101	101	101
query92	59	51	53	51
query93	2020	623	614	614
query94	777	290	286	286
query95	372	273	269	269
query96	614	288	281	281
query97	3221	3042	3066	3042
query98	220	203	196	196
query99	1649	1301	1296	1296
Total cold run time: 258761 ms
Total hot run time: 169174 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.96 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5382b3cafbe8efc678058606997c108cf61a1d73, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.07	0.06
query5	0.53	0.48	0.49
query6	1.13	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.58	0.51	0.51
query10	0.55	0.56	0.56
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.62	0.61	0.60
query14	0.77	0.80	0.80
query15	0.92	0.86	0.85
query16	0.35	0.35	0.36
query17	0.99	0.99	0.99
query18	0.22	0.21	0.21
query19	1.86	1.71	1.72
query20	0.01	0.01	0.01
query21	15.42	0.78	0.66
query22	4.55	7.60	1.24
query23	17.85	1.42	1.28
query24	2.24	0.22	0.22
query25	0.19	0.08	0.08
query26	0.33	0.21	0.22
query27	0.46	0.24	0.24
query28	13.16	1.00	0.97
query29	12.51	3.29	3.28
query30	0.26	0.06	0.05
query31	2.86	0.41	0.41
query32	3.25	0.49	0.48
query33	2.94	2.95	2.96
query34	15.43	4.27	4.26
query35	4.28	4.28	4.29
query36	0.68	0.48	0.49
query37	0.19	0.17	0.16
query38	0.17	0.15	0.16
query39	0.04	0.04	0.04
query40	0.16	0.13	0.13
query41	0.10	0.05	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 108.08 s
Total hot run time: 29.96 s

starocean999
starocean999 reviewed Aug 7, 2024
View reviewed changes
...ava/org/apache/doris/nereids/rules/exploration/mv/AbstractMaterializedViewAggregateRule.java Outdated
* if the slot which query top project use can not cover the slot which query bottom aggregate group by slot
* should compensate group by to make sure the data is right.
*/
private static boolean needCompensateGroupBy(Set<? extends Expression> queryTopPlanGroupBySet,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does queryTopPlanGroupBySet mean top project slots? if it does, we may use a better name such as topProjectList?

@morrySnow morrySnow added dev/2.1.x and removed doing labels Aug 7, 2024
morrySnow
morrySnow previously approved these changes Aug 7, 2024
View reviewed changes
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 7, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 7, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 7, 2024

PR approved by anyone and no changes requested.

@seawinde
Copy link
Contributor Author

seawinde commented Aug 7, 2024

run buildall

@github-actions github-actions bot added doing and removed approved Indicates a PR has been approved by one committer. labels Aug 7, 2024
zfr9527
zfr9527 approved these changes Aug 7, 2024
View reviewed changes
Copy link
Contributor

@zfr9527 zfr9527 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@doris-robot
Copy link

TPC-H: Total hot run time: 41748 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6a74ccb64cf2d437b35b87fcf82a4d8a129e225c, data reload: false

------ Round 1 ----------------------------------
q1	17656	4128	4037	4037
q2	2030	202	197	197
q3	10548	1270	1365	1270
q4	10186	853	914	853
q5	7660	2959	2955	2955
q6	223	140	140	140
q7	1053	615	610	610
q8	9434	1905	1954	1905
q9	8487	6566	6565	6565
q10	8772	3856	3850	3850
q11	444	248	248	248
q12	446	235	232	232
q13	17779	2940	2942	2940
q14	276	242	250	242
q15	529	495	506	495
q16	522	390	390	390
q17	971	907	893	893
q18	7983	7519	7311	7311
q19	1629	1223	1209	1209
q20	558	336	340	336
q21	5319	4794	4800	4794
q22	353	281	276	276
Total cold run time: 112858 ms
Total hot run time: 41748 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4068	4025	4048	4025
q2	327	221	224	221
q3	2994	3021	3138	3021
q4	1980	2017	1966	1966
q5	5572	5492	5458	5458
q6	218	134	136	134
q7	2140	1868	1849	1849
q8	3306	3351	3356	3351
q9	8669	8704	8724	8704
q10	4032	3934	3994	3934
q11	554	477	469	469
q12	783	606	581	581
q13	16444	3129	3142	3129
q14	304	297	270	270
q15	537	487	505	487
q16	468	428	433	428
q17	1750	1716	1743	1716
q18	8318	7746	7669	7669
q19	1705	1739	1736	1736
q20	2064	1868	1855	1855
q21	5635	5436	5236	5236
q22	525	445	471	445
Total cold run time: 72393 ms
Total hot run time: 56684 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169072 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6a74ccb64cf2d437b35b87fcf82a4d8a129e225c, data reload: false

query1	916	390	385	385
query2	6472	1689	1664	1664
query3	6652	213	219	213
query4	19651	17643	17275	17275
query5	3652	518	504	504
query6	269	190	157	157
query7	4587	291	289	289
query8	260	191	211	191
query9	8523	2323	2311	2311
query10	447	281	269	269
query11	10410	10053	9998	9998
query12	125	90	87	87
query13	1634	383	384	383
query14	8728	6292	6884	6292
query15	224	167	172	167
query16	6984	448	440	440
query17	951	569	559	559
query18	1927	285	284	284
query19	191	149	146	146
query20	92	87	93	87
query21	204	103	99	99
query22	4360	4043	3977	3977
query23	33810	34036	33723	33723
query24	9334	3112	3117	3112
query25	672	393	408	393
query26	1551	154	156	154
query27	3101	280	305	280
query28	7717	1992	1964	1964
query29	1140	434	435	434
query30	239	162	155	155
query31	940	765	782	765
query32	103	57	59	57
query33	679	333	326	326
query34	941	500	495	495
query35	897	768	769	768
query36	1049	921	897	897
query37	176	83	79	79
query38	3034	2883	2769	2769
query39	857	813	807	807
query40	281	110	111	110
query41	46	44	46	44
query42	125	101	98	98
query43	476	411	422	411
query44	1187	733	716	716
query45	218	182	183	182
query46	1111	803	768	768
query47	1801	1724	1746	1724
query48	354	292	284	284
query49	962	420	417	417
query50	910	435	420	420
query51	6987	6679	6697	6679
query52	102	87	92	87
query53	247	184	186	184
query54	661	450	444	444
query55	76	75	74	74
query56	279	254	246	246
query57	1145	1049	1043	1043
query58	281	282	271	271
query59	2581	2416	2366	2366
query60	315	286	268	268
query61	93	90	90	90
query62	884	662	656	656
query63	213	179	179	179
query64	5694	1902	1922	1902
query65	3175	3087	3084	3084
query66	1293	337	330	330
query67	15238	14871	14903	14871
query68	5464	575	580	575
query69	680	368	303	303
query70	1117	1057	1070	1057
query71	465	279	276	276
query72	7520	2725	2487	2487
query73	887	326	323	323
query74	6061	5693	5681	5681
query75	3602	2732	2777	2732
query76	3238	1271	1260	1260
query77	547	306	302	302
query78	9409	8901	8946	8901
query79	2723	537	526	526
query80	981	553	502	502
query81	546	225	224	224
query82	1425	142	135	135
query83	253	168	174	168
query84	265	87	77	77
query85	1456	315	304	304
query86	479	299	294	294
query87	3291	3129	3085	3085
query88	3697	2377	2369	2369
query89	394	286	291	286
query90	1756	191	189	189
query91	125	101	102	101
query92	57	96	50	50
query93	2136	633	620	620
query94	810	291	292	291
query95	375	262	260	260
query96	608	287	286	286
query97	3201	3056	3074	3056
query98	214	205	195	195
query99	1681	1267	1254	1254
Total cold run time: 265697 ms
Total hot run time: 169072 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.15 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6a74ccb64cf2d437b35b87fcf82a4d8a129e225c, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.22	0.04	0.04
query4	1.69	0.07	0.06
query5	0.48	0.48	0.49
query6	1.13	0.72	0.72
query7	0.02	0.01	0.01
query8	0.06	0.05	0.04
query9	0.56	0.52	0.51
query10	0.58	0.57	0.56
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.62	0.60	0.60
query14	0.77	0.80	0.78
query15	0.93	0.86	0.87
query16	0.35	0.36	0.35
query17	1.00	0.98	1.04
query18	0.22	0.21	0.21
query19	1.79	1.74	1.73
query20	0.01	0.01	0.01
query21	15.42	0.74	0.64
query22	4.31	6.03	1.47
query23	17.93	1.33	1.34
query24	2.29	0.22	0.21
query25	0.18	0.08	0.08
query26	0.33	0.21	0.22
query27	0.46	0.24	0.23
query28	13.18	1.01	0.97
query29	12.60	3.27	3.28
query30	0.25	0.06	0.05
query31	2.87	0.41	0.41
query32	3.22	0.50	0.49
query33	2.95	2.93	3.00
query34	15.46	4.26	4.27
query35	4.32	4.31	4.31
query36	0.67	0.49	0.47
query37	0.18	0.17	0.16
query38	0.16	0.15	0.14
query39	0.04	0.03	0.04
query40	0.16	0.12	0.13
query41	0.10	0.05	0.05
query42	0.06	0.04	0.04
query43	0.05	0.04	0.03
Total cold run time: 108.03 s
Total hot run time: 30.15 s

@seawinde seawinde force-pushed the fix_select_literal_when_group_by branch from 6a74ccb to d53d29c Compare August 8, 2024 08:01
@seawinde
Copy link
Contributor Author

seawinde commented Aug 8, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39142 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d53d29c357f3e3860857ef453ac6be193012550f, data reload: false

------ Round 1 ----------------------------------
q1	17636	4355	4293	4293
q2	2020	180	181	180
q3	10478	1152	1094	1094
q4	10155	802	712	712
q5	7509	2524	2545	2524
q6	231	140	135	135
q7	963	587	579	579
q8	9221	1899	1897	1897
q9	8611	6520	6525	6520
q10	7086	2198	2141	2141
q11	463	249	239	239
q12	500	213	211	211
q13	18701	3017	2967	2967
q14	279	231	240	231
q15	522	490	492	490
q16	513	393	396	393
q17	972	614	709	614
q18	8089	7493	7400	7400
q19	4950	1098	1037	1037
q20	651	341	316	316
q21	5275	4353	4178	4178
q22	1089	1010	991	991
Total cold run time: 115914 ms
Total hot run time: 39142 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4490	4391	4255	4255
q2	384	275	267	267
q3	2811	2583	2684	2583
q4	1997	1686	1740	1686
q5	5499	5506	5520	5506
q6	230	129	129	129
q7	2076	1691	1769	1691
q8	3302	3429	3410	3410
q9	8757	8730	9087	8730
q10	3525	3282	3328	3282
q11	569	500	502	500
q12	764	611	592	592
q13	15845	3144	3232	3144
q14	332	290	293	290
q15	513	498	488	488
q16	483	434	443	434
q17	1849	1514	1515	1514
q18	8102	7922	7832	7832
q19	1710	1689	1402	1402
q20	2091	1895	1916	1895
q21	9777	5226	5181	5181
q22	1089	1020	1008	1008
Total cold run time: 76195 ms
Total hot run time: 55819 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202500 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d53d29c357f3e3860857ef453ac6be193012550f, data reload: false

query1	949	405	389	389
query2	6454	2049	1875	1875
query3	6626	207	212	207
query4	33979	23168	23019	23019
query5	3652	472	498	472
query6	269	175	177	175
query7	4575	309	289	289
query8	246	202	200	200
query9	8511	2373	2339	2339
query10	928	877	901	877
query11	16213	14903	14992	14903
query12	129	96	98	96
query13	1641	387	374	374
query14	10668	7787	7624	7624
query15	378	364	334	334
query16	7986	453	480	453
query17	1720	585	564	564
query18	2208	426	433	426
query19	256	202	194	194
query20	121	111	114	111
query21	212	114	110	110
query22	4540	4300	4429	4300
query23	34260	33854	33501	33501
query24	11175	3021	2850	2850
query25	584	386	391	386
query26	698	155	153	153
query27	2130	276	272	272
query28	6010	1997	1980	1980
query29	744	409	391	391
query30	263	147	151	147
query31	959	726	769	726
query32	95	52	53	52
query33	688	278	269	269
query34	860	463	470	463
query35	933	867	793	793
query36	1045	930	921	921
query37	131	76	76	76
query38	4210	4165	4209	4165
query39	1421	1373	1385	1373
query40	200	113	111	111
query41	47	46	42	42
query42	111	94	94	94
query43	497	482	482	482
query44	1108	726	735	726
query45	406	379	383	379
query46	1154	794	735	735
query47	1891	1770	1785	1770
query48	370	296	290	290
query49	837	415	414	414
query50	811	394	400	394
query51	6855	6664	6670	6664
query52	101	85	87	85
query53	247	187	175	175
query54	861	451	450	450
query55	76	74	74	74
query56	260	241	240	240
query57	1145	1043	1057	1043
query58	224	228	232	228
query59	3029	2800	2794	2794
query60	285	284	262	262
query61	99	94	92	92
query62	811	647	614	614
query63	214	178	180	178
query64	9255	2410	1927	1927
query65	3189	3127	3157	3127
query66	743	329	324	324
query67	15445	14805	14905	14805
query68	6393	536	542	536
query69	453	427	400	400
query70	1168	1130	1068	1068
query71	551	282	269	269
query72	19898	16311	16456	16311
query73	816	324	320	320
query74	9207	8730	8755	8730
query75	4855	2694	2653	2653
query76	4737	984	1007	984
query77	801	326	342	326
query78	9769	9053	8954	8954
query79	2740	524	522	522
query80	2760	493	499	493
query81	598	224	225	224
query82	753	133	133	133
query83	300	152	163	152
query84	274	79	89	79
query85	1511	298	297	297
query86	457	295	271	271
query87	4699	4529	4552	4529
query88	4107	2498	2474	2474
query89	390	275	282	275
query90	2007	198	199	198
query91	156	128	119	119
query92	71	51	49	49
query93	3227	539	532	532
query94	1023	287	259	259
query95	360	267	262	262
query96	628	277	280	277
query97	3263	2994	3047	2994
query98	220	202	193	193
query99	1568	1294	1249	1249
Total cold run time: 318498 ms
Total hot run time: 202500 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.36 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d53d29c357f3e3860857ef453ac6be193012550f, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.06
query4	1.66	0.09	0.10
query5	0.48	0.48	0.48
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.55	0.50	0.49
query10	0.55	0.56	0.55
query11	0.16	0.12	0.12
query12	0.15	0.13	0.13
query13	0.59	0.59	0.59
query14	0.77	0.76	0.79
query15	0.88	0.82	0.81
query16	0.37	0.37	0.36
query17	0.96	1.02	0.99
query18	0.23	0.23	0.22
query19	1.82	1.71	1.68
query20	0.01	0.02	0.01
query21	15.39	0.74	0.66
query22	3.45	8.71	1.56
query23	18.29	1.35	1.29
query24	2.16	0.22	0.23
query25	0.16	0.08	0.09
query26	0.31	0.21	0.22
query27	0.46	0.22	0.21
query28	13.28	1.02	1.00
query29	12.61	3.29	3.28
query30	0.25	0.06	0.06
query31	2.86	0.39	0.38
query32	3.31	0.47	0.48
query33	2.89	2.94	2.91
query34	17.12	4.38	4.35
query35	4.42	4.46	4.43
query36	0.65	0.49	0.46
query37	0.19	0.16	0.16
query38	0.15	0.14	0.14
query39	0.05	0.04	0.04
query40	0.15	0.13	0.13
query41	0.10	0.04	0.04
query42	0.06	0.05	0.04
query43	0.04	0.05	0.04
Total cold run time: 109.08 s
Total hot run time: 30.36 s

@seawinde seawinde force-pushed the fix_select_literal_when_group_by branch from d53d29c to 7b38851 Compare August 9, 2024 06:14
@seawinde
Copy link
Contributor Author

seawinde commented Aug 9, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39270 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7b38851c5529884843405ea1fc18450b65d7282c, data reload: false

------ Round 1 ----------------------------------
q1	17626	4437	4287	4287
q2	2009	172	171	171
q3	10512	1175	1018	1018
q4	10140	771	771	771
q5	7497	2482	2502	2482
q6	231	138	138	138
q7	976	625	594	594
q8	9219	1906	1920	1906
q9	8813	6566	6536	6536
q10	7079	2203	2178	2178
q11	446	245	251	245
q12	495	225	223	223
q13	18720	2972	2990	2972
q14	283	234	240	234
q15	534	495	498	495
q16	517	393	385	385
q17	969	661	678	661
q18	8128	7418	7363	7363
q19	4075	989	952	952
q20	653	334	328	328
q21	5324	4343	4501	4343
q22	1093	1008	988	988
Total cold run time: 115339 ms
Total hot run time: 39270 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4505	4316	4301	4301
q2	388	278	281	278
q3	2849	2636	2679	2636
q4	1980	1728	1739	1728
q5	5480	5616	5460	5460
q6	241	145	143	143
q7	2110	1729	1747	1729
q8	3309	3531	3479	3479
q9	8807	8679	8913	8679
q10	3487	3300	3338	3300
q11	598	479	487	479
q12	781	658	634	634
q13	17053	3190	3184	3184
q14	328	283	296	283
q15	529	509	490	490
q16	496	441	448	441
q17	1883	1514	1496	1496
q18	8164	7943	7814	7814
q19	1776	1588	1629	1588
q20	2144	1892	1908	1892
q21	10334	5395	5330	5330
q22	1087	1027	1044	1027
Total cold run time: 78329 ms
Total hot run time: 56391 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191280 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 248b34e8e8dce435f3f34d8d811925215aff5b6d, data reload: false

query1	962	396	396	396
query2	6744	2064	1992	1992
query3	6667	244	258	244
query4	34331	23130	23365	23130
query5	4320	751	734	734
query6	315	213	216	213
query7	4618	348	319	319
query8	485	441	446	441
query9	8559	2540	2524	2524
query10	523	365	352	352
query11	17711	14948	14909	14909
query12	186	137	138	137
query13	1717	448	457	448
query14	10405	7382	7130	7130
query15	307	191	195	191
query16	7713	534	535	534
query17	1673	611	592	592
query18	2060	338	335	335
query19	388	174	175	174
query20	142	137	136	136
query21	250	143	145	143
query22	4481	4080	4008	4008
query23	34246	33457	33457	33457
query24	11078	2949	2913	2913
query25	650	426	438	426
query26	1019	185	180	180
query27	2450	303	312	303
query28	7336	2144	2111	2111
query29	805	452	440	440
query30	341	193	186	186
query31	1035	847	834	834
query32	121	84	83	83
query33	811	356	344	344
query34	897	508	526	508
query35	916	767	754	754
query36	1142	969	975	969
query37	172	107	105	105
query38	3956	3922	3881	3881
query39	1522	1462	1477	1462
query40	248	160	154	154
query41	142	139	139	139
query42	142	119	120	119
query43	551	528	527	527
query44	1272	789	809	789
query45	229	199	201	199
query46	1145	777	797	777
query47	1914	1834	1856	1834
query48	426	342	339	339
query49	1181	595	602	595
query50	882	474	481	474
query51	6811	6724	6725	6724
query52	117	108	113	108
query53	309	244	229	229
query54	1017	513	493	493
query55	92	92	91	91
query56	350	307	312	307
query57	1215	1124	1126	1124
query58	311	301	317	301
query59	3173	2899	2910	2899
query60	357	340	327	327
query61	149	152	154	152
query62	903	710	717	710
query63	261	230	231	230
query64	5518	2456	1814	1814
query65	3231	3160	3174	3160
query66	1177	692	680	680
query67	15296	14830	14950	14830
query68	5064	582	595	582
query69	682	406	331	331
query70	1155	1181	1190	1181
query71	514	311	315	311
query72	7550	2301	2171	2171
query73	812	359	369	359
query74	9149	8819	8762	8762
query75	4574	2725	2769	2725
query76	3571	1070	1055	1055
query77	876	462	458	458
query78	9776	9149	9519	9149
query79	6552	564	566	564
query80	2645	609	619	609
query81	643	258	273	258
query82	963	162	162	162
query83	377	215	216	215
query84	302	102	97	97
query85	1832	366	401	366
query86	476	319	329	319
query87	4369	4209	4253	4209
query88	4853	2508	2484	2484
query89	447	329	326	326
query90	1999	240	232	232
query91	160	128	134	128
query92	85	77	75	75
query93	5144	552	547	547
query94	1017	310	320	310
query95	387	294	293	293
query96	636	283	282	282
query97	3246	3033	3070	3033
query98	252	233	234	233
query99	1600	1315	1291	1291
Total cold run time: 313181 ms
Total hot run time: 191280 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.44 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 248b34e8e8dce435f3f34d8d811925215aff5b6d, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.66	0.10	0.08
query5	0.50	0.47	0.47
query6	1.14	0.73	0.74
query7	0.02	0.01	0.02
query8	0.06	0.04	0.05
query9	0.54	0.50	0.49
query10	0.53	0.55	0.55
query11	0.16	0.12	0.13
query12	0.16	0.12	0.12
query13	0.61	0.61	0.60
query14	0.76	0.81	0.79
query15	0.89	0.83	0.82
query16	0.37	0.36	0.38
query17	0.97	0.99	0.96
query18	0.23	0.22	0.22
query19	1.73	1.71	1.66
query20	0.01	0.02	0.01
query21	15.41	0.67	0.66
query22	4.33	6.82	2.35
query23	18.27	1.48	1.29
query24	2.13	0.22	0.21
query25	0.16	0.08	0.08
query26	0.32	0.23	0.23
query27	0.46	0.23	0.23
query28	13.36	1.02	1.01
query29	12.61	3.29	3.30
query30	0.45	0.25	0.22
query31	2.80	0.40	0.40
query32	3.24	0.48	0.48
query33	2.94	2.91	2.97
query34	17.14	4.33	4.36
query35	4.39	4.46	4.42
query36	0.67	0.48	0.49
query37	0.21	0.17	0.17
query38	0.17	0.16	0.17
query39	0.06	0.06	0.05
query40	0.17	0.14	0.14
query41	0.10	0.06	0.06
query42	0.07	0.06	0.07
query43	0.07	0.06	0.05
Total cold run time: 110.22 s
Total hot run time: 31.44 s

@seawinde seawinde force-pushed the fix_select_literal_when_group_by branch from 248b34e to 65a5b27 Compare August 18, 2024 23:58
@seawinde
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37842 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 65a5b279506c27124890b2bdaebb484b285a5267, data reload: false

------ Round 1 ----------------------------------
q1	17849	4818	4313	4313
q2	2049	207	198	198
q3	11717	995	1184	995
q4	10533	714	843	714
q5	7788	2850	2812	2812
q6	263	159	156	156
q7	1016	650	638	638
q8	9578	2052	2108	2052
q9	8582	6526	6529	6526
q10	7044	2200	2271	2200
q11	476	267	271	267
q12	423	262	266	262
q13	18945	2957	3014	2957
q14	290	251	244	244
q15	545	521	518	518
q16	508	422	404	404
q17	969	682	740	682
q18	7468	6852	6678	6678
q19	6825	1068	1077	1068
q20	710	333	346	333
q21	3822	2803	2924	2803
q22	1104	1046	1022	1022
Total cold run time: 118504 ms
Total hot run time: 37842 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4555	4329	4307	4307
q2	399	297	305	297
q3	2804	2620	2809	2620
q4	1992	1702	1687	1687
q5	5631	5777	5617	5617
q6	244	151	144	144
q7	2170	1776	1765	1765
q8	3281	3485	3447	3447
q9	8781	8776	8747	8747
q10	3578	3396	3299	3299
q11	631	556	528	528
q12	847	667	667	667
q13	17303	3152	3156	3152
q14	318	290	296	290
q15	551	518	519	518
q16	487	442	465	442
q17	1840	1576	1520	1520
q18	8192	7743	7853	7743
q19	2782	1497	1502	1497
q20	2150	1901	1948	1901
q21	13225	5378	5308	5308
q22	1179	1071	1070	1070
Total cold run time: 82940 ms
Total hot run time: 56566 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196155 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 65a5b279506c27124890b2bdaebb484b285a5267, data reload: false

query1	1338	908	891	891
query2	6594	1969	2023	1969
query3	10633	3694	3804	3694
query4	58223	26317	23423	23423
query5	5736	715	708	708
query6	553	213	215	213
query7	6398	330	338	330
query8	569	452	455	452
query9	9262	2590	2570	2570
query10	559	347	337	337
query11	18731	15077	15524	15077
query12	190	128	126	126
query13	1637	430	448	430
query14	12021	7401	7196	7196
query15	312	200	194	194
query16	7558	551	532	532
query17	1159	610	609	609
query18	1956	338	339	338
query19	286	179	174	174
query20	147	136	137	136
query21	247	150	146	146
query22	4636	4382	4242	4242
query23	34575	33917	34361	33917
query24	5742	2980	3008	2980
query25	585	427	421	421
query26	727	183	179	179
query27	1746	326	320	320
query28	3772	2215	2183	2183
query29	724	453	460	453
query30	237	190	203	190
query31	1060	902	863	863
query32	101	80	80	80
query33	523	349	341	341
query34	876	497	515	497
query35	852	786	743	743
query36	1115	938	960	938
query37	155	99	100	99
query38	3965	3834	3844	3834
query39	1540	1469	1439	1439
query40	239	157	161	157
query41	140	137	139	137
query42	134	116	117	116
query43	545	515	512	512
query44	1117	783	795	783
query45	222	196	190	190
query46	1111	803	779	779
query47	1917	1824	1867	1824
query48	409	335	338	335
query49	914	587	583	583
query50	855	456	473	456
query51	6797	6857	6743	6743
query52	121	111	107	107
query53	294	237	230	230
query54	615	504	504	504
query55	89	91	89	89
query56	326	311	307	307
query57	1201	1089	1106	1089
query58	301	303	292	292
query59	3206	2757	2789	2757
query60	349	330	323	323
query61	144	145	169	145
query62	780	677	704	677
query63	253	226	229	226
query64	3329	1869	1840	1840
query65	3279	3172	3203	3172
query66	1024	659	670	659
query67	15189	15032	15053	15032
query68	4709	595	581	581
query69	629	315	319	315
query70	1214	1149	1114	1114
query71	487	320	312	312
query72	7590	2317	2096	2096
query73	793	357	354	354
query74	9188	8782	8854	8782
query75	4145	2791	2721	2721
query76	2898	1089	1008	1008
query77	834	450	451	450
query78	9913	9186	9096	9096
query79	3960	550	547	547
query80	1544	595	591	591
query81	610	260	257	257
query82	905	158	162	158
query83	410	269	218	218
query84	301	97	96	96
query85	1306	354	354	354
query86	447	323	300	300
query87	4318	4247	4361	4247
query88	4790	2463	2439	2439
query89	445	329	314	314
query90	2049	234	230	230
query91	159	128	129	128
query92	85	75	74	74
query93	5530	549	553	549
query94	1009	328	311	311
query95	387	287	310	287
query96	622	293	288	288
query97	3280	3092	3091	3091
query98	246	242	231	231
query99	1626	1288	1304	1288
Total cold run time: 328369 ms
Total hot run time: 196155 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.57 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 65a5b279506c27124890b2bdaebb484b285a5267, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.67	0.08	0.09
query5	0.50	0.48	0.49
query6	1.13	0.73	0.73
query7	0.02	0.02	0.02
query8	0.06	0.05	0.05
query9	0.56	0.50	0.49
query10	0.55	0.56	0.53
query11	0.17	0.13	0.12
query12	0.15	0.13	0.13
query13	0.63	0.59	0.59
query14	0.76	0.78	0.79
query15	0.85	0.83	0.83
query16	0.38	0.35	0.38
query17	0.97	1.03	0.98
query18	0.24	0.22	0.24
query19	1.83	1.69	1.71
query20	0.01	0.01	0.01
query21	15.39	0.67	0.67
query22	4.33	7.04	2.26
query23	18.33	1.32	1.31
query24	2.14	0.23	0.23
query25	0.15	0.08	0.08
query26	0.31	0.25	0.24
query27	0.46	0.23	0.24
query28	13.24	1.04	1.02
query29	12.64	3.32	3.25
query30	0.38	0.20	0.20
query31	2.80	0.41	0.40
query32	3.23	0.50	0.48
query33	2.93	3.00	2.98
query34	16.93	4.39	4.36
query35	4.43	4.43	4.48
query36	0.66	0.49	0.51
query37	0.20	0.19	0.18
query38	0.19	0.17	0.16
query39	0.06	0.05	0.06
query40	0.18	0.15	0.16
query41	0.12	0.07	0.07
query42	0.08	0.07	0.06
query43	0.07	0.06	0.06
Total cold run time: 110.08 s
Total hot run time: 31.57 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 19, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@morrySnow
Copy link
Contributor

run beut

@morrySnow
Copy link
Contributor

run cloudut

@morrySnow morrySnow merged commit bccce84 into apache:master Aug 20, 2024
27 of 28 checks passed
seawinde added a commit to seawinde/doris that referenced this pull request Aug 21, 2024
@seawinde
[fix](mtmv) Fix select literal result wrongly in group by when use ma…
c365cb6 
…terialized view (apache#38958)

This is brought by apache#34274

if mv def is
select o_orderdate from  orders group by o_orderdate;

query is as followiing, the result is wrong.
select 1 from orders group by o_orderdate;
@seawinde seawinde mentioned this pull request Aug 21, 2024
Merged
yiguolei pushed a commit that referenced this pull request Aug 22, 2024
@seawinde
[fix](mtmv) Fix some pr to 21, prs are (#39041)(#38958)(#39541) (#39678)
021982f 
## Proposed changes

pr: #39041
commitId: 2256298

pr: #38958
commitId: c365cb6

pr: #39541
commitId: 89bb669
dataroaring pushed a commit that referenced this pull request Aug 23, 2024
@seawinde @dataroaring
[fix](mtmv) Fix select literal result wrongly in group by when use ma…
c5643e1 
…terialized view (#38958)

This is brought by #34274

if mv def is 
select o_orderdate from  orders group by o_orderdate;

query is as followiing, the result is wrong.
select 1 from orders group by o_orderdate;
@yiguolei yiguolei mentioned this pull request Sep 5, 2024
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@starocean999 starocean999 starocean999 left review comments

@zfr9527 zfr9527 zfr9527 approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.2-merged doing p0_c reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@seawinde @doris-robot @morrySnow @starocean999 @zfr9527 @gavinchou @yiguolei