Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](nereids)fix nullable property of ForEachCombinator #37796

Merged
merged 1 commit into from
Jul 17, 2024
Merged

[fix](nereids)fix nullable property of ForEachCombinator #37796

merged 1 commit into from
Jul 17, 2024

Conversation

starocean999
Copy link
Contributor

Proposed changes

ForEachCombinator should be NullableAggregateFunction instead of AlwaysNullable

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39890 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d8a5e52fe5b7f7c1f18354ff9e44aa6fd07dc2c0, data reload: false

------ Round 1 ----------------------------------
q1	17629	4318	4311	4311
q2	2021	187	186	186
q3	10466	1178	1163	1163
q4	10196	928	854	854
q5	7552	2662	2641	2641
q6	221	138	137	137
q7	952	594	596	594
q8	9211	2048	2073	2048
q9	8871	6557	6536	6536
q10	8787	3797	3753	3753
q11	452	235	240	235
q12	394	227	233	227
q13	17767	3018	2963	2963
q14	277	232	255	232
q15	517	473	475	473
q16	494	396	375	375
q17	952	640	746	640
q18	8123	7463	7554	7463
q19	7525	1403	1371	1371
q20	703	319	330	319
q21	4816	3086	3331	3086
q22	343	291	283	283
Total cold run time: 118269 ms
Total hot run time: 39890 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4461	4266	4264	4264
q2	367	264	272	264
q3	3018	2957	2980	2957
q4	2027	1720	1751	1720
q5	5639	5613	5497	5497
q6	228	136	134	134
q7	2305	1855	1873	1855
q8	3296	3495	3405	3405
q9	8836	8897	8891	8891
q10	4147	3878	3987	3878
q11	626	528	508	508
q12	852	688	703	688
q13	16339	3342	3402	3342
q14	340	293	316	293
q15	539	494	503	494
q16	517	461	472	461
q17	1843	1538	1515	1515
q18	8615	9416	7939	7939
q19	1828	1560	1550	1550
q20	2156	1926	1862	1862
q21	8219	4926	4770	4770
q22	574	523	493	493
Total cold run time: 76772 ms
Total hot run time: 56780 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174704 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d8a5e52fe5b7f7c1f18354ff9e44aa6fd07dc2c0, data reload: false

query1	922	379	364	364
query2	6455	1827	1824	1824
query3	6630	205	218	205
query4	28343	17370	17257	17257
query5	3628	508	511	508
query6	269	189	202	189
query7	4577	298	288	288
query8	246	199	197	197
query9	8681	2393	2357	2357
query10	452	289	271	271
query11	11716	10076	10290	10076
query12	117	89	89	89
query13	1653	372	373	372
query14	10335	7907	7795	7795
query15	253	162	166	162
query16	7617	319	310	310
query17	1794	593	526	526
query18	1827	275	277	275
query19	193	155	153	153
query20	94	82	79	79
query21	203	130	122	122
query22	4245	4075	3975	3975
query23	34119	33474	34580	33474
query24	10793	2956	2927	2927
query25	664	408	391	391
query26	1058	159	151	151
query27	2292	269	287	269
query28	7170	2048	2034	2034
query29	900	644	650	644
query30	251	160	159	159
query31	968	783	743	743
query32	97	55	59	55
query33	766	327	300	300
query34	973	500	513	500
query35	699	590	598	590
query36	1139	984	1007	984
query37	179	91	90	90
query38	2946	2903	2937	2903
query39	952	839	884	839
query40	219	123	127	123
query41	45	47	46	46
query42	118	102	104	102
query43	506	444	455	444
query44	1192	729	751	729
query45	203	163	164	163
query46	1087	720	750	720
query47	1847	1763	1782	1763
query48	373	292	291	291
query49	830	417	420	417
query50	784	395	407	395
query51	6952	6813	6819	6813
query52	100	98	90	90
query53	368	292	296	292
query54	916	449	453	449
query55	76	73	73	73
query56	286	275	273	273
query57	1130	1042	1055	1042
query58	278	256	260	256
query59	2781	2642	2574	2574
query60	313	282	291	282
query61	97	93	95	93
query62	759	640	633	633
query63	319	295	296	295
query64	9415	2232	2714	2232
query65	3160	3115	3093	3093
query66	702	340	351	340
query67	15563	14948	15260	14948
query68	4575	532	541	532
query69	577	401	371	371
query70	1147	1131	1179	1131
query71	417	293	289	289
query72	7931	5764	5810	5764
query73	747	330	327	327
query74	6245	5702	5737	5702
query75	3921	2664	2736	2664
query76	3047	918	959	918
query77	665	337	337	337
query78	11640	9344	8892	8892
query79	8221	530	521	521
query80	1438	508	525	508
query81	598	223	230	223
query82	850	133	141	133
query83	341	176	178	176
query84	275	92	93	92
query85	770	487	300	300
query86	460	317	322	317
query87	3363	3146	3106	3106
query88	5351	2498	2487	2487
query89	505	390	397	390
query90	2018	191	198	191
query91	132	100	105	100
query92	62	53	52	52
query93	4535	512	505	505
query94	1365	223	219	219
query95	412	391	324	324
query96	613	277	276	276
query97	3195	3024	3019	3019
query98	220	193	200	193
query99	1636	1262	1227	1227
Total cold run time: 293907 ms
Total hot run time: 174704 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.86 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d8a5e52fe5b7f7c1f18354ff9e44aa6fd07dc2c0, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.22	0.04	0.04
query4	1.69	0.06	0.07
query5	0.50	0.49	0.50
query6	1.14	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.53	0.47	0.46
query10	0.54	0.55	0.56
query11	0.15	0.10	0.11
query12	0.15	0.13	0.12
query13	0.59	0.58	0.58
query14	0.75	0.78	0.78
query15	0.86	0.81	0.81
query16	0.35	0.37	0.35
query17	0.94	0.99	0.98
query18	0.22	0.21	0.23
query19	1.93	1.82	1.73
query20	0.01	0.01	0.01
query21	15.41	0.75	0.66
query22	4.35	7.17	2.23
query23	18.24	1.37	1.28
query24	2.09	0.25	0.22
query25	0.14	0.08	0.09
query26	0.30	0.21	0.20
query27	0.45	0.23	0.23
query28	13.30	1.02	1.00
query29	12.65	3.30	3.30
query30	0.26	0.06	0.05
query31	2.88	0.40	0.39
query32	3.25	0.48	0.48
query33	2.90	2.91	2.89
query34	16.92	4.32	4.39
query35	4.39	4.38	4.40
query36	0.65	0.46	0.48
query37	0.18	0.15	0.16
query38	0.15	0.15	0.15
query39	0.04	0.04	0.03
query40	0.16	0.13	0.12
query41	0.08	0.05	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.65 s
Total hot run time: 30.86 s

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40488 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e4df75770c4fc26a13096de9097c79502c888d54, data reload: false

------ Round 1 ----------------------------------
q1	17756	4351	4253	4253
q2	2025	188	202	188
q3	10520	1221	1139	1139
q4	10191	769	749	749
q5	7583	2722	2660	2660
q6	218	136	137	136
q7	959	596	606	596
q8	9240	2092	2063	2063
q9	8876	6633	6629	6629
q10	8805	3809	3776	3776
q11	437	246	234	234
q12	421	224	226	224
q13	18027	3029	3011	3011
q14	289	252	239	239
q15	523	491	505	491
q16	550	400	408	400
q17	959	609	710	609
q18	9064	8121	7976	7976
q19	9112	1498	1358	1358
q20	729	331	343	331
q21	5457	3120	3302	3120
q22	359	306	308	306
Total cold run time: 122100 ms
Total hot run time: 40488 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4637	4351	4378	4351
q2	394	284	279	279
q3	3151	2900	2925	2900
q4	2035	1688	1754	1688
q5	5518	5471	5602	5471
q6	223	130	145	130
q7	2197	1838	1859	1838
q8	3270	3387	3408	3387
q9	8810	8827	8748	8748
q10	3993	3893	3846	3846
q11	624	490	507	490
q12	782	648	651	648
q13	16026	3156	3219	3156
q14	314	301	284	284
q15	523	486	478	478
q16	488	428	434	428
q17	1854	1542	1555	1542
q18	8105	8088	7851	7851
q19	1735	1598	1708	1598
q20	2192	1856	1850	1850
q21	6928	4865	4850	4850
q22	787	523	501	501
Total cold run time: 74586 ms
Total hot run time: 56314 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174434 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e4df75770c4fc26a13096de9097c79502c888d54, data reload: false

query1	917	380	366	366
query2	6359	1908	1956	1908
query3	6632	222	213	213
query4	28107	17368	17420	17368
query5	3580	489	480	480
query6	270	189	151	151
query7	4577	290	288	288
query8	238	192	191	191
query9	8689	2407	2402	2402
query10	438	295	271	271
query11	11761	9843	10011	9843
query12	132	82	83	82
query13	1628	368	372	368
query14	10165	8398	7588	7588
query15	218	162	172	162
query16	7674	316	304	304
query17	1797	546	536	536
query18	1926	294	271	271
query19	193	145	148	145
query20	89	79	83	79
query21	203	126	125	125
query22	4352	4275	4090	4090
query23	34080	33590	33624	33590
query24	9053	2958	2993	2958
query25	608	379	400	379
query26	701	152	151	151
query27	2228	284	276	276
query28	5824	2111	2125	2111
query29	900	659	633	633
query30	258	154	161	154
query31	962	761	771	761
query32	96	55	52	52
query33	677	318	318	318
query34	932	511	498	498
query35	676	581	586	581
query36	1116	970	1024	970
query37	139	83	86	83
query38	3031	2862	2818	2818
query39	921	856	811	811
query40	200	121	118	118
query41	49	44	47	44
query42	118	103	104	103
query43	520	467	454	454
query44	1068	732	721	721
query45	189	161	167	161
query46	1081	738	730	730
query47	1838	1801	1775	1775
query48	370	297	290	290
query49	841	411	415	411
query50	767	403	390	390
query51	6954	6863	6716	6716
query52	105	90	89	89
query53	354	288	282	282
query54	861	463	491	463
query55	78	77	71	71
query56	292	276	270	270
query57	1120	1074	1032	1032
query58	251	266	253	253
query59	2965	2639	2599	2599
query60	290	271	277	271
query61	98	95	92	92
query62	786	636	652	636
query63	314	287	295	287
query64	9186	2241	7497	2241
query65	3178	3133	3143	3133
query66	775	345	337	337
query67	15318	15084	15121	15084
query68	5924	570	555	555
query69	657	460	379	379
query70	1185	1110	1117	1110
query71	444	287	284	284
query72	7604	5824	5630	5630
query73	754	326	338	326
query74	6151	5646	5652	5646
query75	3592	2749	2734	2734
query76	3152	893	981	893
query77	596	311	323	311
query78	11165	8998	8975	8975
query79	8929	526	534	526
query80	1680	501	494	494
query81	591	222	235	222
query82	754	136	138	136
query83	218	178	172	172
query84	276	93	89	89
query85	1432	480	287	287
query86	438	290	293	290
query87	3316	3111	3078	3078
query88	4676	2371	2352	2352
query89	513	384	387	384
query90	1882	195	189	189
query91	128	101	102	101
query92	59	47	106	47
query93	6672	523	524	523
query94	775	218	214	214
query95	404	308	323	308
query96	608	276	274	274
query97	3166	3038	3035	3035
query98	215	201	195	195
query99	1689	1247	1260	1247
Total cold run time: 291758 ms
Total hot run time: 174434 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.35 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e4df75770c4fc26a13096de9097c79502c888d54, data reload: false

query1	0.04	0.04	0.02
query2	0.08	0.04	0.03
query3	0.22	0.05	0.05
query4	1.69	0.07	0.07
query5	0.50	0.48	0.48
query6	1.14	0.73	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.55	0.49	0.50
query10	0.56	0.54	0.53
query11	0.15	0.11	0.12
query12	0.15	0.13	0.12
query13	0.59	0.58	0.58
query14	0.76	0.79	0.80
query15	0.86	0.81	0.82
query16	0.36	0.36	0.37
query17	1.04	1.04	1.05
query18	0.22	0.22	0.22
query19	1.84	1.77	1.74
query20	0.02	0.01	0.01
query21	15.46	0.74	0.66
query22	3.89	7.20	2.59
query23	18.31	1.44	1.24
query24	2.15	0.23	0.22
query25	0.17	0.09	0.08
query26	0.30	0.21	0.22
query27	0.46	0.24	0.24
query28	13.28	1.02	1.01
query29	12.63	3.30	3.29
query30	0.26	0.06	0.05
query31	2.85	0.39	0.39
query32	3.27	0.48	0.49
query33	2.91	2.91	2.88
query34	17.16	4.37	4.36
query35	4.39	4.44	4.40
query36	0.65	0.47	0.48
query37	0.19	0.15	0.15
query38	0.14	0.14	0.15
query39	0.04	0.04	0.03
query40	0.15	0.12	0.12
query41	0.09	0.04	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.69 s
Total hot run time: 31.35 s

@starocean999 starocean999 marked this pull request as ready for review July 16, 2024 08:30
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 17, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@starocean999 starocean999 mentioned this pull request Jul 17, 2024
Merged
yiguolei pushed a commit that referenced this pull request Jul 17, 2024
@starocean999
[fix](nereids)fix nullable property of ForEachCombinator (#37980)
b2a4cff 
## Proposed changes

pick from master #37796

<!--Describe your changes.-->
@yiguolei yiguolei merged commit 2a88288 into apache:master Jul 17, 2024
27 of 29 checks passed
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
@starocean999 @dataroaring
[fix](nereids)fix nullable property of ForEachCombinator (#37796)
cdf00a8 
## Proposed changes

ForEachCombinator should be NullableAggregateFunction instead of
AlwaysNullable

<!--Describe your changes.-->
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@englefly englefly englefly approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.1-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@starocean999 @doris-robot @englefly @morrySnow @yiguolei @dataroaring