Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Opt](load) don't print stack when some errors occur for stream load … #38658

Merged
merged 1 commit into from
Aug 1, 2024
Merged

[Opt](load) don't print stack when some errors occur for stream load … #38658

merged 1 commit into from
Aug 1, 2024
+8 −9

Conversation

liaoxin01
Copy link
Contributor

cherry pick from #38332

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@liaoxin01
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 1, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.73% (8100/21466)
Line Coverage: 29.37% (66323/225799)
Region Coverage: 28.88% (34219/118477)
Branch Coverage: 24.77% (17582/70982)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ad4177e1a46571ce8fa7ebf8f9d09d9e74034199_ad4177e1a46571ce8fa7ebf8f9d09d9e74034199/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 49822 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ad4177e1a46571ce8fa7ebf8f9d09d9e74034199, data reload: false

------ Round 1 ----------------------------------
q1	18034	4397	4334	4334
q2	2071	166	152	152
q3	10353	1905	1952	1905
q4	10377	1231	1316	1231
q5	8367	3835	3897	3835
q6	256	131	122	122
q7	2041	1609	1612	1609
q8	9292	2733	2709	2709
q9	10554	10385	10269	10269
q10	8628	3542	3543	3542
q11	428	245	252	245
q12	468	302	303	302
q13	18353	3958	3996	3958
q14	353	330	326	326
q15	516	456	461	456
q16	672	582	558	558
q17	1143	963	949	949
q18	7321	6902	6971	6902
q19	1795	1657	1623	1623
q20	536	292	291	291
q21	4416	4141	4064	4064
q22	533	440	448	440
Total cold run time: 116507 ms
Total hot run time: 49822 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4347	4324	4373	4324
q2	327	228	226	226
q3	4200	4158	4124	4124
q4	2754	2739	2741	2739
q5	7211	7140	7099	7099
q6	239	120	115	115
q7	3259	2816	2780	2780
q8	4374	4466	4483	4466
q9	16989	17437	16795	16795
q10	4212	4246	4232	4232
q11	776	686	679	679
q12	1040	861	872	861
q13	5718	3726	3763	3726
q14	451	427	428	427
q15	500	456	455	455
q16	739	677	670	670
q17	3768	3858	3870	3858
q18	8774	8668	8709	8668
q19	1710	1737	1658	1658
q20	2372	2103	2101	2101
q21	8461	8590	8480	8480
q22	1059	973	1007	973
Total cold run time: 83280 ms
Total hot run time: 79456 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203290 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ad4177e1a46571ce8fa7ebf8f9d09d9e74034199, data reload: false

query1	918	426	373	373
query2	6563	2878	2674	2674
query3	6920	213	202	202
query4	21351	17971	17948	17948
query5	19752	6562	6504	6504
query6	293	223	229	223
query7	4167	312	332	312
query8	445	431	408	408
query9	3149	2662	2617	2617
query10	432	317	300	300
query11	11389	10792	10638	10638
query12	125	81	77	77
query13	5608	717	707	707
query14	17810	13329	13297	13297
query15	369	243	239	239
query16	6464	286	263	263
query17	1734	1456	864	864
query18	2320	417	428	417
query19	203	148	157	148
query20	80	80	84	80
query21	205	102	104	102
query22	5180	5081	4936	4936
query23	32817	32068	31980	31980
query24	6995	6493	6464	6464
query25	531	439	428	428
query26	526	173	165	165
query27	1869	301	303	301
query28	6159	2341	2313	2313
query29	2795	2825	2691	2691
query30	239	167	166	166
query31	889	754	718	718
query32	66	60	59	59
query33	398	265	255	255
query34	847	480	496	480
query35	1105	962	931	931
query36	1432	1484	1182	1182
query37	94	59	65	59
query38	3102	2886	2949	2886
query39	1365	1329	1323	1323
query40	204	94	100	94
query41	46	44	45	44
query42	84	89	85	85
query43	832	784	905	784
query44	1125	725	712	712
query45	251	239	243	239
query46	1234	970	986	970
query47	1820	1867	2016	1867
query48	996	733	720	720
query49	626	371	372	371
query50	860	637	591	591
query51	4803	4677	4631	4631
query52	98	86	94	86
query53	446	339	324	324
query54	2682	2477	2502	2477
query55	99	80	86	80
query56	260	212	211	211
query57	1177	1048	1109	1048
query58	224	208	198	198
query59	4141	4098	4167	4098
query60	211	200	216	200
query61	99	96	97	96
query62	840	520	514	514
query63	493	341	349	341
query64	2489	1540	1347	1347
query65	3660	3572	3580	3572
query66	816	383	388	383
query67	15825	14868	16797	14868
query68	8540	638	666	638
query69	575	343	329	329
query70	1642	1559	1326	1326
query71	400	305	327	305
query72	6580	3485	3473	3473
query73	726	324	322	322
query74	6376	5894	5871	5871
query75	5129	3618	3824	3618
query76	5146	1154	1182	1154
query77	830	252	262	252
query78	12520	11920	11797	11797
query79	8996	645	651	645
query80	1258	416	403	403
query81	483	239	234	234
query82	1673	99	94	94
query83	183	137	133	133
query84	259	72	71	71
query85	927	327	342	327
query86	355	292	296	292
query87	3185	3011	3037	3011
query88	5184	2284	2269	2269
query89	389	287	280	280
query90	1922	216	205	205
query91	171	163	137	137
query92	63	53	58	53
query93	5670	605	566	566
query94	733	208	206	206
query95	1111	1052	1038	1038
query96	646	327	324	324
query97	6483	6388	6257	6257
query98	176	182	177	177
query99	2779	873	802	802
Total cold run time: 314257 ms
Total hot run time: 203290 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.44 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ad4177e1a46571ce8fa7ebf8f9d09d9e74034199, data reload: false

query1	0.02	0.02	0.03
query2	0.06	0.03	0.03
query3	0.25	0.05	0.04
query4	1.79	0.07	0.07
query5	0.55	0.53	0.53
query6	1.22	0.63	0.62
query7	0.02	0.01	0.01
query8	0.03	0.02	0.02
query9	0.54	0.50	0.47
query10	0.54	0.52	0.52
query11	0.12	0.08	0.09
query12	0.12	0.09	0.09
query13	0.63	0.61	0.61
query14	0.78	0.78	0.78
query15	0.77	0.76	0.76
query16	0.36	0.36	0.35
query17	0.98	1.03	1.00
query18	0.20	0.29	0.23
query19	1.94	1.82	1.86
query20	0.02	0.01	0.01
query21	15.48	0.54	0.52
query22	2.07	1.99	1.50
query23	16.79	0.92	0.96
query24	3.83	3.31	0.96
query25	0.39	0.13	0.05
query26	0.60	0.16	0.15
query27	0.05	0.04	0.03
query28	7.06	0.78	0.72
query29	12.66	2.24	2.41
query30	0.64	0.53	0.57
query31	2.82	0.39	0.38
query32	3.37	0.50	0.49
query33	3.12	3.07	3.11
query34	15.29	4.79	4.76
query35	4.84	4.84	4.82
query36	1.07	1.00	1.02
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.15	0.15	0.13
query41	0.07	0.02	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 101.4 s
Total hot run time: 30.44 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit ad4177e1a46571ce8fa7ebf8f9d09d9e74034199 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@liaoxin01 liaoxin01 merged commit f49a731 into apache:branch-2.0 Aug 1, 2024
22 of 25 checks passed
@liaoxin01 liaoxin01 deleted the pick_38332_to_origin_branch-2.0 branch August 1, 2024 03:48
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
@liaoxin01
[Opt](load) don't print stack when some errors occur for stream load … (
2de6510 
apache#38658)

cherry pick from apache#38332
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@liaoxin01 @doris-robot