Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improvement](statistics)External table getRowCount return -1 when row count is not available or row count is 0. #43009

Merged
merged 1 commit into from
Nov 1, 2024
Merged

[improvement](statistics)External table getRowCount return -1 when row count is not available or row count is 0. #43009

merged 1 commit into from
Nov 1, 2024

Conversation

Jibing-Li
Copy link
Contributor

External table getRowCount return -1 when row count is not available or row count is 0. So the behavior of external table could match with internal olap table.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Jibing-Li Jibing-Li marked this pull request as ready for review October 31, 2024 09:47
@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41339 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0d9696015de01c925bba0f8c577bbeb06dbd808f, data reload: false

------ Round 1 ----------------------------------
q1	17561	7458	7339	7339
q2	2044	168	167	167
q3	10624	1096	1177	1096
q4	10579	847	917	847
q5	7749	3048	3108	3048
q6	233	146	150	146
q7	1021	617	590	590
q8	9345	1964	2051	1964
q9	6600	6456	6440	6440
q10	7073	2488	2427	2427
q11	435	245	255	245
q12	415	232	218	218
q13	17771	3061	3004	3004
q14	246	211	209	209
q15	592	523	513	513
q16	636	576	579	576
q17	973	593	518	518
q18	7237	6758	6657	6657
q19	1330	1104	1019	1019
q20	469	186	182	182
q21	3971	3141	3287	3141
q22	1127	993	1006	993
Total cold run time: 108031 ms
Total hot run time: 41339 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7323	7314	7287	7287
q2	330	231	225	225
q3	3000	2948	2939	2939
q4	2112	1928	1814	1814
q5	5762	5690	5771	5690
q6	232	145	143	143
q7	2247	1818	1816	1816
q8	3384	3570	3437	3437
q9	8980	8949	8889	8889
q10	3594	3559	3531	3531
q11	594	494	493	493
q12	873	657	600	600
q13	12460	3224	3228	3224
q14	312	276	282	276
q15	570	507	523	507
q16	686	648	662	648
q17	1876	1619	1605	1605
q18	8264	7909	7660	7660
q19	1724	1609	1648	1609
q20	2134	1865	1915	1865
q21	5621	5427	5626	5427
q22	1149	1080	1050	1050
Total cold run time: 73227 ms
Total hot run time: 60735 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196233 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0d9696015de01c925bba0f8c577bbeb06dbd808f, data reload: false

query1	1171	917	927	917
query2	6246	2095	2025	2025
query3	10786	4025	4099	4025
query4	67793	28656	23559	23559
query5	4905	454	439	439
query6	401	174	178	174
query7	5491	293	293	293
query8	292	220	233	220
query9	8619	2730	2723	2723
query10	422	252	248	248
query11	17357	15275	15818	15275
query12	150	107	103	103
query13	1457	446	427	427
query14	10162	7256	6712	6712
query15	214	190	180	180
query16	7149	491	501	491
query17	1003	550	550	550
query18	1813	298	299	298
query19	208	150	147	147
query20	113	110	111	110
query21	204	100	102	100
query22	4801	4351	4532	4351
query23	34748	34178	34263	34178
query24	5964	2754	2737	2737
query25	491	374	370	370
query26	647	156	151	151
query27	1665	285	284	284
query28	4256	2439	2431	2431
query29	671	402	394	394
query30	238	160	154	154
query31	980	798	818	798
query32	63	53	56	53
query33	462	254	258	254
query34	906	520	530	520
query35	826	729	747	729
query36	1092	968	966	966
query37	121	76	68	68
query38	4419	4274	4274	4274
query39	1464	1443	1411	1411
query40	198	94	94	94
query41	54	43	42	42
query42	111	105	98	98
query43	534	498	492	492
query44	1157	813	830	813
query45	181	162	165	162
query46	1113	724	680	680
query47	1971	1843	1805	1805
query48	425	318	321	318
query49	699	378	376	376
query50	809	401	383	383
query51	7280	7194	7128	7128
query52	100	88	86	86
query53	250	172	176	172
query54	505	403	389	389
query55	78	78	75	75
query56	248	245	237	237
query57	1281	1177	1162	1162
query58	211	202	206	202
query59	3220	3062	3050	3050
query60	280	239	246	239
query61	101	99	102	99
query62	790	697	673	673
query63	209	185	182	182
query64	1342	625	599	599
query65	3309	3196	3228	3196
query66	707	321	306	306
query67	15878	15782	15741	15741
query68	2660	589	584	584
query69	400	256	252	252
query70	1176	1155	1149	1149
query71	361	271	253	253
query72	5403	4140	4113	4113
query73	763	359	362	359
query74	9661	9079	9162	9079
query75	3407	2682	2672	2672
query76	1647	1137	1031	1031
query77	423	293	279	279
query78	10391	9470	9400	9400
query79	2385	612	617	612
query80	1438	448	488	448
query81	540	237	239	237
query82	1254	115	113	113
query83	244	138	140	138
query84	277	75	69	69
query85	1008	292	279	279
query86	425	299	300	299
query87	4768	4719	4631	4631
query88	3820	2186	2165	2165
query89	409	290	286	286
query90	1960	185	187	185
query91	136	100	100	100
query92	61	49	48	48
query93	2850	544	552	544
query94	873	250	284	250
query95	349	246	244	244
query96	636	280	290	280
query97	2935	2705	2700	2700
query98	226	203	191	191
query99	1604	1315	1305	1305
Total cold run time: 317301 ms
Total hot run time: 196233 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.23 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0d9696015de01c925bba0f8c577bbeb06dbd808f, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.22	0.07	0.07
query4	1.64	0.10	0.10
query5	0.41	0.42	0.41
query6	1.14	0.66	0.66
query7	0.02	0.02	0.01
query8	0.04	0.05	0.02
query9	0.57	0.50	0.49
query10	0.55	0.55	0.58
query11	0.15	0.10	0.10
query12	0.13	0.11	0.11
query13	0.61	0.59	0.60
query14	2.70	2.72	2.78
query15	0.90	0.82	0.83
query16	0.41	0.41	0.40
query17	1.00	1.02	1.08
query18	0.20	0.20	0.20
query19	2.01	1.90	2.00
query20	0.01	0.01	0.01
query21	15.36	0.61	0.58
query22	2.44	2.50	1.70
query23	17.03	1.04	0.86
query24	2.82	1.68	1.80
query25	0.24	0.23	0.05
query26	0.51	0.15	0.14
query27	0.05	0.04	0.06
query28	9.92	1.08	1.08
query29	12.52	3.30	3.31
query30	0.25	0.06	0.06
query31	2.87	0.38	0.39
query32	3.27	0.47	0.46
query33	2.99	3.06	3.02
query34	16.96	4.45	4.43
query35	4.58	4.48	4.44
query36	0.64	0.47	0.48
query37	0.08	0.06	0.06
query38	0.04	0.03	0.04
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.72 s
Total hot run time: 33.23 s

@Jibing-Li
Copy link
Contributor Author

run external

2 similar comments
@Jibing-Li
Copy link
Contributor Author

run external

@Jibing-Li
Copy link
Contributor Author

run external

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 1, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 1, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 1, 2024

PR approved by anyone and no changes requested.

@Jibing-Li Jibing-Li merged commit 601d801 into apache:master Nov 1, 2024
25 of 27 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 1, 2024
@Jibing-Li
[improvement](statistics)External table getRowCount return -1 when ro…
3f5be04 
…w count is not available or row count is 0. (#43009)

External table getRowCount return -1 when row count is not available or
row count is 0. So the behavior of external table could match with
internal olap table.
@github-actions github-actions bot mentioned this pull request Nov 1, 2024
Closed
@Jibing-Li Jibing-Li deleted the externalrowcount branch November 1, 2024 06:35
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Nov 1, 2024
@Jibing-Li
[improvement](statistics)External table getRowCount return -1 when ro…
522d63e 
…w count is not available or row count is 0. (apache#43009)

External table getRowCount return -1 when row count is not available or
row count is 0. So the behavior of external table could match with
internal olap table.
@Jibing-Li Jibing-Li mentioned this pull request Nov 1, 2024
Merged
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Nov 1, 2024
@Jibing-Li
[improvement](statistics)External table getRowCount return -1 when ro…
a3bd58b 
…w count is not available or row count is 0. (apache#43009)

External table getRowCount return -1 when row count is not available or
row count is 0. So the behavior of external table could match with
internal olap table.
@Jibing-Li Jibing-Li mentioned this pull request Nov 1, 2024
Merged
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Nov 1, 2024
@Jibing-Li
[improvement](statistics)External table getRowCount return -1 when ro…
27f72a4 
…w count is not available or row count is 0. (apache#43009)

External table getRowCount return -1 when row count is not available or
row count is 0. So the behavior of external table could match with
internal olap table.
Jibing-Li added a commit that referenced this pull request Nov 1, 2024
@Jibing-Li
[improvement](statistics)External table getRowCount return -1 when ro…
594865c 
…w count is not available or row count is 0. (#43009) (#43081)

backport: #43009
Jibing-Li added a commit that referenced this pull request Nov 2, 2024
@Jibing-Li
[improvement](statistics)External table getRowCount return -1 when ro…
3b9a240 
…w count is not available or row count is 0. (#43009) (#43087)

backport: #43009
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Nov 12, 2024
@Jibing-Li
[improvement](statistics)External table getRowCount return -1 when ro…
7c11cea 
…w count is not available or row count is 0. (apache#43009)

External table getRowCount return -1 when row count is not available or
row count is 0. So the behavior of external table could match with
internal olap table.
@Jibing-Li Jibing-Li mentioned this pull request Nov 12, 2024
Merged
Jibing-Li added a commit that referenced this pull request Nov 12, 2024
@Jibing-Li
[improvement](statistics)External table getRowCount return -1 when ro…
6e021dc 
…w count is not available or row count is 0. (#43009) (#43771)

backport: #43009
@gavinchou gavinchou mentioned this pull request Nov 26, 2024
Open
@yiguolei yiguolei mentioned this pull request Jan 19, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrySnow morrySnow morrySnow approved these changes

@xzj7019 xzj7019 xzj7019 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.16-merged dev/2.1.8-merged dev/3.0.3-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Jibing-Li @doris-robot @morrySnow @xzj7019