Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[feature](inverted index) String type inverted index match function completion #38170

Merged
merged 1 commit into from
Jul 31, 2024
Merged

[feature](inverted index) String type inverted index match function completion #38170

merged 1 commit into from
Jul 31, 2024

Conversation

zzzxl1993
Copy link
Contributor

Proposed changes

  1. Inverted index of string type supports match_phrase_prefix and match_regexp.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zzzxl1993
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40148 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 11f71ef17bfd66a96a840faff8b306723535dc0d, data reload: false

------ Round 1 ----------------------------------
q1	17630	4564	4298	4298
q2	2019	197	189	189
q3	10439	1217	1135	1135
q4	10183	855	837	837
q5	7558	2749	2709	2709
q6	221	134	137	134
q7	955	608	612	608
q8	9224	2074	2135	2074
q9	8957	6549	6562	6549
q10	8855	3778	3821	3778
q11	454	239	235	235
q12	402	222	232	222
q13	18622	2985	3005	2985
q14	276	231	236	231
q15	541	480	481	480
q16	504	381	385	381
q17	977	653	778	653
q18	8176	7510	7473	7473
q19	6754	1410	1411	1410
q20	698	314	327	314
q21	4854	3166	3229	3166
q22	356	287	290	287
Total cold run time: 118655 ms
Total hot run time: 40148 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4356	4259	4243	4243
q2	389	263	265	263
q3	3097	2864	3016	2864
q4	1970	1753	1743	1743
q5	5581	5598	5421	5421
q6	221	129	137	129
q7	2278	1903	1843	1843
q8	3303	3465	3412	3412
q9	8792	8894	8839	8839
q10	4116	3764	3966	3764
q11	593	494	511	494
q12	810	618	618	618
q13	16291	3178	3233	3178
q14	347	299	279	279
q15	528	502	493	493
q16	519	443	439	439
q17	1830	1525	1534	1525
q18	8137	7961	7829	7829
q19	1724	1478	1627	1478
q20	2222	1908	1874	1874
q21	5096	4708	4832	4708
q22	577	525	521	521
Total cold run time: 72777 ms
Total hot run time: 55957 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174108 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 11f71ef17bfd66a96a840faff8b306723535dc0d, data reload: false

query1	922	371	368	368
query2	6420	1786	1787	1786
query3	6625	205	218	205
query4	28396	17482	17315	17315
query5	3700	497	478	478
query6	280	168	179	168
query7	4595	283	282	282
query8	239	193	193	193
query9	8583	2400	2386	2386
query10	441	293	277	277
query11	10580	10190	10112	10112
query12	118	84	86	84
query13	1661	378	368	368
query14	10278	7702	7886	7702
query15	235	172	170	170
query16	7639	496	452	452
query17	1580	593	586	586
query18	1687	272	275	272
query19	198	146	149	146
query20	91	82	82	82
query21	210	125	130	125
query22	4349	3965	3932	3932
query23	34199	33671	33549	33549
query24	11023	2934	2895	2895
query25	653	377	392	377
query26	1138	155	150	150
query27	2424	274	274	274
query28	6727	2071	2068	2068
query29	878	639	656	639
query30	265	155	159	155
query31	990	749	770	749
query32	90	56	55	55
query33	767	345	335	335
query34	906	497	506	497
query35	898	754	749	749
query36	1141	960	1001	960
query37	152	87	90	87
query38	3011	2941	2828	2828
query39	924	850	856	850
query40	211	122	122	122
query41	47	44	43	43
query42	117	99	98	98
query43	506	474	465	465
query44	1198	730	728	728
query45	194	160	159	159
query46	1111	718	712	712
query47	1867	1756	1768	1756
query48	377	295	291	291
query49	851	416	427	416
query50	794	394	400	394
query51	6878	6840	6787	6787
query52	111	89	98	89
query53	359	291	297	291
query54	878	451	438	438
query55	76	74	75	74
query56	288	260	275	260
query57	1112	1014	1037	1014
query58	258	246	256	246
query59	2728	2587	2712	2587
query60	299	284	285	284
query61	109	94	128	94
query62	788	652	658	652
query63	328	288	284	284
query64	9401	2256	1684	1684
query65	3187	3116	3100	3100
query66	742	342	335	335
query67	15402	15068	14957	14957
query68	4483	535	543	535
query69	563	430	369	369
query70	1137	1136	1171	1136
query71	414	294	277	277
query72	7141	5648	5707	5648
query73	763	323	331	323
query74	6287	5732	5651	5651
query75	3552	2777	2694	2694
query76	2457	975	952	952
query77	468	307	316	307
query78	9831	8882	10020	8882
query79	2048	534	519	519
query80	2277	487	494	487
query81	598	223	228	223
query82	606	142	142	142
query83	300	168	172	168
query84	272	83	85	83
query85	2004	313	301	301
query86	481	309	332	309
query87	3264	3141	3105	3105
query88	4119	2374	2379	2374
query89	466	372	400	372
query90	1799	200	200	200
query91	129	174	100	100
query92	63	54	52	52
query93	1710	506	501	501
query94	1182	297	295	295
query95	414	326	325	325
query96	599	276	276	276
query97	3210	3009	3052	3009
query98	236	205	188	188
query99	1510	1239	1271	1239
Total cold run time: 279613 ms
Total hot run time: 174108 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.35 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 11f71ef17bfd66a96a840faff8b306723535dc0d, data reload: false

query1	0.05	0.04	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.08	0.07
query5	0.49	0.48	0.48
query6	1.15	0.74	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.50
query10	0.55	0.54	0.55
query11	0.15	0.12	0.11
query12	0.15	0.13	0.12
query13	0.60	0.59	0.58
query14	0.76	0.80	0.77
query15	0.86	0.82	0.82
query16	0.38	0.35	0.36
query17	0.98	1.04	1.06
query18	0.23	0.21	0.21
query19	1.76	1.71	1.73
query20	0.02	0.01	0.01
query21	15.39	0.78	0.65
query22	3.89	8.47	1.51
query23	18.32	1.31	1.32
query24	2.11	0.22	0.23
query25	0.17	0.09	0.08
query26	0.30	0.21	0.21
query27	0.46	0.24	0.23
query28	13.26	1.02	0.99
query29	12.64	3.37	3.33
query30	0.26	0.06	0.05
query31	2.89	0.38	0.40
query32	3.28	0.48	0.47
query33	2.87	2.97	2.94
query34	17.10	4.31	4.39
query35	4.50	4.41	4.41
query36	0.65	0.47	0.49
query37	0.18	0.16	0.15
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.15	0.12	0.12
query41	0.10	0.05	0.05
query42	0.06	0.04	0.05
query43	0.04	0.05	0.04
Total cold run time: 109.55 s
Total hot run time: 30.35 s

qidaye
qidaye approved these changes Jul 23, 2024
View reviewed changes
Copy link
Contributor

@qidaye qidaye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 23, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

csun5285
csun5285 approved these changes Jul 26, 2024
View reviewed changes
Copy link
Contributor

@csun5285 csun5285 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qidaye qidaye merged commit 6fdce0d into apache:master Jul 31, 2024
27 of 30 checks passed
@xiaokang xiaokang added usercase Important user case type label dev/2.1.x dev/3.0.x labels Jul 31, 2024
zzzxl1993 added a commit to zzzxl1993/doris that referenced this pull request Aug 5, 2024
@zzzxl1993
[feature](inverted index) String type inverted index match function c…
5b31f12 
…ompletion (apache#38170)

1. Inverted index of string type supports match_phrase_prefix and
match_regexp.
@zzzxl1993 zzzxl1993 mentioned this pull request Aug 5, 2024
Merged
zzzxl1993 added a commit to zzzxl1993/doris that referenced this pull request Aug 5, 2024
@zzzxl1993
[feature](inverted index) String type inverted index match function c…
4cb316b 
…ompletion (apache#38170)

1. Inverted index of string type supports match_phrase_prefix and
match_regexp.
@zzzxl1993 zzzxl1993 mentioned this pull request Aug 5, 2024
Closed
zzzxl1993 added a commit to zzzxl1993/doris that referenced this pull request Aug 5, 2024
@zzzxl1993
[feature](inverted index) String type inverted index match function c…
d572d5d 
…ompletion (apache#38170)

1. Inverted index of string type supports match_phrase_prefix and
match_regexp.
yiguolei pushed a commit that referenced this pull request Aug 6, 2024
@zzzxl1993
[feature](inverted index) String type inverted index match function c… (
bcea541 
#38872)

#38170
dataroaring pushed a commit that referenced this pull request Aug 6, 2024
@zzzxl1993 @dataroaring
[feature](inverted index) String type inverted index match function c…
f082119 
…ompletion (#38170)

1. Inverted index of string type supports match_phrase_prefix and
match_regexp.
zzzxl1993 added a commit to zzzxl1993/doris that referenced this pull request Aug 6, 2024
@zzzxl1993
[feature](inverted index) String type inverted index match function c… (
96994fe 
apache#38872)

apache#38170
feiniaofeiafei pushed a commit to feiniaofeiafei/doris that referenced this pull request Aug 9, 2024
@zzzxl1993 @feiniaofeiafei
[feature](inverted index) String type inverted index match function c…
0b9dbc6 
…ompletion (apache#38170)

1. Inverted index of string type supports match_phrase_prefix and
match_regexp.
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@qidaye qidaye qidaye approved these changes

@csun5285 csun5285 csun5285 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.1-merged reviewed usercase Important user case type label
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@zzzxl1993 @doris-robot @qidaye @csun5285 @xiaokang @yiguolei @dataroaring