Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](inverted index) fix multi match result error #38931

Merged
merged 1 commit into from
Aug 7, 2024

Conversation

zzzxl1993
Copy link
Contributor

Proposed changes

  1. multi_match result merging logic is incorrect

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zzzxl1993
Copy link
Contributor Author

run buildall

Copy link
Contributor

github-actions bot commented Aug 6, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 41596 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9f9ffa465d0578d000ae05d95b20029173ebd253, data reload: false

------ Round 1 ----------------------------------
q1	17641	4131	4073	4073
q2	2019	202	192	192
q3	10450	1337	1330	1330
q4	10164	780	918	780
q5	7655	2953	2988	2953
q6	218	139	136	136
q7	1058	612	611	611
q8	9442	1889	1970	1889
q9	8466	6603	6597	6597
q10	8823	3868	3841	3841
q11	448	245	250	245
q12	433	229	223	223
q13	17761	2947	2933	2933
q14	279	242	241	241
q15	533	485	504	485
q16	535	404	388	388
q17	968	889	891	889
q18	8039	7368	7280	7280
q19	1963	1203	1220	1203
q20	554	321	322	321
q21	5290	4845	4710	4710
q22	356	284	276	276
Total cold run time: 113095 ms
Total hot run time: 41596 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4093	3994	3990	3990
q2	325	219	215	215
q3	3017	3014	3208	3014
q4	2012	2016	1951	1951
q5	5639	5447	5480	5447
q6	217	129	128	128
q7	2108	1766	1839	1766
q8	3295	3386	3324	3324
q9	8675	8632	8818	8632
q10	3917	4067	3896	3896
q11	559	439	480	439
q12	740	578	581	578
q13	16356	3062	3084	3062
q14	291	280	264	264
q15	531	484	491	484
q16	464	427	412	412
q17	1770	1751	1788	1751
q18	8191	7750	7747	7747
q19	1738	1727	1709	1709
q20	2055	1912	1865	1865
q21	5646	5423	5300	5300
q22	549	465	473	465
Total cold run time: 72188 ms
Total hot run time: 56439 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 168811 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9f9ffa465d0578d000ae05d95b20029173ebd253, data reload: false

query1	912	372	368	368
query2	6470	1703	1731	1703
query3	6655	213	234	213
query4	20116	17646	17155	17155
query5	3684	499	499	499
query6	264	174	170	170
query7	4593	309	297	297
query8	261	201	210	201
query9	8508	2359	2350	2350
query10	425	302	274	274
query11	10371	9984	10091	9984
query12	125	89	89	89
query13	1639	384	399	384
query14	9943	6714	6957	6714
query15	203	171	171	171
query16	6827	479	472	472
query17	923	546	541	541
query18	1874	275	282	275
query19	188	144	139	139
query20	95	84	87	84
query21	219	99	98	98
query22	4317	4053	3961	3961
query23	33738	33685	33280	33280
query24	9415	3063	3152	3063
query25	649	398	396	396
query26	722	157	158	157
query27	2294	284	279	279
query28	5751	2034	2028	2028
query29	900	446	417	417
query30	244	159	159	159
query31	954	775	762	762
query32	103	57	59	57
query33	677	315	330	315
query34	932	513	498	498
query35	874	768	778	768
query36	1074	899	921	899
query37	169	93	87	87
query38	2960	2838	2783	2783
query39	867	800	847	800
query40	200	110	114	110
query41	45	44	46	44
query42	129	104	101	101
query43	480	421	434	421
query44	1168	714	728	714
query45	201	175	175	175
query46	1075	811	788	788
query47	1792	1725	1710	1710
query48	368	295	292	292
query49	858	420	416	416
query50	888	444	438	438
query51	6800	6665	6671	6665
query52	98	89	88	88
query53	248	178	174	174
query54	648	440	443	440
query55	77	76	75	75
query56	265	251	260	251
query57	1119	1037	1033	1033
query58	274	278	277	277
query59	2652	2367	2431	2367
query60	298	266	273	266
query61	97	124	91	91
query62	882	646	658	646
query63	217	179	181	179
query64	4642	1895	1849	1849
query65	3163	3078	3087	3078
query66	841	335	334	334
query67	15015	14859	14689	14689
query68	4419	570	577	570
query69	679	374	310	310
query70	1102	1056	1039	1039
query71	413	288	282	282
query72	7709	2831	2636	2636
query73	787	328	329	328
query74	5968	5637	5665	5637
query75	3396	2716	2713	2713
query76	2822	1217	1288	1217
query77	551	312	314	312
query78	9550	8937	8817	8817
query79	1695	529	533	529
query80	2041	536	526	526
query81	561	231	234	231
query82	640	133	133	133
query83	268	174	175	174
query84	267	86	86	86
query85	1208	355	303	303
query86	463	300	293	293
query87	3276	3066	3112	3066
query88	3707	2412	2435	2412
query89	385	285	292	285
query90	1811	194	193	193
query91	122	100	97	97
query92	60	49	49	49
query93	2068	611	612	611
query94	743	288	297	288
query95	377	273	268	268
query96	601	283	284	283
query97	3218	3071	3062	3062
query98	226	198	186	186
query99	1612	1282	1274	1274
Total cold run time: 258220 ms
Total hot run time: 168811 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.71 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9f9ffa465d0578d000ae05d95b20029173ebd253, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.03
query3	0.22	0.04	0.04
query4	1.68	0.06	0.06
query5	0.48	0.49	0.48
query6	1.15	0.72	0.71
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.57	0.50	0.52
query10	0.57	0.58	0.58
query11	0.16	0.11	0.12
query12	0.15	0.12	0.13
query13	0.61	0.62	0.60
query14	0.76	0.79	0.80
query15	0.91	0.85	0.88
query16	0.36	0.35	0.35
query17	1.01	1.05	1.01
query18	0.21	0.22	0.20
query19	1.81	1.72	1.71
query20	0.01	0.01	0.01
query21	15.38	0.77	0.66
query22	3.90	8.42	1.12
query23	17.74	1.30	1.25
query24	2.27	0.22	0.21
query25	0.18	0.08	0.08
query26	0.33	0.22	0.22
query27	0.46	0.24	0.23
query28	13.17	1.00	0.97
query29	12.55	3.34	3.34
query30	0.25	0.06	0.05
query31	2.87	0.40	0.40
query32	3.25	0.49	0.49
query33	2.96	2.94	2.90
query34	15.47	4.28	4.20
query35	4.31	4.30	4.29
query36	0.69	0.47	0.48
query37	0.19	0.16	0.16
query38	0.15	0.15	0.14
query39	0.04	0.04	0.03
query40	0.16	0.13	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 107.38 s
Total hot run time: 29.71 s

Copy link
Contributor

github-actions bot commented Aug 7, 2024

clang-tidy review says "All clean, LGTM! 👍"

@zzzxl1993
Copy link
Contributor Author

run buildall

Copy link
Contributor

github-actions bot commented Aug 7, 2024

clang-tidy review says "All clean, LGTM! 👍"

@zzzxl1993
Copy link
Contributor Author

run buildall

Copy link
Contributor

github-actions bot commented Aug 7, 2024

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
Copy link
Contributor

github-actions bot commented Aug 7, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 41946 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 054229f3be531f88044ac027eeda9212244f956c, data reload: false

------ Round 1 ----------------------------------
q1	17691	4136	4100	4100
q2	2026	201	200	200
q3	10450	1292	1350	1292
q4	10174	906	911	906
q5	7610	3055	3045	3045
q6	224	138	137	137
q7	1078	632	612	612
q8	9453	1877	1988	1877
q9	8602	6645	6682	6645
q10	8773	3858	3856	3856
q11	443	250	245	245
q12	412	234	229	229
q13	17773	2998	2955	2955
q14	272	244	245	244
q15	524	488	503	488
q16	534	401	392	392
q17	986	952	927	927
q18	8124	7490	7262	7262
q19	1508	1234	1226	1226
q20	560	307	330	307
q21	5363	4797	4716	4716
q22	356	286	285	285
Total cold run time: 112936 ms
Total hot run time: 41946 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4049	4033	3995	3995
q2	331	235	221	221
q3	3011	3035	3177	3035
q4	2023	2034	2008	2008
q5	5629	5517	5512	5512
q6	215	135	131	131
q7	2151	1822	1851	1822
q8	3346	3396	3383	3383
q9	8763	8782	8828	8782
q10	4019	3978	4010	3978
q11	583	463	481	463
q12	759	624	647	624
q13	16065	3164	3150	3150
q14	293	282	278	278
q15	521	503	481	481
q16	475	426	427	426
q17	1775	1720	1717	1717
q18	8235	8041	7742	7742
q19	2263	1747	1713	1713
q20	2055	1844	1848	1844
q21	5807	5423	5453	5423
q22	549	457	465	457
Total cold run time: 72917 ms
Total hot run time: 57185 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169464 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 054229f3be531f88044ac027eeda9212244f956c, data reload: false

query1	911	377	373	373
query2	6456	1723	1638	1638
query3	6654	216	220	216
query4	20086	17475	17246	17246
query5	3634	523	527	523
query6	277	167	158	158
query7	4597	309	304	304
query8	265	196	200	196
query9	8522	2324	2313	2313
query10	437	272	268	268
query11	10519	10069	9973	9973
query12	122	91	91	91
query13	1633	397	380	380
query14	9630	6943	6207	6207
query15	206	164	172	164
query16	6983	472	452	452
query17	949	580	564	564
query18	1924	295	298	295
query19	202	152	150	150
query20	94	87	91	87
query21	206	99	100	99
query22	4439	4108	4073	4073
query23	33765	33986	33532	33532
query24	10302	3174	3106	3106
query25	701	485	394	394
query26	1701	155	160	155
query27	2841	294	294	294
query28	7483	2008	1993	1993
query29	1267	472	446	446
query30	235	156	155	155
query31	930	763	777	763
query32	111	58	57	57
query33	664	321	347	321
query34	902	495	500	495
query35	879	753	769	753
query36	1066	904	906	904
query37	290	84	85	84
query38	3055	2860	2851	2851
query39	911	843	818	818
query40	253	116	118	116
query41	49	47	43	43
query42	117	99	108	99
query43	477	419	424	419
query44	1190	750	738	738
query45	211	180	178	178
query46	1095	812	803	803
query47	1805	1710	1753	1710
query48	372	297	299	297
query49	915	432	438	432
query50	908	450	461	450
query51	6819	6744	6681	6681
query52	108	90	90	90
query53	263	184	180	180
query54	607	470	465	465
query55	77	75	77	75
query56	300	258	257	257
query57	1145	1068	1081	1068
query58	257	272	289	272
query59	2569	2326	2428	2326
query60	312	283	284	283
query61	101	95	94	94
query62	889	643	650	643
query63	215	181	180	180
query64	5700	1930	1884	1884
query65	3184	3114	3125	3114
query66	1321	331	334	331
query67	15236	14890	14957	14890
query68	4244	580	584	580
query69	436	298	303	298
query70	1088	1087	1075	1075
query71	436	293	320	293
query72	7014	2693	2522	2522
query73	763	333	332	332
query74	6078	5643	5661	5643
query75	3409	2767	2729	2729
query76	2752	1231	1298	1231
query77	476	346	341	341
query78	9567	9009	8934	8934
query79	2751	530	529	529
query80	996	539	529	529
query81	580	227	227	227
query82	881	136	130	130
query83	248	171	176	171
query84	269	80	136	80
query85	1558	313	301	301
query86	486	305	292	292
query87	3323	3076	3129	3076
query88	3952	2507	2493	2493
query89	389	302	294	294
query90	1850	205	202	202
query91	127	100	101	100
query92	58	51	53	51
query93	2271	613	628	613
query94	789	308	304	304
query95	378	267	271	267
query96	619	292	285	285
query97	3248	3059	3113	3059
query98	220	246	200	200
query99	1666	1293	1274	1274
Total cold run time: 265266 ms
Total hot run time: 169464 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.84 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 054229f3be531f88044ac027eeda9212244f956c, data reload: false

query1	0.05	0.05	0.04
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.67	0.07	0.07
query5	0.49	0.49	0.49
query6	1.19	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.57	0.52	0.53
query10	0.59	0.57	0.56
query11	0.16	0.12	0.12
query12	0.14	0.12	0.13
query13	0.62	0.62	0.60
query14	0.77	0.79	0.79
query15	0.90	0.86	0.88
query16	0.35	0.36	0.36
query17	1.00	0.99	1.02
query18	0.21	0.21	0.21
query19	1.89	1.78	1.76
query20	0.01	0.03	0.01
query21	15.39	0.76	0.68
query22	4.16	8.19	1.03
query23	17.92	1.32	1.28
query24	2.24	0.22	0.22
query25	0.18	0.08	0.08
query26	0.32	0.21	0.22
query27	0.46	0.23	0.23
query28	13.16	0.99	0.97
query29	12.54	3.26	3.27
query30	0.26	0.06	0.05
query31	2.88	0.40	0.41
query32	3.25	0.48	0.47
query33	2.91	2.93	2.93
query34	15.46	4.27	4.31
query35	4.28	4.31	4.33
query36	0.68	0.48	0.48
query37	0.19	0.16	0.16
query38	0.16	0.15	0.16
query39	0.04	0.04	0.04
query40	0.15	0.13	0.13
query41	0.10	0.05	0.05
query42	0.06	0.04	0.06
query43	0.05	0.05	0.04
Total cold run time: 107.81 s
Total hot run time: 29.84 s

Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

github-actions bot commented Aug 7, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 7, 2024
Copy link
Contributor

github-actions bot commented Aug 7, 2024

PR approved by anyone and no changes requested.

@qidaye qidaye merged commit 0c8fd93 into apache:master Aug 7, 2024
27 of 30 checks passed
@xiaokang xiaokang removed the doing label Aug 10, 2024
dataroaring pushed a commit that referenced this pull request Aug 11, 2024
1. multi_match result merging logic is incorrect
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Aug 14, 2024
1. multi_match result merging logic is incorrect
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
1. multi_match result merging logic is incorrect
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.2-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants