Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug](cache) Fix query cache report error message #47883

Merged
merged 1 commit into from
Feb 14, 2025

Conversation

HappenLee
Copy link
Contributor

@HappenLee HappenLee commented Feb 13, 2025

What problem does this PR solve?

the plan:

 QUERY_CACHE:                                                                                                                                                                                                                                                                                                                           |
|     CACHE_NODE_ID: 2                                                                                                                                                                                                                                                                                                                     |
|     DIGEST: 557413bec3209f50fd640eb3b0534c12ccd9df35f9d4c86620416936985e2679                                                                                                                                                                                                                                                             |
|                                                                                                                                                                                                                                                                                                                                          |
|   STREAM DATA SINK                                                                                                                                                                                                                                                                                                                       |
|     EXCHANGE ID: 03                                                                                                                                                                                                                                                                                                                      |
|     UNPARTITIONED                                                                                                                                                                                                                                                                                                                        |
|                                                                                                                                                                                                                                                                                                                                          |
|   2:VAGGREGATE (merge finalize)(273)                                                                                                                                                                                                                                                                                                     |
|   |  output: min(partial_min(pk)[#6])[#9], max(partial_max(pk)[#7])[#10]                                                                                                                                                                                                                                                                 |
|   |  group by: pk[#5]                                                                                                                                                                                                                                                                                                                    |
|   |  sortByGroupKey:false                                                                                                                                                                                                                                                                                                                |
|   |  cardinality=0                                                                                                                                                                                                                                                                                                                       |
|   |  final projections: field1[#9], field2[#10], pk[#8]                                                                                                                                                                                                                                                                                  |
|   |  final project output tuple id: 4                                                                                                                                                                                                                                                                                                    |
|   |  distribute expr lists: pk[#5]                                                                                                                                                                                                                                                                                                       |
|   |                                                                                                                                                                                                                                                                                                                                      |
|   1:VAGGREGATE (update serialize)(268)                                                                                                                                                                                                                                                                                                   |
|   |  output: partial_min(pk[#4])[#6], partial_max(pk[#4])[#7]                                                                                                                                                                                                                                                                            |
|   |  group by: pk[#4]                                                                                                                                                                                                                                                                                                                    |
|   |  sortByGroupKey:false                                                                                                                                                                                                                                                                                                                |
|   |  cardinality=0                                                                                                                                                                                                                                                                                                                       |
|   |  distribute expr lists: pk[#4]                                                                                                                                                                                                                                                                                                       |
|   |                                                                                                                                                                                                                                                                                           

FE choose the id agg 2, but BE still use id agg 1. BE must support the case

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Feb 13, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@HappenLee HappenLee marked this pull request as ready for review February 13, 2025 09:48
@HappenLee
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31718 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1f874cf3f599d7e2b6247e883bca2c7258440dbd, data reload: false

------ Round 1 ----------------------------------
q1	17612	5266	5041	5041
q2	2057	294	160	160
q3	10422	1285	706	706
q4	10226	1009	532	532
q5	7509	2350	2350	2350
q6	189	168	136	136
q7	884	755	597	597
q8	9310	1327	1062	1062
q9	4863	4880	4896	4880
q10	6809	2300	1897	1897
q11	477	276	248	248
q12	338	361	217	217
q13	17781	3685	3090	3090
q14	226	238	202	202
q15	499	474	450	450
q16	633	627	604	604
q17	571	876	332	332
q18	6879	6227	6304	6227
q19	1789	970	560	560
q20	315	320	187	187
q21	2760	2154	1940	1940
q22	365	328	300	300
Total cold run time: 102514 ms
Total hot run time: 31718 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5189	5098	5124	5098
q2	237	324	234	234
q3	2175	2660	2273	2273
q4	1488	1848	1411	1411
q5	4243	4175	4215	4175
q6	209	168	127	127
q7	1856	1817	1778	1778
q8	2607	2474	2599	2474
q9	7200	7072	7112	7072
q10	3043	3221	2714	2714
q11	576	503	475	475
q12	682	759	600	600
q13	3528	3904	3303	3303
q14	295	302	289	289
q15	501	447	456	447
q16	634	674	665	665
q17	1102	1559	1358	1358
q18	7440	7261	7201	7201
q19	784	866	883	866
q20	1949	1997	1902	1902
q21	5396	5058	4800	4800
q22	624	592	535	535
Total cold run time: 51758 ms
Total hot run time: 49797 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 183574 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1f874cf3f599d7e2b6247e883bca2c7258440dbd, data reload: false

query1	978	376	374	374
query2	6532	1860	1838	1838
query3	6798	222	212	212
query4	26438	23587	23134	23134
query5	4329	651	505	505
query6	328	193	207	193
query7	4608	492	302	302
query8	300	253	211	211
query9	8603	2558	2532	2532
query10	481	295	253	253
query11	16262	15523	14859	14859
query12	171	110	109	109
query13	1649	526	377	377
query14	10181	6311	6324	6311
query15	204	195	181	181
query16	7439	631	484	484
query17	1197	701	555	555
query18	1838	401	307	307
query19	200	190	163	163
query20	128	124	114	114
query21	210	129	105	105
query22	4208	4282	4338	4282
query23	34495	33307	32960	32960
query24	7808	2350	2361	2350
query25	518	436	383	383
query26	1237	266	149	149
query27	2345	480	323	323
query28	4125	2415	2377	2377
query29	695	528	407	407
query30	227	179	156	156
query31	954	855	837	837
query32	80	63	63	63
query33	552	344	293	293
query34	770	867	499	499
query35	802	821	736	736
query36	983	983	877	877
query37	117	93	72	72
query38	4141	4110	4151	4110
query39	1425	1395	1391	1391
query40	201	119	98	98
query41	52	50	53	50
query42	128	109	106	106
query43	477	498	490	490
query44	1266	787	777	777
query45	176	167	160	160
query46	867	1059	630	630
query47	1747	1775	1730	1730
query48	407	413	299	299
query49	778	513	394	394
query50	676	733	394	394
query51	4187	4176	4177	4176
query52	104	99	93	93
query53	229	247	180	180
query54	495	475	404	404
query55	83	82	83	82
query56	279	267	257	257
query57	1147	1113	1091	1091
query58	255	233	235	233
query59	2632	2769	2732	2732
query60	273	276	254	254
query61	123	117	120	117
query62	822	722	666	666
query63	218	190	187	187
query64	4285	986	764	764
query65	3251	3205	3139	3139
query66	1105	423	315	315
query67	15811	15403	15312	15312
query68	6098	764	500	500
query69	475	295	275	275
query70	1176	1094	1100	1094
query71	392	296	270	270
query72	5142	3579	3846	3579
query73	739	719	348	348
query74	8978	8931	9206	8931
query75	3110	3151	2666	2666
query76	3280	1151	735	735
query77	484	375	315	315
query78	9987	10011	9399	9399
query79	3201	826	578	578
query80	893	551	492	492
query81	566	280	245	245
query82	1124	130	94	94
query83	173	169	149	149
query84	240	94	72	72
query85	784	354	305	305
query86	408	300	281	281
query87	4415	4524	4363	4363
query88	3931	2192	2175	2175
query89	397	322	279	279
query90	1898	198	199	198
query91	130	143	109	109
query92	79	67	57	57
query93	2612	999	569	569
query94	799	412	286	286
query95	359	270	254	254
query96	489	558	266	266
query97	2749	2882	2724	2724
query98	236	198	198	198
query99	1358	1381	1254	1254
Total cold run time: 273132 ms
Total hot run time: 183574 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.24 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1f874cf3f599d7e2b6247e883bca2c7258440dbd, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.03
query3	0.23	0.08	0.06
query4	1.60	0.10	0.10
query5	0.42	0.41	0.39
query6	1.15	0.67	0.67
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.60	0.52	0.52
query10	0.57	0.57	0.58
query11	0.15	0.10	0.10
query12	0.14	0.11	0.12
query13	0.61	0.60	0.60
query14	2.69	2.71	2.80
query15	0.93	0.85	0.87
query16	0.37	0.37	0.38
query17	0.99	1.05	1.01
query18	0.22	0.20	0.19
query19	1.92	1.78	2.04
query20	0.02	0.01	0.01
query21	15.35	0.94	0.54
query22	0.74	1.22	0.67
query23	14.94	1.34	0.63
query24	11.85	0.97	0.39
query25	0.34	0.19	0.07
query26	0.90	0.18	0.14
query27	0.05	0.04	0.05
query28	6.23	0.81	0.46
query29	12.61	4.01	3.30
query30	0.25	0.08	0.06
query31	2.84	0.62	0.39
query32	3.24	0.56	0.47
query33	2.96	3.04	3.07
query34	15.83	5.17	4.56
query35	4.52	4.53	4.57
query36	0.68	0.50	0.50
query37	0.09	0.06	0.06
query38	0.06	0.04	0.04
query39	0.03	0.03	0.02
query40	0.18	0.14	0.12
query41	0.07	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.6 s
Total hot run time: 30.24 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 42.98% (11309/26312)
Line Coverage: 32.91% (94971/288561)
Region Coverage: 32.06% (48674/151832)
Branch Coverage: 27.91% (24550/87952)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1f874cf3f599d7e2b6247e883bca2c7258440dbd_1f874cf3f599d7e2b6247e883bca2c7258440dbd/report/index.html

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 13, 2025
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@HappenLee HappenLee merged commit c58fc18 into apache:master Feb 14, 2025
26 of 28 checks passed
lzyy2024 pushed a commit to lzyy2024/doris that referenced this pull request Feb 21, 2025
the plan:
```
 QUERY_CACHE:                                                                                                                                                                                                                                                                                                                           |
|     CACHE_NODE_ID: 2                                                                                                                                                                                                                                                                                                                     |
|     DIGEST: 557413bec3209f50fd640eb3b0534c12ccd9df35f9d4c86620416936985e2679                                                                                                                                                                                                                                                             |
|                                                                                                                                                                                                                                                                                                                                          |
|   STREAM DATA SINK                                                                                                                                                                                                                                                                                                                       |
|     EXCHANGE ID: 03                                                                                                                                                                                                                                                                                                                      |
|     UNPARTITIONED                                                                                                                                                                                                                                                                                                                        |
|                                                                                                                                                                                                                                                                                                                                          |
|   2:VAGGREGATE (merge finalize)(273)                                                                                                                                                                                                                                                                                                     |
|   |  output: min(partial_min(pk)[apache#6])[apache#9], max(partial_max(pk)[apache#7])[apache#10]                                                                                                                                                                                                                                                                 |
|   |  group by: pk[apache#5]                                                                                                                                                                                                                                                                                                                    |
|   |  sortByGroupKey:false                                                                                                                                                                                                                                                                                                                |
|   |  cardinality=0                                                                                                                                                                                                                                                                                                                       |
|   |  final projections: field1[apache#9], field2[apache#10], pk[apache#8]                                                                                                                                                                                                                                                                                  |
|   |  final project output tuple id: 4                                                                                                                                                                                                                                                                                                    |
|   |  distribute expr lists: pk[apache#5]                                                                                                                                                                                                                                                                                                       |
|   |                                                                                                                                                                                                                                                                                                                                      |
|   1:VAGGREGATE (update serialize)(268)                                                                                                                                                                                                                                                                                                   |
|   |  output: partial_min(pk[apache#4])[apache#6], partial_max(pk[apache#4])[apache#7]                                                                                                                                                                                                                                                                            |
|   |  group by: pk[apache#4]                                                                                                                                                                                                                                                                                                                    |
|   |  sortByGroupKey:false                                                                                                                                                                                                                                                                                                                |
|   |  cardinality=0                                                                                                                                                                                                                                                                                                                       |
|   |  distribute expr lists: pk[apache#4]                                                                                                                                                                                                                                                                                                       |
|   |                                                                                                                                                                                                                                                                                           
```

FE choose the id agg 2, but BE still use id agg 1. BE should support the case
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants