Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[branch-2.0](function) fix wrong result when convert_tz is out of bound (#37358) #38312

Merged
merged 1 commit into from
Jul 31, 2024
Merged

[branch-2.0](function) fix wrong result when convert_tz is out of bound (#37358) #38312

merged 1 commit into from
Jul 31, 2024

Conversation

zclllyybb
Copy link
Contributor

pick #37358

before:

mysql> select CONVERT_TZ(cast('0000-01-01 00:00:00.00001'  as DATETIMEV1), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533)));
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| convert_tz(cast('0000-01-01 00:00:00.00001' as DATETIME), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533))) |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| q535-12-31 08:01:19                                                                                                                               |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.12 sec)

now:

mysql> select CONVERT_TZ(cast('0000-01-01 00:00:00.00001'  as DATETIMEV1), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533)));
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| convert_tz(cast('0000-01-01 00:00:00.00001' as DATETIME), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533))) |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| NULL                                                                                                                                              |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.09 sec)

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

github-actions[bot]
github-actions bot reviewed Jul 24, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/vec/functions/function_convert_tz.h
@@ -18,19 +18,15 @@
#pragma once

#include <cctz/time_zone.h>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: 'cctz/time_zone.h' file not found [clang-diagnostic-error]

#include <cctz/time_zone.h>
         ^

@zclllyybb zclllyybb force-pushed the pick_convert_tz_20 branch from 4eeab38 to 7d2a0c1 Compare July 24, 2024 15:48
@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49932 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7d2a0c11634de3bf4ac0b72800f35997b318f80c, data reload: false

------ Round 1 ----------------------------------
q1	17559	4345	4374	4345
q2	2079	163	151	151
q3	10263	1883	1907	1883
q4	10106	1259	1320	1259
q5	8485	3901	3900	3900
q6	231	124	127	124
q7	2053	1599	1608	1599
q8	9561	2747	2692	2692
q9	13757	11091	10481	10481
q10	8625	3517	3525	3517
q11	417	250	255	250
q12	475	306	304	304
q13	18348	3921	4055	3921
q14	350	321	330	321
q15	507	449	458	449
q16	661	593	572	572
q17	1121	957	901	901
q18	7311	6880	6969	6880
q19	1829	1638	1570	1570
q20	554	313	294	294
q21	4380	4077	4123	4077
q22	544	451	442	442
Total cold run time: 119216 ms
Total hot run time: 49932 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4298	4271	4337	4271
q2	318	234	224	224
q3	4176	4155	4129	4129
q4	2750	2751	2742	2742
q5	7108	7113	7058	7058
q6	240	120	121	120
q7	3244	2815	2872	2815
q8	4385	4470	4482	4470
q9	16743	16684	16735	16684
q10	4216	4264	4286	4264
q11	739	700	703	700
q12	1036	835	855	835
q13	6912	3773	3750	3750
q14	443	426	421	421
q15	501	450	461	450
q16	731	698	682	682
q17	3831	3899	3834	3834
q18	8895	8816	8756	8756
q19	1726	1640	1669	1640
q20	2374	2146	2086	2086
q21	8546	8571	8586	8571
q22	1061	969	978	969
Total cold run time: 84273 ms
Total hot run time: 79471 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.80% (8115/21466)
Line Coverage: 29.47% (66519/225746)
Region Coverage: 28.94% (34284/118469)
Branch Coverage: 24.82% (17614/70962)
Coverage Report: http://coverage.selectdb-in.cc/coverage/7d2a0c11634de3bf4ac0b72800f35997b318f80c_7d2a0c11634de3bf4ac0b72800f35997b318f80c/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 206620 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7d2a0c11634de3bf4ac0b72800f35997b318f80c, data reload: false

query1	990	391	374	374
query2	6529	2845	2589	2589
query3	6926	223	206	206
query4	21165	17995	18053	17995
query5	19735	6459	6534	6459
query6	274	212	226	212
query7	4159	298	311	298
query8	437	436	436	436
query9	3110	2678	2630	2630
query10	404	317	298	298
query11	11259	10667	10661	10661
query12	126	89	76	76
query13	5607	704	708	704
query14	17546	13309	13432	13309
query15	354	244	262	244
query16	6465	282	259	259
query17	1759	1441	876	876
query18	2324	419	419	419
query19	216	152	167	152
query20	78	82	81	81
query21	195	96	93	93
query22	4951	4898	4939	4898
query23	32673	32280	32149	32149
query24	6925	6492	6446	6446
query25	530	417	421	417
query26	523	168	168	168
query27	1862	296	299	296
query28	6143	2392	2330	2330
query29	2855	2683	2744	2683
query30	242	168	169	168
query31	917	735	771	735
query32	70	66	64	64
query33	404	269	262	262
query34	863	486	471	471
query35	1107	921	953	921
query36	1494	1373	1324	1324
query37	87	61	61	61
query38	3119	2935	2981	2935
query39	1390	1333	1311	1311
query40	200	97	101	97
query41	53	45	45	45
query42	79	82	86	82
query43	744	699	798	699
query44	1146	714	725	714
query45	246	243	232	232
query46	1234	943	977	943
query47	2059	1882	1760	1760
query48	1009	710	711	710
query49	624	373	378	373
query50	858	632	621	621
query51	4806	4679	4656	4656
query52	90	79	76	76
query53	450	329	326	326
query54	2656	2433	2455	2433
query55	83	94	80	80
query56	234	226	214	214
query57	1252	1077	1067	1067
query58	214	209	205	205
query59	4469	4043	4170	4043
query60	222	213	212	212
query61	96	92	101	92
query62	828	470	449	449
query63	475	341	349	341
query64	2563	1574	1419	1419
query65	3625	3578	3552	3552
query66	775	390	381	381
query67	15575	18057	18519	18057
query68	8366	675	663	663
query69	579	357	345	345
query70	1648	1643	1386	1386
query71	424	308	324	308
query72	6515	3440	3482	3440
query73	746	320	329	320
query74	6361	5845	5838	5838
query75	4627	3760	3680	3680
query76	4705	1169	1227	1169
query77	603	258	266	258
query78	12340	11698	11602	11602
query79	8111	639	689	639
query80	1196	410	405	405
query81	508	237	241	237
query82	1511	103	104	103
query83	173	133	128	128
query84	264	75	72	72
query85	975	342	330	330
query86	341	292	296	292
query87	3248	3031	2988	2988
query88	5159	2347	2336	2336
query89	379	301	312	301
query90	1886	218	200	200
query91	168	143	150	143
query92	59	52	54	52
query93	4917	563	616	563
query94	762	213	214	213
query95	1131	1077	1091	1077
query96	649	330	335	330
query97	6565	6317	6474	6317
query98	196	170	170	170
query99	3009	970	826	826
Total cold run time: 310599 ms
Total hot run time: 206620 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.75 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7d2a0c11634de3bf4ac0b72800f35997b318f80c, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.02	0.02
query3	0.24	0.05	0.05
query4	1.78	0.07	0.07
query5	0.53	0.52	0.52
query6	1.23	0.62	0.61
query7	0.01	0.01	0.01
query8	0.04	0.02	0.03
query9	0.54	0.49	0.49
query10	0.54	0.55	0.52
query11	0.11	0.08	0.08
query12	0.11	0.08	0.09
query13	0.63	0.62	0.62
query14	0.78	0.79	0.81
query15	0.78	0.75	0.75
query16	0.36	0.38	0.39
query17	1.02	1.00	1.01
query18	0.23	0.26	0.25
query19	1.94	1.83	1.87
query20	0.01	0.01	0.01
query21	15.50	0.55	0.57
query22	1.97	2.12	1.40
query23	17.34	1.21	1.06
query24	5.89	1.23	4.40
query25	0.36	0.11	0.05
query26	0.96	0.17	0.14
query27	0.05	0.03	0.03
query28	3.73	0.79	0.70
query29	12.70	2.33	2.17
query30	0.57	0.54	0.52
query31	2.82	0.38	0.37
query32	3.38	0.50	0.50
query33	3.12	3.07	3.04
query34	15.27	4.80	4.78
query35	4.85	4.82	4.84
query36	1.05	1.04	1.01
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 100.92 s
Total hot run time: 30.75 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 7d2a0c11634de3bf4ac0b72800f35997b318f80c with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@xiaokang xiaokang merged commit d618d9d into apache:branch-2.0 Jul 31, 2024
22 of 24 checks passed
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
@zclllyybb
[branch-2.0](function) fix wrong result when convert_tz is out of bou…
9287d3a 
…nd (apache#37358) (apache#38312)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

Assignees
No one assigned
Labels
kind/test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@zclllyybb @doris-robot @xiaokang