Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix](Load)Audit logs avoid recording certain sensitive information #38769 #38783

Merged
merged 1 commit into from
Aug 2, 2024

Conversation

CalvinKirs
Copy link
Member

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added the area/planner Issues or PRs related to the query planner label Aug 2, 2024
@CalvinKirs
Copy link
Member Author

run buildall

@CalvinKirs
Copy link
Member Author

#38769

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 2, 2024
Copy link
Contributor

github-actions bot commented Aug 2, 2024

PR approved by at least one committer and no changes requested.

Copy link
Contributor

github-actions bot commented Aug 2, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 49901 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ddc630716a4629cd47667e16e2fcac9f6efed2b0, data reload: false

------ Round 1 ----------------------------------
q1	17596	4418	4423	4418
q2	2069	159	152	152
q3	10257	1946	1962	1946
q4	10146	1243	1353	1243
q5	8400	3897	3892	3892
q6	232	151	124	124
q7	2017	1605	1609	1605
q8	9514	2745	2713	2713
q9	13803	10276	10243	10243
q10	8646	3550	3540	3540
q11	414	258	256	256
q12	476	300	303	300
q13	18335	3966	4038	3966
q14	352	339	338	338
q15	510	460	463	460
q16	676	577	567	567
q17	1141	947	931	931
q18	7189	6767	6838	6767
q19	1776	1630	1604	1604
q20	567	299	310	299
q21	4435	4157	4104	4104
q22	537	442	433	433
Total cold run time: 119088 ms
Total hot run time: 49901 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4366	4398	4301	4301
q2	320	222	217	217
q3	4155	4145	4122	4122
q4	2738	2731	2765	2731
q5	7141	7148	7098	7098
q6	251	125	121	121
q7	3264	2845	2814	2814
q8	4377	4483	4456	4456
q9	16882	16668	16822	16668
q10	4216	4301	4263	4263
q11	742	710	685	685
q12	1030	855	854	854
q13	6629	3763	3713	3713
q14	460	424	430	424
q15	507	453	462	453
q16	762	680	683	680
q17	3795	3845	3840	3840
q18	8808	8762	8852	8762
q19	1746	1677	1714	1677
q20	2379	2180	2104	2104
q21	8501	8393	8474	8393
q22	1044	1048	969	969
Total cold run time: 84113 ms
Total hot run time: 79345 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 204152 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ddc630716a4629cd47667e16e2fcac9f6efed2b0, data reload: false

query1	933	388	407	388
query2	6548	2790	2729	2729
query3	6914	205	202	202
query4	20175	17921	17918	17918
query5	19731	6471	6539	6471
query6	312	220	231	220
query7	4270	305	302	302
query8	424	431	381	381
query9	3109	2687	2631	2631
query10	428	302	307	302
query11	11375	10774	10658	10658
query12	129	80	75	75
query13	5599	693	716	693
query14	18575	13280	13566	13280
query15	355	254	245	245
query16	6439	309	267	267
query17	1657	1453	893	893
query18	2290	414	409	409
query19	219	152	153	152
query20	82	80	80	80
query21	199	97	93	93
query22	5280	5109	4975	4975
query23	32433	32084	31954	31954
query24	6934	6492	6493	6492
query25	533	423	408	408
query26	532	166	158	158
query27	1864	303	303	303
query28	6129	2407	2349	2349
query29	2961	2640	2808	2640
query30	242	166	166	166
query31	915	756	743	743
query32	68	62	62	62
query33	403	270	268	268
query34	865	466	488	466
query35	1127	911	948	911
query36	1267	1117	1325	1117
query37	89	60	59	59
query38	3072	2910	2886	2886
query39	1368	1313	1311	1311
query40	208	98	98	98
query41	47	46	44	44
query42	85	84	84	84
query43	754	771	648	648
query44	1117	721	719	719
query45	251	237	247	237
query46	1237	968	971	968
query47	1723	1695	1840	1695
query48	1017	721	715	715
query49	633	363	371	363
query50	874	633	595	595
query51	4759	4694	4640	4640
query52	85	76	83	76
query53	451	334	337	334
query54	2644	2491	2475	2475
query55	100	79	89	79
query56	252	212	218	212
query57	1212	1185	1176	1176
query58	221	200	205	200
query59	4255	4286	4159	4159
query60	221	215	202	202
query61	99	95	97	95
query62	826	465	453	453
query63	491	344	356	344
query64	2468	1459	1529	1459
query65	3646	3580	3558	3558
query66	766	382	378	378
query67	16257	16943	15698	15698
query68	9482	659	651	651
query69	568	339	345	339
query70	1700	1549	1427	1427
query71	412	301	312	301
query72	6521	3518	3485	3485
query73	751	322	321	321
query74	6453	5868	5861	5861
query75	5354	3706	3713	3706
query76	5932	1158	1159	1158
query77	985	255	255	255
query78	12629	11745	13068	11745
query79	13858	661	620	620
query80	1399	410	404	404
query81	506	239	238	238
query82	698	100	95	95
query83	192	138	133	133
query84	258	70	74	70
query85	969	331	332	331
query86	350	302	321	302
query87	3255	2996	3003	2996
query88	5206	2299	2309	2299
query89	446	276	289	276
query90	2745	209	202	202
query91	191	146	139	139
query92	60	54	52	52
query93	6116	571	538	538
query94	1336	210	214	210
query95	1096	1051	1046	1046
query96	664	322	331	322
query97	6533	6252	6353	6252
query98	184	170	173	170
query99	2908	918	847	847
Total cold run time: 322301 ms
Total hot run time: 204152 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.89 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ddc630716a4629cd47667e16e2fcac9f6efed2b0, data reload: false

query1	0.03	0.02	0.02
query2	0.08	0.02	0.02
query3	0.27	0.04	0.05
query4	1.76	0.05	0.07
query5	0.54	0.53	0.53
query6	1.23	0.65	0.61
query7	0.01	0.01	0.01
query8	0.04	0.02	0.03
query9	0.52	0.50	0.49
query10	0.53	0.55	0.54
query11	0.12	0.08	0.08
query12	0.12	0.08	0.08
query13	0.62	0.62	0.61
query14	0.79	0.78	0.78
query15	0.78	0.75	0.75
query16	0.35	0.36	0.36
query17	1.00	1.02	1.01
query18	0.24	0.22	0.25
query19	1.92	1.88	1.81
query20	0.01	0.01	0.01
query21	15.47	0.55	0.56
query22	2.05	2.45	1.72
query23	16.78	1.05	0.83
query24	5.33	1.35	2.58
query25	0.38	0.08	0.05
query26	0.74	0.15	0.16
query27	0.05	0.04	0.04
query28	5.88	0.77	0.74
query29	12.63	2.35	2.18
query30	0.61	0.52	0.52
query31	2.81	0.38	0.37
query32	3.36	0.50	0.49
query33	3.10	3.03	3.06
query34	15.24	4.78	4.75
query35	4.87	4.83	4.86
query36	1.05	0.99	1.00
query37	0.05	0.04	0.04
query38	0.04	0.02	0.02
query39	0.01	0.02	0.01
query40	0.16	0.14	0.15
query41	0.07	0.02	0.02
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 101.68 s
Total hot run time: 30.89 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit ddc630716a4629cd47667e16e2fcac9f6efed2b0 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s

@CalvinKirs CalvinKirs merged commit 86292c7 into apache:branch-2.0 Aug 2, 2024
24 of 27 checks passed
@CalvinKirs CalvinKirs requested a review from xiaokang August 2, 2024 10:09
@CalvinKirs CalvinKirs deleted the branch-2.0-38769 branch August 2, 2024 10:09
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
…pache#38769 (apache#38783)

…

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. area/planner Issues or PRs related to the query planner reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants