Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix](Load)Audit logs avoid recording certain sensitive information #38769

Merged
merged 1 commit into from
Aug 4, 2024
Merged

[Fix](Load)Audit logs avoid recording certain sensitive information #38769

merged 1 commit into from
Aug 4, 2024

Conversation

CalvinKirs
Copy link
Member

No description provided.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added the doing label Aug 2, 2024
@CalvinKirs
Copy link
Member Author

run buildall

morningman
morningman approved these changes Aug 2, 2024
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 2, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 41374 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 075951b41c84ea1c1cca0ad005d434a1312c16aa, data reload: false

------ Round 1 ----------------------------------
q1	17600	4074	4015	4015
q2	2032	200	202	200
q3	10449	1386	1283	1283
q4	10148	799	915	799
q5	7637	2985	2944	2944
q6	219	136	139	136
q7	1078	623	624	623
q8	9429	1776	1953	1776
q9	8591	6582	6564	6564
q10	8760	3836	3853	3836
q11	435	245	249	245
q12	441	230	230	230
q13	17776	2949	2980	2949
q14	274	245	240	240
q15	529	492	486	486
q16	525	398	393	393
q17	956	941	904	904
q18	8004	7317	7168	7168
q19	1383	1217	1201	1201
q20	567	316	342	316
q21	5406	4788	4804	4788
q22	350	278	279	278
Total cold run time: 112589 ms
Total hot run time: 41374 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4050	3976	4078	3976
q2	330	223	221	221
q3	2983	2984	3076	2984
q4	2021	2036	2006	2006
q5	5565	5493	5444	5444
q6	219	139	133	133
q7	2147	1749	1806	1749
q8	3313	3393	3327	3327
q9	8619	8641	8799	8641
q10	3975	4054	4012	4012
q11	552	457	455	455
q12	782	606	577	577
q13	16447	3165	3162	3162
q14	306	257	268	257
q15	541	492	494	492
q16	489	405	422	405
q17	1758	1730	1734	1730
q18	8120	7689	7624	7624
q19	1715	1724	1716	1716
q20	2021	1891	1847	1847
q21	5888	5562	5467	5467
q22	522	469	478	469
Total cold run time: 72363 ms
Total hot run time: 56694 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169878 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 075951b41c84ea1c1cca0ad005d434a1312c16aa, data reload: false

query1	905	370	370	370
query2	6488	1706	1622	1622
query3	6664	211	216	211
query4	19885	17554	17274	17274
query5	3688	508	512	508
query6	288	179	166	166
query7	4592	295	291	291
query8	255	197	214	197
query9	8526	2359	2333	2333
query10	429	281	294	281
query11	10440	9870	9991	9870
query12	120	92	88	88
query13	1640	384	391	384
query14	9013	7097	7126	7097
query15	203	162	164	162
query16	6859	471	478	471
query17	956	616	570	570
query18	1940	291	289	289
query19	197	163	149	149
query20	92	85	87	85
query21	211	101	101	101
query22	4202	3951	3921	3921
query23	34040	33678	33805	33678
query24	10162	3109	3046	3046
query25	703	399	416	399
query26	1753	155	158	155
query27	3080	285	287	285
query28	7540	2010	1996	1996
query29	1359	431	429	429
query30	241	156	161	156
query31	943	758	776	758
query32	101	54	57	54
query33	706	338	346	338
query34	920	514	531	514
query35	889	768	764	764
query36	1046	905	896	896
query37	296	81	85	81
query38	2897	2843	2814	2814
query39	893	796	808	796
query40	252	116	114	114
query41	46	43	44	43
query42	123	104	99	99
query43	461	424	443	424
query44	1206	751	732	732
query45	213	180	184	180
query46	1116	805	771	771
query47	1789	1711	1707	1707
query48	372	298	297	297
query49	944	430	432	430
query50	889	432	445	432
query51	6812	6551	6697	6551
query52	106	89	89	89
query53	253	192	183	183
query54	615	462	455	455
query55	77	78	76	76
query56	288	249	248	248
query57	1145	1036	1034	1034
query58	283	264	280	264
query59	2568	2381	2436	2381
query60	301	276	275	275
query61	92	92	93	92
query62	890	664	659	659
query63	208	190	189	189
query64	5649	1915	1906	1906
query65	3189	3131	3134	3131
query66	1324	329	331	329
query67	15276	14770	14748	14748
query68	4460	557	569	557
query69	537	380	325	325
query70	1150	1051	1118	1051
query71	457	279	289	279
query72	7209	2683	2562	2562
query73	789	344	340	340
query74	6223	5854	5638	5638
query75	3387	2749	2728	2728
query76	2575	1194	1259	1194
query77	412	308	318	308
query78	9433	8994	8966	8966
query79	1890	533	530	530
query80	1012	527	526	526
query81	564	224	223	223
query82	1025	131	130	130
query83	248	172	174	172
query84	272	77	90	77
query85	1298	325	352	325
query86	383	321	300	300
query87	3289	3127	3096	3096
query88	3057	2510	2489	2489
query89	403	298	296	296
query90	1779	195	202	195
query91	129	102	101	101
query92	59	51	51	51
query93	1600	611	607	607
query94	823	289	295	289
query95	371	270	269	269
query96	596	287	284	284
query97	3226	3044	3087	3044
query98	219	202	198	198
query99	1646	1309	1290	1290
Total cold run time: 262170 ms
Total hot run time: 169878 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.25 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 075951b41c84ea1c1cca0ad005d434a1312c16aa, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.04	0.04
query4	1.67	0.07	0.07
query5	0.49	0.48	0.48
query6	1.13	0.71	0.71
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.57	0.52	0.52
query10	0.58	0.56	0.57
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.61	0.60	0.60
query14	0.78	0.78	0.79
query15	0.90	0.84	0.87
query16	0.35	0.35	0.36
query17	0.98	1.00	0.97
query18	0.22	0.21	0.21
query19	1.84	1.76	1.77
query20	0.01	0.01	0.01
query21	15.39	0.75	0.68
query22	4.05	7.62	1.40
query23	17.78	1.46	1.35
query24	2.32	0.22	0.22
query25	0.19	0.08	0.08
query26	0.32	0.21	0.21
query27	0.46	0.23	0.23
query28	13.16	1.01	0.99
query29	12.54	3.33	3.33
query30	0.26	0.06	0.06
query31	2.84	0.41	0.40
query32	3.24	0.50	0.49
query33	2.96	2.98	2.96
query34	15.46	4.28	4.24
query35	4.32	4.30	4.31
query36	0.69	0.48	0.47
query37	0.20	0.16	0.17
query38	0.16	0.15	0.15
query39	0.04	0.04	0.03
query40	0.16	0.14	0.13
query41	0.11	0.04	0.04
query42	0.06	0.05	0.05
query43	0.05	0.04	0.03
Total cold run time: 107.6 s
Total hot run time: 30.25 s

CalvinKirs added a commit to CalvinKirs/incubator-doris that referenced this pull request Aug 2, 2024
@CalvinKirs
[Fix](Load)Audit logs avoid recording certain sensitive information a…
ddc6307 
…pache#38769
CalvinKirs added a commit to CalvinKirs/incubator-doris that referenced this pull request Aug 2, 2024
@CalvinKirs
[Fix](Load)Audit logs avoid recording certain sensitive information a…
835644f 
…pache#38769
@CalvinKirs CalvinKirs mentioned this pull request Aug 2, 2024
Merged
@CalvinKirs
Copy link
Member Author

run cloud_p0

@CalvinKirs CalvinKirs mentioned this pull request Aug 2, 2024
Merged
CalvinKirs added a commit that referenced this pull request Aug 2, 2024
@CalvinKirs
[Fix](Load)Audit logs avoid recording certain sensitive information #…
86292c7 
…38769 (#38783)

…

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
@dataroaring dataroaring added the usercase Important user case type label label Aug 4, 2024
dataroaring
dataroaring approved these changes Aug 4, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

dataroaring pushed a commit that referenced this pull request Aug 4, 2024
@CalvinKirs
[Fix](Load)Audit logs avoid recording certain sensitive information #…
7c70f75 
…38769 (#38784)

…

## Proposed changes

#38769

<!--Describe your changes.-->
@dataroaring dataroaring merged commit f442290 into apache:master Aug 4, 2024
30 of 31 checks passed
dataroaring pushed a commit that referenced this pull request Aug 4, 2024
@CalvinKirs @dataroaring
[Fix](Load)Audit logs avoid recording certain sensitive information (#…
bc954d4 
…38769)
@CalvinKirs CalvinKirs deleted the audit-log branch August 5, 2024 01:56
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@CalvinKirs @Baymine
[Fix](Load)Audit logs avoid recording certain sensitive information a…
069cead 
…pache#38769 (apache#38783)

…

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morningman morningman morningman approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.14-merged dev/2.1.6-merged dev/3.0.1-merged doing reviewed usercase Important user case type label
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@CalvinKirs @doris-robot @morningman @dataroaring @xiaokang