Skip to content

[chore](sk) Encrypt secret key and hide access key for log #55241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
gavinchou merged 1 commit into from
Sep 3, 2025
Merged

[chore](sk) Encrypt secret key and hide access key for log #55241

gavinchou merged 1 commit into from
Sep 3, 2025

Conversation

@wyxxxcat
Copy link
Contributor

@wyxxxcat wyxxxcat commented Aug 25, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 25, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@wyxxxcat
Copy link
Contributor Author

wyxxxcat commented Aug 25, 2025

run buildall

@wyxxxcat
Copy link
Contributor Author

wyxxxcat commented Aug 25, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 25, 2025

Cloud UT Coverage Report

Increment line coverage 57.14% (28/49) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.62% (1419/1697)
Line Coverage 66.95% (25253/37719)
Region Coverage 68.08% (12595/18500)
Branch Coverage 57.84% (6666/11524)

@dataroaring dataroaring added dev/3.0.x dev/3.1.x usercase Important user case type label labels Aug 25, 2025
@wyxxxcat
Copy link
Contributor Author

wyxxxcat commented Aug 25, 2025

run cloud_p0

@wyxxxcat
Copy link
Contributor Author

wyxxxcat commented Aug 26, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 26, 2025

Cloud UT Coverage Report

Increment line coverage 75.38% (49/65) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.60% (1427/1707)
Line Coverage 67.08% (25365/37811)
Region Coverage 68.16% (12654/18566)
Branch Coverage 57.90% (6696/11564)

gavinchou
gavinchou reviewed Aug 28, 2025
View reviewed changes
@wyxxxcat wyxxxcat changed the title [chore](sk) Encrypt secret key for create/get instance and create stage in rpc log [chore](sk) Encrypt secret key and hide access key for log Aug 29, 2025
@wyxxxcat wyxxxcat force-pushed the encrypt_sk branch 4 times, most recently from d43d2ba to 15e4009 Compare August 29, 2025 08:30
gavinchou
gavinchou reviewed Aug 29, 2025
View reviewed changes
@wyxxxcat wyxxxcat force-pushed the encrypt_sk branch 2 times, most recently from befb1b0 to 6b29985 Compare August 31, 2025 10:42
@wyxxxcat
Copy link
Contributor Author

wyxxxcat commented Aug 31, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 31, 2025

Cloud UT Coverage Report

Increment line coverage 88.89% (80/90) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 84.50% (1467/1736)
Line Coverage 67.81% (26225/38674)
Region Coverage 68.72% (13024/18953)
Branch Coverage 58.64% (6963/11874)

@doris-robot
Copy link

doris-robot commented Aug 31, 2025

TPC-H: Total hot run time: 34236 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6b29985493031306e976987342661844ffc0e9f8, data reload: false

------ Round 1 ----------------------------------
q1	17650	5276	5098	5098
q2	2043	315	207	207
q3	10260	1284	721	721
q4	10237	1013	530	530
q5	7556	2455	2396	2396
q6	190	169	144	144
q7	955	776	635	635
q8	9346	1347	1060	1060
q9	7117	5166	5163	5163
q10	6969	2406	1963	1963
q11	509	321	300	300
q12	376	359	232	232
q13	17804	3710	3066	3066
q14	251	255	225	225
q15	593	506	487	487
q16	440	432	388	388
q17	605	869	376	376
q18	7613	7213	7092	7092
q19	1109	1005	577	577
q20	343	343	235	235
q21	3738	3229	2338	2338
q22	1065	1039	1003	1003
Total cold run time: 106769 ms
Total hot run time: 34236 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5153	5112	5267	5112
q2	272	334	231	231
q3	2250	2687	2295	2295
q4	1371	1780	1332	1332
q5	4231	4481	4498	4481
q6	221	180	156	156
q7	2094	2034	1928	1928
q8	2799	2704	2630	2630
q9	7532	7413	7400	7400
q10	3150	3367	2886	2886
q11	570	529	518	518
q12	718	802	700	700
q13	3491	3912	3454	3454
q14	279	295	283	283
q15	558	506	496	496
q16	430	496	448	448
q17	1220	1601	1422	1422
q18	7786	7842	7549	7549
q19	827	800	889	800
q20	2014	2318	1897	1897
q21	4968	4356	4352	4352
q22	1088	1056	1003	1003
Total cold run time: 53022 ms
Total hot run time: 51373 ms

@doris-robot
Copy link

doris-robot commented Aug 31, 2025

TPC-DS: Total hot run time: 187275 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6b29985493031306e976987342661844ffc0e9f8, data reload: false

query1	1059	434	415	415
query2	6596	1784	1827	1784
query3	6755	232	229	229
query4	26707	24093	23488	23488
query5	4997	643	526	526
query6	363	259	261	259
query7	4665	517	312	312
query8	318	266	263	263
query9	8677	2910	2915	2910
query10	515	354	315	315
query11	16011	15020	14776	14776
query12	184	129	122	122
query13	1701	579	478	478
query14	9581	5847	5858	5847
query15	226	196	170	170
query16	7586	654	488	488
query17	1181	776	624	624
query18	2035	424	340	340
query19	201	193	181	181
query20	173	128	120	120
query21	222	126	113	113
query22	4070	4076	3994	3994
query23	34056	33062	33240	33062
query24	8232	2360	2398	2360
query25	592	528	447	447
query26	1239	281	165	165
query27	2738	511	351	351
query28	4396	2279	2240	2240
query29	751	604	501	501
query30	309	227	199	199
query31	913	787	721	721
query32	95	85	92	85
query33	602	381	359	359
query34	792	872	548	548
query35	874	809	775	775
query36	985	1019	930	930
query37	129	114	92	92
query38	4087	4051	4016	4016
query39	1455	1469	1441	1441
query40	231	143	128	128
query41	73	104	62	62
query42	130	111	121	111
query43	516	513	512	512
query44	1352	871	875	871
query45	181	179	174	174
query46	857	1024	655	655
query47	1776	1811	1725	1725
query48	389	441	323	323
query49	761	512	431	431
query50	634	685	406	406
query51	4149	4160	4224	4160
query52	117	117	113	113
query53	248	282	212	212
query54	618	664	535	535
query55	97	92	96	92
query56	338	335	323	323
query57	1191	1255	1114	1114
query58	293	288	281	281
query59	2696	2848	2609	2609
query60	355	351	342	342
query61	163	158	160	158
query62	805	754	674	674
query63	236	195	196	195
query64	4370	1147	822	822
query65	4342	4210	4187	4187
query66	1134	460	378	378
query67	15627	15339	14947	14947
query68	7731	925	592	592
query69	504	343	306	306
query70	1234	1147	1124	1124
query71	448	349	324	324
query72	6033	5007	5066	5007
query73	679	661	370	370
query74	9192	9019	9008	9008
query75	3398	3117	2628	2628
query76	3391	1153	766	766
query77	690	417	340	340
query78	9641	9786	8866	8866
query79	2830	819	610	610
query80	732	582	523	523
query81	518	273	228	228
query82	489	140	112	112
query83	270	262	246	246
query84	270	115	90	90
query85	913	474	428	428
query86	392	315	368	315
query87	4235	4242	4208	4208
query88	3826	2182	2208	2182
query89	397	331	302	302
query90	1952	239	225	225
query91	192	161	137	137
query92	99	73	76	73
query93	2442	979	651	651
query94	704	419	313	313
query95	414	343	337	337
query96	510	580	289	289
query97	2629	2686	2587	2587
query98	264	228	221	221
query99	1351	1416	1281	1281
Total cold run time: 278117 ms
Total hot run time: 187275 ms

@doris-robot
Copy link

doris-robot commented Aug 31, 2025

ClickBench: Total hot run time: 32.96 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6b29985493031306e976987342661844ffc0e9f8, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.05	0.06
query3	0.25	0.09	0.09
query4	1.61	0.12	0.12
query5	0.45	0.41	0.40
query6	1.21	0.63	0.66
query7	0.03	0.03	0.02
query8	0.06	0.05	0.05
query9	0.61	0.51	0.52
query10	0.59	0.59	0.57
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.64	0.63	0.64
query14	0.80	0.85	0.83
query15	0.89	0.86	0.87
query16	0.40	0.41	0.39
query17	1.05	1.10	1.02
query18	0.21	0.19	0.20
query19	1.98	1.85	1.82
query20	0.02	0.01	0.02
query21	15.42	0.99	0.59
query22	0.81	1.13	0.92
query23	14.70	1.42	0.67
query24	6.57	1.49	0.65
query25	0.44	0.20	0.20
query26	0.61	0.16	0.14
query27	0.08	0.06	0.06
query28	9.57	0.94	0.42
query29	12.67	4.02	3.30
query30	3.14	3.04	2.93
query31	2.82	0.60	0.38
query32	3.25	0.56	0.48
query33	3.25	3.09	3.15
query34	16.01	5.49	4.83
query35	4.91	4.93	4.93
query36	0.69	0.51	0.51
query37	0.10	0.07	0.08
query38	0.05	0.05	0.05
query39	0.03	0.03	0.04
query40	0.20	0.15	0.15
query41	0.09	0.03	0.03
query42	0.03	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 106.75 s
Total hot run time: 32.96 s

@wyxxxcat
Copy link
Contributor Author

wyxxxcat commented Sep 2, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 2, 2025

Cloud UT Coverage Report

Increment line coverage 88.89% (80/90) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 84.62% (1469/1736)
Line Coverage 67.84% (26243/38686)
Region Coverage 68.83% (13051/18961)
Branch Coverage 58.76% (6981/11880)

@doris-robot
Copy link

doris-robot commented Sep 2, 2025

TPC-H: Total hot run time: 34110 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 32c706ab89c30b51b0d35dd2315c0d1687668d35, data reload: false

------ Round 1 ----------------------------------
q1	17610	5275	5159	5159
q2	2022	319	213	213
q3	10251	1283	700	700
q4	10280	1096	527	527
q5	7595	2463	2334	2334
q6	188	169	136	136
q7	920	750	633	633
q8	9371	1343	1151	1151
q9	7026	5182	5179	5179
q10	6979	2380	1963	1963
q11	491	309	272	272
q12	370	368	229	229
q13	17800	3635	2980	2980
q14	233	259	228	228
q15	586	518	478	478
q16	445	433	390	390
q17	603	895	375	375
q18	7667	7131	7003	7003
q19	1462	964	562	562
q20	342	330	227	227
q21	3734	3220	2378	2378
q22	1078	1057	993	993
Total cold run time: 107053 ms
Total hot run time: 34110 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5236	5144	5125	5125
q2	244	324	224	224
q3	2170	2682	2311	2311
q4	1372	1764	1370	1370
q5	4230	4466	4534	4466
q6	220	171	133	133
q7	2030	1979	1967	1967
q8	2657	2612	2568	2568
q9	7459	7289	7459	7289
q10	3106	3380	2870	2870
q11	571	520	500	500
q12	728	1018	603	603
q13	3459	3852	3325	3325
q14	317	316	447	316
q15	544	494	483	483
q16	448	499	451	451
q17	1175	1576	1330	1330
q18	7972	7737	7627	7627
q19	820	833	821	821
q20	2006	2102	1869	1869
q21	4851	4376	4386	4376
q22	1087	1039	1037	1037
Total cold run time: 52702 ms
Total hot run time: 51061 ms

@doris-robot
Copy link

doris-robot commented Sep 2, 2025

TPC-DS: Total hot run time: 186488 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 32c706ab89c30b51b0d35dd2315c0d1687668d35, data reload: false

query1	1065	480	411	411
query2	6579	1793	1801	1793
query3	6747	230	222	222
query4	26298	23084	23113	23084
query5	4425	654	508	508
query6	348	250	213	213
query7	4656	516	305	305
query8	321	272	272	272
query9	8684	2908	2921	2908
query10	522	345	290	290
query11	15788	15017	14731	14731
query12	172	124	120	120
query13	1658	547	432	432
query14	9463	5799	5739	5739
query15	223	195	176	176
query16	7400	650	492	492
query17	1217	752	629	629
query18	2037	442	345	345
query19	209	199	176	176
query20	137	131	129	129
query21	218	141	120	120
query22	4152	4195	4072	4072
query23	33925	33012	32947	32947
query24	8266	2425	2384	2384
query25	590	510	456	456
query26	1250	285	162	162
query27	2722	514	358	358
query28	4360	2311	2284	2284
query29	809	613	515	515
query30	291	222	196	196
query31	895	829	738	738
query32	89	79	82	79
query33	581	393	375	375
query34	801	855	544	544
query35	816	848	759	759
query36	964	1012	903	903
query37	128	114	96	96
query38	4050	4073	3934	3934
query39	1490	1424	1447	1424
query40	230	135	131	131
query41	75	64	60	60
query42	134	114	121	114
query43	545	510	486	486
query44	1379	880	868	868
query45	187	175	164	164
query46	879	1016	648	648
query47	1776	1835	1745	1745
query48	410	424	325	325
query49	767	526	404	404
query50	663	687	404	404
query51	4095	4127	4148	4127
query52	125	120	109	109
query53	257	274	204	204
query54	617	608	539	539
query55	97	90	94	90
query56	344	356	333	333
query57	1262	1199	1130	1130
query58	291	289	280	280
query59	2661	2853	2607	2607
query60	364	357	363	357
query61	165	180	161	161
query62	839	740	678	678
query63	231	191	195	191
query64	4509	1202	838	838
query65	4271	4173	4241	4173
query66	1169	448	359	359
query67	15543	15296	15166	15166
query68	7342	920	577	577
query69	512	330	303	303
query70	1264	1120	1125	1120
query71	555	343	318	318
query72	6029	4903	4972	4903
query73	655	620	370	370
query74	8919	9013	8836	8836
query75	3138	3088	2611	2611
query76	3122	1140	748	748
query77	509	411	347	347
query78	9487	9850	8968	8968
query79	1620	809	602	602
query80	682	606	563	563
query81	517	262	235	235
query82	208	144	116	116
query83	282	359	245	245
query84	270	109	92	92
query85	862	469	428	428
query86	387	332	305	305
query87	4274	4320	4106	4106
query88	2913	2256	2239	2239
query89	397	333	293	293
query90	1955	233	238	233
query91	162	172	134	134
query92	94	75	77	75
query93	1205	999	665	665
query94	714	419	325	325
query95	416	429	335	335
query96	483	604	285	285
query97	2600	2694	2583	2583
query98	241	224	213	213
query99	1327	1465	1303	1303
Total cold run time: 271372 ms
Total hot run time: 186488 ms

@doris-robot
Copy link

doris-robot commented Sep 2, 2025

ClickBench: Total hot run time: 32.78 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 32c706ab89c30b51b0d35dd2315c0d1687668d35, data reload: false

query1	0.06	0.04	0.04
query2	0.10	0.05	0.06
query3	0.24	0.08	0.08
query4	1.61	0.11	0.12
query5	0.44	0.42	0.42
query6	1.14	0.64	0.67
query7	0.03	0.03	0.03
query8	0.05	0.04	0.04
query9	0.61	0.53	0.52
query10	0.58	0.57	0.57
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.63	0.62	0.61
query14	0.79	0.84	0.83
query15	0.87	0.85	0.86
query16	0.39	0.40	0.38
query17	1.04	1.05	1.02
query18	0.22	0.21	0.19
query19	1.97	1.92	1.89
query20	0.02	0.02	0.01
query21	15.40	0.93	0.60
query22	0.76	1.06	0.79
query23	14.97	1.39	0.65
query24	6.82	0.76	1.19
query25	0.46	0.33	0.08
query26	0.58	0.16	0.13
query27	0.06	0.06	0.06
query28	10.14	0.96	0.43
query29	12.55	3.85	3.21
query30	3.12	2.99	3.02
query31	2.83	0.59	0.39
query32	3.25	0.55	0.48
query33	3.02	3.04	3.15
query34	16.01	5.45	4.83
query35	4.98	4.92	4.93
query36	0.69	0.53	0.51
query37	0.10	0.08	0.08
query38	0.06	0.04	0.05
query39	0.04	0.03	0.03
query40	0.18	0.16	0.15
query41	0.09	0.03	0.04
query42	0.04	0.03	0.04
query43	0.05	0.04	0.04
Total cold run time: 107.3 s
Total hot run time: 32.78 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 2, 2025

FE UT Coverage Report

Increment line coverage 0.00% (0/34) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 2, 2025

BE UT Coverage Report

Increment line coverage 75.76% (25/33) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 51.66% (17131/33159)
Line Coverage 37.20% (156580/420877)
Region Coverage 31.84% (119421/375095)
Branch Coverage 33.22% (52501/158050)

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 2, 2025

BE Regression && UT Coverage Report

Increment line coverage 87.88% (29/33) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 70.62% (23004/32575)
Line Coverage 56.93% (239526/420749)
Region Coverage 52.23% (198811/380612)
Branch Coverage 53.99% (85863/159031)

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 2, 2025

FE Regression Coverage Report

Increment line coverage 67.65% (23/34) 🎉
Increment coverage report
Complete coverage report

dataroaring
dataroaring approved these changes Sep 3, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 3, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 3, 2025

PR approved by at least one committer and no changes requested.

@gavinchou gavinchou merged commit 77b5cb3 into apache:master Sep 3, 2025
26 of 28 checks passed
github-actions bot pushed a commit that referenced this pull request Sep 3, 2025
@wyxxxcat
[chore](sk) Encrypt secret key and hide access key for log (#55241)
83b0196
@github-actions github-actions bot mentioned this pull request Sep 3, 2025
Merged
github-actions bot pushed a commit that referenced this pull request Sep 3, 2025
@wyxxxcat
[chore](sk) Encrypt secret key and hide access key for log (#55241)
0c03bd8
@github-actions github-actions bot mentioned this pull request Sep 3, 2025
Merged
dataroaring pushed a commit that referenced this pull request Sep 3, 2025
@wyxxxcat @dataroaring
[chore](sk) Encrypt secret key and hide access key for log (#55241)
f0bf2bb
dataroaring pushed a commit that referenced this pull request Sep 4, 2025
@wyxxxcat @dataroaring
[chore](sk) Encrypt secret key and hide access key for log (#55241)
e6d6157
@wyxxxcat wyxxxcat mentioned this pull request Sep 4, 2025
Closed
dataroaring pushed a commit that referenced this pull request Sep 5, 2025
@github-actions @wyxxxcat
branch-3.0: [chore](sk) Encrypt secret key and hide access key fo…
5593057 
…r log #55241 (#55618)

Cherry-picked from #55241

Co-authored-by: Uniqueyou <wangyixuan@selectdb.com>
uchenily pushed a commit to uchenily/doris that referenced this pull request Sep 5, 2025
@wyxxxcat @uchenily
[chore](sk) Encrypt secret key and hide access key for log (apach…
7b63fc0 
…e#55241)
morrySnow pushed a commit that referenced this pull request Sep 5, 2025
@github-actions @wyxxxcat
branch-3.1: [chore](sk) Encrypt secret key and hide access key fo…
07d2425 
…r log #55241 (#55619)

Cherry-picked from #55241

Co-authored-by: Uniqueyou <wangyixuan@selectdb.com>
@morrySnow morrySnow mentioned this pull request Sep 22, 2025
Closed
yiguolei pushed a commit to yiguolei/incubator-doris that referenced this pull request Nov 4, 2025
@wyxxxcat
[cloud-4.0] [chore](sk) Encrypt secret key and hide access key for log …
e44f63a 
…apache#55241 (apache#4364)

pick: apache#55241
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gavinchou gavinchou gavinchou approved these changes

@dataroaring dataroaring dataroaring approved these changes

@w41ter w41ter Awaiting requested review from w41ter w41ter is a code owner

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.9-merged dev/3.1.1-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@wyxxxcat @hello-stephen @doris-robot @gavinchou @dataroaring @morrySnow