Skip to content

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Dec 6, 2024

Cherry-picked from #44916

…Support Kerberos Ticket Auto-Renewal (#44916)

### Background
The current implementation uses the HadoopUGI method, which invokes the
ugiDoAs function for each operation to log in and execute actions based
on the configuration. However, this approach has the following issues:

- Lack of Auto-Renewal: If the Kerberos TGT (Ticket Granting Ticket)
expires, manual re-login is required as there is no support for
automatic ticket renewal.
- Redundant Login Overhead: Each operation requires reinitializing or
checking UserGroupInformation, potentially causing performance
bottlenecks.
- Complex Management: The HadoopUGI design does not unify the lifecycle
management of UGI instances, leading to duplicated logic across the
codebase.
### Objective

- Auto-Renewal: Automatically renew Kerberos credentials when the TGT is
expired or near expiry.
- UGI Caching: Maintain reusable UserGroupInformation instances during
their lifecycle to avoid repetitive logins.
- Unified Management: Simplify the management of UGI instances and
Kerberos credentials.
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Dec 6, 2024
@dataroaring dataroaring reopened this Dec 6, 2024
@doris-robot
Copy link

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40273 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a55d3da37c40079f1934a8b0510de321d5869bb3, data reload: false

------ Round 1 ----------------------------------
q1	17571	7381	7267	7267
q2	2071	165	164	164
q3	10706	1075	1136	1075
q4	10552	783	723	723
q5	7737	2820	2736	2736
q6	231	144	146	144
q7	951	605	604	604
q8	9571	1936	1987	1936
q9	6948	6389	6387	6387
q10	6990	2275	2275	2275
q11	452	266	249	249
q12	403	212	210	210
q13	17795	2960	2945	2945
q14	245	211	213	211
q15	568	495	513	495
q16	666	596	623	596
q17	959	533	561	533
q18	7115	6529	6501	6501
q19	1563	1082	957	957
q20	458	213	189	189
q21	3903	3097	3395	3097
q22	1079	986	979	979
Total cold run time: 108534 ms
Total hot run time: 40273 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7302	7838	7196	7196
q2	327	241	229	229
q3	2845	2918	2904	2904
q4	2032	1807	1768	1768
q5	5656	5701	5716	5701
q6	227	142	138	138
q7	2187	1761	1793	1761
q8	3301	3498	3440	3440
q9	8834	8830	8794	8794
q10	3529	3526	3493	3493
q11	600	487	483	483
q12	810	599	627	599
q13	16484	3117	3077	3077
q14	302	261	287	261
q15	555	525	531	525
q16	687	676	688	676
q17	1828	1589	1572	1572
q18	8077	7963	7658	7658
q19	3171	1574	1542	1542
q20	2054	1877	1806	1806
q21	5363	5315	5284	5284
q22	1110	1010	1025	1010
Total cold run time: 77281 ms
Total hot run time: 59917 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196192 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a55d3da37c40079f1934a8b0510de321d5869bb3, data reload: false

query1	1227	901	922	901
query2	6934	2027	2048	2027
query3	10831	4142	3931	3931
query4	66154	29102	23446	23446
query5	5216	451	440	440
query6	428	174	176	174
query7	5689	316	315	315
query8	313	229	230	229
query9	9472	2702	2677	2677
query10	507	264	253	253
query11	17660	15163	15884	15163
query12	159	103	100	100
query13	1570	435	434	434
query14	9917	7347	7475	7347
query15	217	182	174	174
query16	7553	567	467	467
query17	1313	587	584	584
query18	1888	342	332	332
query19	219	167	168	167
query20	128	120	118	118
query21	208	113	107	107
query22	4902	4630	4704	4630
query23	34407	34100	34302	34100
query24	6126	2927	2935	2927
query25	539	436	420	420
query26	678	168	176	168
query27	1802	293	304	293
query28	4382	2552	2558	2552
query29	693	441	433	433
query30	236	163	161	161
query31	1048	810	831	810
query32	64	51	54	51
query33	441	277	275	275
query34	916	517	518	517
query35	867	739	753	739
query36	1061	956	967	956
query37	119	67	68	67
query38	4035	4009	4051	4009
query39	1520	1491	1455	1455
query40	198	101	97	97
query41	49	55	45	45
query42	105	99	96	96
query43	535	477	486	477
query44	1149	807	797	797
query45	185	165	165	165
query46	1160	746	719	719
query47	2025	1934	1936	1934
query48	468	371	371	371
query49	718	373	380	373
query50	832	415	416	415
query51	7278	7277	7120	7120
query52	91	103	84	84
query53	254	175	186	175
query54	560	443	442	442
query55	74	77	70	70
query56	253	224	226	224
query57	1219	1113	1117	1113
query58	204	194	199	194
query59	3060	2936	2881	2881
query60	270	242	229	229
query61	109	106	107	106
query62	797	665	647	647
query63	213	186	188	186
query64	1696	670	639	639
query65	3273	3146	3175	3146
query66	718	299	296	296
query67	15892	15521	15346	15346
query68	4524	575	560	560
query69	426	249	249	249
query70	1176	1096	1124	1096
query71	409	246	243	243
query72	6434	3906	3888	3888
query73	752	342	339	339
query74	10249	8819	8900	8819
query75	3324	2621	2661	2621
query76	2036	975	1045	975
query77	499	265	262	262
query78	10737	9632	9452	9452
query79	9877	587	596	587
query80	2155	418	424	418
query81	549	234	239	234
query82	1216	111	115	111
query83	303	137	141	137
query84	290	77	73	73
query85	1592	284	280	280
query86	502	285	298	285
query87	4406	4266	4203	4203
query88	5788	2375	2396	2375
query89	540	290	286	286
query90	2063	179	179	179
query91	183	142	143	142
query92	67	47	49	47
query93	6987	537	533	533
query94	923	283	295	283
query95	361	241	240	240
query96	615	286	278	278
query97	3385	3204	3171	3171
query98	220	206	208	206
query99	1618	1284	1297	1284
Total cold run time: 339030 ms
Total hot run time: 196192 ms

@morningman morningman merged commit 3ed7c55 into branch-3.0 Dec 7, 2024
13 checks passed
@github-actions github-actions bot deleted the auto-pick-44916-branch-3.0 branch December 7, 2024 06:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants