[bugfix](hive) fix_hive_metastore_client_socket_timeout #47378

justfortaste · 2025-01-23T13:28:24Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-01-23T13:28:29Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSExternalCatalog.java

lide-reed

LGTM

github-actions · 2025-02-27T06:48:07Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-02-27T06:48:09Z

PR approved by anyone and no changes requested.

liutang123 · 2025-03-06T06:17:34Z

run buildall

doris-robot · 2025-03-06T07:32:28Z

TPC-H: Total hot run time: 32607 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 88abd8e5410a369366ffcaee8e9ccf4d7d014068, data reload: false

------ Round 1 ----------------------------------
q1	17622	5200	5057	5057
q2	2051	303	177	177
q3	10469	1309	715	715
q4	10228	1018	530	530
q5	7659	2429	2403	2403
q6	206	169	133	133
q7	888	742	599	599
q8	9309	1255	1128	1128
q9	5006	4796	4886	4796
q10	6798	2315	1885	1885
q11	491	272	257	257
q12	339	359	215	215
q13	17762	3653	3039	3039
q14	227	224	204	204
q15	531	504	484	484
q16	630	607	578	578
q17	572	867	363	363
q18	6866	6505	6370	6370
q19	1754	954	547	547
q20	313	330	191	191
q21	2774	2154	1985	1985
q22	1070	1019	951	951
Total cold run time: 103565 ms
Total hot run time: 32607 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5185	5119	5137	5119
q2	253	335	237	237
q3	2179	2661	2302	2302
q4	1393	1834	1335	1335
q5	4258	4121	4163	4121
q6	213	160	121	121
q7	1860	1911	1766	1766
q8	2608	2611	2613	2611
q9	7269	7133	7149	7133
q10	3029	3209	2755	2755
q11	591	513	499	499
q12	693	753	658	658
q13	3464	3909	3232	3232
q14	293	315	270	270
q15	522	474	489	474
q16	651	669	630	630
q17	1168	1653	1307	1307
q18	7823	7535	7463	7463
q19	808	784	983	784
q20	1971	2050	1891	1891
q21	5556	5006	4864	4864
q22	1101	1085	1006	1006
Total cold run time: 52888 ms
Total hot run time: 50578 ms

doris-robot · 2025-03-06T07:44:02Z

TPC-DS: Total hot run time: 185789 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 88abd8e5410a369366ffcaee8e9ccf4d7d014068, data reload: false

query1	1001	378	386	378
query2	6534	1950	1883	1883
query3	6805	212	218	212
query4	26940	23416	23528	23416
query5	4280	674	473	473
query6	309	203	191	191
query7	4614	501	304	304
query8	278	225	235	225
query9	8636	2567	2536	2536
query10	472	320	250	250
query11	15588	15217	14862	14862
query12	158	106	106	106
query13	1656	517	378	378
query14	9475	6183	6536	6183
query15	204	190	183	183
query16	7242	636	487	487
query17	1198	750	586	586
query18	1967	409	309	309
query19	198	187	155	155
query20	119	116	118	116
query21	208	123	114	114
query22	4284	4393	4192	4192
query23	34124	33165	33317	33165
query24	7730	2389	2417	2389
query25	574	516	390	390
query26	1231	278	157	157
query27	2540	477	326	326
query28	4333	2398	2401	2398
query29	754	541	425	425
query30	277	224	189	189
query31	951	843	764	764
query32	71	65	63	63
query33	548	374	322	322
query34	791	844	485	485
query35	863	808	750	750
query36	969	996	884	884
query37	123	96	74	74
query38	4416	4126	4089	4089
query39	1433	1372	1385	1372
query40	203	114	101	101
query41	55	53	52	52
query42	125	106	101	101
query43	494	509	480	480
query44	1278	775	777	775
query45	177	171	167	167
query46	830	1024	618	618
query47	1773	1834	1783	1783
query48	362	399	310	310
query49	773	492	423	423
query50	691	723	396	396
query51	4168	4334	4122	4122
query52	109	110	91	91
query53	239	259	182	182
query54	488	485	414	414
query55	79	78	81	78
query56	276	274	273	273
query57	1130	1156	1096	1096
query58	247	231	232	231
query59	2707	2699	2468	2468
query60	292	280	255	255
query61	121	121	123	121
query62	851	709	681	681
query63	240	189	191	189
query64	4339	1004	659	659
query65	4376	4333	4353	4333
query66	1126	412	312	312
query67	15725	15602	15329	15329
query68	8784	874	516	516
query69	479	294	266	266
query70	1179	1135	1095	1095
query71	451	292	259	259
query72	5575	3590	3756	3590
query73	818	741	336	336
query74	9005	9182	9101	9101
query75	4019	3158	2718	2718
query76	3714	1178	754	754
query77	785	378	293	293
query78	10144	10362	9338	9338
query79	1900	895	594	594
query80	634	521	440	440
query81	475	271	224	224
query82	671	131	97	97
query83	176	176	159	159
query84	250	105	94	94
query85	799	353	308	308
query86	335	303	292	292
query87	4432	4416	4411	4411
query88	2878	2268	2208	2208
query89	397	316	288	288
query90	1930	215	208	208
query91	146	146	112	112
query92	76	62	57	57
query93	1141	1036	582	582
query94	678	416	310	310
query95	359	265	254	254
query96	480	571	270	270
query97	3334	3368	3284	3284
query98	223	209	206	206
query99	1473	1429	1304	1304
Total cold run time: 274908 ms
Total hot run time: 185789 ms

doris-robot · 2025-03-06T07:49:29Z

ClickBench: Total hot run time: 30.91 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 88abd8e5410a369366ffcaee8e9ccf4d7d014068, data reload: false

query1	0.04	0.04	0.03
query2	0.06	0.04	0.03
query3	0.24	0.06	0.07
query4	1.62	0.11	0.10
query5	0.55	0.54	0.56
query6	1.18	0.72	0.72
query7	0.03	0.02	0.02
query8	0.04	0.04	0.04
query9	0.58	0.54	0.51
query10	0.56	0.58	0.57
query11	0.16	0.11	0.11
query12	0.15	0.11	0.12
query13	0.62	0.60	0.59
query14	2.68	2.83	2.72
query15	0.92	0.86	0.85
query16	0.39	0.38	0.39
query17	1.00	1.02	1.00
query18	0.21	0.20	0.19
query19	1.95	1.81	2.05
query20	0.01	0.01	0.01
query21	15.37	0.87	0.55
query22	0.76	1.07	0.64
query23	15.08	1.39	0.59
query24	6.84	1.98	0.95
query25	0.52	0.17	0.19
query26	0.56	0.16	0.13
query27	0.06	0.05	0.04
query28	10.24	0.87	0.44
query29	12.61	3.94	3.26
query30	0.25	0.09	0.06
query31	2.82	0.60	0.38
query32	3.23	0.55	0.48
query33	3.02	3.00	3.01
query34	15.81	5.09	4.51
query35	4.52	4.52	4.52
query36	0.65	0.49	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.02	0.02	0.02
query40	0.17	0.14	0.14
query41	0.09	0.03	0.02
query42	0.03	0.03	0.02
query43	0.04	0.03	0.02
Total cold run time: 105.82 s
Total hot run time: 30.91 s

…47378 (#50222) Cherry-picked from #47378 Co-authored-by: zhangyuan <ayuanzhang@tencent.com>

lide-reed reviewed Jan 23, 2025

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSExternalCatalog.java Outdated Show resolved Hide resolved

[bugfix](hive) fix_hive_metastore_client_socket_timeout

88abd8e

justfortaste force-pushed the fix_hive_metastore_client_socket_timeout branch from 26757b0 to 88abd8e Compare February 27, 2025 06:41

lide-reed approved these changes Feb 27, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 27, 2025

github-actions bot added the reviewed label Feb 27, 2025

liutang123 approved these changes Mar 6, 2025

View reviewed changes

lide-reed merged commit 6521cb6 into apache:master Mar 7, 2025
24 of 26 checks passed

lide-reed added dev/3.0.5-merged dev/2.1.9-merged labels Mar 13, 2025

lide-reed removed dev/2.1.9-merged dev/3.0.5-merged labels Apr 18, 2025

This was referenced Apr 21, 2025

[fix](hive) Fix hive metastore client socket timeout #50192

Merged

[bugfix](hive) fix_hive_metastore_client_socket_timeout #50193

Closed

yiguolei added dev/2.1.10-merged usercase Important user case type label dev/3.0.x labels Apr 21, 2025

github-actions bot pushed a commit that referenced this pull request Apr 21, 2025

[bugfix](hive) fix_hive_metastore_client_socket_timeout (#47378)

b732345

github-actions bot mentioned this pull request Apr 21, 2025

branch-3.0: [bugfix](hive) fix_hive_metastore_client_socket_timeout #47378 #50222

Merged

dataroaring pushed a commit that referenced this pull request Apr 22, 2025

branch-3.0: [bugfix](hive) fix_hive_metastore_client_socket_timeout #…

dfad916

…47378 (#50222) Cherry-picked from #47378 Co-authored-by: zhangyuan <ayuanzhang@tencent.com>

dataroaring added dev/3.0.6-merged and removed dev/3.0.x labels Apr 22, 2025

koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025

[bugfix](hive) fix_hive_metastore_client_socket_timeout (apache#47378)

055a710

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[bugfix](hive) fix_hive_metastore_client_socket_timeout #47378

[bugfix](hive) fix_hive_metastore_client_socket_timeout #47378

Uh oh!

justfortaste commented Jan 23, 2025

Uh oh!

Thearas commented Jan 23, 2025

Uh oh!

Uh oh!

lide-reed left a comment

Uh oh!

github-actions bot commented Feb 27, 2025

Uh oh!

github-actions bot commented Feb 27, 2025

Uh oh!

liutang123 commented Mar 6, 2025

Uh oh!

doris-robot commented Mar 6, 2025

Uh oh!

doris-robot commented Mar 6, 2025

Uh oh!

doris-robot commented Mar 6, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

[bugfix](hive) fix_hive_metastore_client_socket_timeout #47378

[bugfix](hive) fix_hive_metastore_client_socket_timeout #47378

Uh oh!

Conversation

justfortaste commented Jan 23, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Jan 23, 2025

Uh oh!

Uh oh!

lide-reed left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 27, 2025

Uh oh!

github-actions bot commented Feb 27, 2025

Uh oh!

liutang123 commented Mar 6, 2025

Uh oh!

doris-robot commented Mar 6, 2025

Uh oh!

doris-robot commented Mar 6, 2025

Uh oh!

doris-robot commented Mar 6, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants