[fix](S3)Downgrade the AWS SDK version to avoid impacting other S3-compatible services. #58566

CalvinKirs · 2025-12-01T07:48:33Z

…

This PR addresses an issue related to AWS SDK for Java v2 S3 client behavior described in: aws/aws-sdk-java-v2#5805

Starting from newer versions of the SDK, the S3 client introduces unexpected behavior that causes our application to fail during normal S3 operations. Due to this problem, we were forced to downgrade aws-s3 from the latest version to 2.29.x, which restored correct functionality. The regression affects stability in our production environment and prevents us from upgrading the SDK.

#57226

…mpatible services. This PR addresses an issue related to AWS SDK for Java v2 S3 client behavior described in: aws/aws-sdk-java-v2#5805 Starting from newer versions of the SDK, the S3 client introduces unexpected behavior that causes our application to fail during normal S3 operations. Due to this problem, we were forced to downgrade aws-s3 from the latest version to 2.29.x, which restored correct functionality. The regression affects stability in our production environment and prevents us from upgrading the SDK.

Thearas · 2025-12-01T07:48:40Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

CalvinKirs · 2025-12-01T07:48:46Z

run buildall

CalvinKirs · 2025-12-01T08:08:55Z

run buildall

doris-robot · 2025-12-01T08:48:52Z

TPC-H: Total hot run time: 34064 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c613ff3595b839b33d781d07109262956802f5e2, data reload: false

------ Round 1 ----------------------------------
q1	17610	5131	4910	4910
q2	2029	362	216	216
q3	10200	1337	758	758
q4	10219	829	338	338
q5	7542	2459	2137	2137
q6	191	174	141	141
q7	968	801	640	640
q8	9373	1372	1006	1006
q9	7065	5313	5345	5313
q10	6856	2212	1804	1804
q11	543	315	297	297
q12	392	377	239	239
q13	17801	3716	3040	3040
q14	239	236	223	223
q15	611	516	514	514
q16	916	880	797	797
q17	609	754	548	548
q18	7555	7075	7056	7056
q19	1537	971	587	587
q20	337	348	233	233
q21	3891	2699	2331	2331
q22	1007	1000	936	936
Total cold run time: 107491 ms
Total hot run time: 34064 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5088	4985	4957	4957
q2	314	410	335	335
q3	2193	2674	2331	2331
q4	1374	1768	1304	1304
q5	4289	4450	4700	4450
q6	218	183	138	138
q7	2083	2049	1821	1821
q8	2662	2572	2498	2498
q9	7577	7518	7488	7488
q10	3081	3284	2829	2829
q11	580	515	500	500
q12	689	783	618	618
q13	3571	3933	3254	3254
q14	300	313	289	289
q15	560	513	527	513
q16	932	965	882	882
q17	1161	1414	1460	1414
q18	7939	7584	7618	7584
q19	873	802	831	802
q20	2057	2161	1959	1959
q21	5124	4411	4128	4128
q22	1053	1035	957	957
Total cold run time: 53718 ms
Total hot run time: 51051 ms

doris-robot · 2025-12-01T09:00:17Z

TPC-DS: Total hot run time: 182308 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c613ff3595b839b33d781d07109262956802f5e2, data reload: false

query1	1084	409	395	395
query2	6579	1194	1193	1193
query3	6744	227	227	227
query4	25449	23362	22723	22723
query5	4485	673	518	518
query6	337	249	239	239
query7	4670	517	317	317
query8	340	256	251	251
query9	8774	2674	2688	2674
query10	531	371	319	319
query11	15352	14774	15023	14774
query12	182	123	118	118
query13	1698	577	467	467
query14	9701	6058	6039	6039
query15	218	205	193	193
query16	7487	712	539	539
query17	1264	791	643	643
query18	2041	444	351	351
query19	216	224	184	184
query20	141	135	127	127
query21	221	139	118	118
query22	3948	3950	3953	3950
query23	32973	32050	31941	31941
query24	8470	2461	2458	2458
query25	630	517	469	469
query26	1260	272	167	167
query27	2748	509	348	348
query28	4309	2185	2133	2133
query29	806	660	496	496
query30	311	244	217	217
query31	822	719	610	610
query32	83	82	76	76
query33	633	375	332	332
query34	842	891	564	564
query35	816	842	754	754
query36	896	936	837	837
query37	124	114	94	94
query38	3851	3835	3851	3835
query39	1470	1413	1406	1406
query40	228	135	125	125
query41	68	63	63	63
query42	130	120	116	116
query43	429	461	422	422
query44	1318	761	761	761
query45	201	193	188	188
query46	896	1034	670	670
query47	1670	1703	1648	1648
query48	394	441	329	329
query49	751	515	428	428
query50	674	725	440	440
query51	3907	3910	3970	3910
query52	114	113	107	107
query53	259	263	199	199
query54	314	300	281	281
query55	102	98	96	96
query56	334	329	327	327
query57	1122	1170	1106	1106
query58	295	276	275	275
query59	2340	2454	2333	2333
query60	371	352	337	337
query61	164	156	159	156
query62	805	734	665	665
query63	236	202	209	202
query64	4503	1217	1044	1044
query65	4040	4024	3965	3965
query66	1161	472	368	368
query67	15188	15219	14738	14738
query68	7757	968	636	636
query69	532	355	325	325
query70	1120	1037	1049	1037
query71	449	362	334	334
query72	5928	4960	4915	4915
query73	694	589	347	347
query74	8756	8789	8643	8643
query75	3041	3034	2563	2563
query76	3323	1154	763	763
query77	507	417	320	320
query78	9552	9493	8864	8864
query79	2259	841	578	578
query80	637	573	473	473
query81	517	276	243	243
query82	480	142	113	113
query83	264	261	243	243
query84	262	120	98	98
query85	950	502	443	443
query86	387	299	279	279
query87	3986	4068	3934	3934
query88	4111	2281	2272	2272
query89	390	331	299	299
query90	1936	247	218	218
query91	172	175	145	145
query92	89	69	72	69
query93	1934	1007	663	663
query94	782	451	342	342
query95	514	413	409	409
query96	521	547	283	283
query97	2659	2666	2586	2586
query98	246	226	210	210
query99	1312	1373	1266	1266
Total cold run time: 270587 ms
Total hot run time: 182308 ms

doris-robot · 2025-12-01T09:05:19Z

ClickBench: Total hot run time: 27.52 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c613ff3595b839b33d781d07109262956802f5e2, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.05	0.04
query3	0.26	0.09	0.10
query4	1.60	0.11	0.10
query5	0.28	0.25	0.26
query6	1.18	0.64	0.64
query7	0.04	0.03	0.02
query8	0.05	0.04	0.04
query9	0.58	0.51	0.51
query10	0.56	0.56	0.56
query11	0.15	0.11	0.11
query12	0.16	0.11	0.12
query13	0.63	0.60	0.61
query14	0.99	0.98	0.98
query15	0.81	0.80	0.81
query16	0.40	0.39	0.42
query17	1.03	1.06	1.05
query18	0.23	0.20	0.21
query19	1.95	1.82	1.87
query20	0.01	0.01	0.01
query21	15.48	0.27	0.14
query22	4.83	0.05	0.05
query23	15.97	0.27	0.10
query24	1.25	0.68	0.30
query25	0.08	0.06	0.06
query26	0.14	0.13	0.14
query27	0.05	0.06	0.07
query28	3.19	1.24	1.02
query29	12.59	3.98	3.32
query30	0.28	0.13	0.12
query31	2.81	0.60	0.39
query32	3.23	0.54	0.48
query33	3.00	3.06	3.05
query34	16.54	5.21	4.54
query35	4.55	4.54	4.59
query36	0.68	0.53	0.49
query37	0.10	0.07	0.06
query38	0.07	0.05	0.04
query39	0.05	0.03	0.04
query40	0.18	0.14	0.15
query41	0.08	0.03	0.03
query42	0.05	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 96.3 s
Total hot run time: 27.52 s

github-actions · 2025-12-02T02:39:48Z

PR approved by anyone and no changes requested.

morningman

LGTM

github-actions · 2025-12-02T03:15:54Z

PR approved by at least one committer and no changes requested.

…mpatible services. (#58566) … This PR addresses an issue related to AWS SDK for Java v2 S3 client behavior described in: aws/aws-sdk-java-v2#5805 Starting from newer versions of the SDK, the S3 client introduces unexpected behavior that causes our application to fail during normal S3 operations. Due to this problem, we were forced to downgrade aws-s3 from the latest version to 2.29.x, which restored correct functionality. The regression affects stability in our production environment and prevents us from upgrading the SDK. #57226

… other S3-compatible services. #58566 (#58608) Cherry-picked from #58566 Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>

… other S3-compatible services. #58566 (#58607) Cherry-picked from #58566 Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>

…mpatible services. (apache#58566) … This PR addresses an issue related to AWS SDK for Java v2 S3 client behavior described in: aws/aws-sdk-java-v2#5805 Starting from newer versions of the SDK, the S3 client introduces unexpected behavior that causes our application to fail during normal S3 operations. Due to this problem, we were forced to downgrade aws-s3 from the latest version to 2.29.x, which restored correct functionality. The regression affects stability in our production environment and prevents us from upgrading the SDK. apache#57226

CalvinKirs requested a review from morningman as a code owner December 1, 2025 07:48

1

c613ff3

hubgeter approved these changes Dec 2, 2025

View reviewed changes

github-actions bot added the reviewed label Dec 2, 2025

morningman approved these changes Dec 2, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 2, 2025

CalvinKirs added dev/3.1.x dev/4.0.x and removed approved Indicates a PR has been approved by one committer. reviewed labels Dec 2, 2025

CalvinKirs merged commit 21d56a5 into apache:master Dec 2, 2025
30 of 31 checks passed

github-actions bot mentioned this pull request Dec 2, 2025

branch-3.1: [fix](S3)Downgrade the AWS SDK version to avoid impacting other S3-compatible services. #58566 #58607

Merged

github-actions bot mentioned this pull request Dec 2, 2025

branch-4.0: [fix](S3)Downgrade the AWS SDK version to avoid impacting other S3-compatible services. #58566 #58608

Merged

yiguolei pushed a commit that referenced this pull request Dec 2, 2025

branch-4.0: [fix](S3)Downgrade the AWS SDK version to avoid impacting…

3fed2e1

… other S3-compatible services. #58566 (#58608) Cherry-picked from #58566 Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>

yiguolei added dev/4.0.2-merged and removed dev/4.0.x labels Dec 2, 2025

morrySnow pushed a commit that referenced this pull request Dec 4, 2025

branch-3.1: [fix](S3)Downgrade the AWS SDK version to avoid impacting…

f51544b

… other S3-compatible services. #58566 (#58607) Cherry-picked from #58566 Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>

morrySnow added dev/3.1.4-merged and removed dev/3.1.x labels Dec 4, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[fix](S3)Downgrade the AWS SDK version to avoid impacting other S3-compatible services. #58566

[fix](S3)Downgrade the AWS SDK version to avoid impacting other S3-compatible services. #58566

Uh oh!

CalvinKirs commented Dec 1, 2025 •

edited

Loading

Uh oh!

Thearas commented Dec 1, 2025

Uh oh!

CalvinKirs commented Dec 1, 2025

Uh oh!

CalvinKirs commented Dec 1, 2025

Uh oh!

doris-robot commented Dec 1, 2025

Uh oh!

doris-robot commented Dec 1, 2025

Uh oh!

doris-robot commented Dec 1, 2025

Uh oh!

github-actions bot commented Dec 2, 2025

Uh oh!

morningman left a comment

Uh oh!

github-actions bot commented Dec 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

[fix](S3)Downgrade the AWS SDK version to avoid impacting other S3-compatible services. #58566

[fix](S3)Downgrade the AWS SDK version to avoid impacting other S3-compatible services. #58566

Uh oh!

Conversation

CalvinKirs commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Thearas commented Dec 1, 2025

Uh oh!

CalvinKirs commented Dec 1, 2025

Uh oh!

CalvinKirs commented Dec 1, 2025

Uh oh!

doris-robot commented Dec 1, 2025

Uh oh!

doris-robot commented Dec 1, 2025

Uh oh!

doris-robot commented Dec 1, 2025

Uh oh!

github-actions bot commented Dec 2, 2025

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

CalvinKirs commented Dec 1, 2025 •

edited

Loading