Skip to content

[fix](storage vault) Fix missing use_path_style when create storage vault #45155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
gavinchou merged 2 commits into from
Dec 17, 2024
Merged

[fix](storage vault) Fix missing use_path_style when create storage vault #45155

gavinchou merged 2 commits into from
Dec 17, 2024

Conversation

@gavinchou
Copy link
Contributor

@gavinchou gavinchou commented Dec 7, 2024

What problem does this PR solve?

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

doris-robot commented Dec 7, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@gavinchou gavinchou force-pushed the gavin-fix-missing-use-path-style branch from fcd08a1 to 96b948c Compare December 7, 2024 20:37
@gavinchou
Copy link
Contributor Author

gavinchou commented Dec 7, 2024

run buildall

gavinchou added a commit that referenced this pull request Dec 7, 2024
@gavinchou
[fix](storage vault) Fix missing use_path_style when create storage v…
5098a4a 
…ault (#45155)
@doris-robot
Copy link

doris-robot commented Dec 7, 2024

TPC-H: Total hot run time: 39946 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 96b948c89818e03d7ce99d0c20a7ad2e86ad546b, data reload: false

------ Round 1 ----------------------------------
q1	17631	7484	7270	7270
q2	2038	184	187	184
q3	10566	1063	1168	1063
q4	10553	761	744	744
q5	7607	2732	2710	2710
q6	234	147	150	147
q7	995	662	591	591
q8	9237	1866	1930	1866
q9	6664	6500	6484	6484
q10	7036	2297	2313	2297
q11	480	260	261	260
q12	426	229	222	222
q13	17788	3037	3032	3032
q14	248	214	215	214
q15	565	517	521	517
q16	655	589	606	589
q17	967	596	525	525
q18	7249	6934	6621	6621
q19	1336	1011	954	954
q20	468	186	179	179
q21	4008	3179	3173	3173
q22	389	304	306	304
Total cold run time: 107140 ms
Total hot run time: 39946 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7250	7238	7229	7229
q2	335	231	230	230
q3	2903	2767	2937	2767
q4	2058	1810	1827	1810
q5	5695	5631	5639	5631
q6	232	142	141	141
q7	2243	1810	1834	1810
q8	3356	3559	3473	3473
q9	8989	8984	8974	8974
q10	3586	3590	3556	3556
q11	601	494	500	494
q12	790	591	601	591
q13	10993	3252	3200	3200
q14	311	279	275	275
q15	584	540	511	511
q16	675	646	633	633
q17	1856	1645	1628	1628
q18	8219	7772	7601	7601
q19	1707	1627	1539	1539
q20	2114	1876	1893	1876
q21	5633	5526	5531	5526
q22	656	577	566	566
Total cold run time: 70786 ms
Total hot run time: 60061 ms

@gavinchou
Copy link
Contributor Author

gavinchou commented Dec 7, 2024

run buildall

gavinchou added a commit that referenced this pull request Dec 7, 2024
@gavinchou
[fix](storage vault) Fix missing use_path_style when create storage v…
62a58bf 
…ault (#45155)
@doris-robot
Copy link

doris-robot commented Dec 7, 2024

TPC-H: Total hot run time: 40245 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dbe00c233e8adff942410d6ebddbbbeb0af32ef1, data reload: false

------ Round 1 ----------------------------------
q1	17617	7454	7337	7337
q2	2046	179	173	173
q3	10625	1170	1175	1170
q4	10536	740	744	740
q5	7621	2762	2759	2759
q6	238	149	149	149
q7	1011	656	615	615
q8	9236	1811	1895	1811
q9	6801	6573	6517	6517
q10	7034	2322	2354	2322
q11	463	254	263	254
q12	415	228	237	228
q13	17782	3011	3000	3000
q14	246	208	218	208
q15	558	533	504	504
q16	661	577	576	576
q17	991	562	517	517
q18	7336	6702	6833	6702
q19	1327	1049	1013	1013
q20	485	176	185	176
q21	4063	3206	3159	3159
q22	378	315	331	315
Total cold run time: 107470 ms
Total hot run time: 40245 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7233	7259	7326	7259
q2	327	238	231	231
q3	2994	2825	2789	2789
q4	1976	1703	1725	1703
q5	5421	5403	5440	5403
q6	221	137	139	137
q7	2158	1715	1716	1715
q8	3285	3419	3466	3419
q9	8693	8634	8675	8634
q10	3505	3432	3416	3416
q11	639	482	488	482
q12	812	600	620	600
q13	7803	3018	3018	3018
q14	284	256	270	256
q15	576	526	517	517
q16	696	623	633	623
q17	1816	1596	1571	1571
q18	7921	7322	7499	7322
q19	1654	1461	1474	1461
q20	2057	1844	1828	1828
q21	5408	5298	5225	5225
q22	633	563	565	563
Total cold run time: 66112 ms
Total hot run time: 58172 ms

@doris-robot
Copy link

doris-robot commented Dec 7, 2024

TPC-DS: Total hot run time: 191415 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dbe00c233e8adff942410d6ebddbbbeb0af32ef1, data reload: false

query1	962	383	369	369
query2	6456	2072	2064	2064
query3	6391	214	226	214
query4	33710	23614	23637	23614
query5	4140	475	429	429
query6	263	177	174	174
query7	4482	304	309	304
query8	272	228	232	228
query9	9108	2645	2662	2645
query10	482	255	248	248
query11	18127	15374	15278	15278
query12	161	103	109	103
query13	1503	402	405	402
query14	9311	7663	7010	7010
query15	270	181	181	181
query16	8038	476	504	476
query17	1596	581	581	581
query18	2038	300	302	300
query19	340	161	156	156
query20	118	111	113	111
query21	210	103	108	103
query22	4697	4261	4323	4261
query23	35064	34036	35342	34036
query24	11207	2430	2431	2430
query25	542	372	383	372
query26	1133	149	158	149
query27	2750	278	274	274
query28	8017	2444	2422	2422
query29	656	415	401	401
query30	301	153	154	153
query31	1050	798	818	798
query32	92	55	68	55
query33	773	296	287	287
query34	988	513	545	513
query35	895	754	729	729
query36	1108	953	945	945
query37	200	70	81	70
query38	4394	4139	4271	4139
query39	1605	1416	1426	1416
query40	276	106	98	98
query41	47	42	44	42
query42	109	98	97	97
query43	546	508	493	493
query44	1258	802	817	802
query45	192	161	174	161
query46	1188	730	708	708
query47	1976	1856	1878	1856
query48	398	299	308	299
query49	1263	396	397	396
query50	815	384	390	384
query51	7192	7279	7234	7234
query52	95	92	89	89
query53	259	178	177	177
query54	1037	390	421	390
query55	78	77	78	77
query56	263	226	232	226
query57	1251	1122	1127	1122
query58	232	215	214	214
query59	3319	3104	2954	2954
query60	265	263	240	240
query61	115	104	101	101
query62	901	679	680	679
query63	215	183	189	183
query64	4932	701	617	617
query65	3282	3227	3263	3227
query66	1301	311	310	310
query67	16265	15690	15677	15677
query68	5233	553	550	550
query69	416	249	313	249
query70	1207	1128	1134	1128
query71	334	282	268	268
query72	6080	4124	4103	4103
query73	767	357	363	357
query74	10499	8966	9120	8966
query75	3464	2671	2649	2649
query76	3071	1076	980	980
query77	473	277	279	277
query78	10609	9531	9508	9508
query79	1610	614	609	609
query80	1158	424	447	424
query81	535	229	226	226
query82	808	117	118	117
query83	258	144	148	144
query84	249	71	66	66
query85	1362	298	327	298
query86	367	303	297	297
query87	4683	4567	4583	4567
query88	3378	2186	2173	2173
query89	409	297	304	297
query90	2077	187	183	183
query91	135	104	100	100
query92	60	51	51	51
query93	1179	555	545	545
query94	1101	298	288	288
query95	360	254	241	241
query96	609	276	285	276
query97	2857	2688	2718	2688
query98	220	200	191	191
query99	1568	1316	1306	1306
Total cold run time: 299656 ms
Total hot run time: 191415 ms

@doris-robot
Copy link

doris-robot commented Dec 7, 2024

ClickBench: Total hot run time: 33.13 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dbe00c233e8adff942410d6ebddbbbeb0af32ef1, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.03
query3	0.23	0.08	0.07
query4	1.62	0.10	0.11
query5	0.43	0.41	0.40
query6	1.16	0.65	0.65
query7	0.02	0.02	0.01
query8	0.03	0.03	0.03
query9	0.60	0.53	0.50
query10	0.56	0.56	0.56
query11	0.14	0.11	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	2.82	2.83	2.74
query15	0.91	0.82	0.83
query16	0.38	0.38	0.38
query17	1.07	1.07	1.05
query18	0.23	0.22	0.21
query19	1.88	1.92	2.03
query20	0.02	0.01	0.02
query21	15.36	0.60	0.56
query22	2.89	2.36	1.76
query23	16.94	1.02	0.77
query24	2.99	1.60	1.48
query25	0.22	0.08	0.22
query26	0.63	0.14	0.13
query27	0.05	0.04	0.04
query28	10.53	1.17	1.07
query29	12.52	3.20	3.18
query30	0.24	0.06	0.06
query31	2.86	0.38	0.39
query32	3.29	0.47	0.47
query33	2.97	3.04	3.11
query34	16.87	4.48	4.52
query35	4.62	4.51	4.56
query36	0.67	0.51	0.50
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.09	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.11 s
Total hot run time: 33.13 s

dataroaring
dataroaring approved these changes Dec 11, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 11, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Dec 11, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 11, 2024

PR approved by anyone and no changes requested.

@gavinchou gavinchou merged commit b252c0f into apache:master Dec 17, 2024
23 of 25 checks passed
github-actions bot pushed a commit that referenced this pull request Dec 17, 2024
@gavinchou
[fix](storage vault) Fix missing use_path_style when create storage v…
0c5a530 
…ault (#45155)
@github-actions github-actions bot mentioned this pull request Dec 17, 2024
Merged
dataroaring pushed a commit that referenced this pull request Dec 23, 2024
@github-actions @gavinchou
branch-3.0: [fix](storage vault) Fix missing use_path_style when crea…
ca9bf8c 
…te storage vault #45155 (#45560)

Cherry-picked from #45155

Co-authored-by: Gavin Chou <gavin@selectdb.com>
@gavinchou gavinchou mentioned this pull request Feb 18, 2025
Closed
@yagagagaga yagagagaga mentioned this pull request May 6, 2025
Merged
16 tasks
gavinchou pushed a commit that referenced this pull request May 7, 2025
@yagagagaga
[fix](vault) fix UnsupportedOperationException if no use_path_style…
210b960 
… in CreateStorageVaultStmt (#50615)

Related PR: #50356 #45155

Problem Summary:

```sql
CREATE STORAGE VAULT test PROPERTIES
    (
      "type"="S3",
      "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
      "s3.access_key" = "*****",
      "s3.secret_key" = "*****",
      "s3.region" = "ap-guangzhou",
      "s3.root.path" = "prefix",
      "s3.bucket" = "test",
      "provider" = "COS"
    )
```

throw Exception:
```java
2025-05-06 15:05:16,395 WARN (mysql-nio-pool-0|235) [StmtExecutor.handleDdlStmt():3216] DDL statement(CREATE STORAGE VAULT test PROPERTIES
    (
      "type"="S3",
      "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
      "s3.access_key" = "*****",
      "s3.secret_key" = "*****",
      "s3.region" = "ap-guangzhou",
      "s3.root.path" = "prefix",
      "s3.bucket" = "test",
      "provider" = "COS"
    )) process failed.
java.lang.UnsupportedOperationException: null
        at com.google.common.collect.ImmutableMap.put(ImmutableMap.java:814) ~[guava-33.2.1-jre.jar:?]
        at org.apache.doris.catalog.StorageVault.getStorageVaultInstance(StorageVault.java:151) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.StorageVault.fromStmt(StorageVault.java:103) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.StorageVaultMgr.createStorageVaultResource(StorageVaultMgr.java:69) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.DdlExecutor.execute(DdlExecutor.java:440) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleDdlStmt(StmtExecutor.java:3195) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1124) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:644) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:574) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:559) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:349) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:249) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:233) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:261) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:444) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:833) ~[?:?]
```
yagagagaga added a commit to yagagagaga/doris that referenced this pull request May 7, 2025
@yagagagaga
[fix](vault) fix UnsupportedOperationException if no use_path_style…
4a0a436 
… in CreateStorageVaultStmt (apache#50615)

Related PR: apache#50356 apache#45155

Problem Summary:

```sql
CREATE STORAGE VAULT test PROPERTIES
    (
      "type"="S3",
      "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
      "s3.access_key" = "*****",
      "s3.secret_key" = "*****",
      "s3.region" = "ap-guangzhou",
      "s3.root.path" = "prefix",
      "s3.bucket" = "test",
      "provider" = "COS"
    )
```

throw Exception:
```java
2025-05-06 15:05:16,395 WARN (mysql-nio-pool-0|235) [StmtExecutor.handleDdlStmt():3216] DDL statement(CREATE STORAGE VAULT test PROPERTIES
    (
      "type"="S3",
      "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
      "s3.access_key" = "*****",
      "s3.secret_key" = "*****",
      "s3.region" = "ap-guangzhou",
      "s3.root.path" = "prefix",
      "s3.bucket" = "test",
      "provider" = "COS"
    )) process failed.
java.lang.UnsupportedOperationException: null
        at com.google.common.collect.ImmutableMap.put(ImmutableMap.java:814) ~[guava-33.2.1-jre.jar:?]
        at org.apache.doris.catalog.StorageVault.getStorageVaultInstance(StorageVault.java:151) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.StorageVault.fromStmt(StorageVault.java:103) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.StorageVaultMgr.createStorageVaultResource(StorageVaultMgr.java:69) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.DdlExecutor.execute(DdlExecutor.java:440) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleDdlStmt(StmtExecutor.java:3195) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1124) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:644) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:574) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:559) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:349) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:249) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:233) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:261) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:444) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:833) ~[?:?]
```

(cherry picked from commit 210b960)
morningman pushed a commit to morningman/doris that referenced this pull request May 7, 2025
@yagagagaga @morningman
[fix](vault) fix UnsupportedOperationException if no use_path_style…
4d25e92 
… in CreateStorageVaultStmt (apache#50615)

Related PR: apache#50356 apache#45155

Problem Summary:

```sql
CREATE STORAGE VAULT test PROPERTIES
    (
      "type"="S3",
      "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
      "s3.access_key" = "*****",
      "s3.secret_key" = "*****",
      "s3.region" = "ap-guangzhou",
      "s3.root.path" = "prefix",
      "s3.bucket" = "test",
      "provider" = "COS"
    )
```

throw Exception:
```java
2025-05-06 15:05:16,395 WARN (mysql-nio-pool-0|235) [StmtExecutor.handleDdlStmt():3216] DDL statement(CREATE STORAGE VAULT test PROPERTIES
    (
      "type"="S3",
      "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
      "s3.access_key" = "*****",
      "s3.secret_key" = "*****",
      "s3.region" = "ap-guangzhou",
      "s3.root.path" = "prefix",
      "s3.bucket" = "test",
      "provider" = "COS"
    )) process failed.
java.lang.UnsupportedOperationException: null
        at com.google.common.collect.ImmutableMap.put(ImmutableMap.java:814) ~[guava-33.2.1-jre.jar:?]
        at org.apache.doris.catalog.StorageVault.getStorageVaultInstance(StorageVault.java:151) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.StorageVault.fromStmt(StorageVault.java:103) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.StorageVaultMgr.createStorageVaultResource(StorageVaultMgr.java:69) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.DdlExecutor.execute(DdlExecutor.java:440) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleDdlStmt(StmtExecutor.java:3195) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1124) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:644) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:574) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:559) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:349) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:249) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:233) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:261) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:444) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:833) ~[?:?]
```
morningman pushed a commit to morningman/doris that referenced this pull request May 7, 2025
@yagagagaga @morningman
[fix](vault) fix UnsupportedOperationException if no use_path_style…
6c52b12 
… in CreateStorageVaultStmt (apache#50615)

Related PR: apache#50356 apache#45155

Problem Summary:

```sql
CREATE STORAGE VAULT test PROPERTIES
    (
      "type"="S3",
      "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
      "s3.access_key" = "*****",
      "s3.secret_key" = "*****",
      "s3.region" = "ap-guangzhou",
      "s3.root.path" = "prefix",
      "s3.bucket" = "test",
      "provider" = "COS"
    )
```

throw Exception:
```java
2025-05-06 15:05:16,395 WARN (mysql-nio-pool-0|235) [StmtExecutor.handleDdlStmt():3216] DDL statement(CREATE STORAGE VAULT test PROPERTIES
    (
      "type"="S3",
      "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
      "s3.access_key" = "*****",
      "s3.secret_key" = "*****",
      "s3.region" = "ap-guangzhou",
      "s3.root.path" = "prefix",
      "s3.bucket" = "test",
      "provider" = "COS"
    )) process failed.
java.lang.UnsupportedOperationException: null
        at com.google.common.collect.ImmutableMap.put(ImmutableMap.java:814) ~[guava-33.2.1-jre.jar:?]
        at org.apache.doris.catalog.StorageVault.getStorageVaultInstance(StorageVault.java:151) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.StorageVault.fromStmt(StorageVault.java:103) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.StorageVaultMgr.createStorageVaultResource(StorageVaultMgr.java:69) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.DdlExecutor.execute(DdlExecutor.java:440) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleDdlStmt(StmtExecutor.java:3195) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1124) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:644) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:574) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:559) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:349) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:249) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:233) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:261) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:444) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:833) ~[?:?]
```
morningman pushed a commit to morningman/doris that referenced this pull request May 14, 2025
@yagagagaga @morningman
[fix](vault) fix UnsupportedOperationException if no use_path_style…
d3608b4 
… in CreateStorageVaultStmt (apache#50615)

Related PR: apache#50356 apache#45155

Problem Summary:

```sql
CREATE STORAGE VAULT test PROPERTIES
    (
      "type"="S3",
      "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
      "s3.access_key" = "*****",
      "s3.secret_key" = "*****",
      "s3.region" = "ap-guangzhou",
      "s3.root.path" = "prefix",
      "s3.bucket" = "test",
      "provider" = "COS"
    )
```

throw Exception:
```java
2025-05-06 15:05:16,395 WARN (mysql-nio-pool-0|235) [StmtExecutor.handleDdlStmt():3216] DDL statement(CREATE STORAGE VAULT test PROPERTIES
    (
      "type"="S3",
      "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
      "s3.access_key" = "*****",
      "s3.secret_key" = "*****",
      "s3.region" = "ap-guangzhou",
      "s3.root.path" = "prefix",
      "s3.bucket" = "test",
      "provider" = "COS"
    )) process failed.
java.lang.UnsupportedOperationException: null
        at com.google.common.collect.ImmutableMap.put(ImmutableMap.java:814) ~[guava-33.2.1-jre.jar:?]
        at org.apache.doris.catalog.StorageVault.getStorageVaultInstance(StorageVault.java:151) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.StorageVault.fromStmt(StorageVault.java:103) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.StorageVaultMgr.createStorageVaultResource(StorageVaultMgr.java:69) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.DdlExecutor.execute(DdlExecutor.java:440) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleDdlStmt(StmtExecutor.java:3195) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1124) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:644) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:574) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:559) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:349) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:249) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:233) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:261) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:444) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:833) ~[?:?]
```
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@yagagagaga @koarz
[fix](vault) fix UnsupportedOperationException if no use_path_style…
dc3c412 
… in CreateStorageVaultStmt (apache#50615)

Related PR: apache#50356 apache#45155

Problem Summary:

```sql
CREATE STORAGE VAULT test PROPERTIES
    (
      "type"="S3",
      "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
      "s3.access_key" = "*****",
      "s3.secret_key" = "*****",
      "s3.region" = "ap-guangzhou",
      "s3.root.path" = "prefix",
      "s3.bucket" = "test",
      "provider" = "COS"
    )
```

throw Exception:
```java
2025-05-06 15:05:16,395 WARN (mysql-nio-pool-0|235) [StmtExecutor.handleDdlStmt():3216] DDL statement(CREATE STORAGE VAULT test PROPERTIES
    (
      "type"="S3",
      "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
      "s3.access_key" = "*****",
      "s3.secret_key" = "*****",
      "s3.region" = "ap-guangzhou",
      "s3.root.path" = "prefix",
      "s3.bucket" = "test",
      "provider" = "COS"
    )) process failed.
java.lang.UnsupportedOperationException: null
        at com.google.common.collect.ImmutableMap.put(ImmutableMap.java:814) ~[guava-33.2.1-jre.jar:?]
        at org.apache.doris.catalog.StorageVault.getStorageVaultInstance(StorageVault.java:151) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.StorageVault.fromStmt(StorageVault.java:103) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.StorageVaultMgr.createStorageVaultResource(StorageVaultMgr.java:69) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.DdlExecutor.execute(DdlExecutor.java:440) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleDdlStmt(StmtExecutor.java:3195) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1124) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:644) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:574) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:559) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:349) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:249) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:233) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:261) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:444) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:833) ~[?:?]
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@luwei16 luwei16 luwei16 approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.3-merged dev/3.0.4-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@gavinchou @doris-robot @luwei16 @dataroaring