Skip to content

[improve](glue)Enforce HTTPS protocol for Glue endpoint #58306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
CalvinKirs merged 2 commits into from
Nov 25, 2025
Merged

[improve](glue)Enforce HTTPS protocol for Glue endpoint #58306

CalvinKirs merged 2 commits into from
Nov 25, 2025

Conversation

@CalvinKirs
Copy link
Member

@CalvinKirs CalvinKirs commented Nov 24, 2025

FYI https://docs.aws.amazon.com/general/latest/gr/glue.html

This PR adds a validation to ensure that the Glue endpoint uses the HTTPS protocol.

  • Adds a check for Glue endpoint URLs.
  • Throws an IllegalArgumentException if the endpoint does not start with "https://".
  • Error message: "glue.endpoint must use https protocol, please set glue.endpoint to https://"

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@CalvinKirs
[improve](glue)Enforce HTTPS protocol for Glue endpoint
a6b47cb 
FYI https://docs.aws.amazon.com/general/latest/gr/glue.html

This PR adds a validation to ensure that the Glue endpoint uses the HTTPS protocol.

- Adds a check for Glue endpoint URLs.
- Throws an IllegalArgumentException if the endpoint does not start with "https://".
- Error message: "glue.endpoint must use https protocol, please set glue.endpoint to https://"
@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 24, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 25, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Nov 25, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 25, 2025

PR approved by anyone and no changes requested.

@CalvinKirs
Copy link
Member Author

CalvinKirs commented Nov 25, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Nov 25, 2025

TPC-H: Total hot run time: 34770 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d2cf45f905a57933f551a729a93634b96fa37ff2, data reload: false

------ Round 1 ----------------------------------
q1	17632	5032	4848	4848
q2	2080	315	209	209
q3	10229	1292	728	728
q4	10236	901	366	366
q5	7525	2395	2282	2282
q6	186	173	138	138
q7	931	774	626	626
q8	9355	1281	981	981
q9	7089	5303	5267	5267
q10	6798	2260	1827	1827
q11	502	311	282	282
q12	330	361	231	231
q13	17771	3667	3005	3005
q14	227	246	216	216
q15	599	525	502	502
q16	1022	1015	963	963
q17	622	861	360	360
q18	7828	7617	8017	7617
q19	1236	950	569	569
q20	377	363	241	241
q21	4359	3346	2450	2450
q22	1134	1062	1070	1062
Total cold run time: 108068 ms
Total hot run time: 34770 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5088	5206	5117	5117
q2	323	418	339	339
q3	2430	2791	2514	2514
q4	1452	1925	1448	1448
q5	4634	4483	4549	4483
q6	204	167	124	124
q7	2087	1968	1844	1844
q8	2599	2781	2679	2679
q9	7487	7377	7548	7377
q10	3037	3258	2752	2752
q11	580	515	500	500
q12	697	767	610	610
q13	3275	3595	2982	2982
q14	264	289	275	275
q15	537	489	478	478
q16	1035	1059	1023	1023
q17	1091	1471	1365	1365
q18	7557	7193	7036	7036
q19	759	791	866	791
q20	1913	1995	1814	1814
q21	4722	4273	4280	4273
q22	1087	1046	1009	1009
Total cold run time: 52858 ms
Total hot run time: 50833 ms

@doris-robot
Copy link

doris-robot commented Nov 25, 2025

TPC-DS: Total hot run time: 184078 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d2cf45f905a57933f551a729a93634b96fa37ff2, data reload: false

query1	1039	399	402	399
query2	6590	1634	1584	1584
query3	6768	224	219	219
query4	26144	22646	22244	22244
query5	4408	656	473	473
query6	335	251	224	224
query7	4670	503	309	309
query8	324	259	243	243
query9	8685	2607	2594	2594
query10	519	347	309	309
query11	15054	14910	14826	14826
query12	189	115	117	115
query13	1680	573	457	457
query14	11177	8765	8870	8765
query15	210	202	182	182
query16	7677	667	494	494
query17	1221	771	612	612
query18	2056	433	338	338
query19	219	207	191	191
query20	139	126	123	123
query21	225	136	132	132
query22	4109	4259	4190	4190
query23	33179	32082	31934	31934
query24	8518	2395	2381	2381
query25	620	540	480	480
query26	1259	295	167	167
query27	2723	499	356	356
query28	4380	2158	2142	2142
query29	815	641	512	512
query30	310	242	219	219
query31	876	718	625	625
query32	88	80	75	75
query33	645	375	328	328
query34	807	866	539	539
query35	807	840	717	717
query36	935	940	881	881
query37	129	110	86	86
query38	3338	3298	3362	3298
query39	1477	1415	1398	1398
query40	217	133	118	118
query41	63	60	62	60
query42	124	111	112	111
query43	473	478	442	442
query44	1240	747	752	747
query45	198	187	178	178
query46	880	1035	649	649
query47	1714	1737	1639	1639
query48	387	424	338	338
query49	781	517	429	429
query50	643	682	409	409
query51	3919	3992	3902	3902
query52	109	115	107	107
query53	243	259	191	191
query54	299	301	271	271
query55	93	85	86	85
query56	324	319	303	303
query57	1143	1171	1100	1100
query58	279	267	268	267
query59	2418	2470	2412	2412
query60	355	347	328	328
query61	152	161	162	161
query62	797	729	667	667
query63	226	188	183	183
query64	4499	1192	903	903
query65	4065	4048	3981	3981
query66	1093	432	329	329
query67	15547	15202	14863	14863
query68	8214	900	615	615
query69	500	350	303	303
query70	1254	1269	1177	1177
query71	454	336	310	310
query72	5837	4995	4950	4950
query73	648	609	367	367
query74	8654	8855	8340	8340
query75	3576	3334	2810	2810
query76	3429	1131	728	728
query77	547	405	298	298
query78	9529	9569	8883	8883
query79	2437	858	606	606
query80	692	588	487	487
query81	511	268	238	238
query82	326	157	132	132
query83	275	263	251	251
query84	263	107	92	92
query85	931	479	436	436
query86	379	307	290	290
query87	3432	3528	3401	3401
query88	3725	2280	2242	2242
query89	389	357	299	299
query90	1988	235	226	226
query91	169	168	144	144
query92	79	74	62	62
query93	1980	1019	655	655
query94	751	461	325	325
query95	499	409	409	409
query96	496	586	294	294
query97	2905	2954	2893	2893
query98	234	219	205	205
query99	1340	1392	1276	1276
Total cold run time: 273068 ms
Total hot run time: 184078 ms

@doris-robot
Copy link

doris-robot commented Nov 25, 2025

ClickBench: Total hot run time: 27.52 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d2cf45f905a57933f551a729a93634b96fa37ff2, data reload: false

query1	0.05	0.05	0.05
query2	0.09	0.05	0.05
query3	0.25	0.08	0.08
query4	1.61	0.11	0.11
query5	0.26	0.25	0.24
query6	1.17	0.64	0.64
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.58	0.53	0.52
query10	0.56	0.57	0.58
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.61	0.60	0.60
query14	1.00	0.99	1.00
query15	0.86	0.85	0.84
query16	0.40	0.41	0.40
query17	1.05	1.04	1.00
query18	0.21	0.20	0.19
query19	1.86	1.79	1.77
query20	0.02	0.02	0.01
query21	15.50	0.18	0.13
query22	5.15	0.07	0.05
query23	15.66	0.27	0.10
query24	2.90	1.18	0.39
query25	0.07	0.06	0.07
query26	0.14	0.14	0.13
query27	0.06	0.05	0.06
query28	5.15	1.13	0.94
query29	12.56	3.97	3.32
query30	0.28	0.14	0.12
query31	2.81	0.58	0.39
query32	3.23	0.55	0.46
query33	2.96	3.02	3.31
query34	15.80	5.08	4.59
query35	4.57	4.56	4.58
query36	0.68	0.49	0.48
query37	0.10	0.07	0.07
query38	0.06	0.05	0.05
query39	0.04	0.03	0.03
query40	0.18	0.15	0.14
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 99.04 s
Total hot run time: 27.52 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Nov 25, 2025

FE Regression Coverage Report

Increment line coverage 0.00% (0/7) 🎉
Increment coverage report
Complete coverage report

@CalvinKirs CalvinKirs merged commit 73ccbaa into apache:master Nov 25, 2025
29 of 30 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 25, 2025
@CalvinKirs
[improve](glue)Enforce HTTPS protocol for Glue endpoint (#58306)
54b9270 
FYI https://docs.aws.amazon.com/general/latest/gr/glue.html

This PR adds a validation to ensure that the Glue endpoint uses the
HTTPS protocol.

- Adds a check for Glue endpoint URLs.
- Throws an IllegalArgumentException if the endpoint does not start with
"https://".
- Error message: "glue.endpoint must use https protocol, please set
glue.endpoint to https://"


None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [x] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot mentioned this pull request Nov 25, 2025
Merged
github-actions bot pushed a commit that referenced this pull request Nov 25, 2025
@CalvinKirs
[improve](glue)Enforce HTTPS protocol for Glue endpoint (#58306)
14536a4 
FYI https://docs.aws.amazon.com/general/latest/gr/glue.html

This PR adds a validation to ensure that the Glue endpoint uses the
HTTPS protocol.

- Adds a check for Glue endpoint URLs.
- Throws an IllegalArgumentException if the endpoint does not start with
"https://".
- Error message: "glue.endpoint must use https protocol, please set
glue.endpoint to https://"


None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [x] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot mentioned this pull request Nov 25, 2025
Merged
morrySnow pushed a commit that referenced this pull request Nov 26, 2025
@github-actions @CalvinKirs
branch-3.1: [improve](glue)Enforce HTTPS protocol for Glue endpoint #…
b234bd0 
…58306 (#58366)

Cherry-picked from #58306

Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
yiguolei pushed a commit that referenced this pull request Dec 2, 2025
@github-actions @CalvinKirs
branch-4.0: [improve](glue)Enforce HTTPS protocol for Glue endpoint #…
45081ca 
…58306 (#58367)

Cherry-picked from #58306

Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
nagisa-kunhah pushed a commit to nagisa-kunhah/doris that referenced this pull request Dec 14, 2025
@CalvinKirs @nagisa-kunhah
[improve](glue)Enforce HTTPS protocol for Glue endpoint (apache#58306)
252e2f8 
FYI https://docs.aws.amazon.com/general/latest/gr/glue.html

This PR adds a validation to ensure that the Glue endpoint uses the
HTTPS protocol.

- Adds a check for Glue endpoint URLs.
- Throws an IllegalArgumentException if the endpoint does not start with
"https://".
- Error message: "glue.endpoint must use https protocol, please set
glue.endpoint to https://"


None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [x] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@zy-kkk zy-kkk zy-kkk approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.4-merged dev/4.0.2-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@CalvinKirs @hello-stephen @doris-robot @morningman @zy-kkk @yiguolei @morrySnow