Skip to content

[opt](hive) set hive.recursive_directories default to true #55737

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 2 commits into from
Sep 8, 2025
Merged

[opt](hive) set hive.recursive_directories default to true #55737

morningman merged 2 commits into from
Sep 8, 2025

Conversation

@morningman
Copy link
Contributor

@morningman morningman commented Sep 6, 2025 •
edited
Loading

What problem does this PR solve?

set hive.recursive_directories default to true, to be act same as other engine.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 6, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman
Copy link
Contributor Author

morningman commented Sep 6, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Sep 6, 2025

TPC-H: Total hot run time: 34728 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9c523fe25634f976fe3a1fbd42017f93d3f43151, data reload: false

------ Round 1 ----------------------------------
q1	17625	5244	5092	5092
q2	1974	345	212	212
q3	10251	1313	731	731
q4	10235	1048	530	530
q5	7568	2361	2402	2361
q6	187	168	137	137
q7	931	766	644	644
q8	9352	1371	1114	1114
q9	6963	5113	5129	5113
q10	6901	2416	1964	1964
q11	486	322	289	289
q12	354	370	214	214
q13	17793	3640	3048	3048
q14	248	247	218	218
q15	555	507	494	494
q16	1005	1018	954	954
q17	622	871	371	371
q18	7419	7182	7078	7078
q19	1246	961	588	588
q20	353	366	235	235
q21	3902	2625	2360	2360
q22	1096	1023	981	981
Total cold run time: 107066 ms
Total hot run time: 34728 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5189	5124	5151	5124
q2	252	331	231	231
q3	2201	2652	2320	2320
q4	1343	1805	1341	1341
q5	4237	4546	4599	4546
q6	231	174	133	133
q7	2022	2064	1810	1810
q8	2643	2706	2672	2672
q9	7535	7421	7320	7320
q10	3123	3293	2941	2941
q11	587	502	524	502
q12	731	813	629	629
q13	3581	4188	3236	3236
q14	312	320	271	271
q15	515	472	461	461
q16	1081	1143	1097	1097
q17	1228	1577	1400	1400
q18	7863	7658	7666	7658
q19	856	842	1009	842
q20	2022	2061	1930	1930
q21	5147	4571	4398	4398
q22	1066	1034	1024	1024
Total cold run time: 53765 ms
Total hot run time: 51886 ms

@doris-robot
Copy link

doris-robot commented Sep 6, 2025

TPC-DS: Total hot run time: 188996 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9c523fe25634f976fe3a1fbd42017f93d3f43151, data reload: false

query1	1072	437	401	401
query2	6567	1657	1683	1657
query3	6759	229	228	228
query4	26302	23101	23017	23017
query5	4476	671	542	542
query6	369	258	233	233
query7	4662	528	320	320
query8	320	266	285	266
query9	8681	2986	2946	2946
query10	503	364	313	313
query11	15892	14959	14784	14784
query12	182	122	130	122
query13	1689	580	447	447
query14	11320	9397	9155	9155
query15	212	193	171	171
query16	7684	648	555	555
query17	1399	769	630	630
query18	2047	422	333	333
query19	207	204	178	178
query20	136	125	120	120
query21	219	133	120	120
query22	4034	4179	3982	3982
query23	34006	32794	32845	32794
query24	8208	2440	2430	2430
query25	564	510	449	449
query26	1240	283	172	172
query27	2705	537	360	360
query28	4376	2287	2256	2256
query29	808	611	494	494
query30	292	228	213	213
query31	937	808	727	727
query32	94	82	84	82
query33	580	401	349	349
query34	814	851	539	539
query35	812	838	767	767
query36	971	1019	917	917
query37	138	117	102	102
query38	3512	3547	3483	3483
query39	1515	1415	1418	1415
query40	226	143	129	129
query41	66	63	62	62
query42	129	121	123	121
query43	505	514	461	461
query44	1369	882	870	870
query45	189	184	173	173
query46	871	1022	651	651
query47	1762	1834	1769	1769
query48	385	432	329	329
query49	749	511	410	410
query50	680	687	404	404
query51	3901	3880	3803	3803
query52	126	122	112	112
query53	255	273	205	205
query54	632	634	560	560
query55	98	92	96	92
query56	383	352	335	335
query57	1196	1215	1125	1125
query58	314	290	301	290
query59	2748	2668	2547	2547
query60	426	363	344	344
query61	173	163	163	163
query62	827	709	668	668
query63	240	193	201	193
query64	4473	1195	839	839
query65	4012	3956	3958	3956
query66	1101	436	359	359
query67	15678	15356	15177	15177
query68	9035	936	594	594
query69	492	338	300	300
query70	1422	1332	1404	1332
query71	592	360	326	326
query72	6005	5098	5004	5004
query73	740	627	369	369
query74	8947	9086	8911	8911
query75	4220	3257	2814	2814
query76	3789	1177	774	774
query77	799	412	345	345
query78	9712	9685	8789	8789
query79	2034	859	637	637
query80	663	600	554	554
query81	489	263	229	229
query82	447	177	146	146
query83	271	267	255	255
query84	257	114	96	96
query85	885	477	433	433
query86	347	345	313	313
query87	3735	3780	3596	3596
query88	3450	2255	2259	2255
query89	405	336	294	294
query90	1961	232	226	226
query91	162	172	135	135
query92	95	81	74	74
query93	1366	1019	670	670
query94	705	428	329	329
query95	411	347	335	335
query96	499	584	280	280
query97	2962	2994	2840	2840
query98	255	222	219	219
query99	1669	1393	1306	1306
Total cold run time: 278639 ms
Total hot run time: 188996 ms

@doris-robot
Copy link

doris-robot commented Sep 6, 2025

ClickBench: Total hot run time: 30.41 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9c523fe25634f976fe3a1fbd42017f93d3f43151, data reload: false

query1	0.06	0.04	0.04
query2	0.09	0.05	0.05
query3	0.25	0.08	0.09
query4	1.61	0.11	0.11
query5	0.27	0.26	0.26
query6	1.17	0.66	0.65
query7	0.03	0.03	0.03
query8	0.05	0.04	0.04
query9	0.61	0.53	0.52
query10	0.60	0.57	0.58
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.63	0.63	0.62
query14	1.02	1.02	1.05
query15	0.87	0.84	0.87
query16	0.41	0.40	0.40
query17	1.04	1.07	1.05
query18	0.22	0.20	0.20
query19	1.92	1.88	1.83
query20	0.02	0.01	0.02
query21	15.44	0.94	0.58
query22	0.78	1.27	0.82
query23	14.79	1.40	0.61
query24	6.75	1.18	1.34
query25	0.52	0.13	0.12
query26	0.55	0.18	0.13
query27	0.06	0.06	0.05
query28	10.50	0.90	0.42
query29	12.62	3.97	3.27
query30	0.29	0.12	0.11
query31	2.83	0.62	0.38
query32	3.24	0.57	0.48
query33	3.08	3.06	3.11
query34	15.84	5.51	4.88
query35	4.88	4.91	4.91
query36	0.71	0.52	0.50
query37	0.10	0.08	0.07
query38	0.06	0.04	0.05
query39	0.04	0.02	0.03
query40	0.19	0.15	0.14
query41	0.09	0.03	0.02
query42	0.04	0.03	0.03
query43	0.05	0.04	0.04
Total cold run time: 104.63 s
Total hot run time: 30.41 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 6, 2025

FE UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 6, 2025

FE Regression Coverage Report

Increment line coverage 100.00% (1/1) 🎉
Increment coverage report
Complete coverage report

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 8, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

PR approved by anyone and no changes requested.

@morningman morningman merged commit 28c537c into apache:master Sep 8, 2025
34 of 35 checks passed
morningman added a commit to morningman/doris that referenced this pull request Sep 11, 2025
@morningman
[opt](hive) set hive.recursive_directories default to true (apache#55737
9198ae8 
)

set `hive.recursive_directories` default to true, to be act same as
other engine.
@morningman morningman mentioned this pull request Sep 11, 2025
Merged
morningman added a commit that referenced this pull request Sep 11, 2025
@morningman
branch-3.1: [opt](hive) set hive.recursive_directories default to true (
311a010 
#55737) (#55905)

bp #55737
@morrySnow morrySnow mentioned this pull request Sep 22, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CalvinKirs CalvinKirs CalvinKirs approved these changes

@zy-kkk zy-kkk zy-kkk approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.1-merged kind/behavior-changed reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@morningman @hello-stephen @doris-robot @CalvinKirs @zy-kkk