Skip to content

[opt](outfile) change suffix of compressed file #55803

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
zy-kkk merged 1 commit into from
Sep 11, 2025
Merged

[opt](outfile) change suffix of compressed file #55803

zy-kkk merged 1 commit into from
Sep 11, 2025

Conversation

@morningman
Copy link
Contributor

@morningman morningman commented Sep 9, 2025 •
edited
Loading

What problem does this PR solve?

Followup #55392
Change gzip to gz, bzip2 to bz2.
So that tools like gunzip can recognize it directly

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Sep 9, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman
Copy link
Contributor Author

morningman commented Sep 9, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Sep 9, 2025

TPC-H: Total hot run time: 35087 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dd03c8db4b97f0747f21cc990447c3177aa25fc5, data reload: false

------ Round 1 ----------------------------------
q1	17601	5225	5140	5140
q2	1993	332	235	235
q3	10235	1338	745	745
q4	10235	1007	546	546
q5	7515	2478	2336	2336
q6	191	180	140	140
q7	989	757	652	652
q8	9370	1381	1130	1130
q9	7013	5169	5174	5169
q10	6894	2408	1975	1975
q11	474	298	284	284
q12	359	372	232	232
q13	17787	3753	3065	3065
q14	245	256	226	226
q15	556	502	492	492
q16	1015	1000	991	991
q17	605	868	367	367
q18	7596	7202	7186	7186
q19	1225	947	568	568
q20	351	338	234	234
q21	3856	3182	2413	2413
q22	1064	1034	961	961
Total cold run time: 107169 ms
Total hot run time: 35087 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5171	5158	5190	5158
q2	250	329	225	225
q3	2179	2716	2310	2310
q4	1344	1802	1325	1325
q5	4279	4652	4596	4596
q6	222	177	138	138
q7	2023	2013	1806	1806
q8	2718	2676	2596	2596
q9	7317	7212	7215	7212
q10	3148	3259	2960	2960
q11	580	527	530	527
q12	713	771	664	664
q13	3600	3959	3369	3369
q14	313	329	289	289
q15	565	518	503	503
q16	1087	1153	1056	1056
q17	1233	1645	1357	1357
q18	7910	7649	7622	7622
q19	851	830	868	830
q20	2023	2082	1976	1976
q21	5103	4456	4337	4337
q22	1060	1019	995	995
Total cold run time: 53689 ms
Total hot run time: 51851 ms

@doris-robot
Copy link

doris-robot commented Sep 9, 2025

TPC-DS: Total hot run time: 189369 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dd03c8db4b97f0747f21cc990447c3177aa25fc5, data reload: false

query1	1096	450	413	413
query2	6594	1729	1705	1705
query3	6763	230	235	230
query4	26602	23301	23044	23044
query5	4398	680	537	537
query6	343	243	241	241
query7	4655	516	299	299
query8	310	246	249	246
query9	8606	2923	2909	2909
query10	482	366	290	290
query11	15781	15059	15172	15059
query12	175	127	115	115
query13	1677	555	429	429
query14	10932	9303	9157	9157
query15	212	196	179	179
query16	7378	660	518	518
query17	1241	743	615	615
query18	2016	428	341	341
query19	210	196	175	175
query20	135	134	127	127
query21	205	133	123	123
query22	4088	4155	4146	4146
query23	33905	32843	33114	32843
query24	8139	2424	2432	2424
query25	557	516	446	446
query26	1236	282	175	175
query27	2712	523	345	345
query28	4385	2271	2266	2266
query29	830	618	498	498
query30	301	228	196	196
query31	904	837	719	719
query32	93	84	78	78
query33	614	405	362	362
query34	810	871	521	521
query35	792	830	750	750
query36	986	1044	929	929
query37	132	136	92	92
query38	3493	3521	3526	3521
query39	1479	1426	1426	1426
query40	231	137	124	124
query41	70	64	62	62
query42	133	124	116	116
query43	508	523	484	484
query44	1331	887	869	869
query45	190	177	182	177
query46	862	1015	649	649
query47	1800	1805	1769	1769
query48	385	443	319	319
query49	770	519	412	412
query50	655	697	416	416
query51	3976	3915	3878	3878
query52	124	115	109	109
query53	256	284	199	199
query54	622	607	556	556
query55	101	96	91	91
query56	343	344	325	325
query57	1213	1205	1126	1126
query58	292	283	276	276
query59	2572	2631	2619	2619
query60	365	377	363	363
query61	208	192	187	187
query62	831	737	671	671
query63	237	199	208	199
query64	4539	1162	858	858
query65	4034	3962	4017	3962
query66	1146	447	371	371
query67	15563	15394	15088	15088
query68	7274	936	581	581
query69	501	335	295	295
query70	1384	1272	1252	1252
query71	544	357	325	325
query72	5970	5041	5101	5041
query73	686	632	356	356
query74	9299	8857	8740	8740
query75	3286	3224	2806	2806
query76	3181	1249	750	750
query77	501	411	344	344
query78	9387	9649	8851	8851
query79	2080	851	590	590
query80	681	582	527	527
query81	512	269	225	225
query82	220	178	160	160
query83	273	269	266	266
query84	262	119	92	92
query85	865	467	484	467
query86	386	315	314	314
query87	3760	3774	3666	3666
query88	2952	2204	2223	2204
query89	400	337	316	316
query90	1969	225	222	222
query91	161	171	135	135
query92	96	77	75	75
query93	2102	984	644	644
query94	702	404	340	340
query95	422	343	343	343
query96	481	582	282	282
query97	2899	3000	2879	2879
query98	251	219	222	219
query99	1337	1387	1298	1298
Total cold run time: 273448 ms
Total hot run time: 189369 ms

@doris-robot
Copy link

doris-robot commented Sep 9, 2025

ClickBench: Total hot run time: 30.5 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dd03c8db4b97f0747f21cc990447c3177aa25fc5, data reload: false

query1	0.05	0.05	0.04
query2	0.09	0.06	0.06
query3	0.25	0.08	0.08
query4	1.61	0.12	0.12
query5	0.28	0.27	0.25
query6	1.19	0.65	0.66
query7	0.03	0.03	0.03
query8	0.06	0.05	0.04
query9	0.62	0.53	0.53
query10	0.60	0.58	0.58
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.63	0.62	0.62
query14	1.04	1.04	1.07
query15	0.89	0.85	0.86
query16	0.40	0.40	0.41
query17	1.04	1.10	1.09
query18	0.21	0.20	0.22
query19	1.99	1.89	1.84
query20	0.01	0.01	0.02
query21	15.43	0.97	0.60
query22	0.75	1.18	0.85
query23	14.79	1.37	0.64
query24	7.08	1.04	1.29
query25	0.47	0.22	0.16
query26	0.62	0.17	0.14
query27	0.07	0.06	0.05
query28	10.20	0.92	0.43
query29	12.60	3.97	3.30
query30	0.28	0.14	0.11
query31	2.83	0.60	0.38
query32	3.24	0.56	0.48
query33	3.07	3.05	3.13
query34	15.91	5.52	4.85
query35	4.93	4.88	4.92
query36	0.72	0.52	0.51
query37	0.11	0.07	0.07
query38	0.07	0.05	0.04
query39	0.04	0.04	0.03
query40	0.18	0.14	0.16
query41	0.08	0.03	0.03
query42	0.04	0.04	0.03
query43	0.04	0.03	0.04
Total cold run time: 104.85 s
Total hot run time: 30.5 s

@morningman
Copy link
Contributor Author

morningman commented Sep 11, 2025

run check_coverage

@github-actions
Copy link
Contributor

github-actions bot commented Sep 11, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Sep 11, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 11, 2025

PR approved by anyone and no changes requested.

@zy-kkk zy-kkk merged commit d1e858d into apache:master Sep 11, 2025
32 of 35 checks passed
github-actions bot pushed a commit that referenced this pull request Sep 11, 2025
@morningman
[opt](outfile) change suffix of compressed file (#55803)
923f702 
Followup #55392
Change gzip to gz, bzip2 to bz2.
So that tools like `gunzip` can recognize it directly
@github-actions github-actions bot mentioned this pull request Sep 11, 2025
Merged
morrySnow pushed a commit that referenced this pull request Sep 12, 2025
@github-actions @morningman
branch-3.1: [opt](outfile) change suffix of compressed file #55803 (#…
b209e00 
…55908)

Cherry-picked from #55803

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CalvinKirs CalvinKirs CalvinKirs approved these changes

@zy-kkk zy-kkk zy-kkk approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.1-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@morningman @Thearas @doris-robot @CalvinKirs @zy-kkk @morrySnow