[Feature](Export) support native format #58711

eldenmoon · 2025-12-04T07:48:32Z

support export doris native format file

// Doris Native format file-level constants.
//
// File layout (byte stream):
//
//   +-------------------------------+---------------------------+---------------------------+ ...
//   | File header                   | Data block #0             | Data block #1             | ...
//   +-------------------------------+---------------------------+---------------------------+ ...
//
//   File header (12 bytes total):
//     - [0..7]   : magic bytes "DORISN1\0"  (DORIS_NATIVE_MAGIC)
//     - [8..11]  : uint32_t format_version (DORIS_NATIVE_FORMAT_VERSION, little-endian)
//
//   Each data block i:
//     - uint64_t block_size   : length in bytes of serialized PBlock (little-endian)
//     - uint8_t[block_size]   : PBlock protobuf payload produced by Block::serialize()
//
// NativeReader:
//   - Detects the optional file header by checking the first 8 bytes against DORIS_NATIVE_MAGIC.
//   - If the header is present, it skips 12 bytes and then starts reading blocks as
//     [uint64_t block_size][PBlock bytes]...
//   - If the header is absent (legacy files), it starts reading blocks from offset 0.
//
// VNativeTransformer:
//   - Writes the header once in open(), then appends each block in write() as
//     [uint64_t block_size][PBlock bytes]...
//
// These constants are shared between writer, reader and tests to keep the on-disk
// format definition in a single place.
// Header layout:
// [magic bytes "DORISN1\0"][uint32_t format_version]

docs: apache/doris-website#3190

Thearas · 2025-12-04T07:48:42Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

eldenmoon · 2025-12-04T07:49:07Z

run buildall

eldenmoon · 2025-12-04T07:53:21Z

run buildall

eldenmoon · 2025-12-04T07:56:43Z

run buildall

hello-stephen · 2025-12-04T08:23:45Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.65% (1707/2143)
Line Coverage	66.39% (29918/45063)
Region Coverage	66.97% (14904/22256)
Branch Coverage	57.34% (7930/13830)

hello-stephen · 2025-12-04T09:04:26Z

FE UT Coverage Report

Increment line coverage 0.00% (0/19) 🎉
Increment coverage report
Complete coverage report

doris-robot · 2025-12-04T10:09:58Z

TPC-H: Total hot run time: 34522 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9877d06ac993d2608a635d114c3712ac4768c67b, data reload: false

------ Round 1 ----------------------------------
q1	17428	5091	4937	4937
q2	2064	301	192	192
q3	10181	1316	750	750
q4	10205	787	309	309
q5	7542	2426	2171	2171
q6	198	175	141	141
q7	972	820	647	647
q8	9349	1436	1110	1110
q9	7112	5337	5418	5337
q10	6836	2191	1784	1784
q11	503	324	284	284
q12	344	367	230	230
q13	17776	3680	3048	3048
q14	234	237	220	220
q15	602	519	515	515
q16	885	871	815	815
q17	683	828	494	494
q18	7519	7194	7116	7116
q19	917	954	623	623
q20	378	348	226	226
q21	4065	3503	2615	2615
q22	1029	1007	958	958
Total cold run time: 106822 ms
Total hot run time: 34522 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4978	4975	4963	4963
q2	315	405	335	335
q3	2192	2663	2333	2333
q4	1379	1837	1290	1290
q5	4250	4556	4494	4494
q6	226	177	140	140
q7	2108	2033	1854	1854
q8	2719	2494	2568	2494
q9	7499	7595	7587	7587
q10	3084	3235	2859	2859
q11	592	539	510	510
q12	683	784	632	632
q13	3581	3955	3310	3310
q14	313	392	268	268
q15	554	519	517	517
q16	911	958	869	869
q17	1198	1521	1566	1521
q18	7931	7794	7596	7596
q19	865	833	898	833
q20	1971	1992	1796	1796
q21	4658	4333	4213	4213
q22	1132	1068	985	985
Total cold run time: 53139 ms
Total hot run time: 51399 ms

doris-robot · 2025-12-04T10:21:19Z

TPC-DS: Total hot run time: 180211 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9877d06ac993d2608a635d114c3712ac4768c67b, data reload: false

query5	4441	663	505	505
query6	354	244	219	219
query7	4671	494	270	270
query8	322	254	251	251
query9	8713	2633	2601	2601
query10	534	312	281	281
query11	15451	15039	14564	14564
query12	183	119	118	118
query13	1697	500	383	383
query14	6130	3304	3172	3172
query14_1	2948	2922	2940	2922
query15	218	208	182	182
query16	7514	483	467	467
query17	1268	727	610	610
query18	2021	468	342	342
query19	215	199	165	165
query20	132	120	122	120
query21	220	141	122	122
query22	3906	3948	3874	3874
query23	16704	16264	16069	16069
query23_1	16145	16249	16125	16125
query24	7164	1627	1209	1209
query24_1	1226	1233	1275	1233
query25	609	477	422	422
query26	1260	277	176	176
query27	2911	481	306	306
query28	4371	2129	2135	2129
query29	825	575	447	447
query30	312	241	213	213
query31	821	692	621	621
query32	88	70	73	70
query33	685	351	298	298
query34	879	892	529	529
query35	808	842	736	736
query36	889	905	845	845
query37	124	92	82	82
query38	3904	3813	3755	3755
query39	781	729	720	720
query39_1	713	706	706	706
query40	223	128	120	120
query41	66	65	63	63
query42	124	101	94	94
query43	444	417	389	389
query44	1357	754	759	754
query45	206	191	185	185
query46	912	1013	589	589
query47	1699	1720	1660	1660
query48	401	320	239	239
query49	764	428	353	353
query50	687	297	257	257
query51	3899	3899	3815	3815
query52	116	100	88	88
query53	251	235	182	182
query54	334	264	241	241
query55	97	81	79	79
query56	343	296	288	288
query57	1166	1157	1113	1113
query58	302	266	255	255
query59	2370	2399	2410	2399
query60	363	326	301	301
query61	160	162	162	162
query62	792	677	650	650
query63	228	177	190	177
query64	4573	1181	915	915
query65	4039	3961	3999	3961
query66	1248	444	339	339
query67	15383	15017	14820	14820
query68	6675	940	660	660
query69	531	305	262	262
query70	1110	975	1042	975
query71	436	299	288	288
query72	5853	5114	4869	4869
query73	685	540	302	302
query74	8588	8753	8674	8674
query75	3045	3060	2546	2546
query76	3352	1172	748	748
query77	516	413	319	319
query78	9559	9739	8873	8873
query79	1638	906	579	579
query80	1357	562	473	473
query81	575	269	241	241
query82	401	131	106	106
query83	276	273	250	250
query84	256	125	93	93
query85	934	491	447	447
query86	414	309	278	278
query87	4032	4058	3961	3961
query88	3243	2161	2154	2154
query89	401	319	278	278
query90	2085	184	149	149
query91	175	163	145	145
query92	83	70	68	68
query93	2340	1020	680	680
query94	847	302	252	252
query95	573	381	340	340
query96	550	508	219	219
query97	2609	2682	2558	2558
query98	242	206	198	198
query99	1351	1315	1232	1232
Total cold run time: 269708 ms
Total hot run time: 180211 ms

doris-robot · 2025-12-04T10:26:30Z

ClickBench: Total hot run time: 28.55 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9877d06ac993d2608a635d114c3712ac4768c67b, data reload: false

query1	0.06	0.05	0.05
query2	0.14	0.07	0.07
query3	0.32	0.08	0.08
query4	1.61	0.10	0.10
query5	0.26	0.25	0.25
query6	1.19	0.63	0.65
query7	0.03	0.02	0.03
query8	0.08	0.07	0.07
query9	0.60	0.52	0.50
query10	0.58	0.57	0.57
query11	0.26	0.17	0.14
query12	0.26	0.15	0.14
query13	0.64	0.63	0.62
query14	1.01	1.02	1.00
query15	0.88	0.81	0.83
query16	0.39	0.42	0.37
query17	0.99	1.06	1.03
query18	0.24	0.22	0.21
query19	1.83	1.78	1.85
query20	0.02	0.01	0.02
query21	15.40	0.27	0.24
query22	4.94	0.10	0.09
query23	15.41	0.39	0.23
query24	2.38	0.48	0.30
query25	0.10	0.09	0.10
query26	0.20	0.18	0.18
query27	0.09	0.09	0.09
query28	3.75	1.37	1.18
query29	12.58	4.11	3.29
query30	0.34	0.13	0.11
query31	2.81	0.67	0.43
query32	3.24	0.62	0.49
query33	3.14	3.06	3.24
query34	16.65	5.31	4.59
query35	4.72	4.71	4.70
query36	0.62	0.49	0.49
query37	0.25	0.09	0.08
query38	0.20	0.06	0.06
query39	0.08	0.05	0.05
query40	0.20	0.18	0.17
query41	0.14	0.07	0.08
query42	0.08	0.05	0.05
query43	0.07	0.06	0.06
Total cold run time: 98.78 s
Total hot run time: 28.55 s

hello-stephen · 2025-12-04T12:48:44Z

FE Regression Coverage Report

Increment line coverage 94.74% (18/19) 🎉
Increment coverage report
Complete coverage report

eldenmoon · 2025-12-05T08:16:09Z

run buildall

hello-stephen · 2025-12-05T08:46:05Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.65% (1707/2143)
Line Coverage	66.40% (29927/45068)
Region Coverage	66.97% (14905/22257)
Branch Coverage	57.32% (7927/13830)

doris-robot · 2025-12-05T10:07:35Z

TPC-H: Total hot run time: 36401 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0213622a1ee5e722e8ed30c5d16c111edf25f525, data reload: false

------ Round 1 ----------------------------------
q1	17648	5061	4931	4931
q2	2058	327	196	196
q3	10198	1340	740	740
q4	10213	850	309	309
q5	7523	2458	2170	2170
q6	191	173	136	136
q7	958	786	633	633
q8	9380	1480	1134	1134
q9	7466	5422	5410	5410
q10	6932	2230	1782	1782
q11	541	330	305	305
q12	409	387	234	234
q13	17820	3760	3034	3034
q14	232	239	215	215
q15	598	524	506	506
q16	924	887	833	833
q17	729	848	505	505
q18	7860	7627	7867	7627
q19	1268	1014	646	646
q20	393	374	239	239
q21	4395	3938	3822	3822
q22	1085	1085	994	994
Total cold run time: 108821 ms
Total hot run time: 36401 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5260	5304	5280	5280
q2	323	428	314	314
q3	2381	3031	2466	2466
q4	1453	1909	1409	1409
q5	4629	4473	4519	4473
q6	213	182	125	125
q7	2026	2026	1735	1735
q8	2761	2502	2518	2502
q9	7572	7499	7617	7499
q10	3058	3244	2872	2872
q11	614	531	503	503
q12	677	711	576	576
q13	3329	3689	3070	3070
q14	273	266	256	256
q15	538	497	484	484
q16	867	893	898	893
q17	1131	1336	1405	1336
q18	7253	7070	7198	7070
q19	835	832	824	824
q20	1884	1963	1833	1833
q21	4601	4249	4202	4202
q22	1084	1010	976	976
Total cold run time: 52762 ms
Total hot run time: 50698 ms

doris-robot · 2025-12-05T10:19:01Z

TPC-DS: Total hot run time: 180365 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0213622a1ee5e722e8ed30c5d16c111edf25f525, data reload: false

query5	4881	662	506	506
query6	351	226	211	211
query7	4662	482	281	281
query8	295	248	235	235
query9	8730	2638	2615	2615
query10	539	354	271	271
query11	15274	14851	14852	14851
query12	177	122	117	117
query13	1677	466	372	372
query14	5997	3267	3097	3097
query14_1	3039	2896	2897	2896
query15	206	201	176	176
query16	7290	509	453	453
query17	1185	698	589	589
query18	2003	427	350	350
query19	217	200	175	175
query20	137	120	121	120
query21	226	140	119	119
query22	4038	4027	3871	3871
query23	16669	16269	15750	15750
query23_1	16144	16109	16327	16109
query24	7231	1627	1219	1219
query24_1	1227	1250	1211	1211
query25	616	480	452	452
query26	1268	280	174	174
query27	2921	464	314	314
query28	4434	2195	2174	2174
query29	828	563	442	442
query30	318	247	217	217
query31	849	694	634	634
query32	85	72	74	72
query33	717	350	299	299
query34	859	888	561	561
query35	803	822	736	736
query36	899	916	834	834
query37	126	92	77	77
query38	3896	3878	3788	3788
query39	755	723	720	720
query39_1	702	708	692	692
query40	237	134	124	124
query41	68	64	63	63
query42	127	100	100	100
query43	440	434	406	406
query44	1320	764	775	764
query45	193	196	186	186
query46	923	967	607	607
query47	1690	1713	1623	1623
query48	409	323	237	237
query49	800	442	387	387
query50	683	299	252	252
query51	3818	3823	3913	3823
query52	115	95	88	88
query53	240	234	178	178
query54	316	255	250	250
query55	95	78	77	77
query56	355	305	296	296
query57	1172	1153	1093	1093
query58	305	269	255	255
query59	2241	2398	2277	2277
query60	370	328	299	299
query61	173	160	189	160
query62	805	706	631	631
query63	242	186	190	186
query64	4700	1318	1033	1033
query65	4084	3996	4015	3996
query66	1319	453	346	346
query67	15464	14910	14746	14746
query68	5516	972	673	673
query69	536	305	270	270
query70	1119	1013	1003	1003
query71	442	305	276	276
query72	5997	4905	4869	4869
query73	654	547	303	303
query74	8810	8817	8583	8583
query75	3046	3050	2557	2557
query76	3257	1146	745	745
query77	499	411	315	315
query78	9608	9635	8851	8851
query79	1038	868	596	596
query80	686	549	476	476
query81	506	272	238	238
query82	423	136	103	103
query83	282	283	265	265
query84	266	110	99	99
query85	895	480	459	459
query86	348	308	271	271
query87	4083	4168	4112	4112
query88	2999	2139	2140	2139
query89	395	331	284	284
query90	1894	164	166	164
query91	192	190	162	162
query92	73	81	68	68
query93	1189	1047	695	695
query94	728	334	321	321
query95	588	412	355	355
query96	543	483	213	213
query97	2626	2659	2583	2583
query98	249	212	206	206
query99	1338	1328	1239	1239
Total cold run time: 265686 ms
Total hot run time: 180365 ms

doris-robot · 2025-12-05T10:24:09Z

ClickBench: Total hot run time: 27.97 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0213622a1ee5e722e8ed30c5d16c111edf25f525, data reload: false

query1	0.05	0.04	0.04
query2	0.10	0.04	0.05
query3	0.25	0.09	0.09
query4	1.60	0.12	0.11
query5	0.28	0.28	0.26
query6	1.14	0.65	0.64
query7	0.03	0.02	0.02
query8	0.06	0.04	0.05
query9	0.57	0.51	0.52
query10	0.56	0.55	0.55
query11	0.16	0.10	0.12
query12	0.15	0.12	0.12
query13	0.62	0.61	0.60
query14	0.99	0.99	1.00
query15	0.82	0.81	0.81
query16	0.40	0.40	0.39
query17	1.06	1.07	1.04
query18	0.24	0.22	0.22
query19	1.94	1.84	1.87
query20	0.02	0.01	0.02
query21	15.45	0.27	0.15
query22	4.57	0.06	0.05
query23	15.90	0.29	0.10
query24	1.02	0.83	1.80
query25	0.08	0.09	0.06
query26	0.14	0.13	0.13
query27	0.06	0.05	0.04
query28	6.17	1.21	1.03
query29	12.60	4.06	3.32
query30	0.28	0.15	0.12
query31	2.83	0.64	0.40
query32	3.23	0.56	0.46
query33	3.03	3.02	3.08
query34	16.98	5.21	4.45
query35	4.50	4.53	4.60
query36	0.66	0.51	0.50
query37	0.10	0.07	0.07
query38	0.08	0.04	0.04
query39	0.05	0.03	0.03
query40	0.17	0.14	0.13
query41	0.09	0.04	0.03
query42	0.05	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 99.13 s
Total hot run time: 27.97 s

hello-stephen · 2025-12-05T11:51:21Z

BE UT Coverage Report

Increment line coverage 46.68% (183/392) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	53.31% (18659/35001)
Line Coverage	39.02% (172566/442261)
Region Coverage	33.66% (133715/397241)
Branch Coverage	34.60% (57504/166198)

hello-stephen · 2025-12-05T12:55:54Z

BE Regression && UT Coverage Report

Increment line coverage 75.00% (288/384) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	72.18% (24773/34321)
Line Coverage	58.85% (260062/441887)
Region Coverage	53.60% (215513/402053)
Branch Coverage	55.17% (92187/167097)

hello-stephen · 2025-12-05T13:14:23Z

FE Regression Coverage Report

Increment line coverage 94.74% (18/19) 🎉
Increment coverage report
Complete coverage report

eldenmoon · 2025-12-08T14:51:29Z

run buildall

hello-stephen · 2025-12-08T15:27:49Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.81% (1727/2164)
Line Coverage	65.93% (30567/46366)
Region Coverage	66.62% (15251/22893)
Branch Coverage	56.91% (8111/14252)

doris-robot · 2025-12-08T16:58:35Z

TPC-H: Total hot run time: 34267 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cd1d812f869686fa13ffc2e9869b53d7d04a24be, data reload: false

------ Round 1 ----------------------------------
q1	17681	5023	4866	4866
q2	2015	316	204	204
q3	10216	1292	735	735
q4	10223	855	308	308
q5	7557	2461	2139	2139
q6	191	178	140	140
q7	954	801	643	643
q8	9351	1457	1194	1194
q9	7405	5340	5298	5298
q10	6873	2224	1766	1766
q11	532	319	286	286
q12	385	361	216	216
q13	17799	3652	3059	3059
q14	234	238	211	211
q15	596	516	512	512
q16	900	866	807	807
q17	669	787	505	505
q18	8021	7079	7104	7079
q19	1096	972	610	610
q20	387	353	217	217
q21	3977	3617	2492	2492
q22	1060	988	980	980
Total cold run time: 108122 ms
Total hot run time: 34267 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4896	4969	4924	4924
q2	332	398	315	315
q3	2126	2637	2289	2289
q4	1355	1745	1308	1308
q5	4243	4932	4539	4539
q6	221	180	128	128
q7	2043	2039	1790	1790
q8	2716	2469	2447	2447
q9	7632	7587	7504	7504
q10	3078	3310	2835	2835
q11	649	570	495	495
q12	700	750	579	579
q13	3559	3961	3321	3321
q14	284	320	329	320
q15	544	518	508	508
q16	905	959	879	879
q17	1196	1502	1482	1482
q18	7788	7843	7485	7485
q19	853	831	855	831
q20	2003	1985	1870	1870
q21	4599	4209	4141	4141
q22	1073	1006	989	989
Total cold run time: 52795 ms
Total hot run time: 50979 ms

doris-robot · 2025-12-08T17:09:54Z

TPC-DS: Total hot run time: 179800 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cd1d812f869686fa13ffc2e9869b53d7d04a24be, data reload: false

query5	4965	645	490	490
query6	342	264	222	222
query7	4659	487	286	286
query8	320	257	245	245
query9	8756	2624	2589	2589
query10	541	314	277	277
query11	15419	14760	14510	14510
query12	176	122	117	117
query13	1691	488	379	379
query14	6272	3306	3025	3025
query14_1	2915	2857	2872	2857
query15	220	199	178	178
query16	7699	480	469	469
query17	1222	720	617	617
query18	2055	432	344	344
query19	214	196	171	171
query20	132	124	120	120
query21	226	141	117	117
query22	4064	4137	4002	4002
query23	16720	16105	15940	15940
query23_1	15916	16052	15980	15980
query24	7219	1637	1209	1209
query24_1	1248	1216	1225	1216
query25	655	503	449	449
query26	1258	276	183	183
query27	2888	498	311	311
query28	4350	2171	2150	2150
query29	805	602	478	478
query30	305	250	214	214
query31	844	717	640	640
query32	85	72	75	72
query33	671	359	312	312
query34	858	902	544	544
query35	791	826	752	752
query36	885	955	827	827
query37	128	90	76	76
query38	3814	3781	3780	3780
query39	762	725	746	725
query39_1	697	698	704	698
query40	232	133	114	114
query41	65	67	63	63
query42	128	97	96	96
query43	447	416	399	399
query44	1362	750	754	750
query45	195	194	185	185
query46	909	966	583	583
query47	1768	1734	1657	1657
query48	413	330	233	233
query49	781	448	359	359
query50	709	304	231	231
query51	3857	3892	3827	3827
query52	117	95	92	92
query53	229	231	176	176
query54	318	256	259	256
query55	96	84	75	75
query56	337	278	290	278
query57	1135	1152	1102	1102
query58	291	264	256	256
query59	2306	2445	2375	2375
query60	356	324	297	297
query61	201	172	169	169
query62	798	660	633	633
query63	232	178	176	176
query64	4484	1186	897	897
query65	4040	3932	3913	3913
query66	1205	445	343	343
query67	15250	15056	14878	14878
query68	8267	926	683	683
query69	519	292	268	268
query70	1100	951	990	951
query71	466	292	270	270
query72	5975	4883	4911	4883
query73	679	547	298	298
query74	8800	8737	8597	8597
query75	3562	3026	2523	2523
query76	3602	1139	765	765
query77	820	446	306	306
query78	9548	9733	8787	8787
query79	1641	857	601	601
query80	705	542	476	476
query81	511	272	238	238
query82	217	126	105	105
query83	291	273	259	259
query84	261	122	99	99
query85	899	498	460	460
query86	380	327	292	292
query87	4074	4105	4015	4015
query88	3003	2170	2154	2154
query89	395	330	289	289
query90	1960	170	170	170
query91	174	163	143	143
query92	81	73	66	66
query93	1206	1061	684	684
query94	689	318	281	281
query95	568	393	324	324
query96	556	481	216	216
query97	2606	2652	2546	2546
query98	252	204	197	197
query99	1365	1317	1184	1184
Total cold run time: 269881 ms
Total hot run time: 179800 ms

doris-robot · 2025-12-08T17:15:06Z

ClickBench: Total hot run time: 28.44 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cd1d812f869686fa13ffc2e9869b53d7d04a24be, data reload: false

query1	0.05	0.05	0.04
query2	0.14	0.07	0.06
query3	0.31	0.08	0.07
query4	1.61	0.09	0.10
query5	0.26	0.25	0.25
query6	1.19	0.64	0.63
query7	0.04	0.03	0.02
query8	0.07	0.06	0.06
query9	0.58	0.51	0.50
query10	0.56	0.55	0.55
query11	0.26	0.14	0.13
query12	0.26	0.15	0.13
query13	0.63	0.63	0.59
query14	1.00	1.03	1.00
query15	0.89	0.82	0.82
query16	0.39	0.39	0.40
query17	1.05	1.05	1.04
query18	0.24	0.22	0.21
query19	1.92	1.86	1.85
query20	0.02	0.02	0.01
query21	15.41	0.28	0.23
query22	4.98	0.11	0.10
query23	15.41	0.39	0.22
query24	2.43	0.46	0.30
query25	0.09	0.09	0.09
query26	0.19	0.17	0.18
query27	0.09	0.08	0.09
query28	3.79	1.36	1.16
query29	12.59	4.14	3.32
query30	0.32	0.13	0.10
query31	2.85	0.66	0.42
query32	3.24	0.60	0.49
query33	3.10	3.10	3.09
query34	16.83	5.22	4.62
query35	4.60	4.66	4.72
query36	0.61	0.50	0.48
query37	0.25	0.08	0.08
query38	0.20	0.06	0.05
query39	0.07	0.05	0.04
query40	0.19	0.17	0.15
query41	0.12	0.05	0.06
query42	0.07	0.06	0.05
query43	0.06	0.06	0.05
Total cold run time: 98.96 s
Total hot run time: 28.44 s

hello-stephen · 2025-12-08T18:26:48Z

BE UT Coverage Report

Increment line coverage 50.97% (236/463) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	53.38% (18731/35091)
Line Coverage	39.09% (173193/443022)
Region Coverage	33.72% (134266/398180)
Branch Coverage	34.64% (57684/166518)

hello-stephen · 2025-12-08T23:06:43Z

BE Regression && UT Coverage Report

Increment line coverage 76.92% (350/455) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	72.24% (24847/34393)
Line Coverage	59.00% (261077/442538)
Region Coverage	53.94% (217329/402903)
Branch Coverage	55.45% (92817/167393)

hello-stephen · 2025-12-08T23:23:40Z

FE Regression Coverage Report

Increment line coverage 94.74% (18/19) 🎉
Increment coverage report
Complete coverage report

regression-test/suites/export_p0/outfile/native/test_outfile_native.groovy

regression-test/suites/export_p0/test_export_variant_10k_columns.groovy

be/src/vec/exec/scan/file_scanner.cpp

be/src/vec/exec/format/native/native_reader.cpp

hello-stephen · 2025-12-16T04:37:35Z

FE Regression Coverage Report

Increment line coverage 94.74% (18/19) 🎉
Increment coverage report
Complete coverage report

eldenmoon · 2025-12-16T06:29:41Z

run buildall

doris-robot · 2025-12-16T06:59:40Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.85% (1752/2194)
Line Coverage	65.74% (30743/46764)
Region Coverage	66.51% (15360/23096)
Branch Coverage	56.86% (8167/14364)

doris-robot · 2025-12-16T07:37:09Z

TPC-H: Total hot run time: 34988 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4ee8537ad659bbcb60c7b682d9569203f097b105, data reload: false

------ Round 1 ----------------------------------
q1	17654	4351	4050	4050
q2	2054	350	243	243
q3	10167	1366	756	756
q4	10239	950	323	323
q5	7545	2181	1971	1971
q6	191	169	134	134
q7	998	856	695	695
q8	9365	1481	1137	1137
q9	7065	5405	5378	5378
q10	6779	2412	1938	1938
q11	522	314	303	303
q12	633	734	578	578
q13	17772	3676	3070	3070
q14	296	292	276	276
q15	599	511	524	511
q16	695	667	638	638
q17	703	842	527	527
q18	7611	7197	7041	7041
q19	1109	971	601	601
q20	403	367	255	255
q21	4268	4030	3582	3582
q22	1062	997	981	981
Total cold run time: 107730 ms
Total hot run time: 34988 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4077	4058	4045	4045
q2	347	399	328	328
q3	2187	2712	2310	2310
q4	1358	1767	1328	1328
q5	4251	4675	4854	4675
q6	233	186	134	134
q7	2062	1963	1824	1824
q8	2748	2633	2513	2513
q9	7666	7544	7561	7544
q10	3243	3261	2788	2788
q11	573	518	501	501
q12	709	748	586	586
q13	3552	4127	3244	3244
q14	295	352	276	276
q15	554	534	514	514
q16	667	671	627	627
q17	1183	1452	1463	1452
q18	8076	7744	7573	7573
q19	877	860	886	860
q20	2036	2069	1981	1981
q21	4986	4607	4404	4404
q22	1091	1044	978	978
Total cold run time: 52771 ms
Total hot run time: 50485 ms

hello-stephen · 2025-12-16T07:46:57Z

FE UT Coverage Report

Increment line coverage 0.00% (0/19) 🎉
Increment coverage report
Complete coverage report

doris-robot · 2025-12-16T07:48:14Z

TPC-DS: Total hot run time: 179068 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4ee8537ad659bbcb60c7b682d9569203f097b105, data reload: false

query5	5891	643	519	519
query6	352	252	229	229
query7	4223	478	281	281
query8	326	270	243	243
query9	8797	2552	2545	2545
query10	544	376	338	338
query11	15299	15062	14879	14879
query12	183	121	120	120
query13	1271	495	418	418
query14	7669	3262	2966	2966
query14_1	2851	2847	2873	2847
query15	248	192	181	181
query16	972	475	428	428
query17	1263	681	586	586
query18	2702	432	333	333
query19	274	224	204	204
query20	121	115	110	110
query21	223	138	118	118
query22	4010	4044	4111	4044
query23	16712	16199	15906	15906
query23_1	16069	16048	15945	15945
query24	7095	1662	1219	1219
query24_1	1239	1231	1243	1231
query25	557	476	442	442
query26	1241	268	165	165
query27	2729	464	307	307
query28	4490	2144	2131	2131
query29	798	564	441	441
query30	312	243	214	214
query31	794	707	649	649
query32	84	69	68	68
query33	537	333	287	287
query34	901	885	543	543
query35	786	813	718	718
query36	880	920	798	798
query37	136	90	79	79
query38	2836	2850	2830	2830
query39	744	737	715	715
query39_1	693	700	701	700
query40	231	133	119	119
query41	67	62	64	62
query42	108	106	109	106
query43	454	431	407	407
query44	1321	767	739	739
query45	192	187	185	185
query46	883	983	600	600
query47	1647	1696	1635	1635
query48	316	338	244	244
query49	623	431	348	348
query50	671	300	219	219
query51	3821	3889	3741	3741
query52	102	108	96	96
query53	320	361	294	294
query54	293	256	248	248
query55	81	80	77	77
query56	286	295	299	295
query57	1136	1143	1072	1072
query58	265	256	252	252
query59	2311	2460	2415	2415
query60	323	304	288	288
query61	158	155	160	155
query62	706	669	633	633
query63	331	309	295	295
query64	4968	1315	991	991
query65	4039	3950	4018	3950
query66	1349	454	323	323
query67	15329	15045	14880	14880
query68	2787	1036	753	753
query69	433	347	312	312
query70	1042	1003	916	916
query71	332	312	286	286
query72	6112	5227	5371	5227
query73	543	648	310	310
query74	8777	8816	8552	8552
query75	3113	3140	2796	2796
query76	2824	1164	765	765
query77	377	417	298	298
query78	9507	9656	8805	8805
query79	1141	881	628	628
query80	981	728	598	598
query81	514	270	241	241
query82	1337	138	108	108
query83	354	259	248	248
query84	268	130	108	108
query85	1070	532	463	463
query86	389	299	281	281
query87	3087	3106	2916	2916
query88	3252	2268	2247	2247
query89	464	422	392	392
query90	1862	158	152	152
query91	172	160	144	144
query92	73	65	63	63
query93	960	926	559	559
query94	440	296	290	290
query95	537	329	382	329
query96	591	480	207	207
query97	2274	2315	2208	2208
query98	205	204	191	191
query99	1303	1283	1233	1233
Total cold run time: 256592 ms
Total hot run time: 179068 ms

doris-robot · 2025-12-16T07:53:29Z

ClickBench: Total hot run time: 29.18 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4ee8537ad659bbcb60c7b682d9569203f097b105, data reload: false

query1	0.05	0.04	0.04
query2	0.14	0.06	0.06
query3	0.34	0.08	0.08
query4	1.60	0.10	0.09
query5	0.25	0.24	0.25
query6	1.19	0.65	0.63
query7	0.03	0.03	0.03
query8	0.07	0.07	0.06
query9	0.59	0.52	0.51
query10	0.55	0.56	0.55
query11	0.27	0.13	0.14
query12	0.26	0.14	0.15
query13	0.62	0.61	0.61
query14	1.02	1.01	1.00
query15	0.88	0.82	0.82
query16	0.40	0.38	0.42
query17	0.99	1.07	1.03
query18	0.24	0.22	0.21
query19	1.87	1.87	1.87
query20	0.02	0.02	0.01
query21	15.41	0.28	0.24
query22	4.98	0.10	0.10
query23	15.44	0.40	0.22
query24	2.43	0.45	0.30
query25	0.10	0.09	0.10
query26	0.19	0.18	0.19
query27	0.09	0.09	0.09
query28	3.78	1.37	1.18
query29	12.61	4.08	3.32
query30	0.34	0.13	0.11
query31	2.81	0.64	0.43
query32	3.24	0.59	0.49
query33	3.03	3.05	3.11
query34	16.66	5.17	4.62
query35	4.74	5.38	5.24
query36	0.66	0.53	0.54
query37	0.25	0.08	0.08
query38	0.23	0.05	0.06
query39	0.08	0.05	0.05
query40	0.20	0.17	0.18
query41	0.13	0.06	0.06
query42	0.08	0.05	0.05
query43	0.06	0.04	0.05
Total cold run time: 98.92 s
Total hot run time: 29.18 s

hello-stephen · 2025-12-16T10:35:15Z

FE Regression Coverage Report

Increment line coverage 94.74% (18/19) 🎉
Increment coverage report
Complete coverage report

doris-robot · 2025-12-17T11:02:15Z

BE UT Coverage Report

Increment line coverage 65.01% (275/423) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	53.47% (18866/35285)
Line Coverage	39.26% (174844/445302)
Region Coverage	33.79% (135205/400107)
Branch Coverage	34.72% (58283/167869)

hello-stephen · 2025-12-17T11:05:59Z

BE Regression && UT Coverage Report

Increment line coverage 80.09% (338/422) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	72.26% (24985/34575)
Line Coverage	58.99% (262320/444716)
Region Coverage	53.80% (217797/404836)
Branch Coverage	55.37% (93412/168708)

doris-robot · 2025-12-17T11:06:32Z

BE UT Coverage Report

Increment line coverage 65.01% (275/423) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	53.43% (18853/35285)
Line Coverage	39.24% (174731/445302)
Region Coverage	33.79% (135194/400107)
Branch Coverage	34.70% (58253/167869)

github-actions · 2025-12-17T11:09:32Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-12-17T11:09:35Z

PR approved by anyone and no changes requested.

hello-stephen · 2025-12-17T11:12:09Z

BE Regression && UT Coverage Report

Increment line coverage 80.09% (338/422) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	72.23% (24972/34575)
Line Coverage	58.96% (262197/444716)
Region Coverage	53.78% (217721/404836)
Branch Coverage	55.34% (93367/168708)

airborne12

LGTM

eldenmoon requested a review from zclllyybb as a code owner December 4, 2025 07:48

eldenmoon force-pushed the export-native branch from 408e53c to 155ffd7 Compare December 4, 2025 07:51

eldenmoon force-pushed the export-native branch from 155ffd7 to 9877d06 Compare December 4, 2025 07:56

eldenmoon force-pushed the export-native branch from 9877d06 to 0213622 Compare December 5, 2025 08:15

morningman reviewed Dec 16, 2025

View reviewed changes

eldenmoon force-pushed the export-native branch from ce60809 to d0a52ab Compare December 16, 2025 06:29

eldenmoon added 4 commits December 16, 2025 14:29

support native format

718d77d

add more ut

e6dc619

add serde test

4efa200

fix with comment

4ee8537

eldenmoon force-pushed the export-native branch from d0a52ab to 4ee8537 Compare December 16, 2025 06:29

eldenmoon added dev/4.1.x and removed dev/4.0.x labels Dec 16, 2025

zclllyybb approved these changes Dec 17, 2025

View reviewed changes

github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Dec 17, 2025

eldenmoon requested a review from morningman December 17, 2025 11:12

airborne12 approved these changes Dec 18, 2025

View reviewed changes

eldenmoon merged commit 8ef5bab into apache:master Dec 18, 2025
28 of 31 checks passed

eldenmoon deleted the export-native branch December 18, 2025 09:09

[Feature](Export) support native format #58711

[Feature](Export) support native format #58711

Uh oh!

Conversation

eldenmoon commented Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Thearas commented Dec 4, 2025

Uh oh!

eldenmoon commented Dec 4, 2025

Uh oh!

eldenmoon commented Dec 4, 2025

Uh oh!

eldenmoon commented Dec 4, 2025

Uh oh!

hello-stephen commented Dec 4, 2025

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Dec 4, 2025

FE UT Coverage Report

Uh oh!

doris-robot commented Dec 4, 2025

Uh oh!

doris-robot commented Dec 4, 2025

Uh oh!

doris-robot commented Dec 4, 2025

Uh oh!

hello-stephen commented Dec 4, 2025

FE Regression Coverage Report

Uh oh!

eldenmoon commented Dec 5, 2025

Uh oh!

hello-stephen commented Dec 5, 2025

Cloud UT Coverage Report

Uh oh!

doris-robot commented Dec 5, 2025

Uh oh!

doris-robot commented Dec 5, 2025

Uh oh!

doris-robot commented Dec 5, 2025

Uh oh!

hello-stephen commented Dec 5, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Dec 5, 2025

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Dec 5, 2025

FE Regression Coverage Report

Uh oh!

eldenmoon commented Dec 8, 2025

Uh oh!

hello-stephen commented Dec 8, 2025

Cloud UT Coverage Report

Uh oh!

doris-robot commented Dec 8, 2025

Uh oh!

doris-robot commented Dec 8, 2025

Uh oh!

doris-robot commented Dec 8, 2025

Uh oh!

hello-stephen commented Dec 8, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Dec 8, 2025

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Dec 8, 2025

FE Regression Coverage Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hello-stephen commented Dec 16, 2025

FE Regression Coverage Report

Uh oh!

eldenmoon commented Dec 16, 2025

Uh oh!

doris-robot commented Dec 16, 2025

eldenmoon commented Dec 4, 2025 •

edited

Loading