Skip to content

Conversation

@hubgeter
Copy link
Contributor

@hubgeter hubgeter commented May 28, 2025

What problem does this PR solve?

Related PR: #49051

Problem Summary:

Support reading Hudi and Paimon Iceberg tables after the internal schema of struct is changed.

  1. Introduce hive_reader to avoid confusion between hive and parquet/orc reader
  2. Before this, support for reading tables after schema changes of ordinary columns relied on changing the column name in block, so that parquet/orc reader can read specific file columns when get_next_block, and hudi/iceberg/paimon reader will mix file column names with table column names when using parquet/orc reader.
    This pr clarifies that all calls to parquet/orc reader are based on the concept of table column names, and then introduces TableSchemaChangeHelper::Node to help parquet/orc reader find the specific file columns to be read.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented May 28, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@hubgeter
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.25% (1113/1337)
Line Coverage 66.32% (18750/28271)
Region Coverage 65.94% (9292/14091)
Branch Coverage 55.84% (5010/8972)

@hubgeter
Copy link
Contributor Author

hubgeter commented Jun 5, 2025

run buildall

@doris-robot
Copy link

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.32% (1119/1343)
Line Coverage 66.77% (19183/28732)
Region Coverage 66.45% (9504/14302)
Branch Coverage 56.40% (5153/9136)

@doris-robot
Copy link

TPC-H: Total hot run time: 33718 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0759ac3a8824e535e9d516760dbd97bfc1556a8d, data reload: false

------ Round 1 ----------------------------------
q1	26246	5061	4957	4957
q2	1951	274	186	186
q3	10416	1239	714	714
q4	10236	1011	510	510
q5	7688	2386	2334	2334
q6	185	176	134	134
q7	913	722	601	601
q8	9307	1285	1136	1136
q9	6836	5068	5141	5068
q10	6894	2303	1903	1903
q11	473	290	279	279
q12	353	346	213	213
q13	17774	3676	3070	3070
q14	229	227	214	214
q15	547	476	470	470
q16	418	430	368	368
q17	613	875	370	370
q18	7659	7099	7242	7099
q19	1676	969	565	565
q20	335	326	218	218
q21	3681	3178	2373	2373
q22	1072	998	936	936
Total cold run time: 115502 ms
Total hot run time: 33718 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5155	5059	5053	5053
q2	243	316	223	223
q3	2218	2679	2328	2328
q4	1363	1748	1380	1380
q5	4480	4446	4427	4427
q6	217	172	126	126
q7	2051	1952	1803	1803
q8	2577	2677	2581	2581
q9	7205	7202	6959	6959
q10	3003	3217	2786	2786
q11	583	519	494	494
q12	708	804	637	637
q13	3518	3845	3328	3328
q14	309	301	302	301
q15	522	482	471	471
q16	443	473	450	450
q17	1171	1626	1332	1332
q18	7847	7614	7510	7510
q19	824	863	997	863
q20	1954	1963	1820	1820
q21	4912	4505	4470	4470
q22	1085	1039	1014	1014
Total cold run time: 52388 ms
Total hot run time: 50356 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192968 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0759ac3a8824e535e9d516760dbd97bfc1556a8d, data reload: false

query1	1390	1072	1035	1035
query2	6253	1883	1906	1883
query3	11023	4579	4599	4579
query4	53705	24549	22949	22949
query5	5298	481	467	467
query6	435	220	206	206
query7	5388	512	297	297
query8	298	224	213	213
query9	7479	2648	2690	2648
query10	444	316	276	276
query11	15303	14989	14855	14855
query12	164	111	109	109
query13	1325	532	431	431
query14	10101	6559	6319	6319
query15	215	204	187	187
query16	6996	662	508	508
query17	1081	758	599	599
query18	1535	396	305	305
query19	195	185	172	172
query20	132	125	115	115
query21	208	131	106	106
query22	4308	4409	4209	4209
query23	34201	33622	33413	33413
query24	6704	2446	2441	2441
query25	464	471	419	419
query26	641	272	153	153
query27	2271	515	345	345
query28	2873	2196	2186	2186
query29	588	571	445	445
query30	280	228	188	188
query31	860	858	792	792
query32	69	60	63	60
query33	439	350	298	298
query34	758	879	534	534
query35	800	837	762	762
query36	928	1040	916	916
query37	104	98	72	72
query38	4249	4333	4273	4273
query39	1505	1442	1452	1442
query40	218	125	108	108
query41	61	62	56	56
query42	131	114	107	107
query43	513	529	500	500
query44	1311	831	845	831
query45	200	176	169	169
query46	861	1021	674	674
query47	1818	1901	1814	1814
query48	417	430	336	336
query49	700	487	395	395
query50	666	752	411	411
query51	4221	4297	4256	4256
query52	120	112	97	97
query53	234	278	190	190
query54	575	582	513	513
query55	84	86	86	86
query56	302	317	287	287
query57	1199	1204	1121	1121
query58	265	271	266	266
query59	2911	2860	2856	2856
query60	357	337	317	317
query61	125	151	132	132
query62	769	745	679	679
query63	236	190	233	190
query64	1363	992	672	672
query65	4195	4157	4155	4155
query66	738	403	300	300
query67	15976	15505	15486	15486
query68	4767	897	530	530
query69	495	311	265	265
query70	1187	1123	1107	1107
query71	439	328	298	298
query72	5892	4990	5001	4990
query73	796	637	351	351
query74	9102	9199	8965	8965
query75	3441	3237	2720	2720
query76	3338	1190	761	761
query77	674	365	288	288
query78	10021	10150	9290	9290
query79	2600	816	576	576
query80	822	521	431	431
query81	498	256	214	214
query82	704	127	99	99
query83	267	246	229	229
query84	296	114	94	94
query85	808	350	307	307
query86	476	286	284	284
query87	4349	4601	4477	4477
query88	3488	2269	2249	2249
query89	402	313	285	285
query90	1919	206	202	202
query91	138	137	115	115
query92	75	59	58	58
query93	2597	931	572	572
query94	672	401	312	312
query95	376	298	284	284
query96	496	576	278	278
query97	2702	2765	2683	2683
query98	229	212	201	201
query99	1319	1381	1242	1242
Total cold run time: 296961 ms
Total hot run time: 192968 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.76 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0759ac3a8824e535e9d516760dbd97bfc1556a8d, data reload: false

query1	0.04	0.04	0.03
query2	0.14	0.10	0.11
query3	0.36	0.20	0.20
query4	1.59	0.21	0.20
query5	0.46	0.45	0.46
query6	1.16	0.66	0.67
query7	0.02	0.02	0.01
query8	0.06	0.05	0.05
query9	0.63	0.52	0.52
query10	0.59	0.59	0.57
query11	0.26	0.12	0.13
query12	0.25	0.13	0.14
query13	0.63	0.62	0.63
query14	0.81	0.82	0.82
query15	0.98	0.88	0.87
query16	0.37	0.37	0.36
query17	1.10	1.04	1.02
query18	0.18	0.19	0.18
query19	2.01	1.85	1.83
query20	0.02	0.01	0.01
query21	15.42	0.97	0.68
query22	0.93	1.14	0.88
query23	14.68	1.48	0.74
query24	5.56	0.55	0.28
query25	0.17	0.09	0.09
query26	0.56	0.22	0.19
query27	0.08	0.08	0.08
query28	11.06	1.24	0.58
query29	12.56	4.09	3.42
query30	0.29	0.08	0.06
query31	2.82	0.62	0.43
query32	3.24	0.60	0.51
query33	3.09	3.11	3.09
query34	16.20	5.24	4.40
query35	4.42	4.47	4.41
query36	0.65	0.51	0.50
query37	0.20	0.17	0.17
query38	0.18	0.17	0.15
query39	0.06	0.04	0.05
query40	0.21	0.16	0.16
query41	0.11	0.06	0.05
query42	0.06	0.05	0.06
query43	0.07	0.05	0.05
Total cold run time: 104.28 s
Total hot run time: 29.76 s

@hubgeter hubgeter changed the title [feature](external)support read iceberg/hudi/paimon complex column sc… [feature](external) Support reading Hudi/Paimon/Iceberg tables after schema changes. Jun 6, 2025
@hubgeter
Copy link
Contributor Author

hubgeter commented Jun 8, 2025

run buildall

@doris-robot
Copy link

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.32% (1119/1343)
Line Coverage 66.76% (19182/28732)
Region Coverage 66.40% (9496/14302)
Branch Coverage 56.38% (5151/9136)

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 3.16% (6/190) 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

TPC-H: Total hot run time: 33771 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 29fbf4d99d845f3ab4a883a1f3a2754dde58df94, data reload: false

------ Round 1 ----------------------------------
q1	25918	5045	4979	4979
q2	1959	290	178	178
q3	10387	1258	708	708
q4	10238	1013	517	517
q5	7670	2382	2286	2286
q6	177	161	129	129
q7	885	734	606	606
q8	9295	1221	1071	1071
q9	6759	5095	5087	5087
q10	6868	2354	1890	1890
q11	499	290	280	280
q12	340	351	214	214
q13	17788	3705	3102	3102
q14	231	229	224	224
q15	570	484	503	484
q16	423	438	373	373
q17	581	857	357	357
q18	7593	7277	7192	7192
q19	1942	970	521	521
q20	310	336	220	220
q21	3762	3150	2367	2367
q22	1076	1031	986	986
Total cold run time: 115271 ms
Total hot run time: 33771 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5152	5055	5025	5025
q2	239	318	226	226
q3	2207	2647	2333	2333
q4	1410	1770	1362	1362
q5	4526	4413	4445	4413
q6	221	172	127	127
q7	1988	1929	1719	1719
q8	2570	2458	2482	2458
q9	7297	7198	7166	7166
q10	3003	3178	2763	2763
q11	574	510	505	505
q12	672	754	632	632
q13	3570	3870	3214	3214
q14	286	298	269	269
q15	531	471	470	470
q16	431	493	453	453
q17	1135	1588	1352	1352
q18	7662	7361	7271	7271
q19	773	765	808	765
q20	1958	2032	1882	1882
q21	4917	4568	4308	4308
q22	1094	1055	1008	1008
Total cold run time: 52216 ms
Total hot run time: 49721 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185508 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 29fbf4d99d845f3ab4a883a1f3a2754dde58df94, data reload: false

query1	999	475	476	475
query2	6544	1802	1766	1766
query3	6752	227	221	221
query4	25943	23402	23434	23402
query5	4353	636	475	475
query6	302	216	193	193
query7	4628	490	288	288
query8	270	239	213	213
query9	8593	2614	2610	2610
query10	459	337	271	271
query11	15574	15150	14802	14802
query12	155	103	100	100
query13	1671	513	406	406
query14	9161	6073	6041	6041
query15	192	189	163	163
query16	7176	657	501	501
query17	1196	700	570	570
query18	1978	389	299	299
query19	183	194	155	155
query20	118	122	118	118
query21	221	127	104	104
query22	4097	4225	3934	3934
query23	33940	33016	32987	32987
query24	8463	2342	2320	2320
query25	538	469	402	402
query26	1239	266	155	155
query27	2770	504	339	339
query28	4364	2114	2119	2114
query29	789	570	451	451
query30	289	203	198	198
query31	956	842	776	776
query32	81	64	67	64
query33	577	366	318	318
query34	798	849	539	539
query35	767	824	726	726
query36	967	981	912	912
query37	115	102	80	80
query38	4129	4160	4059	4059
query39	1467	1431	1412	1412
query40	217	122	115	115
query41	74	65	70	65
query42	133	117	108	108
query43	494	513	470	470
query44	1287	831	832	831
query45	181	178	175	175
query46	839	1023	625	625
query47	1765	1845	1729	1729
query48	428	442	317	317
query49	733	493	423	423
query50	649	676	418	418
query51	4093	4125	4160	4125
query52	108	106	96	96
query53	221	248	184	184
query54	569	565	498	498
query55	84	86	124	86
query56	302	282	290	282
query57	1122	1148	1096	1096
query58	264	256	250	250
query59	2589	2669	2577	2577
query60	347	326	307	307
query61	127	124	125	124
query62	819	706	655	655
query63	226	188	184	184
query64	4387	982	667	667
query65	4278	4180	4140	4140
query66	1142	410	346	346
query67	16093	15671	15532	15532
query68	8352	882	531	531
query69	501	322	266	266
query70	1247	1101	1132	1101
query71	458	324	295	295
query72	5503	4777	4936	4777
query73	724	666	361	361
query74	8883	8960	8721	8721
query75	3835	3182	2693	2693
query76	3639	1193	744	744
query77	786	382	297	297
query78	10005	10212	9292	9292
query79	1509	812	572	572
query80	737	522	438	438
query81	471	267	216	216
query82	421	129	98	98
query83	288	244	231	231
query84	307	113	95	95
query85	790	348	302	302
query86	339	292	275	275
query87	4391	4438	4258	4258
query88	2835	2288	2379	2288
query89	374	311	283	283
query90	1921	211	211	211
query91	143	137	110	110
query92	83	57	59	57
query93	1076	940	596	596
query94	675	414	293	293
query95	377	294	285	285
query96	490	583	286	286
query97	2734	2712	2649	2649
query98	227	222	199	199
query99	1447	1402	1304	1304
Total cold run time: 271953 ms
Total hot run time: 185508 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.99 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 29fbf4d99d845f3ab4a883a1f3a2754dde58df94, data reload: false

query1	0.04	0.04	0.02
query2	0.13	0.10	0.10
query3	0.25	0.20	0.20
query4	1.58	0.19	0.11
query5	0.46	0.44	0.42
query6	1.18	0.66	0.65
query7	0.02	0.02	0.01
query8	0.05	0.03	0.04
query9	0.58	0.50	0.52
query10	0.56	0.59	0.56
query11	0.16	0.11	0.11
query12	0.14	0.11	0.12
query13	0.61	0.61	0.60
query14	0.78	0.80	0.80
query15	0.88	0.86	0.87
query16	0.39	0.38	0.39
query17	1.08	1.04	1.02
query18	0.22	0.22	0.21
query19	1.89	1.82	1.84
query20	0.02	0.02	0.01
query21	15.43	0.90	0.55
query22	0.75	1.19	0.68
query23	14.93	1.41	0.60
query24	7.51	1.35	0.85
query25	0.52	0.19	0.11
query26	0.68	0.17	0.15
query27	0.05	0.04	0.05
query28	9.45	0.83	0.42
query29	12.57	4.04	3.35
query30	0.26	0.10	0.07
query31	2.83	0.60	0.40
query32	3.23	0.56	0.46
query33	3.06	3.04	3.14
query34	15.84	5.13	4.46
query35	4.55	4.58	4.49
query36	0.66	0.50	0.48
query37	0.10	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.03	0.02
query40	0.15	0.13	0.12
query41	0.08	0.03	0.02
query42	0.04	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 103.82 s
Total hot run time: 28.99 s

@hubgeter hubgeter force-pushed the imp_full_sc branch 2 times, most recently from 8b8c15d to ac0900b Compare June 9, 2025 16:17
@hubgeter
Copy link
Contributor Author

hubgeter commented Jun 9, 2025

run buildall

@hello-stephen
Copy link
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.32% (1119/1343)
Line Coverage 66.79% (19189/28732)
Region Coverage 66.47% (9507/14302)
Branch Coverage 56.45% (5157/9136)

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 3.62% (8/221) 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

TPC-H: Total hot run time: 33804 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ac0900b04f74d89309f88efd5e53432cece4a1a7, data reload: false

------ Round 1 ----------------------------------
q1	26497	5818	5014	5014
q2	1977	297	185	185
q3	10395	1230	710	710
q4	10230	994	510	510
q5	7683	2352	2348	2348
q6	180	169	132	132
q7	894	753	615	615
q8	9333	1300	1083	1083
q9	6898	5087	5125	5087
q10	6884	2308	1886	1886
q11	488	289	294	289
q12	339	361	218	218
q13	17766	3665	3078	3078
q14	228	228	205	205
q15	563	493	487	487
q16	432	427	378	378
q17	599	851	360	360
q18	7703	7110	7195	7110
q19	2077	963	555	555
q20	327	335	224	224
q21	3698	2565	2337	2337
q22	1045	1046	993	993
Total cold run time: 116236 ms
Total hot run time: 33804 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5151	4997	5088	4997
q2	242	319	217	217
q3	2174	2687	2331	2331
q4	1395	1797	1419	1419
q5	4557	4409	4388	4388
q6	217	170	131	131
q7	1974	1928	1754	1754
q8	2659	2611	2511	2511
q9	7161	7211	7270	7211
q10	3001	3162	2723	2723
q11	587	507	497	497
q12	680	765	629	629
q13	3465	3824	3229	3229
q14	294	295	259	259
q15	534	480	469	469
q16	464	494	447	447
q17	1165	1546	1353	1353
q18	7737	7633	7538	7538
q19	778	810	881	810
q20	1997	1975	1802	1802
q21	4720	4366	4371	4366
q22	1077	1029	970	970
Total cold run time: 52029 ms
Total hot run time: 50051 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185788 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ac0900b04f74d89309f88efd5e53432cece4a1a7, data reload: false

query1	1024	470	492	470
query2	6554	1844	1801	1801
query3	6773	231	220	220
query4	26293	23644	22932	22932
query5	4314	634	485	485
query6	309	213	214	213
query7	4628	486	306	306
query8	266	234	236	234
query9	8667	2680	2678	2678
query10	497	363	271	271
query11	15749	15029	14825	14825
query12	158	109	109	109
query13	1656	537	420	420
query14	9481	6226	6197	6197
query15	202	206	173	173
query16	7224	643	483	483
query17	1200	734	575	575
query18	1990	403	303	303
query19	195	191	165	165
query20	134	121	122	121
query21	222	123	112	112
query22	4143	4300	4069	4069
query23	34219	33217	33081	33081
query24	8425	2406	2412	2406
query25	540	502	400	400
query26	1240	267	148	148
query27	2762	499	336	336
query28	4305	2156	2135	2135
query29	779	550	437	437
query30	281	215	202	202
query31	961	873	763	763
query32	71	68	63	63
query33	565	386	317	317
query34	798	865	540	540
query35	801	820	714	714
query36	968	974	864	864
query37	112	105	83	83
query38	4132	4092	4087	4087
query39	1462	1417	1398	1398
query40	211	125	108	108
query41	65	61	62	61
query42	123	112	106	106
query43	490	507	478	478
query44	1350	834	834	834
query45	180	178	171	171
query46	831	1026	626	626
query47	1742	1782	1712	1712
query48	385	423	315	315
query49	754	496	407	407
query50	650	672	405	405
query51	4051	4146	4066	4066
query52	118	110	104	104
query53	235	258	183	183
query54	587	573	509	509
query55	89	82	84	82
query56	305	297	277	277
query57	1189	1176	1122	1122
query58	269	260	256	256
query59	2625	2656	2562	2562
query60	329	326	315	315
query61	130	163	126	126
query62	814	738	654	654
query63	223	195	192	192
query64	4406	1013	692	692
query65	4265	4199	4215	4199
query66	1160	423	312	312
query67	15781	15593	15413	15413
query68	8387	887	517	517
query69	529	306	274	274
query70	1203	1139	1125	1125
query71	462	334	299	299
query72	5516	4789	4724	4724
query73	715	621	358	358
query74	8882	9152	8698	8698
query75	3891	3233	2699	2699
query76	3670	1206	767	767
query77	816	386	288	288
query78	10137	10396	9407	9407
query79	1917	847	586	586
query80	594	510	454	454
query81	476	254	275	254
query82	414	130	97	97
query83	262	250	236	236
query84	238	109	90	90
query85	812	372	314	314
query86	339	329	282	282
query87	4430	4505	4314	4314
query88	3389	2283	2382	2283
query89	393	323	281	281
query90	1973	215	214	214
query91	155	144	117	117
query92	78	62	62	62
query93	1333	967	590	590
query94	681	400	313	313
query95	384	298	287	287
query96	493	573	285	285
query97	2745	2740	2670	2670
query98	232	208	216	208
query99	1655	1403	1272	1272
Total cold run time: 274658 ms
Total hot run time: 185788 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.33 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ac0900b04f74d89309f88efd5e53432cece4a1a7, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.11	0.11
query3	0.26	0.19	0.19
query4	1.60	0.19	0.10
query5	0.43	0.45	0.44
query6	1.17	0.66	0.66
query7	0.03	0.01	0.02
query8	0.05	0.04	0.04
query9	0.59	0.51	0.52
query10	0.58	0.58	0.57
query11	0.16	0.11	0.11
query12	0.15	0.11	0.11
query13	0.62	0.60	0.60
query14	0.84	0.80	0.80
query15	0.89	0.87	0.86
query16	0.41	0.38	0.37
query17	1.02	1.04	1.04
query18	0.22	0.21	0.21
query19	1.96	1.77	1.86
query20	0.01	0.01	0.02
query21	15.40	0.90	0.57
query22	0.77	1.12	0.72
query23	14.97	1.37	0.60
query24	7.28	1.00	1.21
query25	0.50	0.12	0.11
query26	0.75	0.16	0.16
query27	0.05	0.06	0.05
query28	9.34	0.92	0.44
query29	12.54	4.01	3.35
query30	0.25	0.09	0.06
query31	2.84	0.59	0.39
query32	3.23	0.56	0.46
query33	3.13	3.05	3.14
query34	15.76	5.11	4.48
query35	4.50	4.56	4.54
query36	0.66	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.04
Total cold run time: 103.61 s
Total hot run time: 29.33 s

@hubgeter
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.32% (1119/1343)
Line Coverage 66.76% (19182/28732)
Region Coverage 66.45% (9504/14302)
Branch Coverage 56.40% (5153/9136)

@doris-robot
Copy link

TPC-H: Total hot run time: 33787 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f5f88a8aa531dc66f46c2a22445ae1dd14345a27, data reload: false

------ Round 1 ----------------------------------
q1	26130	5710	5064	5064
q2	1986	297	183	183
q3	10422	1287	699	699
q4	10231	1027	519	519
q5	7735	2392	2370	2370
q6	189	162	132	132
q7	901	727	607	607
q8	9307	1311	1071	1071
q9	6730	5023	5102	5023
q10	6815	2331	1914	1914
q11	467	290	268	268
q12	342	359	217	217
q13	17768	3700	3089	3089
q14	230	229	212	212
q15	561	507	493	493
q16	431	442	378	378
q17	615	852	361	361
q18	7618	7196	7064	7064
q19	2995	1060	585	585
q20	325	344	221	221
q21	3686	3263	2350	2350
q22	1033	989	967	967
Total cold run time: 116517 ms
Total hot run time: 33787 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5289	5079	5103	5079
q2	244	326	228	228
q3	2183	2697	2305	2305
q4	1329	1773	1460	1460
q5	4552	4414	4376	4376
q6	217	174	130	130
q7	2019	1912	1784	1784
q8	2552	2535	2626	2535
q9	7087	7118	7041	7041
q10	2987	3200	2782	2782
q11	573	507	495	495
q12	677	794	641	641
q13	3542	3936	3304	3304
q14	275	299	271	271
q15	537	501	503	501
q16	455	484	429	429
q17	1176	1593	1385	1385
q18	7793	7440	7363	7363
q19	823	843	900	843
q20	2034	2065	1904	1904
q21	4977	4403	4250	4250
q22	1027	1033	999	999
Total cold run time: 52348 ms
Total hot run time: 50105 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185136 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f5f88a8aa531dc66f46c2a22445ae1dd14345a27, data reload: false

query1	1014	479	499	479
query2	6558	1865	1899	1865
query3	6743	220	214	214
query4	26403	23874	22925	22925
query5	4322	622	455	455
query6	311	209	202	202
query7	4629	499	290	290
query8	273	232	226	226
query9	8626	2638	2627	2627
query10	455	327	262	262
query11	15694	15026	14789	14789
query12	155	106	112	106
query13	1675	511	434	434
query14	8555	6213	6297	6213
query15	205	194	171	171
query16	7155	631	507	507
query17	1176	715	568	568
query18	2003	411	308	308
query19	195	194	170	170
query20	127	124	118	118
query21	210	127	109	109
query22	4065	4168	3969	3969
query23	34064	33047	32986	32986
query24	8452	2360	2425	2360
query25	569	479	437	437
query26	1253	275	162	162
query27	2749	500	347	347
query28	4363	2165	2140	2140
query29	769	546	434	434
query30	285	220	185	185
query31	919	807	744	744
query32	77	63	69	63
query33	541	377	294	294
query34	795	851	550	550
query35	793	905	741	741
query36	962	991	873	873
query37	104	104	82	82
query38	4109	4069	4082	4069
query39	1467	1420	1579	1420
query40	209	116	109	109
query41	65	62	58	58
query42	123	110	111	110
query43	513	535	487	487
query44	1324	827	830	827
query45	180	173	165	165
query46	837	1023	636	636
query47	1741	1774	1713	1713
query48	393	422	325	325
query49	748	497	385	385
query50	639	697	426	426
query51	4109	4162	4052	4052
query52	114	104	103	103
query53	227	255	184	184
query54	588	570	507	507
query55	84	85	85	85
query56	304	305	301	301
query57	1123	1142	1081	1081
query58	266	258	261	258
query59	2613	2682	2646	2646
query60	334	321	303	303
query61	130	122	121	121
query62	833	742	657	657
query63	219	191	190	190
query64	4331	993	669	669
query65	4287	4247	4218	4218
query66	1185	417	316	316
query67	15931	15391	15244	15244
query68	8672	892	528	528
query69	472	310	271	271
query70	1207	1083	1091	1083
query71	463	325	296	296
query72	5781	4748	4756	4748
query73	730	623	361	361
query74	9240	9185	8616	8616
query75	3975	3190	2709	2709
query76	3649	1198	763	763
query77	783	370	303	303
query78	9966	10155	9212	9212
query79	2372	829	580	580
query80	639	534	440	440
query81	466	250	227	227
query82	451	125	98	98
query83	302	259	242	242
query84	297	109	95	95
query85	793	356	319	319
query86	329	306	278	278
query87	4480	4437	4332	4332
query88	3006	2287	2282	2282
query89	411	323	284	284
query90	1909	211	208	208
query91	146	206	117	117
query92	76	63	59	59
query93	1624	926	576	576
query94	681	411	307	307
query95	375	294	286	286
query96	500	562	284	284
query97	2747	2792	2681	2681
query98	223	230	200	200
query99	1465	1404	1281	1281
Total cold run time: 274500 ms
Total hot run time: 185136 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.38 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f5f88a8aa531dc66f46c2a22445ae1dd14345a27, data reload: false

query1	0.03	0.03	0.03
query2	0.11	0.11	0.11
query3	0.24	0.19	0.20
query4	1.58	0.20	0.19
query5	0.44	0.44	0.44
query6	1.15	0.65	0.65
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.58	0.50	0.54
query10	0.58	0.55	0.57
query11	0.16	0.11	0.10
query12	0.14	0.12	0.11
query13	0.61	0.59	0.59
query14	0.79	0.80	0.81
query15	0.88	0.87	0.82
query16	0.39	0.37	0.38
query17	1.02	1.01	1.03
query18	0.22	0.21	0.20
query19	1.87	1.77	1.79
query20	0.01	0.01	0.01
query21	15.39	0.89	0.55
query22	0.77	1.12	0.63
query23	15.01	1.40	0.61
query24	6.83	2.24	0.37
query25	0.28	0.11	0.16
query26	0.71	0.16	0.13
query27	0.05	0.05	0.05
query28	9.22	0.89	0.43
query29	12.63	4.01	3.31
query30	0.24	0.09	0.08
query31	2.81	0.58	0.38
query32	3.23	0.55	0.47
query33	3.08	3.01	3.11
query34	15.76	5.15	4.50
query35	4.53	4.48	4.48
query36	0.66	0.49	0.48
query37	0.08	0.07	0.06
query38	0.06	0.03	0.04
query39	0.03	0.03	0.03
query40	0.16	0.14	0.12
query41	0.08	0.02	0.03
query42	0.03	0.03	0.02
query43	0.03	0.02	0.02
Total cold run time: 102.54 s
Total hot run time: 28.38 s

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 1.15% (3/261) 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 49.32% (581/1178) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.21% (15437/26983)
Line Coverage 46.16% (139933/303123)
Region Coverage 45.47% (70875/155855)
Branch Coverage 40.22% (37381/92936)

@doris-robot
Copy link

TPC-H: Total hot run time: 33931 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0a9e43197ad3fa60486372e934ceb186f00d9e2a, data reload: false

------ Round 1 ----------------------------------
q1	17713	5245	5107	5107
q2	1951	296	220	220
q3	10645	1358	732	732
q4	10293	1041	513	513
q5	8359	2409	2316	2316
q6	190	158	129	129
q7	902	734	607	607
q8	9322	1258	1078	1078
q9	7698	5071	5096	5071
q10	6894	2406	1994	1994
q11	477	297	286	286
q12	342	358	207	207
q13	17763	3689	3013	3013
q14	227	233	218	218
q15	540	488	481	481
q16	434	423	375	375
q17	599	861	365	365
q18	7618	7185	7126	7126
q19	1236	954	574	574
q20	334	355	219	219
q21	3970	3207	2343	2343
q22	1017	1000	957	957
Total cold run time: 108524 ms
Total hot run time: 33931 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5166	5107	5124	5107
q2	243	331	216	216
q3	2150	2660	2277	2277
q4	1384	1793	1326	1326
q5	4251	4180	4115	4115
q6	204	164	122	122
q7	1896	1834	1649	1649
q8	2496	2499	2432	2432
q9	6845	6750	6721	6721
q10	2960	3165	2724	2724
q11	561	528	489	489
q12	667	717	583	583
q13	3318	3733	3163	3163
q14	263	270	250	250
q15	503	458	463	458
q16	416	469	421	421
q17	1104	1512	1351	1351
q18	7264	7244	7026	7026
q19	778	827	958	827
q20	1996	1976	1830	1830
q21	4754	4439	4375	4375
q22	1076	1012	968	968
Total cold run time: 50295 ms
Total hot run time: 48430 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184797 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0a9e43197ad3fa60486372e934ceb186f00d9e2a, data reload: false

query1	1028	387	393	387
query2	6565	1688	1700	1688
query3	6752	214	211	211
query4	26410	23493	23281	23281
query5	5048	582	430	430
query6	297	211	216	211
query7	4623	502	291	291
query8	276	217	222	217
query9	8597	2650	2656	2650
query10	504	334	278	278
query11	15705	15006	14815	14815
query12	175	109	111	109
query13	1648	539	407	407
query14	9542	5803	5850	5803
query15	209	192	182	182
query16	7646	649	497	497
query17	1332	721	593	593
query18	2033	408	313	313
query19	202	196	162	162
query20	124	124	126	124
query21	221	126	108	108
query22	4114	4117	4020	4020
query23	34199	33179	32959	32959
query24	8371	2327	2371	2327
query25	516	471	403	403
query26	1240	267	145	145
query27	2733	515	342	342
query28	4318	2164	2120	2120
query29	730	543	427	427
query30	281	224	188	188
query31	904	856	775	775
query32	72	71	62	62
query33	556	363	335	335
query34	809	859	529	529
query35	826	812	751	751
query36	937	974	878	878
query37	118	104	76	76
query38	4325	4121	4088	4088
query39	1482	1428	1415	1415
query40	221	118	103	103
query41	56	55	53	53
query42	132	106	107	106
query43	494	494	483	483
query44	1344	824	820	820
query45	175	170	160	160
query46	836	1018	629	629
query47	1738	1819	1705	1705
query48	384	424	309	309
query49	737	504	429	429
query50	655	685	419	419
query51	4110	4205	4063	4063
query52	109	103	100	100
query53	223	260	184	184
query54	574	587	510	510
query55	87	82	81	81
query56	314	309	295	295
query57	1211	1189	1110	1110
query58	263	255	260	255
query59	2554	2657	2646	2646
query60	325	321	304	304
query61	133	137	122	122
query62	803	739	640	640
query63	215	188	189	188
query64	4314	1004	697	697
query65	4304	4182	4162	4162
query66	1053	410	317	317
query67	15785	15435	15154	15154
query68	8850	886	524	524
query69	465	309	276	276
query70	1209	1138	1019	1019
query71	468	322	288	288
query72	5623	4687	4832	4687
query73	706	606	347	347
query74	9223	8810	9113	8810
query75	4273	3181	2704	2704
query76	3768	1136	711	711
query77	795	382	295	295
query78	10162	10338	9368	9368
query79	1893	820	570	570
query80	610	504	494	494
query81	476	251	225	225
query82	417	126	94	94
query83	247	247	226	226
query84	246	102	87	87
query85	785	352	315	315
query86	331	330	287	287
query87	4339	4389	4298	4298
query88	3256	2279	2282	2279
query89	374	313	285	285
query90	1972	211	206	206
query91	139	155	109	109
query92	77	61	54	54
query93	1431	954	594	594
query94	688	429	293	293
query95	376	291	330	291
query96	493	564	281	281
query97	2706	2746	2653	2653
query98	222	204	212	204
query99	1429	1391	1282	1282
Total cold run time: 276666 ms
Total hot run time: 184797 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.5 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0a9e43197ad3fa60486372e934ceb186f00d9e2a, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.24	0.08	0.08
query4	1.61	0.11	0.12
query5	0.43	0.44	0.42
query6	1.17	0.65	0.67
query7	0.03	0.02	0.02
query8	0.05	0.03	0.04
query9	0.61	0.51	0.52
query10	0.56	0.57	0.57
query11	0.15	0.12	0.11
query12	0.15	0.12	0.11
query13	0.62	0.61	0.61
query14	0.80	0.82	0.82
query15	0.91	0.88	0.88
query16	0.38	0.38	0.39
query17	1.09	1.04	1.03
query18	0.21	0.21	0.21
query19	1.93	1.77	1.94
query20	0.02	0.01	0.01
query21	15.39	0.88	0.53
query22	0.76	1.33	0.96
query23	14.78	1.37	0.63
query24	6.87	1.90	0.58
query25	0.50	0.17	0.08
query26	0.63	0.17	0.13
query27	0.06	0.05	0.05
query28	8.86	0.85	0.46
query29	12.58	4.00	3.31
query30	0.25	0.09	0.07
query31	2.84	0.59	0.38
query32	3.24	0.56	0.46
query33	3.05	3.10	3.10
query34	16.13	5.40	4.74
query35	4.89	4.85	4.81
query36	0.71	0.51	0.48
query37	0.09	0.07	0.07
query38	0.06	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.15	0.13
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.13 s
Total hot run time: 29.5 s

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Jul 4, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jul 4, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 4, 2025

PR approved by anyone and no changes requested.

Copy link
Contributor

@kaka11chen kaka11chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman merged commit b66c78c into apache:master Jul 4, 2025
26 of 27 checks passed
koarz pushed a commit to koarz/doris that referenced this pull request Jul 4, 2025
…schema changes. (apache#51341)

### What problem does this PR solve?
Related PR: apache#49051

Problem Summary:

Support reading Hudi and Paimon Iceberg tables after the internal schema
of struct is changed.
1. Introduce `hive_reader` to avoid confusion between `hive` and
`parquet/orc` reader
2. Before this, support for reading tables after schema changes of
ordinary columns relied on changing the column name in block, so that
parquet/orc reader can read specific file columns when `get_next_block`,
and `hudi/iceberg/paimon reader` will mix `file column names` with
`table column names` when using parquet/orc reader.
This pr clarifies that all calls to `parquet/orc reader` are based on
the concept of `table column names`, and then introduces
`TableSchemaChangeHelper::Node` to help `parquet/orc reader` find the
specific file columns to be read.
seawinde pushed a commit to seawinde/doris that referenced this pull request Jul 4, 2025
…schema changes. (apache#51341)

### What problem does this PR solve?
Related PR: apache#49051

Problem Summary:

Support reading Hudi and Paimon Iceberg tables after the internal schema
of struct is changed.
1. Introduce `hive_reader` to avoid confusion between `hive` and
`parquet/orc` reader
2. Before this, support for reading tables after schema changes of
ordinary columns relied on changing the column name in block, so that
parquet/orc reader can read specific file columns when `get_next_block`,
and `hudi/iceberg/paimon reader` will mix `file column names` with
`table column names` when using parquet/orc reader.
This pr clarifies that all calls to `parquet/orc reader` are based on
the concept of `table column names`, and then introduces
`TableSchemaChangeHelper::Node` to help `parquet/orc reader` find the
specific file columns to be read.
morningman pushed a commit that referenced this pull request Jul 9, 2025
### What problem does this PR solve?

Related PR: #51341

Problem Summary:
In pr #51341, hudiOrcReader was deleted, and this pr reintroduced it to
read hudi orc table.
Although I encountered this error when testing spark-hudi to read orc,
the orc file was indeed generated by spark-hudi.

```
java.lang.UnsupportedOperationException: Base file format is not currently supported (ORC)
        at org.apache.hudi.HoodieBaseRelation.createBaseFileReader(HoodieBaseRelation.scala:574) ~[hudi-spark3.4-bundle_2.12-0.14.0-1.jar:0.14.0-1]
        at org.apache.hudi.BaseFileOnlyRelation.composeRDD(BaseFileOnlyRelation.scala:96) ~[hudi-spark3.4-bundle_2.12-0.14.0-1.jar:0.14.0-1]
        at org.apache.hudi.HoodieBaseRelation.buildScan(HoodieBaseRelation.scala:381) ~[hudi-spark3.4-bundle_2.12-0.14.0-1.jar:0.14.0-1]
        at org.apache.spark.sql.execution.datasources.DataSourceStrategy$.$anonfun$apply$4(DataSourceStrategy.scala:329) ~[spark-sql_2.12-3.4.2.jar:0.14.0-1]
```
hello-stephen pushed a commit that referenced this pull request Jul 10, 2025
…on version. (#53055)

### What problem does this PR solve?

Related PR: #51341

Problem Summary:
In PR #51341, the Docker Paimon was upgraded from version 0.8 to 1.0.1.
Since the required JAR files are pulled from a Maven repository, some
machines may not be able to access the repository.

To fix this, the JAR file has been uploaded to object storage, ensuring
that it can be reliably accessed across different environments.
hubgeter added a commit to hubgeter/doris that referenced this pull request Jul 13, 2025
…schema changes. (apache#51341)

Related PR: apache#49051

Problem Summary:

Support reading Hudi and Paimon Iceberg tables after the internal schema
of struct is changed.
1. Introduce `hive_reader` to avoid confusion between `hive` and
`parquet/orc` reader
2. Before this, support for reading tables after schema changes of
ordinary columns relied on changing the column name in block, so that
parquet/orc reader can read specific file columns when `get_next_block`,
and `hudi/iceberg/paimon reader` will mix `file column names` with
`table column names` when using parquet/orc reader.
This pr clarifies that all calls to `parquet/orc reader` are based on
the concept of `table column names`, and then introduces
`TableSchemaChangeHelper::Node` to help `parquet/orc reader` find the
specific file columns to be read.
hubgeter added a commit to hubgeter/doris that referenced this pull request Jul 15, 2025
…schema changes. (apache#51341)

Related PR: apache#49051

Problem Summary:

Support reading Hudi and Paimon Iceberg tables after the internal schema
of struct is changed.
1. Introduce `hive_reader` to avoid confusion between `hive` and
`parquet/orc` reader
2. Before this, support for reading tables after schema changes of
ordinary columns relied on changing the column name in block, so that
parquet/orc reader can read specific file columns when `get_next_block`,
and `hudi/iceberg/paimon reader` will mix `file column names` with
`table column names` when using parquet/orc reader.
This pr clarifies that all calls to `parquet/orc reader` are based on
the concept of `table column names`, and then introduces
`TableSchemaChangeHelper::Node` to help `parquet/orc reader` find the
specific file columns to be read.
hubgeter added a commit to hubgeter/doris that referenced this pull request Jul 15, 2025
…e#52964)

Related PR: apache#51341

Problem Summary:
In pr apache#51341, hudiOrcReader was deleted, and this pr reintroduced it to
read hudi orc table.
Although I encountered this error when testing spark-hudi to read orc,
the orc file was indeed generated by spark-hudi.

```
java.lang.UnsupportedOperationException: Base file format is not currently supported (ORC)
        at org.apache.hudi.HoodieBaseRelation.createBaseFileReader(HoodieBaseRelation.scala:574) ~[hudi-spark3.4-bundle_2.12-0.14.0-1.jar:0.14.0-1]
        at org.apache.hudi.BaseFileOnlyRelation.composeRDD(BaseFileOnlyRelation.scala:96) ~[hudi-spark3.4-bundle_2.12-0.14.0-1.jar:0.14.0-1]
        at org.apache.hudi.HoodieBaseRelation.buildScan(HoodieBaseRelation.scala:381) ~[hudi-spark3.4-bundle_2.12-0.14.0-1.jar:0.14.0-1]
        at org.apache.spark.sql.execution.datasources.DataSourceStrategy$.$anonfun$apply$4(DataSourceStrategy.scala:329) ~[spark-sql_2.12-3.4.2.jar:0.14.0-1]
```
hubgeter added a commit to hubgeter/doris that referenced this pull request Jul 15, 2025
…on version. (apache#53055)

### What problem does this PR solve?

Related PR: apache#51341

Problem Summary:
In PR apache#51341, the Docker Paimon was upgraded from version 0.8 to 1.0.1.
Since the required JAR files are pulled from a Maven repository, some
machines may not be able to access the repository.

To fix this, the JAR file has been uploaded to object storage, ensuring
that it can be reliably accessed across different environments.
morrySnow pushed a commit that referenced this pull request Jul 16, 2025
…ables after schema changes. #51341 #52964 #52954 #53055 (#53170)

bp #51341: support read hudi/paimon/iceberg schema change
bp #52964: add hudi orc  reader
bp #52954: support timestamp to bigint
bp #53055: fix paimon docker version
suxiaogang223 pushed a commit to suxiaogang223/doris that referenced this pull request Sep 2, 2025
…on version. (apache#53055)

Related PR: apache#51341

Problem Summary:
In PR apache#51341, the Docker Paimon was upgraded from version 0.8 to 1.0.1.
Since the required JAR files are pulled from a Maven repository, some
machines may not be able to access the repository.

To fix this, the JAR file has been uploaded to object storage, ensuring
that it can be reliably accessed across different environments.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.0-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants