[feature](mtmv)Support iceberg mtmv query. #45659

Jibing-Li · 2024-12-19T10:59:51Z

What problem does this PR solve?

Implement MvccTable interface for IcebertExternalTable
IcebergExternalTable overrides the methods in ExternalTable and supports partition pruning
Add snapshot cache in IcebergMetadataCache to store IcebergExternalTable partition infos.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2024-12-19T10:59:56Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

Jibing-Li · 2024-12-19T11:30:03Z

run buildall

doris-robot · 2024-12-19T12:09:53Z

TPC-H: Total hot run time: 39865 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 24f9dba3f65004d0ac59f8a626a9e666d37e5ce4, data reload: false

------ Round 1 ----------------------------------
q1	17571	7477	7338	7338
q2	2043	186	167	167
q3	10735	1083	1151	1083
q4	10568	798	741	741
q5	7584	2735	2645	2645
q6	239	148	147	147
q7	980	637	601	601
q8	9247	1851	1952	1851
q9	6567	6439	6464	6439
q10	7296	2332	2352	2332
q11	475	270	263	263
q12	430	236	237	236
q13	17797	2905	2899	2899
q14	243	214	209	209
q15	562	519	493	493
q16	667	590	585	585
q17	1000	636	588	588
q18	7172	6667	6626	6626
q19	1352	1058	918	918
q20	446	187	187	187
q21	4089	3208	3247	3208
q22	375	309	312	309
Total cold run time: 107438 ms
Total hot run time: 39865 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7313	7474	7319	7319
q2	329	231	237	231
q3	2897	2827	2940	2827
q4	2064	1843	1864	1843
q5	5643	5652	5623	5623
q6	226	152	154	152
q7	2245	1792	1821	1792
q8	3404	3495	3485	3485
q9	8823	9001	8914	8914
q10	3599	3543	3567	3543
q11	610	514	521	514
q12	844	598	610	598
q13	13827	3145	3140	3140
q14	331	280	291	280
q15	566	510	515	510
q16	686	649	653	649
q17	1857	1628	1605	1605
q18	8286	7727	7726	7726
q19	1688	1621	1553	1553
q20	2065	1934	1869	1869
q21	5677	5590	5626	5590
q22	648	581	582	581
Total cold run time: 73628 ms
Total hot run time: 60344 ms

doris-robot · 2024-12-19T12:22:15Z

TPC-DS: Total hot run time: 197927 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 24f9dba3f65004d0ac59f8a626a9e666d37e5ce4, data reload: false

query1	1292	979	925	925
query2	6232	2313	2281	2281
query3	11163	4876	4837	4837
query4	33025	23406	23472	23406
query5	4818	490	464	464
query6	304	188	183	183
query7	3993	325	307	307
query8	294	239	228	228
query9	9608	2708	2712	2708
query10	490	256	237	237
query11	17967	15219	15218	15218
query12	166	105	106	105
query13	1594	441	398	398
query14	10850	7299	8003	7299
query15	263	192	207	192
query16	8170	480	458	458
query17	1671	636	615	615
query18	2155	310	340	310
query19	372	162	169	162
query20	119	117	114	114
query21	202	104	104	104
query22	4708	4382	4431	4382
query23	34602	35050	34159	34159
query24	10454	2556	2495	2495
query25	571	428	440	428
query26	777	159	159	159
query27	2428	341	339	339
query28	7924	2438	2446	2438
query29	693	424	421	421
query30	236	155	152	152
query31	1009	907	868	868
query32	94	56	58	56
query33	736	311	296	296
query34	959	527	546	527
query35	888	765	771	765
query36	1117	968	997	968
query37	125	81	72	72
query38	4335	4132	4263	4132
query39	1678	1495	1489	1489
query40	212	97	96	96
query41	46	44	45	44
query42	121	103	102	102
query43	562	498	505	498
query44	1356	862	836	836
query45	186	170	176	170
query46	1217	749	743	743
query47	2046	1919	1912	1912
query48	443	332	325	325
query49	908	386	393	386
query50	836	413	436	413
query51	7389	7295	7359	7295
query52	112	97	97	97
query53	271	195	195	195
query54	1135	443	455	443
query55	86	78	85	78
query56	316	257	268	257
query57	1242	1172	1168	1168
query58	236	217	226	217
query59	3458	3152	3096	3096
query60	277	249	257	249
query61	107	106	104	104
query62	853	722	691	691
query63	228	199	190	190
query64	4007	711	659	659
query65	3412	3306	3331	3306
query66	765	306	308	306
query67	16455	15453	15531	15453
query68	5280	565	554	554
query69	494	259	249	249
query70	1247	1117	1102	1102
query71	495	262	259	259
query72	6328	4182	3955	3955
query73	812	368	376	368
query74	9889	8936	8921	8921
query75	3575	2642	2677	2642
query76	3695	1116	1116	1116
query77	591	277	294	277
query78	10293	9531	9707	9531
query79	1666	625	599	599
query80	1276	427	440	427
query81	517	232	227	227
query82	534	120	127	120
query83	198	147	149	147
query84	287	73	73	73
query85	1181	309	400	309
query86	360	295	319	295
query87	4832	4458	4352	4352
query88	3435	2340	2198	2198
query89	421	303	292	292
query90	2050	188	190	188
query91	135	103	104	103
query92	63	53	51	51
query93	1914	546	548	546
query94	803	298	286	286
query95	422	263	251	251
query96	614	286	277	277
query97	2892	2675	2722	2675
query98	216	203	193	193
query99	1607	1356	1318	1318
Total cold run time: 304486 ms
Total hot run time: 197927 ms

doris-robot · 2024-12-19T12:27:47Z

ClickBench: Total hot run time: 32.56 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 24f9dba3f65004d0ac59f8a626a9e666d37e5ce4, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.60	0.10	0.10
query5	0.43	0.42	0.42
query6	1.14	0.66	0.66
query7	0.03	0.01	0.01
query8	0.04	0.03	0.04
query9	0.60	0.51	0.50
query10	0.54	0.56	0.56
query11	0.14	0.11	0.11
query12	0.15	0.12	0.12
query13	0.60	0.61	0.61
query14	2.82	2.83	2.75
query15	0.91	0.82	0.82
query16	0.38	0.38	0.38
query17	1.06	1.03	1.01
query18	0.23	0.21	0.21
query19	1.87	1.80	1.96
query20	0.01	0.01	0.01
query21	15.36	0.59	0.60
query22	2.80	2.69	2.07
query23	16.89	1.03	0.88
query24	2.87	1.17	0.39
query25	0.28	0.08	0.10
query26	0.41	0.13	0.13
query27	0.05	0.05	0.04
query28	11.36	1.10	1.08
query29	12.61	3.31	3.28
query30	0.25	0.06	0.06
query31	2.84	0.39	0.38
query32	3.25	0.47	0.47
query33	3.05	3.13	3.20
query34	17.11	4.46	4.53
query35	4.54	4.56	4.49
query36	0.66	0.51	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.12	0.12
query41	0.07	0.02	0.03
query42	0.04	0.03	0.02
query43	0.04	0.02	0.03
Total cold run time: 107.73 s
Total hot run time: 32.56 s

Jibing-Li · 2024-12-19T13:47:18Z

run p0

Jibing-Li · 2024-12-19T13:47:40Z

run external

github-actions · 2024-12-20T03:01:26Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-12-20T03:01:28Z

PR approved by anyone and no changes requested.

fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergExternalTable.java

fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergSchemaCacheKey.java

fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergExternalTable.java

Jibing-Li · 2024-12-23T08:35:07Z

run buildall

zddr

LGTM

fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergExternalTable.java

doris-robot · 2024-12-23T09:06:21Z

TPC-H: Total hot run time: 40050 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9d5bbad3d044713927440c0b697f5daa514185be, data reload: false

------ Round 1 ----------------------------------
q1	17806	7356	7211	7211
q2	2053	185	179	179
q3	11438	1106	1144	1106
q4	10596	760	770	760
q5	7630	2719	2757	2719
q6	250	150	151	150
q7	992	639	620	620
q8	9280	1852	1918	1852
q9	6702	6408	6526	6408
q10	7050	2295	2346	2295
q11	471	253	250	250
q12	423	226	217	217
q13	17973	2965	3146	2965
q14	268	223	243	223
q15	571	511	494	494
q16	670	600	620	600
q17	973	539	559	539
q18	9180	7140	6821	6821
q19	1374	1069	946	946
q20	470	191	181	181
q21	4065	3191	3217	3191
q22	371	330	323	323
Total cold run time: 110606 ms
Total hot run time: 40050 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7223	7180	7144	7144
q2	331	229	229	229
q3	2972	2957	2898	2898
q4	2152	1956	1816	1816
q5	5591	5646	5676	5646
q6	230	146	151	146
q7	2254	1843	1777	1777
q8	3353	3513	3395	3395
q9	8819	8940	8944	8940
q10	3594	3541	3550	3541
q11	584	505	491	491
q12	826	618	604	604
q13	16386	3127	3119	3119
q14	317	270	282	270
q15	570	510	503	503
q16	698	663	651	651
q17	1849	1627	1627	1627
q18	8357	7957	7476	7476
q19	1789	1495	1497	1495
q20	2115	1955	1884	1884
q21	5535	5509	5498	5498
q22	641	596	591	591
Total cold run time: 76186 ms
Total hot run time: 59741 ms

doris-robot · 2024-12-23T09:18:38Z

TPC-DS: Total hot run time: 196422 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9d5bbad3d044713927440c0b697f5daa514185be, data reload: false

query1	1300	955	988	955
query2	6248	2491	2439	2439
query3	10978	4691	4760	4691
query4	33290	23191	23265	23191
query5	4178	465	462	462
query6	285	178	179	178
query7	3995	301	303	301
query8	291	225	215	215
query9	9545	2726	2740	2726
query10	477	250	256	250
query11	18121	15099	15144	15099
query12	155	105	109	105
query13	1581	429	419	419
query14	9434	6832	7592	6832
query15	268	202	190	190
query16	8110	427	457	427
query17	1702	591	599	591
query18	2191	311	318	311
query19	357	176	169	169
query20	127	113	114	113
query21	203	107	139	107
query22	4686	4489	4704	4489
query23	35935	33524	33565	33524
query24	10634	2506	2518	2506
query25	643	409	421	409
query26	1209	158	150	150
query27	2492	335	340	335
query28	7419	2490	2467	2467
query29	848	418	415	415
query30	234	148	143	143
query31	1040	864	831	831
query32	95	59	56	56
query33	764	313	289	289
query34	968	550	535	535
query35	898	800	756	756
query36	1135	968	992	968
query37	146	76	73	73
query38	4166	4161	4454	4161
query39	1556	1457	1478	1457
query40	211	108	110	108
query41	45	46	47	46
query42	117	106	113	106
query43	557	506	505	505
query44	1260	858	841	841
query45	189	170	177	170
query46	1184	731	760	731
query47	2090	1976	1951	1951
query48	421	344	332	332
query49	926	392	388	388
query50	823	421	406	406
query51	7324	7228	7295	7228
query52	107	87	94	87
query53	259	186	192	186
query54	1106	433	454	433
query55	89	83	79	79
query56	296	235	249	235
query57	1311	1195	1226	1195
query58	241	236	224	224
query59	3279	3280	3304	3280
query60	266	245	236	236
query61	115	110	110	110
query62	835	687	680	680
query63	217	193	189	189
query64	3984	745	754	745
query65	3284	3164	3215	3164
query66	748	322	329	322
query67	15962	15728	15556	15556
query68	4794	583	576	576
query69	468	264	260	260
query70	1236	1162	1126	1126
query71	436	291	256	256
query72	6631	4093	3843	3843
query73	794	376	371	371
query74	9871	8853	8723	8723
query75	3443	2652	2645	2645
query76	3676	1050	1111	1050
query77	578	361	289	289
query78	10481	9419	9415	9415
query79	2145	620	617	617
query80	889	434	435	434
query81	552	235	243	235
query82	753	119	120	119
query83	253	145	145	145
query84	249	74	74	74
query85	1560	362	301	301
query86	473	289	273	273
query87	4397	4417	4597	4417
query88	3829	2234	2225	2225
query89	427	288	287	287
query90	1845	190	186	186
query91	144	105	104	104
query92	68	51	50	50
query93	2599	574	567	567
query94	763	258	290	258
query95	349	253	259	253
query96	620	290	290	290
query97	2867	2687	2674	2674
query98	220	195	191	191
query99	1807	1322	1300	1300
Total cold run time: 304167 ms
Total hot run time: 196422 ms

doris-robot · 2024-12-23T09:24:09Z

ClickBench: Total hot run time: 32.78 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9d5bbad3d044713927440c0b697f5daa514185be, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.62	0.11	0.11
query5	0.42	0.43	0.42
query6	1.17	0.66	0.66
query7	0.02	0.01	0.01
query8	0.04	0.04	0.03
query9	0.58	0.50	0.51
query10	0.57	0.61	0.57
query11	0.15	0.10	0.11
query12	0.14	0.11	0.11
query13	0.62	0.61	0.60
query14	2.71	2.71	2.77
query15	0.91	0.83	0.84
query16	0.38	0.38	0.38
query17	1.05	1.04	1.04
query18	0.22	0.21	0.21
query19	1.88	1.80	1.99
query20	0.02	0.02	0.01
query21	15.36	0.61	0.60
query22	2.86	1.78	2.58
query23	17.02	1.03	0.77
query24	3.60	1.36	1.27
query25	0.33	0.15	0.08
query26	0.45	0.14	0.15
query27	0.05	0.04	0.04
query28	10.02	1.12	1.07
query29	12.61	3.24	3.23
query30	0.25	0.06	0.06
query31	2.85	0.40	0.38
query32	3.25	0.46	0.46
query33	3.06	3.08	3.13
query34	17.00	4.42	4.46
query35	4.47	4.51	4.42
query36	0.71	0.48	0.48
query37	0.10	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.03
query40	0.17	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.2 s
Total hot run time: 32.78 s

Jibing-Li · 2024-12-23T11:13:01Z

run buildall

doris-robot · 2024-12-23T11:32:36Z

TPC-H: Total hot run time: 39745 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit be517313061cdd36b1ded625f38d60ffdcbc8b64, data reload: false

------ Round 1 ----------------------------------
q1	17596	7366	7258	7258
q2	2034	194	182	182
q3	10520	1146	1137	1137
q4	10224	812	769	769
q5	7566	2708	2701	2701
q6	233	148	148	148
q7	994	630	608	608
q8	9233	1798	1925	1798
q9	6624	6370	6481	6370
q10	6982	2285	2307	2285
q11	454	263	258	258
q12	412	229	222	222
q13	17760	2927	2965	2927
q14	244	221	213	213
q15	571	502	501	501
q16	658	590	596	590
q17	973	616	494	494
q18	7350	6665	6717	6665
q19	1346	968	1012	968
q20	475	185	182	182
q21	4028	3243	3157	3157
q22	375	312	322	312
Total cold run time: 106652 ms
Total hot run time: 39745 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7207	7224	7219	7219
q2	325	240	229	229
q3	2895	2745	2967	2745
q4	2111	1835	1778	1778
q5	5671	5590	5581	5581
q6	225	143	141	141
q7	2221	1786	1791	1786
q8	3326	3560	3522	3522
q9	8875	8926	8893	8893
q10	3606	3554	3526	3526
q11	609	512	531	512
q12	804	634	591	591
q13	11059	3144	3107	3107
q14	294	266	277	266
q15	553	528	493	493
q16	688	643	640	640
q17	1865	1623	1578	1578
q18	7928	7572	7435	7435
q19	1701	1516	1453	1453
q20	2076	1836	1804	1804
q21	5274	5240	5282	5240
q22	638	578	583	578
Total cold run time: 69951 ms
Total hot run time: 59117 ms

doris-robot · 2024-12-23T11:44:42Z

TPC-DS: Total hot run time: 189984 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit be517313061cdd36b1ded625f38d60ffdcbc8b64, data reload: false

query1	968	382	377	377
query2	6534	2461	2380	2380
query3	6708	212	210	210
query4	34004	23306	23473	23306
query5	4375	471	449	449
query6	284	189	207	189
query7	4723	322	308	308
query8	308	244	239	239
query9	9442	2772	2762	2762
query10	462	254	248	248
query11	18013	15045	15210	15045
query12	156	109	106	106
query13	1680	423	420	420
query14	10189	6772	6857	6772
query15	252	178	192	178
query16	8197	455	434	434
query17	1651	591	590	590
query18	2088	346	306	306
query19	355	151	154	151
query20	115	106	108	106
query21	210	111	101	101
query22	4561	4382	4035	4035
query23	34846	34896	33485	33485
query24	11377	2463	2379	2379
query25	677	386	385	385
query26	1844	154	150	150
query27	2950	333	326	326
query28	8254	2473	2489	2473
query29	1042	406	407	406
query30	302	144	152	144
query31	1059	813	805	805
query32	96	55	58	55
query33	786	297	294	294
query34	972	544	527	527
query35	846	756	744	744
query36	1148	934	966	934
query37	267	80	87	80
query38	4086	4053	4161	4053
query39	1489	1430	1420	1420
query40	292	101	98	98
query41	47	44	46	44
query42	137	106	100	100
query43	552	507	515	507
query44	1296	807	816	807
query45	182	167	175	167
query46	1165	726	729	726
query47	1907	1850	1835	1835
query48	417	348	334	334
query49	1245	400	390	390
query50	815	394	392	392
query51	7159	7152	7131	7131
query52	102	94	90	90
query53	258	179	182	179
query54	1076	410	408	408
query55	82	80	79	79
query56	261	238	232	232
query57	1237	1102	1128	1102
query58	245	229	221	221
query59	3266	3137	2983	2983
query60	282	250	238	238
query61	111	116	115	115
query62	873	678	662	662
query63	217	193	197	193
query64	5237	679	685	679
query65	3243	3199	3261	3199
query66	1194	307	380	307
query67	15777	15543	15519	15519
query68	5862	570	557	557
query69	444	253	250	250
query70	1205	1122	1118	1118
query71	428	254	254	254
query72	6596	4104	4165	4104
query73	777	367	368	367
query74	10181	8907	8857	8857
query75	3452	2627	2640	2627
query76	3560	1016	1041	1016
query77	531	278	269	269
query78	10316	9516	9333	9333
query79	2133	606	619	606
query80	1027	427	433	427
query81	521	242	234	234
query82	848	124	120	120
query83	254	161	146	146
query84	240	71	71	71
query85	1204	303	300	300
query86	425	302	282	282
query87	4685	4522	4529	4522
query88	3889	2249	2221	2221
query89	412	295	296	295
query90	2060	190	190	190
query91	141	104	104	104
query92	69	53	51	51
query93	1540	559	568	559
query94	774	300	286	286
query95	393	250	253	250
query96	617	273	277	273
query97	2869	2683	2685	2683
query98	218	199	201	199
query99	1532	1330	1304	1304
Total cold run time: 305194 ms
Total hot run time: 189984 ms

doris-robot · 2024-12-23T11:50:13Z

ClickBench: Total hot run time: 33.08 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit be517313061cdd36b1ded625f38d60ffdcbc8b64, data reload: false

query1	0.04	0.03	0.04
query2	0.09	0.03	0.03
query3	0.24	0.07	0.06
query4	1.61	0.11	0.10
query5	0.42	0.43	0.39
query6	1.16	0.65	0.66
query7	0.03	0.02	0.01
query8	0.04	0.04	0.03
query9	0.57	0.50	0.52
query10	0.57	0.56	0.55
query11	0.15	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.62	0.59
query14	2.83	2.75	2.74
query15	0.90	0.84	0.83
query16	0.38	0.39	0.38
query17	0.99	1.07	1.00
query18	0.23	0.22	0.21
query19	1.92	1.91	1.92
query20	0.01	0.01	0.01
query21	15.37	0.59	0.58
query22	2.71	2.64	2.10
query23	16.87	0.81	0.93
query24	3.24	1.45	1.19
query25	0.32	0.21	0.06
query26	0.40	0.15	0.13
query27	0.05	0.05	0.04
query28	10.35	1.10	1.07
query29	12.56	3.22	3.22
query30	0.25	0.07	0.06
query31	2.84	0.37	0.38
query32	3.25	0.47	0.46
query33	3.07	3.13	3.12
query34	17.03	4.43	4.44
query35	4.50	4.43	4.49
query36	0.69	0.49	0.48
query37	0.10	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.93 s
Total hot run time: 33.08 s

github-actions · 2024-12-24T03:57:31Z

PR approved by at least one committer and no changes requested.

1. Implement MvccTable interface for IcebertExternalTable 2. IcebergExternalTable overrides the methods in ExternalTable and supports partition pruning 3. Add snapshot cache in IcebergMetadataCache to store IcebergExternalTable partition infos. Issue Number: close #xxx Related PR: #xxx Problem Summary: None

Cherry-pick from #43959 #44419 #44415 #44567 #44673 #44998 #45273 #44911 #44726 #45652 #45659 #46257 #46641 #47026 #47166 #48172 #49956 #50979 --------- Co-authored-by: James <lijibing@selectdb.com> Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>

Jibing-Li force-pushed the mtmv branch 2 times, most recently from efd9485 to 24f9dba Compare December 19, 2024 11:24

Jibing-Li marked this pull request as ready for review December 19, 2024 11:30

morrySnow previously approved these changes Dec 20, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 20, 2024

github-actions bot added the reviewed label Dec 20, 2024

zddr reviewed Dec 20, 2024

View reviewed changes

Jibing-Li dismissed morrySnow’s stale review via 9d5bbad December 23, 2024 08:31

Jibing-Li force-pushed the mtmv branch from 24f9dba to 9d5bbad Compare December 23, 2024 08:31

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 23, 2024

zddr approved these changes Dec 23, 2024

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergExternalTable.java Outdated Show resolved Hide resolved

Support iceberg mtmv query.

be51731

Jibing-Li force-pushed the mtmv branch from 9d5bbad to be51731 Compare December 23, 2024 11:05

morrySnow approved these changes Dec 24, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 24, 2024

Jibing-Li merged commit 6d6ff2d into apache:master Dec 24, 2024
28 of 29 checks passed

Jibing-Li deleted the mtmv branch December 24, 2024 04:12

zddr mentioned this pull request Jun 20, 2025

branch-3.1:[enhance](mtmv)MTMV support iceberg/hudi/paimon #52040

Merged

morrySnow added the dev/3.1.0-merged label Jun 23, 2025

[feature](mtmv)Support iceberg mtmv query. #45659

[feature](mtmv)Support iceberg mtmv query. #45659

Uh oh!

Conversation

Jibing-Li commented Dec 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Dec 19, 2024

Uh oh!

Jibing-Li commented Dec 19, 2024

Uh oh!

doris-robot commented Dec 19, 2024

Uh oh!

doris-robot commented Dec 19, 2024

Uh oh!

doris-robot commented Dec 19, 2024

Uh oh!

Jibing-Li commented Dec 19, 2024

Uh oh!

Jibing-Li commented Dec 19, 2024

Uh oh!

github-actions bot commented Dec 20, 2024

Uh oh!

github-actions bot commented Dec 20, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Jibing-Li commented Dec 23, 2024

Uh oh!

zddr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

doris-robot commented Dec 23, 2024

Uh oh!

doris-robot commented Dec 23, 2024

Uh oh!

doris-robot commented Dec 23, 2024

Uh oh!

Jibing-Li commented Dec 23, 2024

Uh oh!

doris-robot commented Dec 23, 2024

Uh oh!

doris-robot commented Dec 23, 2024

Uh oh!

doris-robot commented Dec 23, 2024

Uh oh!

github-actions bot commented Dec 24, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Jibing-Li commented Dec 19, 2024 •

edited

Loading