Skip to content

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Sep 4, 2025

Cherry-picked from #55466

…(and no data fields) are required (#55466)

### What problem does this PR solve?

This pull request introduces support for "virtual tables" in the Hudi
JNI scanner, allowing the system to handle cases where only partition
columns (and no data fields) are required from a query. The
implementation ensures correct handling of empty field lists throughout
the scanner and vector table logic, and adds regression tests for this
scenario.
@github-actions github-actions bot requested a review from morrySnow as a code owner September 4, 2025 06:04
@morrySnow morrySnow closed this Sep 4, 2025
@morrySnow morrySnow reopened this Sep 4, 2025
@morrySnow
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32528 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a21e4c7c082df73bba884ead06acfbd9dfaaffe7, data reload: false

------ Round 1 ----------------------------------
q1	17623	5546	5446	5446
q2	2045	288	163	163
q3	10425	1275	737	737
q4	10216	853	440	440
q5	8170	2392	2160	2160
q6	182	163	138	138
q7	893	720	613	613
q8	9332	1410	1136	1136
q9	5419	4931	4943	4931
q10	6788	2264	1809	1809
q11	516	277	258	258
q12	331	343	208	208
q13	17764	3598	2997	2997
q14	232	225	220	220
q15	525	459	449	449
q16	421	422	368	368
q17	610	862	371	371
q18	7089	6391	6381	6381
q19	1281	959	539	539
q20	334	351	210	210
q21	3060	2223	1951	1951
q22	1065	1070	1003	1003
Total cold run time: 104321 ms
Total hot run time: 32528 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5529	5486	5502	5486
q2	248	334	237	237
q3	2218	2623	2330	2330
q4	1350	1807	1308	1308
q5	4390	4834	5057	4834
q6	172	172	128	128
q7	2085	1997	1851	1851
q8	2612	2818	2684	2684
q9	7252	7150	7199	7150
q10	2976	3318	2785	2785
q11	581	514	500	500
q12	659	771	613	613
q13	3350	3752	3170	3170
q14	292	292	290	290
q15	534	471	477	471
q16	442	493	434	434
q17	1263	1751	1249	1249
q18	7512	7521	7371	7371
q19	789	1083	1089	1083
q20	1992	2062	1879	1879
q21	5385	4807	4798	4798
q22	1106	1090	1014	1014
Total cold run time: 52737 ms
Total hot run time: 51665 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191226 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a21e4c7c082df73bba884ead06acfbd9dfaaffe7, data reload: false

query1	968	367	366	366
query2	6545	1914	1895	1895
query3	6705	220	216	216
query4	34167	23555	23785	23555
query5	4361	619	467	467
query6	273	190	180	180
query7	4626	493	314	314
query8	287	250	251	250
query9	9448	2627	2619	2619
query10	493	329	264	264
query11	18359	15459	15240	15240
query12	161	106	105	105
query13	1683	563	416	416
query14	9762	6591	7541	6591
query15	244	193	192	192
query16	8122	642	511	511
query17	1759	760	611	611
query18	2211	413	319	319
query19	225	186	172	172
query20	127	128	114	114
query21	212	136	109	109
query22	4577	4921	4480	4480
query23	35108	33931	33872	33872
query24	7636	2645	2618	2618
query25	529	482	411	411
query26	1206	280	170	170
query27	2069	479	340	340
query28	5042	2189	2150	2150
query29	785	581	468	468
query30	260	187	155	155
query31	974	890	832	832
query32	100	63	59	59
query33	525	359	311	311
query34	749	839	521	521
query35	776	791	726	726
query36	1011	1041	949	949
query37	108	99	69	69
query38	3878	3903	3892	3892
query39	1489	1488	1433	1433
query40	209	118	106	106
query41	51	51	48	48
query42	123	104	109	104
query43	508	517	460	460
query44	1444	818	814	814
query45	186	173	174	173
query46	884	1041	672	672
query47	1921	1914	1849	1849
query48	462	437	386	386
query49	816	500	423	423
query50	657	674	424	424
query51	7247	7242	7170	7170
query52	102	97	94	94
query53	228	261	184	184
query54	556	542	472	472
query55	82	77	80	77
query56	266	272	265	265
query57	1249	1240	1170	1170
query58	245	219	212	212
query59	3050	2967	2978	2967
query60	302	267	268	267
query61	131	111	112	111
query62	799	718	684	684
query63	223	196	186	186
query64	4603	971	664	664
query65	3278	3211	3219	3211
query66	1066	413	312	312
query67	15981	15709	15704	15704
query68	8303	849	551	551
query69	489	302	268	268
query70	1142	1150	1120	1120
query71	518	299	273	273
query72	5680	3599	3799	3599
query73	649	744	365	365
query74	10189	9175	9052	9052
query75	3218	3137	2608	2608
query76	3294	1183	763	763
query77	501	362	268	268
query78	10279	10416	9558	9558
query79	3273	902	595	595
query80	664	530	452	452
query81	512	254	223	223
query82	649	122	87	87
query83	173	165	147	147
query84	291	112	82	82
query85	786	376	312	312
query86	404	297	279	279
query87	4362	4326	4258	4258
query88	5296	2439	2408	2408
query89	405	334	295	295
query90	1842	189	186	186
query91	136	159	109	109
query92	65	54	51	51
query93	2080	911	553	553
query94	694	415	312	312
query95	341	266	268	266
query96	494	613	294	294
query97	3206	3314	3218	3218
query98	230	207	206	206
query99	1553	1401	1297	1297
Total cold run time: 295947 ms
Total hot run time: 191226 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.92 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a21e4c7c082df73bba884ead06acfbd9dfaaffe7, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.24	0.06	0.06
query4	1.64	0.11	0.10
query5	0.53	0.51	0.53
query6	1.13	0.73	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.50	0.51
query10	0.56	0.54	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.62	0.61	0.58
query14	0.77	0.84	0.79
query15	0.84	0.84	0.83
query16	0.38	0.39	0.39
query17	0.99	1.01	0.95
query18	0.23	0.23	0.22
query19	1.87	1.81	1.84
query20	0.02	0.01	0.01
query21	15.40	0.94	0.58
query22	0.75	0.72	0.71
query23	15.11	1.37	0.58
query24	3.36	0.98	1.05
query25	0.30	0.16	0.10
query26	0.33	0.16	0.13
query27	0.05	0.04	0.04
query28	13.59	1.03	0.45
query29	12.62	3.92	3.24
query30	0.25	0.10	0.07
query31	2.82	0.63	0.41
query32	3.23	0.55	0.47
query33	3.05	3.01	3.01
query34	16.57	5.13	4.52
query35	4.57	4.51	4.55
query36	0.64	0.49	0.48
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.04	0.03	0.02
query40	0.16	0.12	0.12
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 103.96 s
Total hot run time: 28.92 s

@morrySnow morrySnow merged commit 3ebb9e4 into branch-3.1 Sep 8, 2025
23 checks passed
@github-actions github-actions bot deleted the auto-pick-55466-branch-3.1 branch September 8, 2025 02:54
@morrySnow morrySnow mentioned this pull request Sep 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants