Skip to content

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Sep 4, 2025

Cherry-picked from #55466

…(and no data fields) are required (#55466)

### What problem does this PR solve?

This pull request introduces support for "virtual tables" in the Hudi
JNI scanner, allowing the system to handle cases where only partition
columns (and no data fields) are required from a query. The
implementation ensures correct handling of empty field lists throughout
the scanner and vector table logic, and adds regression tests for this
scenario.
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Sep 4, 2025
@dataroaring dataroaring reopened this Sep 4, 2025
@hello-stephen
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39971 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 423ea89dceb86fa6f45896b61f92af95e442f4ea, data reload: false

------ Round 1 ----------------------------------
q1	17719	6754	6646	6646
q2	2035	218	180	180
q3	10557	1127	1183	1127
q4	10228	758	796	758
q5	7764	2858	2856	2856
q6	209	132	135	132
q7	990	620	620	620
q8	9344	1933	2027	1933
q9	6609	6389	6355	6355
q10	7029	2224	2356	2224
q11	464	268	270	268
q12	418	216	221	216
q13	17791	2989	2961	2961
q14	230	202	203	202
q15	514	464	457	457
q16	480	377	373	373
q17	977	591	537	537
q18	7157	6657	6697	6657
q19	1400	1059	1072	1059
q20	475	197	209	197
q21	3798	3243	3240	3240
q22	1083	994	973	973
Total cold run time: 107271 ms
Total hot run time: 39971 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6624	6546	6533	6533
q2	326	232	240	232
q3	2881	2943	2928	2928
q4	2051	1811	1793	1793
q5	5722	5737	5673	5673
q6	211	130	131	130
q7	2222	1798	1799	1798
q8	3365	3541	3543	3541
q9	8817	8891	8869	8869
q10	3563	3553	3559	3553
q11	573	501	489	489
q12	797	608	594	594
q13	9019	3182	3169	3169
q14	295	272	262	262
q15	506	473	482	473
q16	483	467	455	455
q17	1825	1616	1636	1616
q18	8268	7796	8069	7796
q19	1687	1557	1536	1536
q20	2100	1933	1928	1928
q21	5240	4995	4988	4988
q22	1131	1047	1037	1037
Total cold run time: 67706 ms
Total hot run time: 59393 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192484 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 423ea89dceb86fa6f45896b61f92af95e442f4ea, data reload: false

query1	968	381	386	381
query2	6254	1915	1869	1869
query3	8688	205	200	200
query4	33868	23653	23427	23427
query5	3690	472	456	456
query6	277	179	170	170
query7	4198	317	327	317
query8	287	231	224	224
query9	9594	2587	2577	2577
query10	476	265	264	264
query11	17979	15662	15346	15346
query12	157	102	98	98
query13	1574	423	430	423
query14	9782	6694	7684	6694
query15	249	170	177	170
query16	7910	521	530	521
query17	1610	625	589	589
query18	2227	318	322	318
query19	372	167	159	159
query20	128	114	126	114
query21	215	112	116	112
query22	4833	4658	4351	4351
query23	34819	34313	34471	34313
query24	11329	2973	2926	2926
query25	682	437	421	421
query26	1292	175	173	173
query27	2672	347	360	347
query28	7562	2136	2113	2113
query29	890	465	453	453
query30	265	163	168	163
query31	1045	823	836	823
query32	92	58	63	58
query33	774	321	318	318
query34	989	559	524	524
query35	860	738	725	725
query36	1114	947	958	947
query37	125	70	66	66
query38	4136	3967	3999	3967
query39	1519	1454	1463	1454
query40	209	101	100	100
query41	49	55	48	48
query42	117	100	108	100
query43	529	461	480	461
query44	1315	807	824	807
query45	192	173	173	173
query46	1179	733	735	733
query47	2016	1909	1900	1900
query48	477	389	386	386
query49	1050	393	387	387
query50	835	421	430	421
query51	7516	7323	7287	7287
query52	101	95	89	89
query53	260	201	199	199
query54	1164	467	473	467
query55	78	80	79	79
query56	249	246	246	246
query57	1318	1219	1226	1219
query58	224	214	209	209
query59	3181	2992	3017	2992
query60	281	261	259	259
query61	141	131	116	116
query62	881	689	702	689
query63	221	197	194	194
query64	4227	671	685	671
query65	3432	3305	3319	3305
query66	792	296	306	296
query67	16126	15639	15419	15419
query68	4563	578	573	573
query69	440	265	268	265
query70	1155	1090	1104	1090
query71	339	255	255	255
query72	6283	4058	4002	4002
query73	763	350	354	350
query74	10515	9321	9172	9172
query75	3360	2619	2656	2619
query76	2745	1049	1153	1049
query77	395	282	278	278
query78	10574	9726	9603	9603
query79	1903	598	597	597
query80	1124	423	442	423
query81	551	219	223	219
query82	608	90	90	90
query83	228	146	152	146
query84	236	78	85	78
query85	1315	302	289	289
query86	437	281	270	270
query87	4387	4286	4237	4237
query88	3579	2430	2401	2401
query89	406	294	292	292
query90	1908	183	182	182
query91	199	153	148	148
query92	71	54	54	54
query93	1838	552	554	552
query94	844	300	307	300
query95	355	255	260	255
query96	620	277	289	277
query97	3328	3112	3151	3112
query98	215	200	198	198
query99	1529	1285	1307	1285
Total cold run time: 300474 ms
Total hot run time: 192484 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.22 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 423ea89dceb86fa6f45896b61f92af95e442f4ea, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.06	0.07
query4	1.62	0.11	0.11
query5	0.52	0.53	0.50
query6	1.15	0.74	0.75
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.56	0.51	0.50
query10	0.57	0.55	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.10
query13	0.61	0.62	0.58
query14	0.79	0.82	0.82
query15	0.85	0.84	0.83
query16	0.38	0.39	0.40
query17	1.05	1.05	1.05
query18	0.23	0.23	0.23
query19	1.84	1.83	1.87
query20	0.02	0.01	0.01
query21	15.37	0.59	0.59
query22	2.65	1.92	1.53
query23	17.08	1.05	0.80
query24	3.66	1.46	1.18
query25	0.13	0.05	0.09
query26	0.64	0.13	0.14
query27	0.04	0.05	0.04
query28	9.49	0.52	0.46
query29	12.59	3.16	3.18
query30	0.24	0.06	0.06
query31	2.85	0.38	0.39
query32	3.23	0.47	0.46
query33	3.01	3.01	3.05
query34	17.03	4.57	4.58
query35	4.57	4.54	4.53
query36	0.64	0.51	0.47
query37	0.09	0.07	0.06
query38	0.04	0.03	0.04
query39	0.04	0.02	0.02
query40	0.15	0.12	0.12
query41	0.07	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.56 s
Total hot run time: 30.22 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants