Skip to content

[fix](virtual slot) TimeV2 type should not be used as virtual slot ref #54223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 3 commits into from
Aug 4, 2025
Merged

[fix](virtual slot) TimeV2 type should not be used as virtual slot ref #54223

yiguolei merged 3 commits into from
Aug 4, 2025

Conversation

@zhiqiang-hhhh
Copy link
Contributor

@zhiqiang-hhhh zhiqiang-hhhh commented Aug 1, 2025

What problem does this PR solve?

Related PR: #52701

TimeV2 is a runtime type, it can not be used as VirtualSlotRef.

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Aug 1, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zhiqiang-hhhh
Copy link
Contributor Author

zhiqiang-hhhh commented Aug 1, 2025

run buildall

yiguolei
yiguolei previously approved these changes Aug 1, 2025
View reviewed changes
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 1, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Aug 1, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 1, 2025

PR approved by anyone and no changes requested.

morrySnow
morrySnow reviewed Aug 1, 2025
View reviewed changes
...src/main/java/org/apache/doris/nereids/rules/rewrite/PushDownVirtualColumnsIntoOlapScan.java Outdated

// Supported primitive types by TabletColumn::get_field_length_by_type
// Based on the C++ implementation, these are the types that have defined field lengths
private static final ImmutableSet<TPrimitiveType> SUPPORTED_VIRTUAL_COLUMN_TYPES = ImmutableSet.of(
Copy link
Contributor

@morrySnow morrySnow Aug 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use nereids' type, do not use TPrimitiveType

Copy link
Contributor Author

@zhiqiang-hhhh zhiqiang-hhhh Aug 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use nereids' type, do not use TPrimitiveType

done

@doris-robot
Copy link

doris-robot commented Aug 1, 2025

TPC-H: Total hot run time: 34064 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 87ce38556f788e46234270b323177a7ff19644ed, data reload: false

------ Round 1 ----------------------------------
q1	17674	5808	5400	5400
q2	1947	295	191	191
q3	10321	1269	685	685
q4	10290	1013	522	522
q5	9460	2374	2346	2346
q6	233	168	134	134
q7	873	769	617	617
q8	9299	1309	994	994
q9	6883	5123	5219	5123
q10	6899	2372	2010	2010
q11	469	292	271	271
q12	358	361	231	231
q13	17785	3485	3044	3044
q14	255	257	221	221
q15	527	469	470	469
q16	414	432	372	372
q17	561	845	344	344
q18	7362	7078	7075	7075
q19	1241	987	537	537
q20	316	315	210	210
q21	3481	2449	2273	2273
q22	1060	1091	995	995
Total cold run time: 107708 ms
Total hot run time: 34064 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5981	5796	5977	5796
q2	234	311	213	213
q3	2082	2585	2231	2231
q4	1303	1689	1296	1296
q5	4424	4521	4133	4133
q6	214	190	151	151
q7	2053	2104	1775	1775
q8	2515	2471	2506	2471
q9	7389	7334	7156	7156
q10	3184	3478	2880	2880
q11	554	563	509	509
q12	690	806	622	622
q13	3363	3725	3221	3221
q14	297	312	303	303
q15	515	475	470	470
q16	441	483	442	442
q17	1194	1566	1441	1441
q18	8023	7954	7709	7709
q19	14166	910	1028	910
q20	1905	1961	1919	1919
q21	14890	4201	4295	4201
q22	1063	1054	1046	1046
Total cold run time: 76480 ms
Total hot run time: 50895 ms

@doris-robot
Copy link

doris-robot commented Aug 1, 2025

TPC-DS: Total hot run time: 172737 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 87ce38556f788e46234270b323177a7ff19644ed, data reload: false

============================================
query1	1078	392	388	388
query2	6517	1879	1659	1659
query3	6741	222	218	218
query4	26995	23803	23045	23045
query5	4375	646	547	547
query6	341	238	228	228
query7	4641	526	304	304
query8	279	259	241	241
query9	8660	3183	3134	3134
query10	465	371	292	292
query11	15810	15095	14725	14725
query12	184	140	130	130
query13	1671	566	428	428
query14	9451	7714	7846	7714
query15	218	188	172	172
query16	7802	670	471	471
query17	1635	789	649	649
query18	2051	472	333	333
query19	310	221	203	203
query20	157	142	143	142
query21	220	135	106	106
query22	3941	4028	4139	4028
query23	34414	34455	34351	34351
query24	7598	2454	2489	2454
query25	545	506	447	447
query26	732	288	171	171
query27	2579	522	354	354
query28	3801	2347	2322	2322
query29	700	607	491	491
query30	295	243	217	217
query31	878	816	713	713
query32	90	84	80	80
query33	574	431	414	414
query34	819	854	528	528
query35	786	829	769	769
query36	1005	1005	959	959
query37	133	115	91	91
query38	3965	4043	3954	3954
query39	1439	1375	1356	1356
query40	237	146	133	133
query41	64	58	57	57
query42	144	125	127	125
query43	516	536	488	488
query44	1481	892	882	882
query45	198	186	182	182
query46	1013	1106	697	697
query47	1797	1806	1746	1746
query48	402	433	310	310
query49	662	500	422	422
query50	655	722	419	419
query51	4181	4189	4188	4188
query52	130	134	128	128
query53	267	307	223	223
query54	656	647	578	578
query55	91	93	92	92
query56	370	363	347	347
query57	1209	1212	1119	1119
query58	342	341	329	329
query59	2687	2659	2621	2621
query60	412	411	410	410
query61	121	119	120	119
query62	826	751	679	679
query63	260	219	225	219
query64	2839	1069	763	763
query65	4236	4104	4108	4104
query66	909	462	347	347
query67	query68	18437	599	608	599
query69	1009	308	292	292
query70	1457	1139	1078	1078
query71	717	339	326	326
query72	9195	2266	2311	2266
query73	3766	643	361	361
query74	9122	9033	8828	8828
query75	7649	3112	2710	2710
query76	8925	1250	804	804
query77	1146	418	335	335
query78	9549	11291	9197	9197
query79	14785	621	600	600
query80	2186	550	482	482
query81	587	286	228	228
query82	502	149	121	121
query83	411	283	265	265
query84	305	98	86	86
query85	909	365	405	365
query86	363	318	315	315
query87	4338	4231	4171	4171
query88	5551	2265	2245	2245
query89	489	373	323	323
query90	2550	233	239	233
query91	147	139	113	113
query92	88	74	70	70
query93	6418	984	661	661
query94	1153	394	284	284
query95	462	328	328	328
query96	511	581	358	358
query97	2668	2748	2632	2632
query98	258	236	223	223
query99	1494	1481	1283	1283
Total cold run time: 307747 ms
Total hot run time: 172737 ms

@doris-robot
Copy link

doris-robot commented Aug 1, 2025

ClickBench: Total hot run time: 32.46 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 87ce38556f788e46234270b323177a7ff19644ed, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.05
query3	0.24	0.07	0.06
query4	1.61	0.10	0.10
query5	0.44	0.40	0.42
query6	1.14	0.66	0.66
query7	0.03	0.02	0.02
query8	0.05	0.03	0.04
query9	0.56	0.47	0.47
query10	0.53	0.54	0.52
query11	0.16	0.11	0.11
query12	0.14	0.11	0.11
query13	0.64	0.65	0.64
query14	0.96	1.02	1.18
query15	0.89	0.86	0.90
query16	0.38	0.40	0.39
query17	1.10	1.04	1.07
query18	0.23	0.21	0.20
query19	2.06	1.83	1.92
query20	0.02	0.02	0.02
query21	15.37	0.84	0.56
query22	0.76	1.13	0.67
query23	15.01	1.11	0.62
query24	7.30	0.56	0.34
query25	0.50	0.07	0.07
query26	0.58	0.16	0.14
query27	0.07	0.05	0.05
query28	9.66	0.86	0.44
query29	12.57	3.81	3.34
query30	3.04	2.99	3.04
query31	2.81	0.56	0.40
query32	3.26	0.55	0.49
query33	3.06	3.08	3.17
query34	15.97	5.36	4.91
query35	4.86	4.96	4.99
query36	0.70	0.52	0.50
query37	0.10	0.07	0.07
query38	0.06	0.05	0.05
query39	0.04	0.03	0.02
query40	0.17	0.14	0.13
query41	0.09	0.04	0.03
query42	0.03	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 107.35 s
Total hot run time: 32.46 s

@zhiqiang-hhhh
Copy link
Contributor Author

zhiqiang-hhhh commented Aug 1, 2025

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Aug 1, 2025
@zhiqiang-hhhh
Copy link
Contributor Author

zhiqiang-hhhh commented Aug 2, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 2, 2025

TPC-H: Total hot run time: 33528 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9cc0d6f157f0c07c2fa2f3b0f85c5870cda2f144, data reload: false

------ Round 1 ----------------------------------
q1	17613	5324	5153	5153
q2	1918	281	192	192
q3	10335	1300	707	707
q4	10213	1017	516	516
q5	7519	2363	2232	2232
q6	174	162	131	131
q7	895	769	623	623
q8	9339	1326	1088	1088
q9	7103	5075	5158	5075
q10	6889	2330	1945	1945
q11	453	271	258	258
q12	345	381	229	229
q13	17766	3406	3025	3025
q14	231	230	226	226
q15	533	470	461	461
q16	418	441	382	382
q17	557	810	347	347
q18	7295	7051	6937	6937
q19	1234	980	553	553
q20	313	310	214	214
q21	3528	3014	2282	2282
q22	1021	1093	952	952
Total cold run time: 105692 ms
Total hot run time: 33528 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5321	5321	5328	5321
q2	234	304	213	213
q3	2061	2563	2198	2198
q4	1310	1712	1291	1291
q5	4097	4214	4462	4214
q6	217	176	130	130
q7	2107	1913	1735	1735
q8	2550	2437	2485	2437
q9	7311	7030	7490	7030
q10	3213	3370	2900	2900
q11	543	497	489	489
q12	733	744	610	610
q13	3398	3811	3230	3230
q14	273	294	283	283
q15	518	466	468	466
q16	449	492	441	441
q17	1196	1437	1469	1437
q18	10037	7577	7648	7577
q19	11274	867	861	861
q20	1963	2138	1853	1853
q21	15005	4235	4203	4203
q22	1025	999	987	987
Total cold run time: 74835 ms
Total hot run time: 49906 ms

@doris-robot
Copy link

doris-robot commented Aug 2, 2025

TPC-DS: Total hot run time: 170267 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9cc0d6f157f0c07c2fa2f3b0f85c5870cda2f144, data reload: false

============================================
query1	1028	365	402	365
query2	6604	1894	1752	1752
query3	6743	219	220	219
query4	26982	23633	23007	23007
query5	4374	627	511	511
query6	319	230	220	220
query7	4643	517	304	304
query8	283	259	229	229
query9	8612	2958	2895	2895
query10	498	354	301	301
query11	15544	14999	14784	14784
query12	182	138	133	133
query13	1670	566	428	428
query14	8537	5807	5862	5807
query15	205	193	165	165
query16	7143	685	462	462
query17	1224	778	655	655
query18	2127	464	328	328
query19	229	225	201	201
query20	147	137	141	137
query21	220	130	124	124
query22	3974	4029	3828	3828
query23	34383	34178	34313	34178
query24	7158	2376	2420	2376
query25	517	522	430	430
query26	713	289	162	162
query27	2375	511	372	372
query28	3013	2357	2313	2313
query29	655	611	502	502
query30	288	236	207	207
query31	871	824	765	765
query32	93	76	86	76
query33	531	438	374	374
query34	796	848	521	521
query35	807	831	771	771
query36	1016	1061	927	927
query37	128	106	89	89
query38	3971	3943	3957	3943
query39	1434	1529	1351	1351
query40	238	145	133	133
query41	62	59	54	54
query42	141	125	128	125
query43	529	512	485	485
query44	1394	869	850	850
query45	199	187	180	180
query46	956	1049	664	664
query47	1764	1835	1747	1747
query48	419	453	316	316
query49	685	504	417	417
query50	637	707	420	420
query51	4152	4155	4171	4155
query52	133	132	125	125
query53	255	288	210	210
query54	658	662	556	556
query55	91	89	93	89
query56	364	355	356	355
query57	1188	1224	1134	1134
query58	339	328	338	328
query59	2594	2597	2559	2559
query60	404	396	395	395
query61	125	120	121	120
query62	764	758	640	640
query63	254	215	218	215
query64	2355	1103	759	759
query65	4204	4112	4161	4112
query66	1002	449	335	335
query67	query68	17545	633	591	591
query69	1006	327	290	290
query70	1474	1106	1130	1106
query71	720	335	326	326
query72	9129	2267	2315	2267
query73	3771	643	357	357
query74	9068	8951	8897	8897
query75	7606	3158	2725	2725
query76	8843	1225	785	785
query77	1155	418	342	342
query78	9573	10222	9557	9557
query79	14564	596	584	584
query80	1704	552	573	552
query81	566	274	221	221
query82	499	152	119	119
query83	368	293	276	276
query84	308	102	81	81
query85	1052	362	333	333
query86	368	318	304	304
query87	4401	4260	4207	4207
query88	5715	2314	2255	2255
query89	473	358	318	318
query90	2586	228	235	228
query91	146	139	111	111
query92	89	72	67	67
query93	6228	968	663	663
query94	1114	385	281	281
query95	425	323	323	323
query96	500	587	295	295
query97	2693	2756	2608	2608
query98	254	240	227	227
query99	1448	1432	1281	1281
Total cold run time: 301334 ms
Total hot run time: 170267 ms

@doris-robot
Copy link

doris-robot commented Aug 2, 2025

ClickBench: Total hot run time: 33.04 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9cc0d6f157f0c07c2fa2f3b0f85c5870cda2f144, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.05	0.04
query3	0.24	0.07	0.07
query4	1.64	0.11	0.11
query5	0.44	0.42	0.43
query6	1.15	0.68	0.67
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.55	0.49	0.47
query10	0.54	0.52	0.52
query11	0.15	0.11	0.10
query12	0.14	0.11	0.11
query13	0.65	0.66	0.63
query14	0.93	1.06	1.18
query15	0.88	0.88	0.89
query16	0.40	0.39	0.38
query17	1.12	1.07	1.02
query18	0.21	0.20	0.21
query19	1.90	1.84	1.78
query20	0.01	0.01	0.02
query21	15.39	0.85	0.54
query22	0.75	1.13	0.72
query23	15.00	1.19	0.65
query24	6.88	1.38	0.71
query25	0.61	0.21	0.07
query26	0.60	0.14	0.13
query27	0.05	0.05	0.05
query28	9.69	0.85	0.44
query29	12.56	3.88	3.40
query30	3.06	3.00	2.98
query31	2.82	0.59	0.40
query32	3.26	0.58	0.49
query33	3.04	3.15	3.25
query34	16.14	5.39	4.93
query35	4.91	4.96	4.99
query36	0.71	0.53	0.51
query37	0.10	0.07	0.07
query38	0.06	0.05	0.04
query39	0.04	0.02	0.03
query40	0.16	0.14	0.13
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 107.13 s
Total hot run time: 33.04 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 2, 2025

FE UT Coverage Report

Increment line coverage 75.00% (12/16) 🎉
Increment coverage report
Complete coverage report

@github-actions
Copy link
Contributor

github-actions bot commented Aug 4, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 4, 2025
airborne12
airborne12 approved these changes Aug 4, 2025
View reviewed changes
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei yiguolei merged commit 4af6e74 into apache:master Aug 4, 2025
28 of 30 checks passed
@zhiqiang-hhhh zhiqiang-hhhh deleted the fix-time-type-virtual-slot branch August 4, 2025 10:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@airborne12 airborne12 airborne12 approved these changes

@yiguolei yiguolei yiguolei approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@zhiqiang-hhhh @Thearas @doris-robot @hello-stephen @airborne12 @yiguolei @morrySnow