branch-3.1: [Enhancement] reject schema change on hidden columns #53376 #55385

codeDing18 · 2025-08-27T14:08:44Z

picked from #53376

Thearas · 2025-08-27T14:08:52Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

codeDing18 · 2025-08-27T14:09:00Z

run buildall

codeDing18 · 2025-08-28T15:02:41Z

run buildall

codeDing18 · 2025-08-29T01:26:16Z

run buildall

doris-robot · 2025-08-29T02:03:11Z

TPC-H: Total hot run time: 32564 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 540f0d8417a9d6cf80373f513aacacfa33e3b527, data reload: false

------ Round 1 ----------------------------------
q1	17600	5489	5395	5395
q2	2025	399	267	267
q3	12170	1240	759	759
q4	10527	909	465	465
q5	9341	2381	2109	2109
q6	179	162	133	133
q7	885	749	609	609
q8	9349	1397	1134	1134
q9	5227	4955	4967	4955
q10	6759	2258	1830	1830
q11	487	283	257	257
q12	325	355	216	216
q13	17769	3609	3014	3014
q14	218	239	212	212
q15	533	477	461	461
q16	429	429	367	367
q17	590	861	376	376
q18	6638	6364	6432	6364
q19	1296	942	539	539
q20	324	332	200	200
q21	2795	2122	1941	1941
q22	1047	1028	961	961
Total cold run time: 106513 ms
Total hot run time: 32564 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5523	5460	5530	5460
q2	226	342	233	233
q3	2240	2616	2311	2311
q4	1414	1840	1361	1361
q5	4475	5053	5066	5053
q6	176	171	132	132
q7	2082	1973	1861	1861
q8	2663	2892	2755	2755
q9	7379	7296	7313	7296
q10	3070	3317	2657	2657
q11	564	500	497	497
q12	668	790	624	624
q13	3397	3770	3210	3210
q14	280	313	277	277
q15	522	476	476	476
q16	428	485	450	450
q17	1228	1732	1257	1257
q18	7597	7296	7370	7296
q19	778	1118	998	998
q20	2027	2044	1887	1887
q21	5441	4971	4569	4569
q22	1119	1067	986	986
Total cold run time: 53297 ms
Total hot run time: 51646 ms

doris-robot · 2025-08-29T02:15:21Z

TPC-DS: Total hot run time: 192634 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 540f0d8417a9d6cf80373f513aacacfa33e3b527, data reload: false

query1	944	405	422	405
query2	6209	1908	1832	1832
query3	8689	203	201	201
query4	33208	23823	23562	23562
query5	3861	592	466	466
query6	283	193	184	184
query7	4209	481	311	311
query8	310	246	247	246
query9	9402	2602	2611	2602
query10	476	324	255	255
query11	18026	15399	15150	15150
query12	160	108	104	104
query13	1551	540	408	408
query14	10022	6948	7028	6948
query15	241	191	186	186
query16	8123	689	482	482
query17	1538	759	596	596
query18	2095	418	333	333
query19	221	182	155	155
query20	127	118	122	118
query21	197	130	105	105
query22	4579	4685	4440	4440
query23	35159	34566	34201	34201
query24	7290	2721	2734	2721
query25	530	520	429	429
query26	1086	295	178	178
query27	2062	473	354	354
query28	5258	2231	2187	2187
query29	780	591	461	461
query30	248	194	166	166
query31	988	910	859	859
query32	95	59	61	59
query33	553	390	325	325
query34	740	861	542	542
query35	762	812	722	722
query36	1026	1063	931	931
query37	103	100	73	73
query38	4025	3993	4026	3993
query39	1530	1480	1454	1454
query40	209	121	114	114
query41	55	54	55	54
query42	128	113	105	105
query43	513	523	496	496
query44	1363	825	828	825
query45	190	181	178	178
query46	907	1063	679	679
query47	1970	2043	1990	1990
query48	413	431	369	369
query49	773	514	433	433
query50	684	695	437	437
query51	7396	7399	7272	7272
query52	104	107	91	91
query53	232	255	197	197
query54	549	572	499	499
query55	84	86	81	81
query56	271	270	262	262
query57	1296	1270	1200	1200
query58	253	222	234	222
query59	3074	3119	3120	3119
query60	294	289	269	269
query61	131	126	113	113
query62	809	747	695	695
query63	235	204	196	196
query64	4452	1007	654	654
query65	3356	3284	3340	3284
query66	908	425	320	320
query67	16551	15780	15654	15654
query68	7557	824	537	537
query69	493	314	268	268
query70	1219	1154	1095	1095
query71	391	310	259	259
query72	5777	3730	3844	3730
query73	634	750	354	354
query74	10520	9370	8996	8996
query75	3246	3163	2673	2673
query76	3229	1192	788	788
query77	611	378	296	296
query78	10773	10352	9612	9612
query79	3111	883	605	605
query80	615	524	433	433
query81	502	262	218	218
query82	550	120	88	88
query83	158	161	146	146
query84	243	95	82	82
query85	781	357	296	296
query86	360	326	311	311
query87	4353	4338	4209	4209
query88	5045	2376	2384	2376
query89	405	335	297	297
query90	1776	194	207	194
query91	138	138	111	111
query92	63	56	51	51
query93	1030	835	549	549
query94	698	403	320	320
query95	345	285	290	285
query96	489	610	286	286
query97	3221	3328	3133	3133
query98	227	208	210	208
query99	1514	1423	1276	1276
Total cold run time: 293980 ms
Total hot run time: 192634 ms

doris-robot · 2025-08-29T02:20:34Z

ClickBench: Total hot run time: 28.3 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 540f0d8417a9d6cf80373f513aacacfa33e3b527, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.22	0.07	0.06
query4	1.65	0.11	0.11
query5	0.51	0.49	0.50
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.03
query9	0.56	0.51	0.51
query10	0.55	0.57	0.55
query11	0.13	0.10	0.11
query12	0.15	0.11	0.11
query13	0.61	0.59	0.60
query14	0.78	0.81	0.78
query15	0.85	0.83	0.83
query16	0.38	0.39	0.41
query17	1.07	1.09	1.05
query18	0.24	0.23	0.23
query19	1.87	1.87	1.84
query20	0.02	0.00	0.01
query21	15.38	0.94	0.59
query22	0.74	0.74	0.63
query23	15.20	1.37	0.58
query24	3.01	1.67	0.27
query25	0.13	0.05	0.06
query26	0.26	0.15	0.13
query27	0.05	0.04	0.04
query28	12.85	1.14	0.43
query29	12.59	3.90	3.29
query30	0.25	0.09	0.06
query31	2.83	0.59	0.38
query32	3.22	0.55	0.47
query33	2.99	3.07	3.01
query34	16.48	5.15	4.57
query35	4.65	4.55	4.53
query36	0.63	0.49	0.48
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.16	0.12	0.12
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 102.64 s
Total hot run time: 28.3 s

hello-stephen · 2025-08-29T02:53:11Z

FE UT Coverage Report

Increment line coverage 62.22% (28/45) 🎉
Increment coverage report
Complete coverage report

codeDing18 · 2025-08-29T04:48:48Z

run buildall

codeDing18 · 2025-08-29T09:16:01Z

run buildall

…che#53376) Issue Number: close apache#53019 picked from apache#53376

codeDing18 · 2025-08-30T11:43:10Z

run buildall

doris-robot · 2025-08-30T12:19:12Z

TPC-H: Total hot run time: 32961 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f95d23e564a39fbd9221da2f5576559985928de0, data reload: false

------ Round 1 ----------------------------------
q1	17569	5474	5430	5430
q2	2041	443	293	293
q3	12085	1265	758	758
q4	10523	885	467	467
q5	9638	2415	2164	2164
q6	190	170	138	138
q7	898	748	627	627
q8	9323	1469	1271	1271
q9	5206	4921	4936	4921
q10	6796	2266	1813	1813
q11	503	274	266	266
q12	342	359	213	213
q13	17777	3623	3075	3075
q14	233	221	210	210
q15	530	476	457	457
q16	427	423	386	386
q17	589	887	362	362
q18	6843	6310	6375	6310
q19	1217	957	595	595
q20	333	345	210	210
q21	2871	2285	2022	2022
q22	1049	979	973	973
Total cold run time: 106983 ms
Total hot run time: 32961 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5501	5487	5453	5453
q2	240	334	237	237
q3	2335	2710	2399	2399
q4	1394	1820	1419	1419
q5	4486	5083	5009	5009
q6	190	162	129	129
q7	2065	1953	1843	1843
q8	2660	2822	2722	2722
q9	7220	7261	7338	7261
q10	3030	3330	2770	2770
q11	566	551	510	510
q12	678	814	599	599
q13	3458	3772	3248	3248
q14	289	289	291	289
q15	520	473	462	462
q16	459	500	440	440
q17	1250	1767	1264	1264
q18	7568	7322	7202	7202
q19	834	1013	1162	1013
q20	2053	2073	1916	1916
q21	5414	4892	4629	4629
q22	1052	1010	1006	1006
Total cold run time: 53262 ms
Total hot run time: 51820 ms

doris-robot · 2025-08-30T12:31:21Z

TPC-DS: Total hot run time: 192766 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f95d23e564a39fbd9221da2f5576559985928de0, data reload: false

query1	944	405	431	405
query2	6230	1992	1940	1940
query3	8678	203	205	203
query4	33663	23764	23858	23764
query5	3704	612	467	467
query6	310	202	198	198
query7	4200	519	335	335
query8	307	251	256	251
query9	9576	2599	2591	2591
query10	445	321	254	254
query11	18040	15372	15085	15085
query12	154	104	106	104
query13	1552	552	409	409
query14	8888	7604	7676	7604
query15	249	197	180	180
query16	8051	691	534	534
query17	1566	778	604	604
query18	2157	406	348	348
query19	234	211	183	183
query20	126	126	117	117
query21	216	132	105	105
query22	4523	4485	4397	4397
query23	35066	34284	34110	34110
query24	7432	2726	2762	2726
query25	551	505	445	445
query26	801	301	182	182
query27	2040	513	362	362
query28	4968	2239	2197	2197
query29	744	620	467	467
query30	241	197	170	170
query31	971	945	853	853
query32	99	65	65	65
query33	506	379	323	323
query34	806	873	552	552
query35	805	827	775	775
query36	1011	1067	974	974
query37	105	96	69	69
query38	4133	4094	3965	3965
query39	1523	1478	1494	1478
query40	208	120	110	110
query41	51	54	49	49
query42	140	111	112	111
query43	527	529	504	504
query44	1364	831	826	826
query45	184	187	172	172
query46	950	1106	715	715
query47	1960	1987	1939	1939
query48	418	429	358	358
query49	746	504	423	423
query50	741	720	448	448
query51	7399	7340	7208	7208
query52	104	104	96	96
query53	234	261	195	195
query54	566	559	483	483
query55	86	86	83	83
query56	296	268	261	261
query57	1293	1247	1183	1183
query58	240	230	231	230
query59	3010	3210	3082	3082
query60	306	291	275	275
query61	116	108	111	108
query62	786	765	722	722
query63	254	207	204	204
query64	3775	1037	659	659
query65	3344	3319	3309	3309
query66	787	417	314	314
query67	16283	15854	15491	15491
query68	7673	849	566	566
query69	512	309	269	269
query70	1208	1135	1069	1069
query71	376	303	295	295
query72	5789	3759	2496	2496
query73	664	781	343	343
query74	10272	9349	9427	9349
query75	3231	3160	2669	2669
query76	3025	1190	810	810
query77	518	375	283	283
query78	10400	10388	9657	9657
query79	3791	866	605	605
query80	705	538	439	439
query81	509	264	222	222
query82	590	119	87	87
query83	161	162	143	143
query84	251	105	87	87
query85	787	372	288	288
query86	405	307	300	300
query87	4418	4361	4273	4273
query88	5158	2433	2389	2389
query89	417	342	301	301
query90	1790	191	193	191
query91	139	143	111	111
query92	73	58	54	54
query93	2164	905	557	557
query94	696	417	317	317
query95	349	278	276	276
query96	496	623	292	292
query97	3183	3295	3168	3168
query98	243	210	203	203
query99	1613	1435	1308	1308
Total cold run time: 293389 ms
Total hot run time: 192766 ms

doris-robot · 2025-08-30T12:36:48Z

ClickBench: Total hot run time: 29.05 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f95d23e564a39fbd9221da2f5576559985928de0, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.62	0.11	0.10
query5	0.51	0.50	0.51
query6	1.13	0.73	0.73
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.56	0.52	0.50
query10	0.55	0.54	0.55
query11	0.16	0.10	0.10
query12	0.14	0.11	0.12
query13	0.61	0.60	0.59
query14	0.81	0.79	0.79
query15	0.84	0.85	0.82
query16	0.41	0.40	0.38
query17	1.09	1.05	1.04
query18	0.25	0.22	0.22
query19	1.93	1.83	1.90
query20	0.02	0.01	0.00
query21	15.37	0.91	0.58
query22	0.74	0.75	0.65
query23	15.12	1.49	0.54
query24	3.66	1.13	1.71
query25	0.21	0.14	0.26
query26	0.33	0.15	0.13
query27	0.07	0.04	0.04
query28	13.95	1.03	0.45
query29	12.61	3.92	3.22
query30	0.24	0.09	0.06
query31	2.83	0.61	0.40
query32	3.22	0.55	0.45
query33	2.99	3.03	3.00
query34	16.53	5.15	4.55
query35	4.57	4.52	4.55
query36	0.65	0.51	0.46
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.04	0.02	0.03
query40	0.16	0.13	0.12
query41	0.08	0.02	0.03
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.62 s
Total hot run time: 29.05 s

hello-stephen · 2025-08-30T13:01:00Z

FE UT Coverage Report

Increment line coverage 73.02% (46/63) 🎉
Increment coverage report
Complete coverage report

codeDing18 requested a review from morrySnow as a code owner August 27, 2025 14:08

codeDing18 force-pushed the branch-3.1-me branch from 4961888 to b15e95d Compare August 28, 2025 15:02

codeDing18 force-pushed the branch-3.1-me branch from b15e95d to 540f0d8 Compare August 29, 2025 01:20

codeDing18 force-pushed the branch-3.1-me branch from 540f0d8 to 2fcadbd Compare August 29, 2025 04:48

codeDing18 force-pushed the branch-3.1-me branch from 2fcadbd to a891d3b Compare August 29, 2025 08:07

branch-3.1: [Enhancement] reject schema change on hidden columns (apa…

f95d23e

…che#53376) Issue Number: close apache#53019 picked from apache#53376

codeDing18 force-pushed the branch-3.1-me branch from a891d3b to f95d23e Compare August 30, 2025 11:43

starocean999 approved these changes Sep 1, 2025

View reviewed changes

morrySnow approved these changes Sep 4, 2025

View reviewed changes

morrySnow changed the title ~~branch-3.1: [Enhancement] reject schema change on hidden columns (#53…~~ branch-3.1: [Enhancement] reject schema change on hidden columns #53376 Sep 4, 2025

morrySnow merged commit 2623282 into apache:branch-3.1 Sep 4, 2025
22 checks passed

morrySnow mentioned this pull request Sep 22, 2025

Release Note 3.1.1 #56289

Closed

branch-3.1: [Enhancement] reject schema change on hidden columns #53376 #55385

branch-3.1: [Enhancement] reject schema change on hidden columns #53376 #55385

Uh oh!

Conversation

codeDing18 commented Aug 27, 2025 • edited by morrySnow Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Thearas commented Aug 27, 2025

Uh oh!

codeDing18 commented Aug 27, 2025

Uh oh!

codeDing18 commented Aug 28, 2025

Uh oh!

codeDing18 commented Aug 29, 2025

Uh oh!

doris-robot commented Aug 29, 2025

Uh oh!

doris-robot commented Aug 29, 2025

Uh oh!

doris-robot commented Aug 29, 2025

Uh oh!

hello-stephen commented Aug 29, 2025

FE UT Coverage Report

Uh oh!

codeDing18 commented Aug 29, 2025

Uh oh!

codeDing18 commented Aug 29, 2025

Uh oh!

codeDing18 commented Aug 30, 2025

Uh oh!

doris-robot commented Aug 30, 2025

Uh oh!

doris-robot commented Aug 30, 2025

Uh oh!

doris-robot commented Aug 30, 2025

Uh oh!

hello-stephen commented Aug 30, 2025

FE UT Coverage Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

codeDing18 commented Aug 27, 2025 •

edited by morrySnow

Loading