[enhance](variant)enhance max_sparse_column_statistics_size for variant #55124

amorynan · 2025-08-21T09:06:39Z

What problem does this PR solve?

delete be config for max_sparse_column_statistics_size
and add it for variant properties
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-08-21T09:06:50Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

amorynan · 2025-08-21T09:21:30Z

run buildall

hello-stephen · 2025-08-21T09:36:43Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.60% (1412/1689)
Line Coverage	66.86% (24809/37105)
Region Coverage	67.82% (12363/18230)
Branch Coverage	57.62% (6528/11330)

hello-stephen · 2025-08-21T10:24:19Z

FE UT Coverage Report

Increment line coverage 34.04% (16/47) 🎉
Increment coverage report
Complete coverage report

amorynan · 2025-08-21T10:54:18Z

run buildall

doris-robot · 2025-08-21T11:11:45Z

TPC-H: Total hot run time: 33610 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4eb037fcb17ae74eea3db371d8f09679f088f1bf, data reload: false

------ Round 1 ----------------------------------
q1	17597	5228	5046	5046
q2	1928	284	177	177
q3	10323	1251	693	693
q4	10210	1004	502	502
q5	7495	2366	2338	2338
q6	174	158	136	136
q7	887	738	625	625
q8	9291	1313	1071	1071
q9	6983	5102	5102	5102
q10	6870	2392	1963	1963
q11	473	293	269	269
q12	349	346	213	213
q13	17754	3648	3014	3014
q14	231	256	218	218
q15	545	483	478	478
q16	437	423	368	368
q17	588	856	354	354
q18	7403	7071	7003	7003
q19	880	942	549	549
q20	335	331	220	220
q21	3635	2555	2282	2282
q22	1078	1021	989	989
Total cold run time: 105466 ms
Total hot run time: 33610 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5081	5121	5104	5104
q2	244	319	222	222
q3	2150	2713	2282	2282
q4	1333	1769	1347	1347
q5	4216	4369	4553	4369
q6	220	175	149	149
q7	2009	1919	1812	1812
q8	2695	2521	2580	2521
q9	7557	7366	7201	7201
q10	3105	3287	2862	2862
q11	618	527	496	496
q12	730	763	622	622
q13	3532	3939	3344	3344
q14	294	335	288	288
q15	542	479	464	464
q16	510	547	456	456
q17	1239	1606	1419	1419
q18	7880	7840	7495	7495
q19	814	849	1125	849
q20	1986	2103	1990	1990
q21	5032	4303	4311	4303
q22	1105	1055	1049	1049
Total cold run time: 52892 ms
Total hot run time: 50644 ms

hello-stephen · 2025-08-21T11:15:32Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.60% (1412/1689)
Line Coverage	66.84% (24800/37105)
Region Coverage	67.80% (12360/18230)
Branch Coverage	57.54% (6519/11330)

doris-robot · 2025-08-21T11:23:16Z

TPC-DS: Total hot run time: 183771 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4eb037fcb17ae74eea3db371d8f09679f088f1bf, data reload: false

query1	1007	381	417	381
query2	6516	1746	1694	1694
query3	6744	217	224	217
query4	26023	23164	22779	22779
query5	4351	628	518	518
query6	323	212	203	203
query7	4647	494	300	300
query8	285	229	216	216
query9	8519	2854	2853	2853
query10	486	333	282	282
query11	15826	14991	14801	14801
query12	166	116	114	114
query13	1661	581	422	422
query14	9431	5716	5795	5716
query15	213	203	165	165
query16	7501	639	455	455
query17	1191	705	606	606
query18	2058	406	307	307
query19	177	187	156	156
query20	123	120	112	112
query21	209	119	104	104
query22	4094	4206	3953	3953
query23	33958	32861	32894	32861
query24	8107	2325	2337	2325
query25	532	480	403	403
query26	1236	271	163	163
query27	2745	502	351	351
query28	4380	2246	2224	2224
query29	724	543	482	482
query30	280	218	188	188
query31	894	798	699	699
query32	84	76	74	74
query33	547	388	336	336
query34	831	835	502	502
query35	779	825	739	739
query36	979	1002	913	913
query37	116	103	88	88
query38	4063	4060	3981	3981
query39	1457	1433	1396	1396
query40	223	121	120	120
query41	59	55	56	55
query42	114	106	114	106
query43	486	499	471	471
query44	1308	839	834	834
query45	179	167	166	166
query46	857	1008	631	631
query47	1735	1782	1697	1697
query48	383	406	309	309
query49	715	486	387	387
query50	630	690	394	394
query51	4236	4113	4033	4033
query52	111	113	97	97
query53	234	253	208	208
query54	586	586	531	531
query55	107	88	87	87
query56	315	306	290	290
query57	1165	1179	1107	1107
query58	280	276	266	266
query59	2637	2731	2638	2638
query60	340	340	334	334
query61	132	124	120	120
query62	798	723	654	654
query63	227	186	196	186
query64	4259	1008	750	750
query65	4288	4109	4241	4109
query66	1082	417	321	321
query67	15532	15288	14990	14990
query68	7825	918	578	578
query69	476	316	287	287
query70	1181	1137	1133	1133
query71	541	333	309	309
query72	5524	4767	5013	4767
query73	709	670	354	354
query74	9369	9248	8974	8974
query75	3858	3055	2578	2578
query76	3593	1137	730	730
query77	775	399	320	320
query78	9658	9642	8786	8786
query79	2393	825	592	592
query80	621	549	481	481
query81	512	248	296	248
query82	335	133	107	107
query83	248	264	234	234
query84	249	99	78	78
query85	833	367	335	335
query86	393	310	293	293
query87	4233	4314	4183	4183
query88	3769	2211	2191	2191
query89	395	318	280	280
query90	1881	223	212	212
query91	149	138	109	109
query92	97	70	66	66
query93	1929	982	648	648
query94	634	400	317	317
query95	402	320	308	308
query96	488	583	272	272
query97	2623	2689	2537	2537
query98	255	212	209	209
query99	1305	1391	1285	1285
Total cold run time: 273534 ms
Total hot run time: 183771 ms

doris-robot · 2025-08-21T11:28:50Z

ClickBench: Total hot run time: 33.23 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4eb037fcb17ae74eea3db371d8f09679f088f1bf, data reload: false

query1	0.04	0.03	0.03
query2	0.11	0.06	0.05
query3	0.29	0.06	0.06
query4	1.60	0.08	0.08
query5	0.42	0.42	0.41
query6	1.18	0.64	0.64
query7	0.03	0.01	0.01
query8	0.06	0.05	0.05
query9	0.63	0.52	0.53
query10	0.58	0.57	0.59
query11	0.26	0.12	0.12
query12	0.25	0.13	0.13
query13	0.64	0.62	0.64
query14	0.83	0.89	0.82
query15	0.97	0.87	0.88
query16	0.37	0.37	0.38
query17	1.05	1.04	1.04
query18	0.22	0.21	0.22
query19	2.01	1.82	1.84
query20	0.02	0.01	0.01
query21	15.38	1.02	0.71
query22	0.94	1.08	0.95
query23	14.69	1.53	0.80
query24	5.31	0.59	0.32
query25	0.17	0.10	0.09
query26	0.55	0.21	0.17
query27	0.09	0.09	0.08
query28	11.06	1.14	0.56
query29	12.57	4.05	3.32
query30	3.08	3.03	2.96
query31	2.83	0.61	0.43
query32	3.22	0.59	0.51
query33	3.09	3.04	3.18
query34	17.12	5.53	4.76
query35	4.82	4.89	4.91
query36	0.64	0.52	0.51
query37	0.20	0.18	0.19
query38	0.18	0.16	0.17
query39	0.05	0.06	0.04
query40	0.20	0.17	0.17
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.05	0.05
Total cold run time: 107.96 s
Total hot run time: 33.23 s

be/src/olap/rowset/segment_v2/segment.cpp

be/src/olap/rowset/segment_v2/variant/variant_column_reader.cpp

be/src/olap/rowset/segment_v2/variant_stats_calculator.cpp

fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java

fe/fe-core/src/main/java/org/apache/doris/nereids/types/VariantType.java

...st/java/org/apache/doris/nereids/trees/expressions/functions/ComputeSignatureHelperTest.java

gensrc/proto/olap_file.proto

hello-stephen · 2025-08-21T12:15:22Z

FE UT Coverage Report

Increment line coverage 34.04% (16/47) 🎉
Increment coverage report
Complete coverage report

fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java

regression-test/suites/fault_injection_p0/test_variant_compaction_with_sparse_limit.groovy

regression-test/suites/variant_p0/predefine/test_variant_compaction_with_sparse_limit.groovy

eldenmoon · 2025-08-25T08:19:27Z

run buildall

amorynan · 2025-08-25T08:40:18Z

run buildall

doris-robot · 2025-08-25T08:59:38Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.62% (1419/1697)
Line Coverage	66.97% (25239/37687)
Region Coverage	68.09% (12584/18482)
Branch Coverage	57.90% (6668/11516)

hello-stephen · 2025-08-25T09:42:47Z

FE UT Coverage Report

Increment line coverage 35.90% (14/39) 🎉
Increment coverage report
Complete coverage report

doris-robot · 2025-08-25T10:28:31Z

TPC-H: Total hot run time: 33991 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 96ba2ec80545d90472fcccce3633bf8a3386e610, data reload: false

------ Round 1 ----------------------------------
q1	17033	5213	5086	5086
q2	1913	289	184	184
q3	10095	1291	700	700
q4	10204	1033	506	506
q5	7567	2460	2355	2355
q6	190	167	133	133
q7	895	752	610	610
q8	9307	1352	1047	1047
q9	7011	5234	5168	5168
q10	6962	2388	1960	1960
q11	497	289	278	278
q12	349	346	216	216
q13	17799	3671	3018	3018
q14	229	239	221	221
q15	563	496	480	480
q16	435	422	371	371
q17	613	864	370	370
q18	8103	7217	7193	7193
q19	1095	960	555	555
q20	343	336	217	217
q21	3759	3220	2341	2341
q22	1042	1044	982	982
Total cold run time: 106004 ms
Total hot run time: 33991 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5186	5127	5151	5127
q2	243	327	229	229
q3	2142	2718	2322	2322
q4	1353	1768	1305	1305
q5	4234	4426	4598	4426
q6	221	191	136	136
q7	2043	1962	1828	1828
q8	2617	2572	2471	2471
q9	7449	7367	7321	7321
q10	3177	3316	2915	2915
q11	604	539	498	498
q12	703	763	667	667
q13	3552	3948	3187	3187
q14	292	313	365	313
q15	613	486	483	483
q16	434	512	439	439
q17	1187	1592	1387	1387
q18	7861	7863	7445	7445
q19	839	854	838	838
q20	1925	1999	1883	1883
q21	4790	4346	4389	4346
q22	1044	1055	1021	1021
Total cold run time: 52509 ms
Total hot run time: 50587 ms

doris-robot · 2025-08-25T10:40:02Z

TPC-DS: Total hot run time: 184188 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 96ba2ec80545d90472fcccce3633bf8a3386e610, data reload: false

query1	984	372	409	372
query2	6522	1845	1937	1845
query3	6744	225	212	212
query4	26322	23675	22823	22823
query5	4331	604	459	459
query6	293	208	206	206
query7	4625	488	282	282
query8	285	234	211	211
query9	8636	2870	2885	2870
query10	481	335	283	283
query11	15813	15029	14726	14726
query12	160	116	110	110
query13	1658	553	428	428
query14	8557	5663	5668	5663
query15	204	185	176	176
query16	7334	652	511	511
query17	1212	747	587	587
query18	1993	415	323	323
query19	193	182	161	161
query20	125	126	130	126
query21	213	125	120	120
query22	4080	4016	3972	3972
query23	33840	32982	32844	32844
query24	8039	2337	2319	2319
query25	540	462	415	415
query26	1229	274	155	155
query27	2725	510	343	343
query28	4271	2244	2248	2244
query29	763	562	434	434
query30	281	227	192	192
query31	888	803	708	708
query32	86	71	77	71
query33	545	383	370	370
query34	795	842	510	510
query35	791	855	739	739
query36	1009	1022	913	913
query37	118	107	87	87
query38	4053	4035	3957	3957
query39	1522	1425	1422	1422
query40	221	123	113	113
query41	58	60	53	53
query42	123	107	115	107
query43	526	513	481	481
query44	1345	847	848	847
query45	177	168	167	167
query46	858	1010	636	636
query47	1773	1821	1720	1720
query48	373	416	296	296
query49	717	478	393	393
query50	638	680	390	390
query51	4142	4039	4094	4039
query52	113	112	99	99
query53	238	252	191	191
query54	593	574	521	521
query55	94	88	84	84
query56	313	310	305	305
query57	1177	1201	1127	1127
query58	273	259	261	259
query59	2812	3012	2714	2714
query60	369	336	317	317
query61	125	121	122	121
query62	817	750	653	653
query63	221	185	190	185
query64	4252	1009	743	743
query65	4312	4189	4197	4189
query66	1174	415	315	315
query67	15451	15248	15087	15087
query68	8476	915	621	621
query69	458	313	274	274
query70	1231	1167	1143	1143
query71	453	334	311	311
query72	5591	4780	4925	4780
query73	781	695	356	356
query74	8932	8970	8712	8712
query75	3857	3075	2665	2665
query76	3739	1120	744	744
query77	789	383	303	303
query78	9545	9655	8837	8837
query79	2159	835	587	587
query80	707	540	470	470
query81	473	253	222	222
query82	382	145	103	103
query83	274	255	238	238
query84	298	112	98	98
query85	887	370	347	347
query86	343	309	321	309
query87	4228	4314	4221	4221
query88	3076	2209	2232	2209
query89	407	314	283	283
query90	1913	227	222	222
query91	144	134	108	108
query92	88	67	70	67
query93	1310	961	639	639
query94	675	396	306	306
query95	398	316	310	310
query96	501	583	273	273
query97	2648	2682	2565	2565
query98	252	214	215	214
query99	1513	1433	1310	1310
Total cold run time: 272073 ms
Total hot run time: 184188 ms

doris-robot · 2025-08-25T10:45:38Z

ClickBench: Total hot run time: 33.44 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 96ba2ec80545d90472fcccce3633bf8a3386e610, data reload: false

query1	0.04	0.03	0.04
query2	0.11	0.05	0.06
query3	0.29	0.06	0.06
query4	1.59	0.08	0.09
query5	0.42	0.40	0.40
query6	1.16	0.65	0.64
query7	0.02	0.01	0.01
query8	0.09	0.06	0.05
query9	0.63	0.53	0.52
query10	0.57	0.58	0.59
query11	0.26	0.13	0.13
query12	0.26	0.13	0.14
query13	0.64	0.64	0.62
query14	0.82	0.86	0.83
query15	0.96	0.87	0.89
query16	0.38	0.39	0.39
query17	1.06	1.08	1.04
query18	0.22	0.22	0.23
query19	1.94	1.84	1.82
query20	0.02	0.02	0.01
query21	15.39	1.03	0.72
query22	0.93	1.11	0.86
query23	14.69	1.55	0.75
query24	5.48	0.56	0.30
query25	0.17	0.09	0.08
query26	0.55	0.21	0.19
query27	0.08	0.08	0.08
query28	11.05	1.26	0.59
query29	12.61	4.09	3.36
query30	3.12	3.06	3.00
query31	2.82	0.62	0.43
query32	3.24	0.60	0.51
query33	3.08	3.17	3.22
query34	16.81	5.54	4.82
query35	4.87	4.88	4.89
query36	0.66	0.52	0.52
query37	0.21	0.18	0.17
query38	0.17	0.16	0.16
query39	0.06	0.04	0.05
query40	0.19	0.18	0.17
query41	0.10	0.05	0.05
query42	0.06	0.06	0.06
query43	0.05	0.05	0.05
Total cold run time: 107.87 s
Total hot run time: 33.44 s

be/src/olap/rowset/segment_v2/variant/variant_column_reader.cpp

be/src/olap/tablet_schema.h

gensrc/thrift/Descriptors.thrift

doris-robot · 2025-09-02T10:47:17Z

TPC-DS: Total hot run time: 187714 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cb03967ed7b863f9496984ae54a80eaaa399354e, data reload: false

query1	1071	433	409	409
query2	6568	1794	1764	1764
query3	6765	228	227	227
query4	26007	23963	23654	23654
query5	5190	677	602	602
query6	355	255	229	229
query7	4665	528	315	315
query8	315	292	272	272
query9	8642	2948	2954	2948
query10	525	377	311	311
query11	16351	14992	14833	14833
query12	188	128	120	120
query13	1694	573	470	470
query14	10037	5903	5886	5886
query15	230	202	173	173
query16	7683	673	514	514
query17	1630	801	663	663
query18	2076	496	331	331
query19	241	194	165	165
query20	133	140	128	128
query21	216	131	118	118
query22	4224	4172	4059	4059
query23	34319	33165	33072	33072
query24	7890	2396	2469	2396
query25	582	525	446	446
query26	1249	281	171	171
query27	2689	512	361	361
query28	4293	2340	2281	2281
query29	782	607	510	510
query30	301	231	204	204
query31	896	833	728	728
query32	89	81	79	79
query33	590	431	367	367
query34	802	878	534	534
query35	828	815	769	769
query36	980	1012	912	912
query37	126	106	92	92
query38	4040	4102	4017	4017
query39	1497	1436	1432	1432
query40	234	134	128	128
query41	66	60	63	60
query42	138	124	127	124
query43	525	513	494	494
query44	1363	875	870	870
query45	185	179	171	171
query46	874	1033	648	648
query47	1765	1793	1711	1711
query48	399	446	328	328
query49	750	499	422	422
query50	660	690	417	417
query51	4163	4188	4044	4044
query52	119	119	107	107
query53	256	265	201	201
query54	618	606	549	549
query55	99	90	92	90
query56	349	341	334	334
query57	1200	1202	1119	1119
query58	292	285	281	281
query59	2681	2729	2639	2639
query60	370	362	351	351
query61	169	160	199	160
query62	770	740	668	668
query63	227	199	207	199
query64	4423	1188	819	819
query65	4292	4223	4216	4216
query66	1088	459	353	353
query67	15678	15433	15083	15083
query68	9619	931	581	581
query69	534	341	306	306
query70	1282	1111	1193	1111
query71	604	358	341	341
query72	5914	4860	5205	4860
query73	784	674	357	357
query74	9186	9061	8937	8937
query75	4366	3104	2712	2712
query76	3975	1238	742	742
query77	962	418	336	336
query78	9522	9703	8877	8877
query79	2567	816	585	585
query80	685	592	515	515
query81	493	263	231	231
query82	224	143	112	112
query83	285	266	266	266
query84	304	110	106	106
query85	931	470	432	432
query86	348	319	316	316
query87	4262	4330	4202	4202
query88	2805	2231	2237	2231
query89	415	341	296	296
query90	2071	227	222	222
query91	164	164	129	129
query92	97	81	72	72
query93	1974	999	659	659
query94	697	419	326	326
query95	415	339	331	331
query96	483	605	278	278
query97	2594	2688	2558	2558
query98	249	226	220	220
query99	1418	1409	1319	1319
Total cold run time: 280741 ms
Total hot run time: 187714 ms

hello-stephen · 2025-09-02T10:48:32Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	84.59% (1466/1733)
Line Coverage	67.78% (26170/38612)
Region Coverage	68.73% (13009/18927)
Branch Coverage	58.71% (6963/11860)

doris-robot · 2025-09-02T10:52:59Z

ClickBench: Total hot run time: 32.73 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cb03967ed7b863f9496984ae54a80eaaa399354e, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.06	0.06
query3	0.25	0.08	0.08
query4	1.60	0.12	0.12
query5	0.45	0.44	0.43
query6	1.19	0.65	0.64
query7	0.03	0.02	0.02
query8	0.06	0.05	0.05
query9	0.62	0.55	0.53
query10	0.58	0.58	0.59
query11	0.17	0.12	0.11
query12	0.15	0.13	0.12
query13	0.63	0.64	0.62
query14	0.81	0.86	0.86
query15	0.89	0.86	0.89
query16	0.39	0.40	0.39
query17	1.09	1.09	1.08
query18	0.21	0.20	0.20
query19	1.96	1.92	1.82
query20	0.01	0.02	0.01
query21	15.44	0.93	0.58
query22	0.78	1.27	0.88
query23	14.77	1.42	0.62
query24	6.94	1.52	0.36
query25	0.31	0.18	0.06
query26	0.55	0.17	0.15
query27	0.06	0.07	0.05
query28	9.42	0.94	0.44
query29	12.69	4.00	3.27
query30	3.11	3.06	2.97
query31	2.82	0.59	0.39
query32	3.25	0.57	0.49
query33	3.12	3.14	3.16
query34	16.21	5.48	4.87
query35	4.88	4.96	5.00
query36	0.71	0.52	0.52
query37	0.10	0.06	0.07
query38	0.07	0.05	0.04
query39	0.04	0.03	0.03
query40	0.19	0.14	0.14
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.05	0.03	0.03
Total cold run time: 106.88 s
Total hot run time: 32.73 s

doris-robot · 2025-09-02T11:34:52Z

BE UT Coverage Report

Increment line coverage 98.00% (49/50) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	51.71% (17149/33161)
Line Coverage	37.24% (156760/420991)
Region Coverage	31.88% (119612/375171)
Branch Coverage	33.24% (52544/158081)

hello-stephen · 2025-09-02T14:51:48Z

BE Regression && UT Coverage Report

Increment line coverage 97.83% (45/46) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	56.61% (18440/32576)
Line Coverage	42.06% (177003/420860)
Region Coverage	36.50% (138938/380687)
Branch Coverage	37.68% (59930/159062)

hello-stephen · 2025-09-02T15:03:44Z

FE Regression Coverage Report

Increment line coverage 71.79% (28/39) 🎉
Increment coverage report
Complete coverage report

amorynan · 2025-09-03T06:39:24Z

run buildall

hello-stephen · 2025-09-03T06:59:36Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	84.62% (1469/1736)
Line Coverage	67.84% (26249/38694)
Region Coverage	68.84% (13058/18968)
Branch Coverage	58.80% (6989/11886)

eldenmoon

LGTM

github-actions · 2025-09-03T07:45:38Z

PR approved by at least one committer and no changes requested.

doris-robot · 2025-09-03T08:16:41Z

TPC-H: Total hot run time: 34310 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit be36d6abd243ff5bf1f5b35e4330bfe9eee35fa7, data reload: false

------ Round 1 ----------------------------------
q1	17329	5223	5132	5132
q2	2037	360	215	215
q3	9995	1291	731	731
q4	10220	1044	528	528
q5	7582	2524	2364	2364
q6	186	173	136	136
q7	925	788	628	628
q8	9354	1328	1100	1100
q9	7029	5187	5197	5187
q10	6947	2410	1997	1997
q11	492	290	283	283
q12	391	356	228	228
q13	17787	3661	3067	3067
q14	243	241	235	235
q15	576	509	502	502
q16	442	435	390	390
q17	605	869	372	372
q18	7978	7319	7071	7071
q19	1327	970	563	563
q20	347	335	231	231
q21	3917	3216	2374	2374
q22	1069	1047	976	976
Total cold run time: 106778 ms
Total hot run time: 34310 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5287	5155	5152	5152
q2	256	337	230	230
q3	2163	2730	2318	2318
q4	1357	1792	1333	1333
q5	4204	4607	4695	4607
q6	227	179	146	146
q7	2056	1991	1856	1856
q8	2716	2529	2628	2529
q9	7427	7321	7447	7321
q10	3090	3324	2918	2918
q11	601	549	528	528
q12	954	779	670	670
q13	3528	4027	3257	3257
q14	294	298	318	298
q15	555	478	478	478
q16	453	490	457	457
q17	1251	1828	1411	1411
q18	7707	7734	7626	7626
q19	835	917	908	908
q20	2031	2136	1904	1904
q21	5248	4571	4691	4571
q22	1146	1109	1038	1038
Total cold run time: 53386 ms
Total hot run time: 51556 ms

doris-robot · 2025-09-03T08:28:23Z

TPC-DS: Total hot run time: 186483 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit be36d6abd243ff5bf1f5b35e4330bfe9eee35fa7, data reload: false

query1	1130	426	403	403
query2	6589	1717	1734	1717
query3	6764	234	223	223
query4	25926	23785	23120	23120
query5	4432	659	508	508
query6	360	245	222	222
query7	4651	512	304	304
query8	297	282	263	263
query9	8635	2915	2872	2872
query10	477	336	305	305
query11	15707	15355	14807	14807
query12	199	123	121	121
query13	1669	562	439	439
query14	9268	5839	5902	5839
query15	235	199	167	167
query16	7279	689	497	497
query17	1245	754	631	631
query18	2026	436	356	356
query19	203	196	173	173
query20	142	126	126	126
query21	212	134	114	114
query22	4068	4075	3932	3932
query23	34240	33025	32930	32930
query24	8175	2390	2420	2390
query25	597	509	449	449
query26	1249	275	168	168
query27	2759	519	371	371
query28	4369	2272	2250	2250
query29	816	609	500	500
query30	300	223	199	199
query31	906	816	762	762
query32	91	80	80	80
query33	570	410	359	359
query34	800	855	520	520
query35	833	874	745	745
query36	954	1029	927	927
query37	124	111	93	93
query38	4116	4010	3998	3998
query39	1501	1432	1450	1432
query40	229	131	127	127
query41	65	59	61	59
query42	129	117	123	117
query43	541	504	499	499
query44	1379	887	877	877
query45	184	178	171	171
query46	864	1021	651	651
query47	1768	1803	1738	1738
query48	392	445	322	322
query49	744	513	418	418
query50	661	700	405	405
query51	4183	4127	4267	4127
query52	126	115	105	105
query53	249	268	202	202
query54	612	613	548	548
query55	94	89	94	89
query56	344	331	334	331
query57	1207	1207	1153	1153
query58	298	280	288	280
query59	2604	2839	2560	2560
query60	363	357	345	345
query61	162	205	165	165
query62	843	716	671	671
query63	233	195	190	190
query64	4421	1149	844	844
query65	4290	4208	4209	4208
query66	1177	447	340	340
query67	15524	15220	15286	15220
query68	4481	962	601	601
query69	555	342	316	316
query70	1247	1174	1165	1165
query71	548	354	324	324
query72	5895	4934	5002	4934
query73	646	587	362	362
query74	9066	9193	8648	8648
query75	3163	3099	2678	2678
query76	3169	1160	738	738
query77	510	428	355	355
query78	9488	9721	8993	8993
query79	1654	822	597	597
query80	1760	575	545	545
query81	546	262	226	226
query82	396	138	116	116
query83	358	256	241	241
query84	258	115	96	96
query85	873	455	428	428
query86	383	313	299	299
query87	4275	4297	4128	4128
query88	2802	2192	2207	2192
query89	401	329	298	298
query90	1754	230	225	225
query91	163	163	132	132
query92	81	78	73	73
query93	1170	973	649	649
query94	688	437	329	329
query95	429	345	342	342
query96	488	598	282	282
query97	2639	2712	2595	2595
query98	252	226	223	223
query99	1370	1429	1279	1279
Total cold run time: 269108 ms
Total hot run time: 186483 ms

doris-robot · 2025-09-03T08:34:07Z

ClickBench: Total hot run time: 33.53 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit be36d6abd243ff5bf1f5b35e4330bfe9eee35fa7, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.06	0.06
query3	0.25	0.08	0.08
query4	1.61	0.12	0.11
query5	0.46	0.41	0.40
query6	1.16	0.65	0.63
query7	0.04	0.03	0.03
query8	0.06	0.05	0.05
query9	0.60	0.55	0.53
query10	0.59	0.59	0.58
query11	0.17	0.12	0.12
query12	0.15	0.12	0.12
query13	0.63	0.64	0.62
query14	0.80	0.84	0.83
query15	0.90	0.87	0.89
query16	0.39	0.41	0.40
query17	1.10	1.05	1.07
query18	0.22	0.20	0.20
query19	1.92	1.88	1.85
query20	0.02	0.01	0.02
query21	15.39	0.94	0.60
query22	0.77	1.30	0.88
query23	14.74	1.41	0.63
query24	6.67	1.66	1.19
query25	0.49	0.17	0.10
query26	0.58	0.18	0.13
query27	0.08	0.05	0.05
query28	10.22	0.90	0.43
query29	12.56	3.96	3.26
query30	3.10	3.07	3.04
query31	2.82	0.60	0.38
query32	3.24	0.56	0.47
query33	3.11	3.12	3.21
query34	16.07	5.43	4.85
query35	4.90	4.94	4.97
query36	0.68	0.53	0.50
query37	0.11	0.08	0.08
query38	0.07	0.05	0.04
query39	0.04	0.03	0.03
query40	0.20	0.15	0.14
query41	0.08	0.03	0.02
query42	0.03	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 107.22 s
Total hot run time: 33.53 s

hello-stephen · 2025-09-03T09:38:06Z

BE UT Coverage Report

Increment line coverage 98.04% (50/51) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	51.84% (17209/33195)
Line Coverage	37.28% (157163/421609)
Region Coverage	31.90% (119929/375984)
Branch Coverage	33.29% (52666/158210)

hello-stephen · 2025-09-03T10:42:55Z

BE Regression && UT Coverage Report

Increment line coverage 97.92% (47/48) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	70.70% (23044/32596)
Line Coverage	57.05% (240372/421343)
Region Coverage	52.48% (200134/381361)
Branch Coverage	54.20% (86226/159103)

hello-stephen · 2025-09-03T10:52:35Z

FE Regression Coverage Report

Increment line coverage 71.79% (28/39) 🎉
Increment coverage report
Complete coverage report

eldenmoon · 2025-09-04T03:19:18Z

run p0 5

…nt (apache#55124) delete be config for max_sparse_column_statistics_size and add it for variant properties

…for variant #55124 (#55752) picked from #55124

enhance max_sparse_column_statistics_size for variant

1fea718

amorynan requested review from dataroaring, gavinchou and yiguolei as code owners August 21, 2025 09:06

update variantType

4eb037f

csun5285 reviewed Aug 21, 2025

View reviewed changes

eldenmoon reviewed Aug 21, 2025

View reviewed changes

fix comment

b239209

fix be cofig usage

96ba2ec

csun5285 reviewed Aug 25, 2025

View reviewed changes

be/src/olap/rowset/segment_v2/variant/variant_column_reader.cpp Outdated Show resolved Hide resolved

be/src/olap/tablet_schema.h Outdated Show resolved Hide resolved

eldenmoon added the dev/3.1.x label Aug 26, 2025

eldenmoon reviewed Aug 26, 2025

View reviewed changes

gensrc/thrift/Descriptors.thrift Outdated Show resolved Hide resolved

fix default value

81fbbdb

fix cloud schema and casez

be36d6a

eldenmoon approved these changes Sep 3, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 3, 2025

yiguolei approved these changes Sep 4, 2025

View reviewed changes

eldenmoon merged commit c7f0726 into apache:master Sep 4, 2025
26 of 28 checks passed

github-actions bot added the dev/3.1.x-conflict label Sep 4, 2025

amorynan pushed a commit to amorynan/doris that referenced this pull request Sep 7, 2025

[enhance](variant)enhance max_sparse_column_statistics_size for varia…

42cdcc6

…nt (apache#55124) delete be config for max_sparse_column_statistics_size and add it for variant properties

amorynan mentioned this pull request Sep 7, 2025

branch-3.1: [fix](variant) enhance max_sparse_column_statistics_size for variant #55124 #55752

Merged

wenzhenghu pushed a commit to wenzhenghu/doris that referenced this pull request Sep 8, 2025

[enhance](variant)enhance max_sparse_column_statistics_size for varia…

c1b18ec

…nt (apache#55124) delete be config for max_sparse_column_statistics_size and add it for variant properties

morrySnow pushed a commit that referenced this pull request Sep 10, 2025

branch-3.1: [fix](variant) enhance max_sparse_column_statistics_size …

980ad60

…for variant #55124 (#55752) picked from #55124

eldenmoon added dev/3.1.1-merged and removed dev/3.1.x dev/3.1.x-conflict labels Sep 19, 2025

morrySnow mentioned this pull request Sep 22, 2025

Release Note 3.1.1 #56289

Closed

[enhance](variant)enhance max_sparse_column_statistics_size for variant #55124

[enhance](variant)enhance max_sparse_column_statistics_size for variant #55124

Uh oh!

Conversation

amorynan commented Aug 21, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Aug 21, 2025

Uh oh!

amorynan commented Aug 21, 2025

Uh oh!

hello-stephen commented Aug 21, 2025

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Aug 21, 2025

FE UT Coverage Report

Uh oh!

amorynan commented Aug 21, 2025

Uh oh!

doris-robot commented Aug 21, 2025

Uh oh!

hello-stephen commented Aug 21, 2025

Cloud UT Coverage Report

Uh oh!

doris-robot commented Aug 21, 2025

Uh oh!

doris-robot commented Aug 21, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hello-stephen commented Aug 21, 2025

FE UT Coverage Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eldenmoon commented Aug 25, 2025

Uh oh!

amorynan commented Aug 25, 2025

Uh oh!

doris-robot commented Aug 25, 2025

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Aug 25, 2025

FE UT Coverage Report

Uh oh!

doris-robot commented Aug 25, 2025

Uh oh!

doris-robot commented Aug 25, 2025

Uh oh!

doris-robot commented Aug 25, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

doris-robot commented Sep 2, 2025

Uh oh!

hello-stephen commented Sep 2, 2025

Cloud UT Coverage Report

Uh oh!

doris-robot commented Sep 2, 2025

Uh oh!

doris-robot commented Sep 2, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Sep 2, 2025

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Sep 2, 2025

FE Regression Coverage Report

Uh oh!