[improvement](statistics)Agg table set preagg on when doing sample analyzing. #49918

Jibing-Li · 2025-04-09T11:34:24Z

What problem does this PR solve?

This pr includes 3 changes.

Nereids support set ScanNode preagg on by hint, like this: select * from table1 /+PREAGGOPEN/
When sample analyze agg table and mor unique table, set preagg on to improve performance.
Skip sample analyzing agg table and mor unique table's value columns.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-04-09T11:34:41Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

Jibing-Li · 2025-04-10T07:44:48Z

run buildall

doris-robot · 2025-04-10T08:03:53Z

TPC-H: Total hot run time: 34384 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d9b13534eeeeae11be5289a6b975fa2e55c7cd8e, data reload: false

------ Round 1 ----------------------------------
q1	26032	5151	5101	5101
q2	2072	289	185	185
q3	10382	1231	681	681
q4	10231	1013	525	525
q5	7530	2397	2421	2397
q6	183	164	131	131
q7	919	757	632	632
q8	9327	1360	1178	1178
q9	6811	5078	5061	5061
q10	6818	2334	1877	1877
q11	480	288	272	272
q12	355	356	217	217
q13	17778	3672	3072	3072
q14	231	240	202	202
q15	530	477	476	476
q16	621	617	597	597
q17	624	849	385	385
q18	7549	7211	7171	7171
q19	1232	942	559	559
q20	354	347	233	233
q21	4149	3437	2453	2453
q22	1074	1010	979	979
Total cold run time: 115282 ms
Total hot run time: 34384 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5118	5140	5082	5082
q2	234	344	228	228
q3	2199	2659	2331	2331
q4	1422	1817	1424	1424
q5	4579	4504	4353	4353
q6	210	161	122	122
q7	1979	1902	1749	1749
q8	2631	2614	2559	2559
q9	7135	7155	7214	7155
q10	2957	3185	2732	2732
q11	583	487	491	487
q12	676	781	624	624
q13	3632	3939	3231	3231
q14	276	293	265	265
q15	506	471	476	471
q16	637	671	649	649
q17	1183	1567	1368	1368
q18	7614	7473	7458	7458
q19	833	838	925	838
q20	1971	1998	1816	1816
q21	5054	4711	4635	4635
q22	1073	1005	994	994
Total cold run time: 52502 ms
Total hot run time: 50571 ms

doris-robot · 2025-04-10T08:15:36Z

TPC-DS: Total hot run time: 188167 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d9b13534eeeeae11be5289a6b975fa2e55c7cd8e, data reload: false

query1	1005	505	489	489
query2	6530	1953	1989	1953
query3	6742	218	220	218
query4	26478	23680	23038	23038
query5	4290	648	501	501
query6	309	219	207	207
query7	4624	527	295	295
query8	312	260	240	240
query9	8618	2567	2584	2567
query10	472	326	269	269
query11	15909	15297	15019	15019
query12	178	116	112	112
query13	1662	530	401	401
query14	10061	6659	6601	6601
query15	214	198	175	175
query16	7644	679	487	487
query17	1217	768	599	599
query18	2021	432	320	320
query19	206	196	181	181
query20	130	123	119	119
query21	218	133	115	115
query22	4169	4214	4022	4022
query23	34249	33159	33040	33040
query24	8677	2497	2502	2497
query25	590	502	428	428
query26	1235	293	161	161
query27	2711	534	345	345
query28	4336	2443	2405	2405
query29	762	678	439	439
query30	293	234	209	209
query31	969	900	770	770
query32	75	65	63	63
query33	578	388	326	326
query34	805	928	526	526
query35	823	880	791	791
query36	992	1041	930	930
query37	127	105	78	78
query38	4229	4250	4199	4199
query39	1484	1450	1440	1440
query40	216	125	109	109
query41	58	54	52	52
query42	120	107	102	102
query43	521	523	478	478
query44	1337	803	803	803
query45	185	185	174	174
query46	922	1092	640	640
query47	1791	1804	1749	1749
query48	365	449	306	306
query49	787	540	434	434
query50	667	721	410	410
query51	4216	4320	4172	4172
query52	113	109	103	103
query53	230	270	185	185
query54	576	579	496	496
query55	83	84	84	84
query56	298	305	292	292
query57	1142	1230	1083	1083
query58	275	274	268	268
query59	2619	2724	2703	2703
query60	343	348	313	313
query61	131	126	128	126
query62	854	796	703	703
query63	234	190	191	190
query64	4303	1039	677	677
query65	4443	4244	4290	4244
query66	1075	450	377	377
query67	15778	15370	15223	15223
query68	8560	921	524	524
query69	455	314	282	282
query70	1236	1152	1086	1086
query71	477	338	318	318
query72	5770	4741	4657	4657
query73	680	577	347	347
query74	9195	8989	8828	8828
query75	4092	3404	2791	2791
query76	3713	1343	818	818
query77	806	391	295	295
query78	10123	10252	9271	9271
query79	1755	917	586	586
query80	707	565	483	483
query81	483	271	233	233
query82	443	131	103	103
query83	308	259	248	248
query84	294	107	88	88
query85	780	360	325	325
query86	352	323	304	304
query87	4579	4462	4355	4355
query88	2881	2243	2249	2243
query89	413	347	298	298
query90	2019	221	221	221
query91	147	145	111	111
query92	84	62	59	59
query93	1101	1002	591	591
query94	664	410	310	310
query95	379	299	288	288
query96	497	607	275	275
query97	3245	3294	3180	3180
query98	229	217	213	213
query99	1533	1484	1365	1365
Total cold run time: 277626 ms
Total hot run time: 188167 ms

doris-robot · 2025-04-10T08:20:56Z

ClickBench: Total hot run time: 31.31 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d9b13534eeeeae11be5289a6b975fa2e55c7cd8e, data reload: false

query1	0.03	0.04	0.03
query2	0.12	0.10	0.11
query3	0.24	0.19	0.19
query4	1.58	0.20	0.20
query5	0.57	0.54	0.55
query6	1.15	0.71	0.71
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.59	0.52	0.52
query10	0.62	0.58	0.57
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.62	0.61	0.60
query14	2.64	2.71	2.84
query15	0.92	0.86	0.85
query16	0.39	0.38	0.38
query17	1.02	1.06	1.02
query18	0.21	0.20	0.20
query19	1.92	1.87	1.84
query20	0.01	0.01	0.02
query21	15.36	0.89	0.56
query22	0.76	1.36	0.74
query23	14.74	1.37	0.66
query24	7.43	1.42	0.84
query25	0.50	0.20	0.12
query26	0.56	0.17	0.15
query27	0.06	0.05	0.04
query28	9.67	0.88	0.43
query29	12.54	3.92	3.27
query30	0.25	0.09	0.06
query31	2.81	0.59	0.39
query32	3.23	0.54	0.47
query33	3.16	3.04	3.07
query34	15.71	5.09	4.50
query35	4.54	4.50	4.52
query36	0.66	0.51	0.49
query37	0.09	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.03
query40	0.17	0.13	0.12
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 105.48 s
Total hot run time: 31.31 s

Jibing-Li · 2025-04-10T15:27:29Z

run buildall

doris-robot · 2025-04-10T16:03:35Z

TPC-H: Total hot run time: 33836 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b9f90f95611005381621404526a5c79eee145886, data reload: false

------ Round 1 ----------------------------------
q1	28293	5018	5086	5018
q2	2043	273	190	190
q3	10396	1213	680	680
q4	10222	986	532	532
q5	7546	2310	2332	2310
q6	184	162	133	133
q7	904	729	600	600
q8	9309	1216	1064	1064
q9	6754	5083	5073	5073
q10	6857	2284	1880	1880
q11	492	284	268	268
q12	347	341	219	219
q13	18096	3703	3088	3088
q14	223	243	218	218
q15	523	479	486	479
q16	623	614	580	580
q17	583	842	377	377
q18	7637	7167	7007	7007
q19	1979	957	541	541
q20	324	325	216	216
q21	4006	3398	2401	2401
q22	1093	1019	962	962
Total cold run time: 118434 ms
Total hot run time: 33836 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5253	5094	5155	5094
q2	237	329	245	245
q3	2146	2675	2293	2293
q4	1464	1944	1478	1478
q5	4487	4382	4367	4367
q6	221	171	128	128
q7	1987	1900	1770	1770
q8	2612	2578	2532	2532
q9	7302	7103	7538	7103
q10	2994	3227	2859	2859
q11	600	526	521	521
q12	719	792	665	665
q13	3677	4148	3677	3677
q14	294	317	289	289
q15	531	474	488	474
q16	663	718	655	655
q17	1176	1554	1403	1403
q18	7884	7611	7450	7450
q19	816	803	864	803
q20	2004	1999	1867	1867
q21	5239	4852	4818	4818
q22	1136	1068	1038	1038
Total cold run time: 53442 ms
Total hot run time: 51529 ms

doris-robot · 2025-04-10T16:15:50Z

TPC-DS: Total hot run time: 192805 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b9f90f95611005381621404526a5c79eee145886, data reload: false

query1	1370	1039	1037	1037
query2	6380	1932	1904	1904
query3	10985	4452	4425	4425
query4	55845	25580	23529	23529
query5	5173	522	453	453
query6	385	205	201	201
query7	5046	490	273	273
query8	333	243	225	225
query9	6462	2572	2582	2572
query10	438	324	260	260
query11	15129	14989	14785	14785
query12	155	109	103	103
query13	1156	508	398	398
query14	11040	6299	6387	6299
query15	209	206	184	184
query16	7049	664	512	512
query17	1077	756	594	594
query18	1554	431	324	324
query19	217	197	167	167
query20	134	124	121	121
query21	208	139	110	110
query22	4413	4629	4308	4308
query23	34013	33564	33371	33371
query24	5718	2424	2452	2424
query25	453	462	414	414
query26	666	271	149	149
query27	1765	478	333	333
query28	2735	2455	2415	2415
query29	594	555	434	434
query30	278	229	198	198
query31	889	890	798	798
query32	77	59	62	59
query33	452	373	307	307
query34	777	888	523	523
query35	780	848	760	760
query36	946	997	902	902
query37	125	117	79	79
query38	4265	4188	4122	4122
query39	1509	1469	1473	1469
query40	210	117	103	103
query41	53	52	53	52
query42	130	110	103	103
query43	501	527	488	488
query44	1319	845	824	824
query45	187	190	168	168
query46	829	1011	626	626
query47	1852	1933	1815	1815
query48	394	422	317	317
query49	716	543	428	428
query50	656	683	402	402
query51	4234	4342	4173	4173
query52	112	108	101	101
query53	238	274	178	178
query54	571	595	507	507
query55	84	92	83	83
query56	295	305	293	293
query57	1165	1180	1095	1095
query58	267	258	271	258
query59	2714	2854	2818	2818
query60	345	320	320	320
query61	132	134	126	126
query62	747	742	688	688
query63	231	182	187	182
query64	1500	1061	698	698
query65	4419	4239	4255	4239
query66	712	405	298	298
query67	15806	15543	15262	15262
query68	7441	895	510	510
query69	597	311	258	258
query70	1202	1127	1105	1105
query71	504	320	289	289
query72	5977	4741	4867	4741
query73	1262	670	345	345
query74	8969	8934	8810	8810
query75	3824	3232	2695	2695
query76	4313	1188	737	737
query77	640	360	361	360
query78	10070	10214	9202	9202
query79	2198	815	567	567
query80	576	521	436	436
query81	476	258	225	225
query82	467	124	102	102
query83	263	251	231	231
query84	292	101	85	85
query85	776	346	303	303
query86	402	321	313	313
query87	4540	4519	4416	4416
query88	3420	2211	2189	2189
query89	419	308	282	282
query90	1947	211	211	211
query91	145	144	114	114
query92	73	64	54	54
query93	1548	921	570	570
query94	679	406	313	313
query95	369	330	280	280
query96	485	563	273	273
query97	3158	3243	3140	3140
query98	233	205	208	205
query99	1426	1385	1258	1258
Total cold run time: 299910 ms
Total hot run time: 192805 ms

doris-robot · 2025-04-10T16:21:09Z

ClickBench: Total hot run time: 31.52 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b9f90f95611005381621404526a5c79eee145886, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.10	0.10
query3	0.26	0.20	0.19
query4	1.60	0.19	0.19
query5	0.60	0.57	0.58
query6	1.20	0.72	0.70
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.57	0.50	0.51
query10	0.57	0.60	0.57
query11	0.15	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.60	0.59
query14	2.70	2.72	2.82
query15	0.93	0.84	0.84
query16	0.38	0.39	0.38
query17	1.00	1.02	1.05
query18	0.21	0.20	0.20
query19	1.89	1.90	1.89
query20	0.01	0.01	0.01
query21	15.40	0.91	0.55
query22	0.76	1.22	0.71
query23	14.88	1.40	0.66
query24	7.21	1.53	1.19
query25	0.52	0.32	0.07
query26	0.62	0.16	0.15
query27	0.06	0.05	0.04
query28	9.96	0.84	0.44
query29	12.59	3.98	3.26
query30	0.26	0.09	0.06
query31	2.84	0.57	0.39
query32	3.23	0.55	0.46
query33	3.02	3.06	3.01
query34	15.89	5.08	4.47
query35	4.52	4.57	4.51
query36	0.66	0.49	0.49
query37	0.09	0.06	0.07
query38	0.06	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 105.94 s
Total hot run time: 31.52 s

Jibing-Li · 2025-04-10T23:27:41Z

run buildall

doris-robot · 2025-04-11T00:04:32Z

TPC-H: Total hot run time: 34273 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b9f90f95611005381621404526a5c79eee145886, data reload: false

------ Round 1 ----------------------------------
q1	25917	5061	5309	5061
q2	2064	271	179	179
q3	10404	1207	676	676
q4	10235	1008	517	517
q5	8003	2414	2287	2287
q6	195	161	131	131
q7	909	739	607	607
q8	9566	1284	1092	1092
q9	6843	5184	5188	5184
q10	6828	2320	1924	1924
q11	496	279	268	268
q12	340	356	222	222
q13	18237	3769	3087	3087
q14	228	228	207	207
q15	550	496	483	483
q16	619	604	580	580
q17	598	846	367	367
q18	7627	7196	7217	7196
q19	1257	949	547	547
q20	338	342	223	223
q21	4379	3399	2474	2474
q22	1070	1045	961	961
Total cold run time: 116703 ms
Total hot run time: 34273 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5191	5133	5174	5133
q2	238	328	238	238
q3	2120	2649	2520	2520
q4	1512	2044	1559	1559
q5	4665	4630	4634	4630
q6	232	177	133	133
q7	2062	2018	1910	1910
q8	2661	2629	2662	2629
q9	7589	7165	7223	7165
q10	2962	3160	2753	2753
q11	576	519	482	482
q12	683	773	665	665
q13	3482	3867	3177	3177
q14	282	303	280	280
q15	525	477	482	477
q16	651	690	636	636
q17	1167	1565	1339	1339
q18	7692	7692	7460	7460
q19	835	807	870	807
q20	1987	1954	1824	1824
q21	5302	4739	4797	4739
q22	1107	1055	1005	1005
Total cold run time: 53521 ms
Total hot run time: 51561 ms

doris-robot · 2025-04-11T00:16:26Z

TPC-DS: Total hot run time: 193381 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b9f90f95611005381621404526a5c79eee145886, data reload: false

query1	1417	1059	1032	1032
query2	6302	2009	1899	1899
query3	11174	4752	4526	4526
query4	25595	23654	23039	23039
query5	4298	634	463	463
query6	309	221	194	194
query7	3992	501	275	275
query8	299	254	233	233
query9	8503	2587	2561	2561
query10	479	319	268	268
query11	15379	15036	14825	14825
query12	163	109	107	107
query13	1558	524	400	400
query14	9218	6166	6253	6166
query15	205	191	172	172
query16	7615	626	493	493
query17	1197	736	613	613
query18	2044	422	326	326
query19	206	198	176	176
query20	125	124	120	120
query21	215	123	112	112
query22	4487	4449	4317	4317
query23	34319	33652	33545	33545
query24	8574	2487	2414	2414
query25	514	460	403	403
query26	1203	275	151	151
query27	2849	505	345	345
query28	4606	2458	2402	2402
query29	707	578	447	447
query30	274	233	196	196
query31	924	867	818	818
query32	71	62	63	62
query33	549	371	341	341
query34	814	871	515	515
query35	815	853	761	761
query36	968	998	915	915
query37	119	100	76	76
query38	4258	4359	4322	4322
query39	1475	1428	1599	1428
query40	216	127	114	114
query41	56	50	51	50
query42	121	105	108	105
query43	510	522	487	487
query44	1314	822	807	807
query45	183	173	187	173
query46	845	1038	650	650
query47	1879	1926	1820	1820
query48	400	408	316	316
query49	780	496	417	417
query50	642	684	412	412
query51	4265	4295	4217	4217
query52	113	107	99	99
query53	229	257	182	182
query54	609	589	511	511
query55	85	83	83	83
query56	282	326	290	290
query57	1217	1204	1142	1142
query58	277	269	267	267
query59	2864	2841	2769	2769
query60	340	328	331	328
query61	171	134	143	134
query62	747	724	661	661
query63	235	188	186	186
query64	4143	1122	731	731
query65	4399	4422	4366	4366
query66	965	417	314	314
query67	16039	15565	15297	15297
query68	8143	902	512	512
query69	438	302	270	270
query70	1224	1118	1101	1101
query71	466	309	286	286
query72	5608	4745	4758	4745
query73	556	595	348	348
query74	9106	8965	9032	8965
query75	4047	3202	2698	2698
query76	4071	1185	735	735
query77	915	379	289	289
query78	9865	10157	9297	9297
query79	1880	799	569	569
query80	664	519	433	433
query81	470	267	245	245
query82	459	123	95	95
query83	288	260	235	235
query84	290	102	80	80
query85	782	371	317	317
query86	325	309	299	299
query87	4526	4606	4489	4489
query88	2883	2255	2222	2222
query89	412	309	273	273
query90	1967	219	213	213
query91	142	142	112	112
query92	82	57	57	57
query93	1992	947	569	569
query94	671	416	308	308
query95	371	292	344	292
query96	485	559	269	269
query97	3214	3221	3205	3205
query98	217	213	207	207
query99	1473	1416	1260	1260
Total cold run time: 279777 ms
Total hot run time: 193381 ms

doris-robot · 2025-04-11T00:21:44Z

ClickBench: Total hot run time: 30.78 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b9f90f95611005381621404526a5c79eee145886, data reload: false

query1	0.03	0.03	0.03
query2	0.12	0.10	0.10
query3	0.25	0.19	0.19
query4	1.58	0.18	0.19
query5	0.59	0.57	0.58
query6	1.19	0.73	0.72
query7	0.03	0.02	0.02
query8	0.04	0.03	0.04
query9	0.58	0.51	0.51
query10	0.58	0.59	0.56
query11	0.16	0.10	0.10
query12	0.15	0.11	0.11
query13	0.61	0.60	0.60
query14	2.71	2.70	2.72
query15	0.92	0.85	0.85
query16	0.39	0.38	0.38
query17	1.02	1.04	1.03
query18	0.21	0.19	0.19
query19	1.95	1.90	1.87
query20	0.01	0.02	0.01
query21	15.36	0.91	0.54
query22	0.76	1.21	0.80
query23	14.74	1.35	0.62
query24	8.08	0.65	0.31
query25	0.42	0.25	0.12
query26	0.58	0.16	0.12
query27	0.05	0.05	0.05
query28	9.37	0.86	0.43
query29	12.54	4.00	3.25
query30	0.24	0.10	0.07
query31	2.82	0.59	0.38
query32	3.23	0.54	0.46
query33	2.97	3.06	3.07
query34	15.87	5.11	4.49
query35	4.55	4.57	4.54
query36	0.65	0.49	0.47
query37	0.08	0.06	0.07
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.18	0.14	0.13
query41	0.09	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.85 s
Total hot run time: 30.78 s

Jibing-Li · 2025-04-11T02:49:41Z

run buildall

doris-robot · 2025-04-11T04:07:36Z

TPC-H: Total hot run time: 34830 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 58a6b6ee7f9ad35096120aebb1cb13b77e57e643, data reload: false

------ Round 1 ----------------------------------
q1	26162	5025	4958	4958
q2	2078	305	180	180
q3	10492	1248	694	694
q4	10233	1029	533	533
q5	7674	2471	2338	2338
q6	185	161	131	131
q7	912	725	603	603
q8	9317	1320	1135	1135
q9	6865	5196	5131	5131
q10	6801	2296	1903	1903
q11	460	284	274	274
q12	338	349	220	220
q13	17779	3684	3084	3084
q14	224	229	216	216
q15	550	487	500	487
q16	599	631	590	590
q17	588	844	370	370
q18	7620	7159	7004	7004
q19	1470	948	548	548
q20	342	337	219	219
q21	4276	3355	3245	3245
q22	1087	1004	967	967
Total cold run time: 116052 ms
Total hot run time: 34830 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5105	5056	5068	5056
q2	237	327	230	230
q3	2140	2702	2264	2264
q4	1414	1838	1529	1529
q5	4530	4441	4365	4365
q6	204	162	123	123
q7	1974	1913	1764	1764
q8	2640	2530	2494	2494
q9	7241	7147	7117	7117
q10	2961	3171	2730	2730
q11	597	494	486	486
q12	688	765	623	623
q13	3479	3844	3256	3256
q14	272	286	281	281
q15	560	483	489	483
q16	663	688	643	643
q17	1141	1528	1428	1428
q18	7579	7422	7406	7406
q19	814	808	837	808
q20	2013	2061	1901	1901
q21	5183	4702	4642	4642
q22	1063	1028	1018	1018
Total cold run time: 52498 ms
Total hot run time: 50647 ms

doris-robot · 2025-04-11T04:19:06Z

TPC-DS: Total hot run time: 186527 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 58a6b6ee7f9ad35096120aebb1cb13b77e57e643, data reload: false

query1	991	489	474	474
query2	6541	1914	1909	1909
query3	6747	224	219	219
query4	25839	23381	23367	23367
query5	4366	641	472	472
query6	322	214	196	196
query7	4626	504	305	305
query8	297	260	236	236
query9	8613	2614	2604	2604
query10	465	326	245	245
query11	15370	15031	15372	15031
query12	176	110	105	105
query13	1672	509	418	418
query14	9949	6050	6237	6050
query15	205	192	165	165
query16	7643	620	464	464
query17	1175	733	552	552
query18	2012	395	302	302
query19	193	173	151	151
query20	114	117	117	117
query21	213	119	103	103
query22	4162	4200	3986	3986
query23	34014	32914	33055	32914
query24	8346	2384	2396	2384
query25	528	444	390	390
query26	1230	271	148	148
query27	2761	505	336	336
query28	4330	2448	2415	2415
query29	722	544	422	422
query30	285	222	197	197
query31	952	873	792	792
query32	78	65	67	65
query33	562	385	320	320
query34	781	870	507	507
query35	784	817	726	726
query36	977	986	890	890
query37	119	103	77	77
query38	4117	4199	4174	4174
query39	1453	1440	1415	1415
query40	207	118	107	107
query41	56	55	52	52
query42	121	104	107	104
query43	497	520	468	468
query44	1288	813	800	800
query45	178	173	164	164
query46	832	1015	651	651
query47	1736	1792	1722	1722
query48	373	406	302	302
query49	774	495	409	409
query50	634	684	403	403
query51	4175	4155	4183	4155
query52	113	113	97	97
query53	228	250	180	180
query54	578	573	513	513
query55	93	83	83	83
query56	313	330	305	305
query57	1121	1113	1088	1088
query58	266	261	259	259
query59	2584	2681	2575	2575
query60	336	324	306	306
query61	154	125	130	125
query62	802	729	646	646
query63	222	189	191	189
query64	4272	998	697	697
query65	4364	4278	4264	4264
query66	1056	421	306	306
query67	15929	15433	15222	15222
query68	5969	879	519	519
query69	459	312	263	263
query70	1233	1120	1073	1073
query71	422	324	297	297
query72	5515	4800	4879	4800
query73	670	629	342	342
query74	8850	9209	8923	8923
query75	3187	3219	2682	2682
query76	3244	1197	761	761
query77	500	382	297	297
query78	10000	10188	9263	9263
query79	1453	812	562	562
query80	589	524	435	435
query81	505	251	220	220
query82	195	129	100	100
query83	255	250	242	242
query84	248	109	86	86
query85	749	354	310	310
query86	368	318	276	276
query87	4518	4575	4322	4322
query88	2766	2223	2219	2219
query89	387	309	281	281
query90	1898	206	213	206
query91	140	137	112	112
query92	78	63	54	54
query93	1132	935	583	583
query94	673	421	310	310
query95	373	291	280	280
query96	483	567	278	278
query97	3145	3221	3126	3126
query98	226	210	218	210
query99	1329	1538	1260	1260
Total cold run time: 268591 ms
Total hot run time: 186527 ms

doris-robot · 2025-04-11T04:24:25Z

ClickBench: Total hot run time: 30.58 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 58a6b6ee7f9ad35096120aebb1cb13b77e57e643, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.11	0.10
query3	0.24	0.20	0.19
query4	1.58	0.21	0.11
query5	0.57	0.54	0.55
query6	1.19	0.72	0.70
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.58	0.52	0.52
query10	0.58	0.59	0.56
query11	0.16	0.11	0.11
query12	0.14	0.11	0.11
query13	0.60	0.60	0.61
query14	2.68	2.68	2.72
query15	0.91	0.85	0.86
query16	0.38	0.37	0.40
query17	1.02	1.03	1.02
query18	0.20	0.20	0.19
query19	1.99	1.86	1.85
query20	0.01	0.02	0.01
query21	15.40	0.94	0.57
query22	0.76	1.24	0.71
query23	14.80	1.39	0.64
query24	7.31	1.24	0.41
query25	0.49	0.28	0.08
query26	0.60	0.17	0.14
query27	0.05	0.05	0.05
query28	9.05	0.88	0.43
query29	12.57	3.95	3.24
query30	0.24	0.08	0.06
query31	2.82	0.61	0.39
query32	3.22	0.54	0.47
query33	3.00	3.12	3.08
query34	15.76	5.10	4.47
query35	4.55	4.50	4.46
query36	0.66	0.50	0.48
query37	0.08	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.13	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.8 s
Total hot run time: 30.58 s

Jibing-Li · 2025-04-21T11:18:04Z

run buildall

doris-robot · 2025-04-21T11:41:28Z

TPC-H: Total hot run time: 34723 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2a092a7f49eab7a6fada239b457d27e9eeb448a3, data reload: false

------ Round 1 ----------------------------------
q1	25809	4997	5020	4997
q2	2065	292	190	190
q3	10367	1260	719	719
q4	10221	993	547	547
q5	7546	2363	2357	2357
q6	181	164	133	133
q7	934	739	605	605
q8	9319	1291	1086	1086
q9	6781	5100	5110	5100
q10	6802	2291	1877	1877
q11	476	281	266	266
q12	350	360	218	218
q13	17749	3669	3091	3091
q14	228	239	206	206
q15	526	475	477	475
q16	443	437	401	401
q17	584	864	362	362
q18	7359	7206	7131	7131
q19	1206	936	579	579
q20	348	340	220	220
q21	3975	3356	3193	3193
q22	1061	982	970	970
Total cold run time: 114330 ms
Total hot run time: 34723 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5070	5098	5050	5050
q2	239	331	230	230
q3	2113	2641	2305	2305
q4	1389	1829	1356	1356
q5	4514	4447	4415	4415
q6	213	162	124	124
q7	1953	1889	1778	1778
q8	2542	2538	2501	2501
q9	7182	7077	7101	7077
q10	2951	3182	2683	2683
q11	579	489	499	489
q12	676	792	622	622
q13	3469	3857	3208	3208
q14	288	316	267	267
q15	523	484	478	478
q16	462	499	463	463
q17	1146	1507	1378	1378
q18	7724	7444	7351	7351
q19	794	848	915	848
q20	1990	2001	1845	1845
q21	5069	4674	4583	4583
q22	1053	1010	952	952
Total cold run time: 51939 ms
Total hot run time: 50003 ms

doris-robot · 2025-04-21T11:53:00Z

TPC-DS: Total hot run time: 185298 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2a092a7f49eab7a6fada239b457d27e9eeb448a3, data reload: false

query1	1000	465	513	465
query2	6573	1794	1779	1779
query3	6741	220	219	219
query4	25982	23317	23433	23317
query5	4369	631	457	457
query6	302	205	201	201
query7	4621	488	279	279
query8	296	249	261	249
query9	8637	2552	2594	2552
query10	486	322	284	284
query11	15297	15089	14804	14804
query12	150	108	109	108
query13	1665	509	406	406
query14	9024	6078	6005	6005
query15	211	190	167	167
query16	7345	652	452	452
query17	1209	741	585	585
query18	1976	403	308	308
query19	191	184	161	161
query20	121	117	116	116
query21	216	126	107	107
query22	4281	4253	4235	4235
query23	33882	32961	32790	32790
query24	8480	2361	2366	2361
query25	546	448	377	377
query26	1237	269	153	153
query27	2770	490	326	326
query28	4312	2108	2108	2108
query29	759	555	424	424
query30	282	205	184	184
query31	938	837	771	771
query32	71	65	61	61
query33	554	391	313	313
query34	797	851	502	502
query35	782	801	713	713
query36	948	956	865	865
query37	109	98	72	72
query38	4173	4230	4117	4117
query39	1434	1381	1370	1370
query40	205	117	107	107
query41	56	55	52	52
query42	120	106	105	105
query43	493	499	477	477
query44	1251	786	803	786
query45	174	177	166	166
query46	833	1011	618	618
query47	1799	1815	1715	1715
query48	362	414	292	292
query49	777	487	421	421
query50	635	684	401	401
query51	4067	4196	4048	4048
query52	110	103	104	103
query53	220	254	183	183
query54	579	577	500	500
query55	88	80	81	80
query56	306	287	294	287
query57	1143	1152	1079	1079
query58	264	257	249	249
query59	2548	2654	2559	2559
query60	349	324	300	300
query61	130	125	127	125
query62	826	730	660	660
query63	216	195	189	189
query64	4318	1025	666	666
query65	4330	4222	4220	4220
query66	1149	405	306	306
query67	15719	15410	15213	15213
query68	8094	876	518	518
query69	485	297	262	262
query70	1205	1142	1080	1080
query71	502	313	286	286
query72	5565	4758	4778	4758
query73	702	611	337	337
query74	8872	8874	8930	8874
query75	3940	3176	2700	2700
query76	3629	1206	761	761
query77	785	382	285	285
query78	9840	10044	9344	9344
query79	2446	812	565	565
query80	639	532	423	423
query81	495	256	222	222
query82	453	123	100	100
query83	253	243	236	236
query84	253	101	85	85
query85	802	348	321	321
query86	383	323	291	291
query87	4291	4368	4230	4230
query88	3775	2193	2192	2192
query89	396	318	294	294
query90	1855	210	284	210
query91	135	140	107	107
query92	76	59	59	59
query93	2029	932	583	583
query94	662	418	301	301
query95	378	295	285	285
query96	476	565	267	267
query97	3141	3184	3167	3167
query98	234	204	207	204
query99	1389	1437	1281	1281
Total cold run time: 274014 ms
Total hot run time: 185298 ms

doris-robot · 2025-04-21T11:58:13Z

ClickBench: Total hot run time: 28.98 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2a092a7f49eab7a6fada239b457d27e9eeb448a3, data reload: false

query1	0.04	0.03	0.03
query2	0.13	0.11	0.10
query3	0.25	0.19	0.20
query4	1.59	0.19	0.10
query5	0.57	0.55	0.55
query6	1.18	0.72	0.71
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.50	0.52
query10	0.56	0.56	0.57
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.61	0.59	0.60
query14	1.14	1.16	1.15
query15	0.86	0.86	0.84
query16	0.38	0.39	0.37
query17	1.00	1.04	1.01
query18	0.20	0.20	0.20
query19	1.87	1.75	1.80
query20	0.01	0.01	0.01
query21	15.42	0.93	0.57
query22	0.76	1.30	0.75
query23	14.74	1.37	0.62
query24	7.42	1.24	0.42
query25	0.53	0.16	0.06
query26	0.59	0.17	0.13
query27	0.05	0.05	0.05
query28	9.00	0.87	0.42
query29	12.54	4.04	3.33
query30	0.25	0.09	0.06
query31	2.83	0.60	0.38
query32	3.23	0.54	0.47
query33	3.02	3.04	3.05
query34	15.74	5.11	4.47
query35	4.50	4.52	4.48
query36	0.67	0.49	0.48
query37	0.08	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.16	0.14	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 103.1 s
Total hot run time: 28.98 s

Jibing-Li · 2025-04-21T15:09:35Z

run p0

…alyzing. (apache#49918) This pr includes 3 changes. 1. Nereids support set ScanNode preagg on by hint, like this: select * from table1 /*+PREAGGOPEN*/ 2. When sample analyze agg table and mor unique table, set preagg on to improve performance. 3. Skip sample analyzing agg table and mor unique table's value columns.

…g sample analyzing. (#49918) (#51672) backport: #49918

…alyzing. (apache#49918) This pr includes 3 changes. 1. Nereids support set ScanNode preagg on by hint, like this: select * from table1 /*+PREAGGOPEN*/ 2. When sample analyze agg table and mor unique table, set preagg on to improve performance. 3. Skip sample analyzing agg table and mor unique table's value columns.

…ng sample analyzing. (#49918) (#51675) backport: #49918

Jibing-Li force-pushed the agg branch 4 times, most recently from e7e8c6d to d9b1353 Compare April 10, 2025 07:36

Jibing-Li marked this pull request as ready for review April 10, 2025 07:42

Jibing-Li force-pushed the agg branch from d9b1353 to b9f90f9 Compare April 10, 2025 11:45

Jibing-Li force-pushed the agg branch from b9f90f9 to 58a6b6e Compare April 11, 2025 02:37

Jibing-Li force-pushed the agg branch from 58a6b6e to 2a092a7 Compare April 21, 2025 11:17

Jibing-Li added dev/2.1.x dev/3.0.x labels Jun 12, 2025

github-actions bot added dev/3.0.x-conflict dev/2.1.x-conflict labels Jun 12, 2025

Jibing-Li mentioned this pull request Jun 12, 2025

branch-3.0:[improvement](statistics)Agg table set preagg on when doing sample analyzing. (#49918) #51672

Merged

Jibing-Li mentioned this pull request Jun 12, 2025

branch-2.1: [improvement](statistics)Agg table set preagg on when doing sample analyzing. (#49918) #51675

Merged

Jibing-Li added the usercase Important user case type label label Jun 13, 2025

dataroaring pushed a commit that referenced this pull request Jun 13, 2025

branch-3.0:[improvement](statistics)Agg table set preagg on when doin…

ce42bb5

…g sample analyzing. (#49918) (#51672) backport: #49918

dataroaring added dev/3.0.7-merged and removed dev/3.0.x dev/3.0.x-conflict labels Jun 13, 2025

yiguolei pushed a commit that referenced this pull request Jun 16, 2025

branch-2.1: [improvement](statistics)Agg table set preagg on when doi…

8734c3f

…ng sample analyzing. (#49918) (#51675) backport: #49918

yiguolei added dev/2.1.11-merged and removed dev/2.1.x dev/2.1.x-conflict labels Jun 16, 2025

morrySnow added dev/3.1.x and removed dev/3.1.x labels Jun 19, 2025

github-actions bot added the dev/3.1.x-conflict label Jun 19, 2025

Jibing-Li added dev/3.1.0-merged and removed dev/3.1.x dev/3.1.x-conflict labels Jun 23, 2025

gavinchou mentioned this pull request Aug 15, 2025

Release Note 3.0.7 #54868

Open

[improvement](statistics)Agg table set preagg on when doing sample analyzing. #49918

[improvement](statistics)Agg table set preagg on when doing sample analyzing. #49918

Uh oh!

Conversation

Jibing-Li commented Apr 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Apr 9, 2025

Uh oh!

Jibing-Li commented Apr 10, 2025

Uh oh!

doris-robot commented Apr 10, 2025

Uh oh!

doris-robot commented Apr 10, 2025

Uh oh!

doris-robot commented Apr 10, 2025

Uh oh!

Jibing-Li commented Apr 10, 2025

Uh oh!

doris-robot commented Apr 10, 2025

Uh oh!

doris-robot commented Apr 10, 2025

Uh oh!

doris-robot commented Apr 10, 2025

Uh oh!

Jibing-Li commented Apr 10, 2025

Uh oh!

doris-robot commented Apr 11, 2025

Uh oh!

doris-robot commented Apr 11, 2025

Uh oh!

doris-robot commented Apr 11, 2025

Uh oh!

Jibing-Li commented Apr 11, 2025

Uh oh!

doris-robot commented Apr 11, 2025

Uh oh!

doris-robot commented Apr 11, 2025

Uh oh!

doris-robot commented Apr 11, 2025

Uh oh!

Jibing-Li commented Apr 21, 2025

Uh oh!

doris-robot commented Apr 21, 2025

Uh oh!

doris-robot commented Apr 21, 2025

Uh oh!

doris-robot commented Apr 21, 2025

Uh oh!

Jibing-Li commented Apr 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Jibing-Li commented Apr 9, 2025 •

edited

Loading