Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][cherrypick](fe) LIST partition table support modify default bucket num #40032

Merged

Conversation

htyoung
Copy link
Contributor

@htyoung htyoung commented Aug 28, 2024

cherry pick from #39696

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@htyoung htyoung changed the title [fix][cherrypick](fe) LIST partition table support modify default buc… [fix][cherrypick](fe) LIST partition table support modify default bucket num Aug 28, 2024
@htyoung
Copy link
Contributor Author

htyoung commented Aug 28, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49610 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c295d8a79f608740ce7aa0914734309d959f203f, data reload: false

------ Round 1 ----------------------------------
q1	17908	4398	4337	4337
q2	2045	190	146	146
q3	10298	1899	1949	1899
q4	10100	1286	1299	1286
q5	8484	3908	3896	3896
q6	238	125	122	122
q7	2027	1612	1617	1612
q8	9480	2730	2707	2707
q9	10933	10375	10170	10170
q10	8587	3491	3499	3491
q11	426	239	253	239
q12	466	294	304	294
q13	18356	3977	4024	3977
q14	342	319	328	319
q15	511	448	464	448
q16	665	568	577	568
q17	1125	966	977	966
q18	7243	6868	6927	6868
q19	1696	1532	1506	1506
q20	533	313	291	291
q21	4478	4134	4075	4075
q22	489	393	401	393
Total cold run time: 116430 ms
Total hot run time: 49610 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4295	4377	4293	4293
q2	322	230	225	225
q3	4139	4146	4131	4131
q4	2780	2752	2743	2743
q5	7197	7113	7160	7113
q6	237	121	122	121
q7	3213	2910	2879	2879
q8	4377	4445	4516	4445
q9	16752	16874	16629	16629
q10	4259	4265	4236	4236
q11	746	701	668	668
q12	1024	836	859	836
q13	6973	3777	3718	3718
q14	441	417	412	412
q15	510	464	455	455
q16	737	681	686	681
q17	3827	3912	3800	3800
q18	8806	8719	8656	8656
q19	1699	1705	1634	1634
q20	2366	2098	2120	2098
q21	8456	8407	8504	8407
q22	1060	905	926	905
Total cold run time: 84216 ms
Total hot run time: 79085 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 200385 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c295d8a79f608740ce7aa0914734309d959f203f, data reload: false

query1	927	383	413	383
query2	6561	2112	2043	2043
query3	6915	211	197	197
query4	20699	18025	17883	17883
query5	19732	6462	6517	6462
query6	289	225	230	225
query7	4161	305	311	305
query8	254	264	254	254
query9	3182	2735	2653	2653
query10	426	290	292	290
query11	11289	10656	10763	10656
query12	125	77	74	74
query13	5601	651	666	651
query14	17603	13088	13443	13088
query15	358	231	224	224
query16	6486	291	272	272
query17	1719	1541	879	879
query18	2329	411	410	410
query19	207	158	152	152
query20	78	78	80	78
query21	193	100	92	92
query22	5278	4971	5062	4971
query23	32461	32149	31768	31768
query24	7052	6522	6576	6522
query25	534	439	444	439
query26	627	157	157	157
query27	2013	289	297	289
query28	6103	2267	2255	2255
query29	2831	2786	2612	2612
query30	242	167	166	166
query31	921	773	744	744
query32	67	67	66	66
query33	414	263	243	243
query34	839	461	467	461
query35	1130	909	941	909
query36	1302	1093	1462	1093
query37	89	60	61	60
query38	3040	2874	2942	2874
query39	1388	1308	1316	1308
query40	226	91	97	91
query41	39	39	36	36
query42	87	81	87	81
query43	574	545	609	545
query44	1123	716	706	706
query45	240	230	236	230
query46	1222	948	966	948
query47	1859	1725	1682	1682
query48	987	690	674	674
query49	627	376	376	376
query50	885	658	610	610
query51	4732	4638	4588	4588
query52	98	89	84	84
query53	452	331	321	321
query54	2663	2450	2494	2450
query55	91	76	98	76
query56	223	209	211	209
query57	1248	1058	1146	1058
query58	208	219	214	214
query59	3586	3359	3426	3359
query60	215	202	222	202
query61	97	97	97	97
query62	770	493	490	490
query63	492	340	345	340
query64	2649	1544	1474	1474
query65	3579	3537	3567	3537
query66	826	390	377	377
query67	15446	14719	14755	14719
query68	11765	663	644	644
query69	573	327	339	327
query70	2169	1421	1435	1421
query71	411	310	316	310
query72	6533	3473	3451	3451
query73	1498	320	324	320
query74	6355	5905	5797	5797
query75	5466	3677	3737	3677
query76	6965	1159	1180	1159
query77	1183	258	266	258
query78	12641	11563	11603	11563
query79	7853	644	645	644
query80	1081	404	403	403
query81	463	237	232	232
query82	825	99	94	94
query83	168	134	131	131
query84	256	69	75	69
query85	882	323	318	318
query86	336	304	310	304
query87	3200	3046	3034	3034
query88	4048	2317	2295	2295
query89	482	293	336	293
query90	1965	215	212	212
query91	175	122	123	122
query92	59	57	56	56
query93	7072	563	567	563
query94	700	211	215	211
query95	1143	1084	1059	1059
query96	656	337	328	328
query97	6427	6439	6305	6305
query98	191	182	182	182
query99	2991	821	926	821
Total cold run time: 317241 ms
Total hot run time: 200385 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.3 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c295d8a79f608740ce7aa0914734309d959f203f, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.03	0.02
query3	0.25	0.05	0.05
query4	1.78	0.08	0.06
query5	0.54	0.53	0.50
query6	1.23	0.61	0.60
query7	0.02	0.01	0.01
query8	0.04	0.02	0.02
query9	0.52	0.49	0.48
query10	0.55	0.54	0.53
query11	0.12	0.09	0.08
query12	0.12	0.09	0.08
query13	0.61	0.61	0.62
query14	0.78	0.79	0.80
query15	0.78	0.76	0.77
query16	0.35	0.36	0.37
query17	1.01	0.99	1.01
query18	0.23	0.25	0.23
query19	1.93	1.82	1.86
query20	0.02	0.01	0.01
query21	15.47	0.54	0.52
query22	1.96	2.31	1.45
query23	17.36	0.94	0.93
query24	6.06	0.74	1.13
query25	0.40	0.13	0.05
query26	0.57	0.14	0.16
query27	0.04	0.04	0.04
query28	7.22	0.77	0.75
query29	12.68	2.30	2.30
query30	0.60	0.55	0.55
query31	2.82	0.39	0.37
query32	3.39	0.49	0.48
query33	3.05	3.10	3.07
query34	15.28	4.78	4.82
query35	4.84	4.84	4.87
query36	1.07	1.01	1.02
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.02	0.02
query40	0.17	0.14	0.14
query41	0.07	0.02	0.01
query42	0.02	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 104.19 s
Total hot run time: 30.3 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit c295d8a79f608740ce7aa0914734309d959f203f with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@morningman morningman merged commit fb2ccf0 into apache:branch-2.0 Aug 30, 2024
24 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants