Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[cherry-pick](branch-2.0)add SessionVariable for enableCooldownReplicaAffinity #42683

Merged
merged 6 commits into from
Nov 8, 2024
Merged

[cherry-pick](branch-2.0)add SessionVariable for enableCooldownReplicaAffinity #42683

merged 6 commits into from
Nov 8, 2024

Conversation

cjj2010
Copy link
Contributor

@cjj2010 cjj2010 commented Oct 29, 2024

pick from master：#41741

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@cjj2010 cjj2010 changed the title [cherry-pick](branch-2.1)add SessionVariable for enableCooldownReplicaAffinity [cherry-pick](branch-2.0)add SessionVariable for enableCooldownReplicaAffinity Oct 29, 2024
@github-actions github-actions bot added area/planner Issues or PRs related to the query planner kind/test labels Oct 29, 2024
@cjj2010
Copy link
Contributor Author

cjj2010 commented Oct 30, 2024

run buildall

@cjj2010 cjj2010 mentioned this pull request Oct 30, 2024
Merged
6 tasks
@cjj2010
Copy link
Contributor Author

cjj2010 commented Oct 30, 2024

run buildall

morningman pushed a commit to apache/doris-website that referenced this pull request Nov 5, 2024
@cjj2010
[doc](cooldown)add enable_cooldown_replica_affinity config (#1245)
1b30789 
# Versions 

- [x] dev
- [x] 3.0
- [x] 2.1
- [x] 2.0

# Languages

- [x] Chinese
- [x] English
 cherry-pick apache/doris#42683;
apache/doris#42677;
apache/doris#42675
@cjj2010
Copy link
Contributor Author

cjj2010 commented Nov 7, 2024

run buildall

@cjj2010
Copy link
Contributor Author

cjj2010 commented Nov 7, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 48941 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d8e7add5dfb7ff5e12a81260ed8c887151b1fa67, data reload: false

------ Round 1 ----------------------------------
q1	17915	4383	4332	4332
q2	2058	162	146	146
q3	10281	1900	1924	1900
q4	10112	1242	1354	1242
q5	8493	3924	3910	3910
q6	259	124	141	124
q7	2049	1584	1597	1584
q8	9507	2726	2714	2714
q9	12950	9793	9700	9700
q10	8645	3565	3547	3547
q11	406	256	247	247
q12	468	305	303	303
q13	18340	3935	4022	3935
q14	364	330	321	321
q15	497	454	461	454
q16	536	474	472	472
q17	1130	976	939	939
q18	7214	6928	6743	6743
q19	1677	1580	1539	1539
q20	522	321	297	297
q21	4418	4182	4089	4089
q22	500	407	403	403
Total cold run time: 118341 ms
Total hot run time: 48941 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4330	4298	4304	4298
q2	325	224	221	221
q3	4186	4150	4104	4104
q4	2748	2745	2732	2732
q5	7106	7124	7160	7124
q6	241	119	120	119
q7	3278	2919	2875	2875
q8	4391	4476	4513	4476
q9	13709	13576	13543	13543
q10	4266	4313	4270	4270
q11	724	698	691	691
q12	1025	852	855	852
q13	7013	3719	3706	3706
q14	450	431	430	430
q15	491	458	462	458
q16	620	588	579	579
q17	3872	3910	3915	3910
q18	8705	8732	8714	8714
q19	1726	1658	1667	1658
q20	2383	2090	2117	2090
q21	8510	8415	8546	8415
q22	1042	958	989	958
Total cold run time: 81141 ms
Total hot run time: 76223 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 211289 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d8e7add5dfb7ff5e12a81260ed8c887151b1fa67, data reload: false

query1	939	383	407	383
query2	6537	2204	2022	2022
query3	6923	197	197	197
query4	22936	21242	22279	21242
query5	19734	6499	6538	6499
query6	287	213	237	213
query7	4345	292	296	292
query8	259	259	223	223
query9	3023	2614	2564	2564
query10	424	315	299	299
query11	15469	14870	15037	14870
query12	123	86	73	73
query13	1039	443	431	431
query14	17318	13606	13403	13403
query15	366	216	224	216
query16	6476	281	263	263
query17	1972	940	883	883
query18	891	317	316	316
query19	208	145	155	145
query20	102	105	95	95
query21	193	96	98	96
query22	5089	5063	5055	5055
query23	34340	33539	33352	33352
query24	7791	6249	6333	6249
query25	521	427	429	427
query26	1268	160	157	157
query27	2431	295	287	287
query28	6091	2257	2211	2211
query29	2859	2753	2797	2753
query30	237	169	166	166
query31	969	737	733	733
query32	67	61	59	59
query33	448	265	252	252
query34	867	454	458	454
query35	1125	944	937	937
query36	1347	1170	1205	1170
query37	177	61	59	59
query38	3134	2940	2952	2940
query39	1372	1335	1300	1300
query40	312	98	90	90
query41	40	36	36	36
query42	91	82	89	82
query43	621	570	568	568
query44	1171	725	715	715
query45	240	233	227	227
query46	1230	945	946	945
query47	2083	1813	1853	1813
query48	504	408	405	405
query49	656	369	379	369
query50	876	598	605	598
query51	4773	4629	4663	4629
query52	96	74	73	73
query53	224	185	190	185
query54	2704	2521	2484	2484
query55	90	88	87	87
query56	224	205	206	205
query57	1308	1089	1186	1089
query58	216	219	216	216
query59	3374	3269	3100	3100
query60	224	219	217	217
query61	97	101	94	94
query62	827	443	455	443
query63	201	175	176	175
query64	3549	1573	1429	1429
query65	3626	3582	3604	3582
query66	769	402	442	402
query67	15974	17135	15070	15070
query68	10967	646	613	613
query69	498	252	264	252
query70	1977	1394	1432	1394
query71	410	307	303	303
query72	7034	4879	4830	4830
query73	769	318	324	318
query74	6456	5936	5856	5856
query75	5353	3695	3736	3695
query76	6660	1135	1214	1135
query77	1160	251	252	251
query78	12632	11611	11736	11611
query79	7142	626	640	626
query80	1245	382	385	382
query81	493	246	236	236
query82	1658	99	97	97
query83	163	138	130	130
query84	259	71	70	70
query85	879	315	315	315
query86	339	290	286	286
query87	3216	3094	3097	3094
query88	4259	2286	2278	2278
query89	480	277	298	277
query90	1972	208	215	208
query91	158	125	128	125
query92	61	53	53	53
query93	7125	555	584	555
query94	718	203	208	203
query95	1982	1935	1745	1745
query96	655	338	317	317
query97	6476	6475	6360	6360
query98	239	209	191	191
query99	2965	886	869	869
Total cold run time: 322197 ms
Total hot run time: 211289 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.91 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d8e7add5dfb7ff5e12a81260ed8c887151b1fa67, data reload: false

query1	0.02	0.02	0.03
query2	0.08	0.03	0.02
query3	0.25	0.06	0.06
query4	1.78	0.06	0.06
query5	0.54	0.53	0.53
query6	1.22	0.62	0.62
query7	0.01	0.02	0.01
query8	0.03	0.02	0.02
query9	0.53	0.50	0.47
query10	0.54	0.54	0.53
query11	0.12	0.08	0.09
query12	0.12	0.09	0.10
query13	0.62	0.62	0.61
query14	0.76	0.78	0.79
query15	0.77	0.76	0.76
query16	0.40	0.39	0.36
query17	0.97	1.00	0.99
query18	0.24	0.26	0.25
query19	1.92	1.86	1.87
query20	0.01	0.01	0.01
query21	15.45	0.54	0.55
query22	2.35	1.78	1.73
query23	17.20	1.07	0.92
query24	4.23	1.01	1.28
query25	0.36	0.07	0.05
query26	0.55	0.16	0.14
query27	0.05	0.05	0.04
query28	8.58	0.74	0.71
query29	12.65	2.36	2.37
query30	0.58	0.54	0.49
query31	2.81	0.40	0.37
query32	3.37	0.49	0.49
query33	3.05	3.09	3.07
query34	15.27	4.78	4.79
query35	4.87	4.85	4.83
query36	1.07	1.01	1.01
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.07	0.01	0.02
query42	0.02	0.02	0.01
query43	0.02	0.02	0.01
Total cold run time: 103.76 s
Total hot run time: 30.91 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit d8e7add5dfb7ff5e12a81260ed8c887151b1fa67 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       20.7 seconds inserted 10000000 Rows, about 483K ops/s

@dataroaring dataroaring merged commit 4b3aa2e into apache:branch-2.0 Nov 8, 2024
21 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
area/planner Issues or PRs related to the query planner kind/behavior-changed kind/test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@cjj2010 @doris-robot @dataroaring