Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature](restore) Support clean_tables/clean_partitions properties for restore job #39028

Merged
merged 2 commits into from
Aug 14, 2024

Conversation

w41ter
Copy link
Contributor

@w41ter w41ter commented Aug 7, 2024

The restore will keep the existing tables/partitions in which the restore target is not contained, this PR adds a property clean_restore to indicate that the restore job needs to recycle those tables/partitions

The CCR part PR is: selectdb/ccr-syncer#128.
The website PR is: apache/doris-website#999

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@w41ter
Copy link
Contributor Author

w41ter commented Aug 7, 2024

run buildall

Copy link
Contributor

github-actions bot commented Aug 7, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 41698 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7b552a9cc92330bd56942ac44cd6524791329614, data reload: false

------ Round 1 ----------------------------------
q1	17615	4349	4027	4027
q2	2025	198	195	195
q3	10525	1306	1344	1306
q4	10198	849	912	849
q5	7643	3042	2983	2983
q6	221	144	144	144
q7	1076	633	631	631
q8	9464	1794	1984	1794
q9	8547	6597	6594	6594
q10	8747	3847	3852	3847
q11	432	253	247	247
q12	413	237	231	231
q13	17767	2957	2947	2947
q14	273	241	239	239
q15	523	479	494	479
q16	528	399	384	384
q17	995	955	947	947
q18	8156	7335	7241	7241
q19	2092	1227	1215	1215
q20	555	326	343	326
q21	5368	4788	4818	4788
q22	350	285	284	284
Total cold run time: 113513 ms
Total hot run time: 41698 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4100	4002	4014	4002
q2	334	232	237	232
q3	3024	3020	3187	3020
q4	2016	2014	1991	1991
q5	5661	5532	5502	5502
q6	221	134	131	131
q7	2142	1826	1811	1811
q8	3315	3391	3378	3378
q9	8727	8674	8807	8674
q10	3962	4003	3907	3907
q11	563	457	466	457
q12	790	574	618	574
q13	16440	3152	3140	3140
q14	332	270	276	270
q15	525	475	484	475
q16	457	431	414	414
q17	1776	1762	1755	1755
q18	8327	8023	7681	7681
q19	1729	1776	1737	1737
q20	2051	1841	1833	1833
q21	5710	5496	5483	5483
q22	517	472	463	463
Total cold run time: 72719 ms
Total hot run time: 56930 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169344 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7b552a9cc92330bd56942ac44cd6524791329614, data reload: false

query1	932	369	374	369
query2	6472	1742	1733	1733
query3	6649	210	219	210
query4	19147	17481	17229	17229
query5	3638	522	526	522
query6	300	181	182	181
query7	4589	297	304	297
query8	254	201	199	199
query9	8515	2329	2320	2320
query10	423	278	272	272
query11	10459	10009	9927	9927
query12	127	87	84	84
query13	1622	372	361	361
query14	9987	7015	7167	7015
query15	210	164	167	164
query16	6964	471	445	445
query17	923	546	540	540
query18	1915	277	305	277
query19	184	142	146	142
query20	90	86	84	84
query21	199	101	98	98
query22	4174	4079	4068	4068
query23	33831	33601	33823	33601
query24	10060	3086	3142	3086
query25	671	407	398	398
query26	1607	159	160	159
query27	2935	272	289	272
query28	7365	1986	1973	1973
query29	1101	439	438	438
query30	246	159	159	159
query31	955	726	762	726
query32	106	58	56	56
query33	678	323	321	321
query34	936	492	493	492
query35	878	796	747	747
query36	1044	913	925	913
query37	181	85	84	84
query38	2899	2826	2806	2806
query39	907	804	823	804
query40	274	120	118	118
query41	46	44	45	44
query42	121	106	103	103
query43	512	423	444	423
query44	1181	739	734	734
query45	202	176	174	174
query46	1087	808	795	795
query47	1814	1703	1738	1703
query48	353	288	291	288
query49	947	429	421	421
query50	903	430	429	429
query51	6758	6790	6693	6693
query52	102	91	92	91
query53	260	184	186	184
query54	623	457	453	453
query55	77	75	74	74
query56	268	256	253	253
query57	1154	1070	1036	1036
query58	264	263	270	263
query59	2672	2318	2351	2318
query60	290	286	276	276
query61	93	91	95	91
query62	881	660	657	657
query63	207	181	177	177
query64	5732	1920	1856	1856
query65	3159	3092	3102	3092
query66	1320	318	333	318
query67	15358	14769	14767	14767
query68	4329	559	580	559
query69	447	315	307	307
query70	1106	1063	1048	1048
query71	380	282	275	275
query72	7069	2665	2451	2451
query73	765	329	327	327
query74	6046	5647	5579	5579
query75	3375	2686	2717	2686
query76	2129	1230	1277	1230
query77	445	321	316	316
query78	9451	8900	8856	8856
query79	1937	542	538	538
query80	1074	492	515	492
query81	580	230	223	223
query82	1063	128	132	128
query83	235	172	164	164
query84	266	77	80	77
query85	1238	306	294	294
query86	397	316	322	316
query87	3318	3074	3073	3073
query88	2886	2402	2396	2396
query89	376	292	283	283
query90	1694	192	193	192
query91	125	103	99	99
query92	58	49	52	49
query93	1540	627	633	627
query94	785	345	301	301
query95	374	264	264	264
query96	584	284	275	275
query97	3263	3091	3050	3050
query98	232	199	196	196
query99	1620	1270	1270	1270
Total cold run time: 260018 ms
Total hot run time: 169344 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.73 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7b552a9cc92330bd56942ac44cd6524791329614, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.03	0.04
query3	0.22	0.05	0.05
query4	1.67	0.07	0.07
query5	0.48	0.48	0.49
query6	1.12	0.71	0.71
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.57	0.50	0.51
query10	0.56	0.56	0.57
query11	0.15	0.12	0.12
query12	0.15	0.12	0.12
query13	0.60	0.59	0.59
query14	0.77	0.78	0.81
query15	0.90	0.86	0.85
query16	0.36	0.35	0.35
query17	1.00	0.99	0.98
query18	0.21	0.20	0.20
query19	1.89	1.76	1.74
query20	0.02	0.01	0.01
query21	15.43	0.75	0.67
query22	4.18	7.36	1.09
query23	17.80	1.28	1.29
query24	2.24	0.23	0.21
query25	0.19	0.07	0.08
query26	0.32	0.22	0.21
query27	0.46	0.24	0.22
query28	13.20	1.00	0.97
query29	12.57	3.33	3.30
query30	0.25	0.06	0.06
query31	2.86	0.40	0.41
query32	3.23	0.49	0.49
query33	2.96	2.96	2.92
query34	15.47	4.29	4.22
query35	4.28	4.30	4.30
query36	0.68	0.49	0.50
query37	0.20	0.16	0.16
query38	0.16	0.15	0.14
query39	0.04	0.04	0.03
query40	0.16	0.13	0.14
query41	0.11	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.05	0.04
Total cold run time: 107.75 s
Total hot run time: 29.73 s

@w41ter w41ter marked this pull request as draft August 8, 2024 03:05
@w41ter w41ter changed the title [feature](restore) Support clean_restore property for restore job [feature](restore) Support clean_tables/clean_partitions properties for restore job Aug 8, 2024
@w41ter
Copy link
Contributor Author

w41ter commented Aug 8, 2024

run buildall

Copy link
Contributor

github-actions bot commented Aug 8, 2024

clang-tidy review says "All clean, LGTM! 👍"

@w41ter w41ter force-pushed the clear_restore branch 2 times, most recently from 418ff79 to c2b33c9 Compare August 8, 2024 04:02
Copy link
Contributor

github-actions bot commented Aug 8, 2024

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
Copy link
Contributor

github-actions bot commented Aug 8, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39778 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 96336a67c5df8d187b2f1931ee2bfffb415a9324, data reload: false

------ Round 1 ----------------------------------
q1	18358	5149	4298	4298
q2	2025	178	173	173
q3	10536	1130	1087	1087
q4	10150	729	717	717
q5	7513	2565	2547	2547
q6	226	135	136	135
q7	962	594	582	582
q8	9214	1924	1967	1924
q9	8884	6525	6614	6525
q10	7056	2213	2205	2205
q11	449	238	237	237
q12	390	214	213	213
q13	17756	2964	2996	2964
q14	297	239	245	239
q15	529	494	486	486
q16	509	384	388	384
q17	997	697	804	697
q18	8194	7604	7517	7517
q19	5453	986	1088	986
q20	675	346	328	328
q21	5491	4663	4518	4518
q22	1118	1035	1016	1016
Total cold run time: 116782 ms
Total hot run time: 39778 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4480	4298	4266	4266
q2	393	275	261	261
q3	2866	2722	2757	2722
q4	2004	1748	1729	1729
q5	5594	5593	5474	5474
q6	234	139	132	132
q7	2182	1795	1804	1795
q8	3326	3439	3448	3439
q9	8816	8819	8737	8737
q10	3592	3226	3182	3182
q11	625	513	521	513
q12	807	684	622	622
q13	16261	3099	3182	3099
q14	307	288	298	288
q15	534	488	489	488
q16	500	430	431	430
q17	1846	1550	1531	1531
q18	8079	7918	7833	7833
q19	1785	1620	1584	1584
q20	2181	1901	1883	1883
q21	5542	5512	5184	5184
q22	1147	1045	1034	1034
Total cold run time: 73101 ms
Total hot run time: 56226 ms

@w41ter
Copy link
Contributor Author

w41ter commented Aug 8, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39636 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c2b33c9120526b78b95c33802e1103d55b85c877, data reload: false

------ Round 1 ----------------------------------
q1	18345	5835	4398	4398
q2	2024	176	176	176
q3	10624	1130	990	990
q4	10343	755	779	755
q5	7528	2569	2528	2528
q6	225	139	138	138
q7	969	589	593	589
q8	9208	1912	1936	1912
q9	8646	6623	6568	6568
q10	7060	2224	2199	2199
q11	486	247	250	247
q12	444	224	227	224
q13	18778	2945	2999	2945
q14	289	236	235	235
q15	538	473	493	473
q16	521	390	392	390
q17	991	650	681	650
q18	8198	7618	7382	7382
q19	3094	1122	1025	1025
q20	664	328	315	315
q21	5307	4493	4668	4493
q22	1141	1061	1004	1004
Total cold run time: 115423 ms
Total hot run time: 39636 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4468	4300	4300	4300
q2	364	269	273	269
q3	2845	2674	2783	2674
q4	1981	1740	1658	1658
q5	5604	5551	5394	5394
q6	225	133	132	132
q7	2152	1857	1790	1790
q8	3329	3482	3464	3464
q9	8797	8907	8850	8850
q10	3546	3302	3182	3182
q11	581	499	499	499
q12	821	644	635	635
q13	16485	3181	3191	3181
q14	322	280	296	280
q15	536	500	497	497
q16	518	431	433	431
q17	1822	1558	1536	1536
q18	8037	7964	7816	7816
q19	1759	1668	1542	1542
q20	2128	1916	1879	1879
q21	7176	5510	5275	5275
q22	1110	1040	1079	1040
Total cold run time: 74606 ms
Total hot run time: 56324 ms

@w41ter
Copy link
Contributor Author

w41ter commented Aug 8, 2024

run buildall

Copy link
Contributor

github-actions bot commented Aug 8, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39725 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c6390c3199baf35d94646e9933e44664d2dcadf1, data reload: false

------ Round 1 ----------------------------------
q1	17635	4479	4353	4353
q2	2023	183	174	174
q3	10498	1154	1131	1131
q4	10140	772	751	751
q5	7501	2613	2547	2547
q6	230	141	140	140
q7	982	595	603	595
q8	9220	2073	1926	1926
q9	8662	6572	6583	6572
q10	7064	2207	2169	2169
q11	481	246	247	246
q12	398	234	225	225
q13	19121	3008	2989	2989
q14	284	237	239	237
q15	532	489	481	481
q16	513	401	388	388
q17	1018	577	723	577
q18	7914	7453	7488	7453
q19	7082	1005	979	979
q20	670	329	349	329
q21	5436	4461	4689	4461
q22	1098	1028	1002	1002
Total cold run time: 118502 ms
Total hot run time: 39725 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4535	4310	4283	4283
q2	369	277	268	268
q3	2897	2652	2645	2645
q4	1889	1667	1655	1655
q5	5263	5306	5292	5292
q6	223	132	132	132
q7	2086	1672	1656	1656
q8	3148	3353	3294	3294
q9	8382	8323	8347	8323
q10	3385	3169	3145	3145
q11	587	489	494	489
q12	771	630	613	613
q13	17577	3012	2980	2980
q14	301	273	272	272
q15	505	481	481	481
q16	466	416	432	416
q17	1813	1507	1509	1507
q18	7667	7413	7310	7310
q19	1726	1581	1606	1581
q20	1973	1782	1781	1781
q21	5341	5170	4981	4981
q22	1064	1001	1014	1001
Total cold run time: 71968 ms
Total hot run time: 54105 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202415 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c6390c3199baf35d94646e9933e44664d2dcadf1, data reload: false

query1	946	426	401	401
query2	6478	2011	1911	1911
query3	6638	206	222	206
query4	29645	23120	23146	23120
query5	4212	501	497	497
query6	287	184	187	184
query7	4584	309	296	296
query8	259	195	198	195
query9	8447	2383	2366	2366
query10	933	879	905	879
query11	17537	14937	14848	14848
query12	151	98	99	98
query13	1661	417	380	380
query14	10649	7996	7774	7774
query15	388	327	314	314
query16	7932	474	487	474
query17	1704	548	536	536
query18	1926	367	359	359
query19	257	184	188	184
query20	110	103	102	102
query21	203	120	103	103
query22	4214	4004	4006	4004
query23	33890	32875	33134	32875
query24	11168	3036	2931	2931
query25	601	369	373	369
query26	1076	154	155	154
query27	2785	280	285	280
query28	7366	2046	2013	2013
query29	808	413	426	413
query30	306	144	145	144
query31	978	747	779	747
query32	95	55	53	53
query33	751	283	276	276
query34	985	470	481	470
query35	989	861	852	852
query36	1088	917	933	917
query37	160	85	78	78
query38	4165	4089	4084	4084
query39	1437	1343	1343	1343
query40	198	117	114	114
query41	49	45	47	45
query42	114	95	95	95
query43	510	477	472	472
query44	1244	746	755	746
query45	394	378	383	378
query46	1123	811	776	776
query47	1839	1749	1757	1749
query48	365	311	308	308
query49	1113	430	424	424
query50	810	412	412	412
query51	6850	6666	6628	6628
query52	97	89	88	88
query53	247	187	185	185
query54	882	451	466	451
query55	78	74	76	74
query56	258	239	248	239
query57	1175	1049	1071	1049
query58	242	225	227	225
query59	3167	2720	2847	2720
query60	284	260	253	253
query61	97	95	100	95
query62	817	649	638	638
query63	215	184	182	182
query64	9748	2454	1938	1938
query65	3205	3156	3141	3141
query66	954	340	330	330
query67	15133	14898	14892	14892
query68	4519	542	556	542
query69	458	393	417	393
query70	1186	1130	1031	1031
query71	400	273	265	265
query72	19186	17022	16697	16697
query73	763	337	342	337
query74	9257	8769	8760	8760
query75	3375	2676	2710	2676
query76	2505	1055	963	963
query77	492	305	311	305
query78	9518	9081	8957	8957
query79	2360	529	546	529
query80	2130	497	499	497
query81	617	230	232	230
query82	686	138	137	137
query83	296	152	156	152
query84	263	83	82	82
query85	2242	313	305	305
query86	473	295	284	284
query87	4698	4460	4414	4414
query88	4141	2504	2495	2495
query89	393	286	292	286
query90	2055	203	204	203
query91	151	127	124	124
query92	70	51	52	51
query93	1956	553	540	540
query94	1140	304	303	303
query95	367	343	261	261
query96	602	284	282	282
query97	3235	3041	3029	3029
query98	213	204	195	195
query99	1504	1213	1262	1213
Total cold run time: 310268 ms
Total hot run time: 202415 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.75 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c6390c3199baf35d94646e9933e44664d2dcadf1, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.05
query3	0.23	0.05	0.05
query4	1.67	0.09	0.08
query5	0.50	0.48	0.48
query6	1.13	0.74	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.55	0.50	0.48
query10	0.53	0.54	0.53
query11	0.15	0.12	0.12
query12	0.15	0.12	0.12
query13	0.58	0.59	0.60
query14	0.76	0.80	0.78
query15	0.85	0.88	0.82
query16	0.38	0.37	0.37
query17	1.03	0.97	1.01
query18	0.23	0.22	0.22
query19	1.77	1.77	1.71
query20	0.02	0.01	0.01
query21	15.41	0.77	0.65
query22	4.26	7.63	2.01
query23	18.33	1.40	1.30
query24	2.19	0.21	0.23
query25	0.15	0.09	0.08
query26	0.29	0.20	0.20
query27	0.46	0.22	0.22
query28	13.27	1.02	1.02
query29	12.68	3.32	3.28
query30	0.27	0.06	0.06
query31	2.86	0.38	0.40
query32	3.28	0.48	0.48
query33	2.91	2.96	2.91
query34	16.98	4.31	4.35
query35	4.41	4.39	4.45
query36	0.66	0.46	0.48
query37	0.19	0.16	0.15
query38	0.15	0.15	0.15
query39	0.05	0.04	0.04
query40	0.15	0.13	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.87 s
Total hot run time: 30.75 s

@w41ter
Copy link
Contributor Author

w41ter commented Aug 9, 2024

run buildall

Copy link
Contributor

github-actions bot commented Aug 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39775 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e8dbc3de62ad2fe26026456f72994cd52a51ddca, data reload: false

------ Round 1 ----------------------------------
q1	17938	4435	4308	4308
q2	2026	182	175	175
q3	11832	1015	1111	1015
q4	10543	825	726	726
q5	7762	2823	2839	2823
q6	225	134	133	133
q7	969	601	592	592
q8	9332	2080	2095	2080
q9	8716	6605	6538	6538
q10	7020	2239	2243	2239
q11	444	251	252	251
q12	406	226	222	222
q13	17761	2969	2993	2969
q14	291	246	242	242
q15	531	484	493	484
q16	490	393	381	381
q17	973	601	688	601
q18	8094	7576	7316	7316
q19	4854	1084	1041	1041
q20	681	331	337	331
q21	5306	4299	4604	4299
q22	1127	1040	1009	1009
Total cold run time: 117321 ms
Total hot run time: 39775 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4447	4292	4283	4283
q2	371	267	278	267
q3	2820	2746	2784	2746
q4	2070	1674	1727	1674
q5	5696	5625	5598	5598
q6	224	143	132	132
q7	2165	1815	1785	1785
q8	3317	3475	3424	3424
q9	8807	8900	8808	8808
q10	3583	3238	3216	3216
q11	615	507	521	507
q12	824	653	664	653
q13	16012	3204	3144	3144
q14	334	290	295	290
q15	531	490	496	490
q16	500	445	447	445
q17	1832	1566	1516	1516
q18	8191	7982	7795	7795
q19	4279	1617	1657	1617
q20	2950	1917	1897	1897
q21	5487	5483	5107	5107
q22	1139	1051	1025	1025
Total cold run time: 76194 ms
Total hot run time: 56419 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189419 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e8dbc3de62ad2fe26026456f72994cd52a51ddca, data reload: false

query1	1247	882	861	861
query2	6451	1849	1921	1849
query3	10747	4030	3912	3912
query4	58911	25772	23046	23046
query5	5662	485	473	473
query6	510	188	170	170
query7	6405	290	292	290
query8	294	194	192	192
query9	8851	2412	2379	2379
query10	492	263	255	255
query11	18048	14948	15291	14948
query12	168	101	97	97
query13	1557	405	377	377
query14	11688	8475	7394	7394
query15	272	175	181	175
query16	7570	515	449	449
query17	1118	618	566	566
query18	1441	310	302	302
query19	282	138	137	137
query20	119	111	111	111
query21	212	97	100	97
query22	4575	4314	4391	4314
query23	34439	33227	33291	33227
query24	5600	2863	2844	2844
query25	511	366	381	366
query26	682	157	155	155
query27	1792	271	276	271
query28	3492	2031	2020	2020
query29	638	413	398	398
query30	231	146	146	146
query31	917	741	745	741
query32	72	53	53	53
query33	456	287	273	273
query34	855	452	478	452
query35	817	718	701	701
query36	1046	941	901	901
query37	135	78	80	78
query38	3922	3871	3813	3813
query39	1471	1391	1402	1391
query40	190	116	109	109
query41	42	45	43	43
query42	118	97	93	93
query43	500	455	479	455
query44	1090	735	742	735
query45	196	164	159	159
query46	1092	704	727	704
query47	1840	1824	1719	1719
query48	366	298	291	291
query49	750	407	425	407
query50	800	396	404	396
query51	6874	6714	6757	6714
query52	100	97	86	86
query53	256	180	184	180
query54	562	440	439	439
query55	76	74	73	73
query56	276	242	276	242
query57	1111	1041	1044	1041
query58	223	219	232	219
query59	2854	2740	2676	2676
query60	277	256	261	256
query61	94	90	93	90
query62	760	664	641	641
query63	207	178	176	176
query64	3316	1708	1769	1708
query65	3203	3139	3126	3126
query66	676	326	328	326
query67	14949	14728	14659	14659
query68	4609	553	556	553
query69	420	273	268	268
query70	1180	1113	1113	1113
query71	369	277	268	268
query72	7013	2348	2019	2019
query73	746	327	323	323
query74	9182	8710	8725	8710
query75	3389	2692	2716	2692
query76	2082	998	955	955
query77	526	320	295	295
query78	9445	9267	9039	9039
query79	2238	528	526	526
query80	949	479	480	479
query81	553	217	220	217
query82	477	138	139	138
query83	247	147	144	144
query84	271	74	73	73
query85	1142	271	272	271
query86	442	302	279	279
query87	4323	4268	4199	4199
query88	4050	2414	2408	2408
query89	381	291	278	278
query90	1901	209	192	192
query91	119	101	97	97
query92	59	48	51	48
query93	1773	538	539	538
query94	817	295	297	295
query95	345	255	255	255
query96	596	276	271	271
query97	3204	3071	3035	3035
query98	208	203	199	199
query99	1574	1339	1255	1255
Total cold run time: 311124 ms
Total hot run time: 189419 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.44 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e8dbc3de62ad2fe26026456f72994cd52a51ddca, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.69	0.07	0.07
query5	0.50	0.48	0.48
query6	1.15	0.72	0.72
query7	0.01	0.01	0.02
query8	0.05	0.04	0.05
query9	0.55	0.48	0.49
query10	0.53	0.53	0.54
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.59	0.59	0.59
query14	0.77	0.78	0.79
query15	0.88	0.82	0.82
query16	0.36	0.38	0.38
query17	1.00	1.05	0.95
query18	0.23	0.22	0.22
query19	1.92	1.80	1.68
query20	0.03	0.00	0.00
query21	15.40	0.77	0.66
query22	4.45	7.58	1.66
query23	18.26	1.49	1.29
query24	2.11	0.23	0.21
query25	0.16	0.09	0.08
query26	0.29	0.21	0.22
query27	0.46	0.23	0.22
query28	13.29	1.01	0.99
query29	12.59	3.38	3.32
query30	0.24	0.05	0.05
query31	2.96	0.39	0.39
query32	3.25	0.49	0.48
query33	2.91	2.96	2.98
query34	16.84	4.37	4.38
query35	4.45	4.39	4.41
query36	0.66	0.49	0.46
query37	0.19	0.15	0.15
query38	0.16	0.15	0.14
query39	0.04	0.03	0.04
query40	0.15	0.12	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.06
query43	0.05	0.04	0.04
Total cold run time: 109.99 s
Total hot run time: 30.44 s

Copy link
Member

@xy720 xy720 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, can you update the docs about these two properties?

@w41ter w41ter requested a review from xy720 August 14, 2024 02:35
@w41ter
Copy link
Contributor Author

w41ter commented Aug 14, 2024

Hi, can you update the docs about these two properties?

updated

Copy link
Member

@xy720 xy720 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 14, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 73787b6 into apache:master Aug 14, 2024
30 of 33 checks passed
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Aug 14, 2024
…or restore job (apache#39028)

The restore will keep the existing tables/partitions in which the
restore target is not contained, this PR adds a property clean_restore
to indicate that the restore job needs to recycle those
tables/partitions

The CCR part PR is: selectdb/ccr-syncer#128.
The website PR is: apache/doris-website#999
@w41ter w41ter deleted the clear_restore branch August 14, 2024 11:13
w41ter added a commit to w41ter/incubator-doris that referenced this pull request Aug 14, 2024
…or restore job (apache#39028)

The restore will keep the existing tables/partitions in which the
restore target is not contained, this PR adds a property clean_restore
to indicate that the restore job needs to recycle those
tables/partitions

The CCR part PR is: selectdb/ccr-syncer#128.
The website PR is: apache/doris-website#999
w41ter added a commit to w41ter/incubator-doris that referenced this pull request Aug 14, 2024
…or restore job (apache#39028)

The restore will keep the existing tables/partitions in which the
restore target is not contained, this PR adds a property clean_restore
to indicate that the restore job needs to recycle those
tables/partitions

The CCR part PR is: selectdb/ccr-syncer#128.
The website PR is: apache/doris-website#999
dataroaring pushed a commit that referenced this pull request Aug 15, 2024
w41ter added a commit to w41ter/incubator-doris that referenced this pull request Aug 15, 2024
…or restore job (apache#39028)

The restore will keep the existing tables/partitions in which the
restore target is not contained, this PR adds a property clean_restore
to indicate that the restore job needs to recycle those
tables/partitions

The CCR part PR is: selectdb/ccr-syncer#128.
The website PR is: apache/doris-website#999
w41ter added a commit to w41ter/incubator-doris that referenced this pull request Aug 15, 2024
…or restore job (apache#39028)

The restore will keep the existing tables/partitions in which the
restore target is not contained, this PR adds a property clean_restore
to indicate that the restore job needs to recycle those
tables/partitions

The CCR part PR is: selectdb/ccr-syncer#128.
The website PR is: apache/doris-website#999
w41ter added a commit to w41ter/incubator-doris that referenced this pull request Aug 15, 2024
…or restore job (apache#39028)

The restore will keep the existing tables/partitions in which the
restore target is not contained, this PR adds a property clean_restore
to indicate that the restore job needs to recycle those
tables/partitions

The CCR part PR is: selectdb/ccr-syncer#128.
The website PR is: apache/doris-website#999
w41ter added a commit that referenced this pull request Aug 15, 2024
dataroaring pushed a commit that referenced this pull request Aug 17, 2024
…or restore job (#39028)

The restore will keep the existing tables/partitions in which the
restore target is not contained, this PR adds a property clean_restore
to indicate that the restore job needs to recycle those
tables/partitions

The CCR part PR is: selectdb/ccr-syncer#128.
The website PR is: apache/doris-website#999
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants