[refactore](cloud) separate cloud restore create tablet rpc into multiple batches #54933

xy720 · 2025-08-18T07:42:27Z

What problem does this PR solve?

Problem Summary:

1、Add a config cloud_restore_create_tablet_batch_size default 256, separate cloud restore create tablet rpc into multiple batches to prevent rpc timeout
2、Fix replica's partition id incorrect in restored table.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-08-18T07:42:32Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

xy720 · 2025-08-18T07:44:15Z

run buildall

hello-stephen · 2025-08-18T08:06:03Z

Cloud UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.97% (1441/1716)
Line Coverage	66.95% (24791/37030)
Region Coverage	67.97% (12349/18167)
Branch Coverage	57.60% (6492/11270)

doris-robot · 2025-08-18T08:33:28Z

TPC-H: Total hot run time: 33809 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit de31c89967c80d3eeb50b6e11a60ada8bc659162, data reload: false

------ Round 1 ----------------------------------
q1	17578	5232	5113	5113
q2	1922	288	182	182
q3	10301	1295	709	709
q4	10220	989	522	522
q5	7479	2384	2323	2323
q6	180	160	131	131
q7	905	737	596	596
q8	9291	1294	1045	1045
q9	6875	5170	5126	5126
q10	6883	2381	1997	1997
q11	473	285	270	270
q12	338	349	228	228
q13	17765	3639	2982	2982
q14	230	232	213	213
q15	550	500	474	474
q16	416	431	373	373
q17	590	852	349	349
q18	7333	7209	7099	7099
q19	1274	938	555	555
q20	346	334	219	219
q21	3869	2550	2337	2337
q22	1095	1025	966	966
Total cold run time: 105913 ms
Total hot run time: 33809 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5205	5132	5094	5094
q2	241	315	223	223
q3	2152	2700	2336	2336
q4	1399	1746	1337	1337
q5	4181	4305	4571	4305
q6	213	168	142	142
q7	1986	2035	1844	1844
q8	2644	2568	2544	2544
q9	7383	7364	7322	7322
q10	3110	3335	2848	2848
q11	585	513	491	491
q12	695	836	676	676
q13	3570	3860	3278	3278
q14	275	292	293	292
q15	518	483	473	473
q16	442	513	454	454
q17	1201	1618	1401	1401
q18	7898	7682	7547	7547
q19	826	795	838	795
q20	1890	2048	1809	1809
q21	4753	4401	4273	4273
q22	1119	1044	1005	1005
Total cold run time: 52286 ms
Total hot run time: 50489 ms

doris-robot · 2025-08-18T08:44:57Z

TPC-DS: Total hot run time: 185344 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit de31c89967c80d3eeb50b6e11a60ada8bc659162, data reload: false

query1	1004	377	414	377
query2	6510	1763	1799	1763
query3	6745	229	218	218
query4	26383	23374	23107	23107
query5	4376	655	491	491
query6	319	217	198	198
query7	4627	511	290	290
query8	275	243	230	230
query9	8628	2862	2863	2862
query10	480	358	284	284
query11	15882	15146	15019	15019
query12	164	117	114	114
query13	1654	542	418	418
query14	9532	5755	5708	5708
query15	205	203	169	169
query16	7489	623	524	524
query17	1171	704	600	600
query18	2007	398	311	311
query19	189	205	160	160
query20	124	119	114	114
query21	210	128	119	119
query22	4129	4103	3979	3979
query23	34417	33174	33160	33160
query24	8160	2389	2398	2389
query25	547	478	392	392
query26	1225	270	153	153
query27	2757	514	351	351
query28	4382	2244	2209	2209
query29	713	609	453	453
query30	289	225	186	186
query31	900	813	740	740
query32	89	75	77	75
query33	553	363	349	349
query34	773	874	516	516
query35	808	824	746	746
query36	954	1021	913	913
query37	122	105	87	87
query38	4061	4067	4099	4067
query39	1479	1409	1387	1387
query40	224	125	118	118
query41	61	57	57	57
query42	129	109	118	109
query43	492	500	459	459
query44	1365	854	850	850
query45	174	172	163	163
query46	835	1015	658	658
query47	1742	1791	1734	1734
query48	400	418	300	300
query49	735	504	391	391
query50	666	697	394	394
query51	4068	4263	4141	4141
query52	115	133	100	100
query53	232	255	200	200
query54	599	579	527	527
query55	89	86	86	86
query56	296	313	298	298
query57	1160	1178	1139	1139
query58	279	266	273	266
query59	2644	2812	2571	2571
query60	350	343	323	323
query61	127	121	127	121
query62	815	725	649	649
query63	234	199	190	190
query64	4295	1023	746	746
query65	4345	4209	4203	4203
query66	1113	412	321	321
query67	15627	15133	14933	14933
query68	6853	930	579	579
query69	485	322	282	282
query70	1177	1156	1145	1145
query71	412	340	313	313
query72	5537	4772	4792	4772
query73	665	634	354	354
query74	8861	8994	9019	8994
query75	3127	3125	2625	2625
query76	3241	1160	769	769
query77	482	422	318	318
query78	9472	9707	8829	8829
query79	2038	830	594	594
query80	641	559	472	472
query81	490	253	224	224
query82	198	145	113	113
query83	255	256	242	242
query84	245	97	85	85
query85	756	369	392	369
query86	379	301	292	292
query87	4219	4272	4133	4133
query88	2811	2193	2188	2188
query89	381	313	287	287
query90	1754	230	225	225
query91	142	143	115	115
query92	75	70	70	70
query93	1247	987	646	646
query94	630	406	307	307
query95	396	314	307	307
query96	484	594	270	270
query97	2574	2662	2624	2624
query98	244	216	212	212
query99	1356	1433	1314	1314
Total cold run time: 269170 ms
Total hot run time: 185344 ms

doris-robot · 2025-08-18T08:50:25Z

ClickBench: Total hot run time: 33.03 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit de31c89967c80d3eeb50b6e11a60ada8bc659162, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.04	0.04
query3	0.25	0.08	0.07
query4	1.61	0.11	0.12
query5	0.43	0.41	0.40
query6	1.16	0.64	0.66
query7	0.03	0.01	0.02
query8	0.05	0.04	0.04
query9	0.60	0.54	0.52
query10	0.57	0.58	0.56
query11	0.16	0.11	0.11
query12	0.14	0.11	0.12
query13	0.63	0.61	0.60
query14	0.80	0.83	0.84
query15	0.87	0.87	0.87
query16	0.38	0.39	0.37
query17	1.01	1.05	1.05
query18	0.22	0.19	0.20
query19	1.92	1.75	1.87
query20	0.01	0.01	0.01
query21	15.40	1.02	0.59
query22	0.78	1.11	0.81
query23	14.95	1.40	0.63
query24	6.73	1.26	1.28
query25	0.51	0.21	0.07
query26	0.55	0.17	0.13
query27	0.06	0.05	0.05
query28	10.14	0.95	0.45
query29	12.58	3.94	3.25
query30	3.04	3.00	3.03
query31	2.82	0.58	0.37
query32	3.26	0.55	0.48
query33	3.08	3.07	3.09
query34	15.92	5.48	4.83
query35	4.91	4.90	4.91
query36	0.69	0.51	0.50
query37	0.09	0.07	0.07
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.14	0.12
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.91 s
Total hot run time: 33.03 s

hello-stephen · 2025-08-18T08:54:59Z

FE UT Coverage Report

Increment line coverage 73.77% (45/61) 🎉
Increment coverage report
Complete coverage report

xy720 · 2025-08-18T12:37:47Z

run cloud_p0

xy720 · 2025-08-27T08:21:03Z

run buildall

doris-robot · 2025-08-27T08:44:24Z

TPC-H: Total hot run time: 34290 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4625c90fdbd6a4046a06701e318f355e6de69a9c, data reload: false

------ Round 1 ----------------------------------
q1	17610	5245	5116	5116
q2	2015	345	217	217
q3	10209	1282	714	714
q4	10236	1030	523	523
q5	7560	2394	2380	2380
q6	184	172	137	137
q7	902	771	656	656
q8	9346	1366	1140	1140
q9	6942	5180	5287	5180
q10	6898	2403	1954	1954
q11	507	300	280	280
q12	347	373	217	217
q13	17771	3681	3023	3023
q14	231	238	215	215
q15	546	507	494	494
q16	445	436	383	383
q17	597	872	366	366
q18	7461	7178	7095	7095
q19	1099	967	572	572
q20	344	337	236	236
q21	3844	3130	2372	2372
q22	1090	1049	1020	1020
Total cold run time: 106184 ms
Total hot run time: 34290 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5335	5174	5112	5112
q2	262	332	227	227
q3	2164	2682	2333	2333
q4	1351	1813	1335	1335
q5	4215	4510	4616	4510
q6	236	182	139	139
q7	2146	1935	1829	1829
q8	2642	2724	2554	2554
q9	7493	7367	7369	7367
q10	3158	3269	2880	2880
q11	569	534	499	499
q12	703	802	668	668
q13	3641	3874	3234	3234
q14	300	309	281	281
q15	512	484	472	472
q16	458	508	472	472
q17	1221	1625	1431	1431
q18	8033	7693	7638	7638
q19	828	841	917	841
q20	2100	2071	1930	1930
q21	5119	4518	4323	4323
q22	1117	1064	1001	1001
Total cold run time: 53603 ms
Total hot run time: 51076 ms

doris-robot · 2025-08-27T08:46:45Z

Cloud UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.60% (1427/1707)
Line Coverage	67.19% (25364/37750)
Region Coverage	68.25% (12636/18514)
Branch Coverage	58.00% (6696/11544)

doris-robot · 2025-08-27T08:56:09Z

TPC-DS: Total hot run time: 187773 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4625c90fdbd6a4046a06701e318f355e6de69a9c, data reload: false

query1	1044	456	419	419
query2	6561	1762	1748	1748
query3	6776	229	225	225
query4	27182	23699	23467	23467
query5	5062	647	542	542
query6	343	254	226	226
query7	4656	508	300	300
query8	297	269	253	253
query9	8656	2921	2912	2912
query10	541	360	296	296
query11	16263	15003	15289	15003
query12	187	123	122	122
query13	1680	570	423	423
query14	9528	5877	6027	5877
query15	216	198	168	168
query16	7715	667	509	509
query17	1199	763	623	623
query18	2053	451	355	355
query19	200	198	175	175
query20	129	124	123	123
query21	223	140	115	115
query22	4004	4173	4003	4003
query23	33900	32903	32787	32787
query24	8124	2426	2388	2388
query25	584	516	490	490
query26	1238	290	173	173
query27	2690	529	369	369
query28	4406	2288	2278	2278
query29	775	631	541	541
query30	295	236	199	199
query31	926	831	749	749
query32	94	84	89	84
query33	590	413	363	363
query34	836	855	536	536
query35	841	860	777	777
query36	975	1029	950	950
query37	133	125	95	95
query38	4072	4107	4026	4026
query39	1502	1486	1433	1433
query40	237	148	139	139
query41	80	75	73	73
query42	137	121	119	119
query43	564	528	467	467
query44	1395	875	902	875
query45	181	217	172	172
query46	879	1025	666	666
query47	1769	1836	1737	1737
query48	407	456	342	342
query49	744	524	416	416
query50	672	698	402	402
query51	4183	4194	4168	4168
query52	122	121	109	109
query53	267	266	197	197
query54	624	612	542	542
query55	98	90	93	90
query56	361	331	323	323
query57	1195	1194	1102	1102
query58	297	281	288	281
query59	2659	2794	2706	2706
query60	368	362	352	352
query61	165	163	158	158
query62	797	734	668	668
query63	230	195	201	195
query64	4396	1117	837	837
query65	4328	4304	4240	4240
query66	1090	447	358	358
query67	15581	15422	15135	15135
query68	8551	944	593	593
query69	498	329	295	295
query70	1246	1121	1134	1121
query71	492	353	321	321
query72	5543	5074	5132	5074
query73	754	622	367	367
query74	9250	9118	8895	8895
query75	4238	3123	2619	2619
query76	3803	1165	745	745
query77	802	397	338	338
query78	9541	9776	8911	8911
query79	3033	803	595	595
query80	688	608	529	529
query81	488	258	228	228
query82	490	140	113	113
query83	297	263	243	243
query84	309	114	100	100
query85	899	480	475	475
query86	359	309	328	309
query87	4291	4253	4240	4240
query88	2971	2258	2242	2242
query89	446	333	298	298
query90	1988	228	219	219
query91	160	162	132	132
query92	90	76	75	75
query93	1675	1026	654	654
query94	700	420	327	327
query95	422	338	327	327
query96	503	591	281	281
query97	2710	2683	2592	2592
query98	248	226	224	224
query99	1465	1401	1300	1300
Total cold run time: 279272 ms
Total hot run time: 187773 ms

doris-robot · 2025-08-27T09:01:49Z

ClickBench: Total hot run time: 32.75 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4625c90fdbd6a4046a06701e318f355e6de69a9c, data reload: false

query1	0.06	0.04	0.04
query2	0.10	0.05	0.06
query3	0.25	0.08	0.08
query4	1.60	0.11	0.11
query5	0.48	0.43	0.41
query6	1.16	0.63	0.67
query7	0.04	0.03	0.03
query8	0.06	0.04	0.05
query9	0.61	0.54	0.53
query10	0.59	0.60	0.58
query11	0.17	0.11	0.11
query12	0.16	0.12	0.13
query13	0.63	0.63	0.64
query14	0.79	0.86	0.82
query15	0.89	0.86	0.87
query16	0.40	0.42	0.41
query17	1.04	1.03	1.05
query18	0.22	0.21	0.20
query19	1.96	1.87	1.78
query20	0.02	0.01	0.01
query21	15.41	0.96	0.59
query22	0.80	1.24	0.71
query23	14.76	1.36	0.62
query24	5.90	2.15	0.67
query25	0.58	0.12	0.08
query26	0.58	0.16	0.14
query27	0.06	0.06	0.06
query28	10.14	0.90	0.44
query29	12.60	3.96	3.28
query30	3.06	2.99	2.96
query31	2.81	0.58	0.38
query32	3.24	0.59	0.47
query33	3.15	3.17	3.18
query34	16.06	5.46	4.92
query35	4.91	4.95	4.91
query36	0.70	0.51	0.50
query37	0.10	0.07	0.08
query38	0.06	0.04	0.05
query39	0.03	0.03	0.03
query40	0.20	0.15	0.14
query41	0.08	0.03	0.03
query42	0.03	0.03	0.03
query43	0.04	0.04	0.04
Total cold run time: 106.53 s
Total hot run time: 32.75 s

xy720 · 2025-08-27T13:19:25Z

run p0

xy720 · 2025-08-27T13:19:29Z

run cloud_p0

xy720 · 2025-08-27T13:19:45Z

run fe_ut

xy720 · 2025-08-28T01:56:34Z

run buildall

hello-stephen · 2025-08-28T02:15:50Z

Cloud UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.63% (1430/1710)
Line Coverage	67.20% (25421/37828)
Region Coverage	68.28% (12671/18558)
Branch Coverage	58.03% (6712/11566)

doris-robot · 2025-08-28T02:28:41Z

TPC-H: Total hot run time: 33920 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a0168fb066d492072c4035b37497746cb96e7bfb, data reload: false

------ Round 1 ----------------------------------
q1	17648	5221	5057	5057
q2	2008	333	218	218
q3	10240	1272	720	720
q4	10240	1024	533	533
q5	7543	2433	2338	2338
q6	192	180	141	141
q7	924	769	629	629
q8	9341	1322	1085	1085
q9	6972	5052	5119	5052
q10	6920	2418	1961	1961
q11	502	327	279	279
q12	345	355	226	226
q13	17779	3682	3038	3038
q14	232	252	219	219
q15	579	495	491	491
q16	438	450	384	384
q17	589	855	376	376
q18	7388	7136	7036	7036
q19	1269	970	568	568
q20	351	342	221	221
q21	3835	3214	2321	2321
q22	1073	1056	1027	1027
Total cold run time: 106408 ms
Total hot run time: 33920 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5197	5115	5106	5106
q2	252	327	228	228
q3	2174	2671	2307	2307
q4	1347	1796	1347	1347
q5	4182	4386	4571	4386
q6	219	182	139	139
q7	2065	2016	1778	1778
q8	2761	2715	2582	2582
q9	7289	7411	7343	7343
q10	3095	3326	2878	2878
q11	602	517	514	514
q12	676	807	655	655
q13	3682	3862	3292	3292
q14	326	305	306	305
q15	538	489	480	480
q16	461	490	461	461
q17	1202	1631	1433	1433
q18	7824	7569	7866	7569
q19	785	806	808	806
q20	2029	2081	1914	1914
q21	5003	4414	4343	4343
q22	1095	1076	1031	1031
Total cold run time: 52804 ms
Total hot run time: 50897 ms

doris-robot · 2025-08-28T02:40:22Z

TPC-DS: Total hot run time: 186541 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a0168fb066d492072c4035b37497746cb96e7bfb, data reload: false

query1	1025	446	432	432
query2	6575	1779	1755	1755
query3	6747	230	222	222
query4	27028	23686	22908	22908
query5	4414	665	528	528
query6	353	247	236	236
query7	4646	523	297	297
query8	305	256	246	246
query9	8606	2885	2868	2868
query10	492	346	309	309
query11	15852	15069	14746	14746
query12	185	132	124	124
query13	1699	580	443	443
query14	9690	5878	5866	5866
query15	213	194	171	171
query16	7679	670	494	494
query17	1260	791	606	606
query18	2042	440	331	331
query19	202	201	168	168
query20	130	123	123	123
query21	217	127	122	122
query22	4408	4455	4245	4245
query23	33828	33090	32783	32783
query24	8227	2390	2435	2390
query25	601	514	455	455
query26	1235	281	168	168
query27	2705	519	358	358
query28	4344	2245	2215	2215
query29	748	602	497	497
query30	291	232	198	198
query31	949	822	722	722
query32	88	83	84	83
query33	569	405	355	355
query34	808	862	526	526
query35	834	830	737	737
query36	992	1015	945	945
query37	121	111	94	94
query38	4153	4001	3962	3962
query39	1506	1461	1440	1440
query40	225	135	132	132
query41	98	67	64	64
query42	132	112	121	112
query43	546	533	486	486
query44	1330	857	856	856
query45	181	184	174	174
query46	858	1035	658	658
query47	1773	1855	1754	1754
query48	405	433	324	324
query49	745	517	408	408
query50	653	683	403	403
query51	4103	4163	4056	4056
query52	117	110	108	108
query53	245	267	205	205
query54	620	603	548	548
query55	91	91	93	91
query56	341	349	325	325
query57	1204	1215	1145	1145
query58	294	291	283	283
query59	2606	2703	2730	2703
query60	367	368	358	358
query61	172	164	181	164
query62	818	732	655	655
query63	241	199	200	199
query64	4397	1157	862	862
query65	4315	4224	4277	4224
query66	1089	446	347	347
query67	15626	15219	14886	14886
query68	8768	922	642	642
query69	485	332	300	300
query70	1270	1131	1134	1131
query71	472	338	361	338
query72	5935	5003	4975	4975
query73	741	605	363	363
query74	9369	9014	9117	9014
query75	4180	3100	2632	2632
query76	3711	1162	746	746
query77	826	413	350	350
query78	9620	9780	8878	8878
query79	2432	837	596	596
query80	674	582	537	537
query81	477	259	225	225
query82	439	146	110	110
query83	300	263	259	259
query84	301	110	97	97
query85	954	464	435	435
query86	344	321	308	308
query87	4270	4233	4225	4225
query88	3334	2224	2234	2224
query89	410	330	290	290
query90	1968	228	227	227
query91	160	161	139	139
query92	92	77	74	74
query93	1654	999	641	641
query94	696	407	339	339
query95	418	335	325	325
query96	496	578	278	278
query97	2729	2646	2607	2607
query98	244	218	219	218
query99	1434	1405	1320	1320
Total cold run time: 278562 ms
Total hot run time: 186541 ms

doris-robot · 2025-08-28T02:46:02Z

ClickBench: Total hot run time: 33.21 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a0168fb066d492072c4035b37497746cb96e7bfb, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.08	0.09
query4	1.61	0.12	0.12
query5	0.47	0.44	0.41
query6	1.18	0.65	0.66
query7	0.04	0.03	0.02
query8	0.06	0.05	0.04
query9	0.60	0.54	0.53
query10	0.59	0.58	0.58
query11	0.17	0.12	0.12
query12	0.15	0.12	0.13
query13	0.63	0.62	0.62
query14	0.83	0.85	0.83
query15	0.86	0.83	0.86
query16	0.38	0.41	0.39
query17	1.09	1.02	1.04
query18	0.22	0.21	0.20
query19	1.98	1.84	1.86
query20	0.02	0.01	0.02
query21	15.39	0.92	0.58
query22	0.79	1.21	0.90
query23	14.77	1.39	0.64
query24	6.60	1.19	1.33
query25	0.47	0.19	0.07
query26	0.59	0.17	0.14
query27	0.06	0.05	0.05
query28	10.32	0.91	0.42
query29	12.57	3.93	3.25
query30	3.08	2.98	2.92
query31	2.83	0.57	0.38
query32	3.23	0.57	0.47
query33	3.18	3.06	3.10
query34	16.06	5.51	4.87
query35	4.96	4.90	4.91
query36	0.70	0.52	0.50
query37	0.11	0.07	0.08
query38	0.06	0.05	0.04
query39	0.03	0.03	0.04
query40	0.19	0.14	0.14
query41	0.09	0.03	0.02
query42	0.04	0.04	0.02
query43	0.04	0.04	0.04
Total cold run time: 107.46 s
Total hot run time: 33.21 s

hello-stephen · 2025-08-28T03:09:41Z

FE UT Coverage Report

Increment line coverage 73.77% (45/61) 🎉
Increment coverage report
Complete coverage report

xy720 · 2025-08-28T08:42:54Z

run p0

xy720 · 2025-08-28T10:02:25Z

run p0

xy720 · 2025-08-28T11:05:36Z

run p0

github-actions · 2025-08-28T12:38:38Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-08-28T12:38:41Z

PR approved by anyone and no changes requested.

dataroaring

LGTM

xy720 · 2025-08-28T17:19:58Z

run p0

xy720 · 2025-08-29T03:07:38Z

run p0

xy720 · 2025-08-29T05:19:48Z

run p0

xy720 · 2025-08-29T07:45:26Z

run p0

xy720 · 2025-08-29T11:27:26Z

run p0

xy720 · 2025-08-29T12:16:02Z

run buildall

hello-stephen · 2025-08-29T12:32:53Z

Cloud UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	84.48% (1464/1733)
Line Coverage	67.68% (26103/38570)
Region Coverage	68.66% (12985/18913)
Branch Coverage	58.58% (6943/11852)

doris-robot · 2025-08-29T12:41:07Z

TPC-H: Total hot run time: 34286 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 93f40cded32274f8766a9b6cd6814f4c263cf9fc, data reload: false

------ Round 1 ----------------------------------
q1	17623	5317	5056	5056
q2	2022	334	235	235
q3	10214	1287	747	747
q4	10230	1003	549	549
q5	7547	2516	2333	2333
q6	192	172	141	141
q7	936	758	645	645
q8	9350	1404	1107	1107
q9	6962	5060	5144	5060
q10	6885	2386	2002	2002
q11	505	301	287	287
q12	342	361	237	237
q13	17758	3710	3023	3023
q14	244	233	241	233
q15	557	502	482	482
q16	430	437	384	384
q17	593	875	358	358
q18	7462	7153	7179	7153
q19	1235	955	562	562
q20	342	341	242	242
q21	3990	3248	2404	2404
q22	1081	1046	1048	1046
Total cold run time: 106500 ms
Total hot run time: 34286 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5142	5136	5086	5086
q2	248	330	227	227
q3	2202	2736	2342	2342
q4	1373	1801	1318	1318
q5	4235	4468	4642	4468
q6	218	173	134	134
q7	2107	1988	1827	1827
q8	2674	2717	2616	2616
q9	7543	7377	7324	7324
q10	3101	3346	2840	2840
q11	581	510	534	510
q12	704	826	615	615
q13	3560	3994	3406	3406
q14	320	335	285	285
q15	562	504	489	489
q16	450	496	459	459
q17	1168	1550	1396	1396
q18	7914	7496	7616	7496
q19	819	804	821	804
q20	2008	2152	1883	1883
q21	5092	4569	4354	4354
q22	1116	1056	1006	1006
Total cold run time: 53137 ms
Total hot run time: 50885 ms

doris-robot · 2025-08-29T12:52:49Z

TPC-DS: Total hot run time: 187198 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 93f40cded32274f8766a9b6cd6814f4c263cf9fc, data reload: false

query1	1056	466	404	404
query2	6564	1807	1770	1770
query3	6766	235	230	230
query4	26487	23737	22842	22842
query5	4372	683	523	523
query6	367	246	225	225
query7	4640	519	307	307
query8	308	253	240	240
query9	8672	2937	2903	2903
query10	481	349	302	302
query11	15818	15016	15139	15016
query12	186	122	126	122
query13	1675	563	435	435
query14	9803	5914	5829	5829
query15	223	197	172	172
query16	7678	638	453	453
query17	1226	842	639	639
query18	2054	429	345	345
query19	210	199	172	172
query20	136	124	119	119
query21	229	130	115	115
query22	4095	4276	4026	4026
query23	33959	33123	33194	33123
query24	8211	2425	2425	2425
query25	573	578	449	449
query26	1234	289	168	168
query27	2723	513	361	361
query28	4409	2310	2276	2276
query29	787	613	504	504
query30	294	234	205	205
query31	939	831	728	728
query32	89	89	83	83
query33	572	401	355	355
query34	803	855	526	526
query35	838	823	774	774
query36	972	1019	909	909
query37	124	112	97	97
query38	4113	4055	3976	3976
query39	1490	1434	1602	1434
query40	229	134	128	128
query41	65	64	63	63
query42	132	119	134	119
query43	554	545	506	506
query44	1362	873	885	873
query45	188	181	189	181
query46	862	1030	682	682
query47	1784	1839	1720	1720
query48	410	451	351	351
query49	774	555	445	445
query50	683	678	410	410
query51	4080	4107	4147	4107
query52	117	113	112	112
query53	250	266	208	208
query54	625	620	545	545
query55	95	99	96	96
query56	361	338	321	321
query57	1196	1208	1130	1130
query58	293	293	292	292
query59	2693	2805	2642	2642
query60	362	355	350	350
query61	168	160	171	160
query62	818	741	669	669
query63	234	200	202	200
query64	4468	1126	836	836
query65	4333	4220	4236	4220
query66	1077	444	344	344
query67	15627	15298	15138	15138
query68	7909	941	590	590
query69	490	345	296	296
query70	1231	1100	1145	1100
query71	479	363	323	323
query72	5869	5182	5106	5106
query73	671	587	367	367
query74	9440	9005	9096	9005
query75	3884	3117	2673	2673
query76	3573	1142	741	741
query77	807	409	348	348
query78	9690	10037	8766	8766
query79	2383	851	620	620
query80	638	577	522	522
query81	493	262	225	225
query82	513	150	112	112
query83	283	264	253	253
query84	273	114	93	93
query85	965	467	426	426
query86	401	326	290	290
query87	4313	4395	4238	4238
query88	3649	2254	2222	2222
query89	408	336	289	289
query90	1927	235	232	232
query91	165	165	136	136
query92	91	76	72	72
query93	1901	1012	664	664
query94	665	424	304	304
query95	428	394	330	330
query96	489	578	282	282
query97	2671	2667	2601	2601
query98	255	228	217	217
query99	1340	1400	1325	1325
Total cold run time: 277310 ms
Total hot run time: 187198 ms

doris-robot · 2025-08-29T12:58:28Z

ClickBench: Total hot run time: 32.81 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 93f40cded32274f8766a9b6cd6814f4c263cf9fc, data reload: false

query1	0.05	0.05	0.05
query2	0.09	0.06	0.05
query3	0.26	0.08	0.08
query4	1.61	0.11	0.11
query5	0.44	0.42	0.41
query6	1.19	0.64	0.67
query7	0.03	0.03	0.02
query8	0.06	0.04	0.05
query9	0.65	0.61	0.54
query10	0.58	0.59	0.59
query11	0.17	0.12	0.12
query12	0.15	0.12	0.13
query13	0.63	0.63	0.62
query14	0.79	0.84	0.84
query15	0.88	0.84	0.87
query16	0.40	0.42	0.39
query17	1.09	1.06	1.07
query18	0.21	0.19	0.20
query19	1.90	1.84	1.82
query20	0.01	0.02	0.01
query21	15.41	0.95	0.57
query22	0.76	1.31	0.78
query23	14.74	1.39	0.64
query24	6.48	1.60	0.73
query25	0.48	0.16	0.12
query26	0.55	0.17	0.14
query27	0.06	0.05	0.06
query28	9.44	0.90	0.44
query29	12.55	3.86	3.22
query30	3.07	3.04	2.98
query31	2.82	0.58	0.40
query32	3.24	0.56	0.49
query33	2.98	3.08	3.11
query34	16.06	5.50	4.81
query35	5.00	4.96	4.95
query36	0.71	0.51	0.50
query37	0.10	0.07	0.07
query38	0.06	0.05	0.04
query39	0.04	0.03	0.03
query40	0.18	0.15	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 106.08 s
Total hot run time: 32.81 s

hello-stephen · 2025-08-29T14:45:31Z

FE Regression Coverage Report

Increment line coverage 77.05% (47/61) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2025-08-29T15:17:03Z

FE UT Coverage Report

Increment line coverage 73.77% (45/61) 🎉
Increment coverage report
Complete coverage report

…iple batches (apache#54933) 1、Add a config cloud_restore_create_tablet_batch_size default 256, separate cloud restore create tablet rpc into multiple batches to prevent rpc timeout 2、Fix replica's partition id incorrect in restored table.

…c into multiple batches (#55691) pick #54933

xy720 requested review from dataroaring, gavinchou and w41ter as code owners August 18, 2025 07:42

xy720 added the dev/3.1.x label Aug 19, 2025

save

4625c90

xy720 force-pushed the fix-cloud-restore-create-tablet branch from de31c89 to 4625c90 Compare August 27, 2025 08:20

Merge branch 'master' into fix-cloud-restore-create-tablet

a0168fb

gavinchou approved these changes Aug 28, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 28, 2025

github-actions bot added the reviewed label Aug 28, 2025

dataroaring approved these changes Aug 28, 2025

View reviewed changes

Merge branch 'master' into fix-cloud-restore-create-tablet

93f40cd

liutang123 approved these changes Sep 1, 2025

View reviewed changes

xy720 merged commit 3d89334 into apache:master Sep 1, 2025
25 of 26 checks passed

github-actions bot added the dev/3.1.x-conflict label Sep 1, 2025

xy720 mentioned this pull request Sep 4, 2025

branch-3.1: [refactor](cloud) separate cloud restore create tablet rpc into multiple batches #54933 #55691

Merged

morrySnow pushed a commit that referenced this pull request Sep 5, 2025

branch-3.1: [refactor](cloud) separate cloud restore create tablet rp…

d41dd79

…c into multiple batches (#55691) pick #54933

morrySnow added dev/3.1.1-merged and removed dev/3.1.x dev/3.1.x-conflict labels Sep 5, 2025

[refactore](cloud) separate cloud restore create tablet rpc into multiple batches #54933

[refactore](cloud) separate cloud restore create tablet rpc into multiple batches #54933

Uh oh!

Conversation

xy720 commented Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Aug 18, 2025

Uh oh!

xy720 commented Aug 18, 2025

Uh oh!

hello-stephen commented Aug 18, 2025

Cloud UT Coverage Report

Uh oh!

doris-robot commented Aug 18, 2025

Uh oh!

doris-robot commented Aug 18, 2025

Uh oh!

doris-robot commented Aug 18, 2025

Uh oh!

hello-stephen commented Aug 18, 2025

FE UT Coverage Report

Uh oh!

xy720 commented Aug 18, 2025

Uh oh!

xy720 commented Aug 27, 2025

Uh oh!

doris-robot commented Aug 27, 2025

Uh oh!

doris-robot commented Aug 27, 2025

Cloud UT Coverage Report

Uh oh!

doris-robot commented Aug 27, 2025

Uh oh!

doris-robot commented Aug 27, 2025

Uh oh!

xy720 commented Aug 27, 2025

Uh oh!

xy720 commented Aug 27, 2025

Uh oh!

xy720 commented Aug 27, 2025

Uh oh!

xy720 commented Aug 28, 2025

Uh oh!

hello-stephen commented Aug 28, 2025

Cloud UT Coverage Report

Uh oh!

doris-robot commented Aug 28, 2025

Uh oh!

doris-robot commented Aug 28, 2025

Uh oh!

doris-robot commented Aug 28, 2025

Uh oh!

hello-stephen commented Aug 28, 2025

FE UT Coverage Report

Uh oh!

xy720 commented Aug 28, 2025

Uh oh!

xy720 commented Aug 28, 2025

Uh oh!

xy720 commented Aug 28, 2025

Uh oh!

github-actions bot commented Aug 28, 2025

Uh oh!

github-actions bot commented Aug 28, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

xy720 commented Aug 28, 2025

Uh oh!

xy720 commented Aug 29, 2025

Uh oh!

xy720 commented Aug 29, 2025

Uh oh!

xy720 commented Aug 29, 2025

xy720 commented Aug 18, 2025 •

edited

Loading