branch-3.0: [fix](cloud) Periodically cleaning secondary be in cloud used by redundant tablets #50200

[github-actions]

Cherry-picked from #50200

[Thearas]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[Thearas]

run buildall

[doris-robot]

TPC-H: Total hot run time: 40262 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bba55189ea17dda6c3d6af76e9ee7fecb580b56e, data reload: false

------ Round 1 ----------------------------------
q1	17613	6850	6661	6661
q2	2070	179	162	162
q3	10599	1087	1246	1087
q4	10568	778	747	747
q5	7767	2877	2848	2848
q6	226	132	135	132
q7	964	614	609	609
q8	9348	1969	2050	1969
q9	6639	6421	6426	6421
q10	7049	2299	2311	2299
q11	466	264	256	256
q12	400	215	211	211
q13	17770	2974	2949	2949
q14	246	207	205	205
q15	495	458	447	447
q16	646	588	593	588
q17	995	646	570	570
q18	7253	6732	6688	6688
q19	1407	1133	1105	1105
q20	462	214	203	203
q21	4008	3129	3155	3129
q22	1121	986	976	976
Total cold run time: 108112 ms
Total hot run time: 40262 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6580	6583	6556	6556
q2	344	241	233	233
q3	2907	2749	2934	2749
q4	2064	1791	1775	1775
q5	5790	5756	5740	5740
q6	204	126	130	126
q7	2240	1829	1825	1825
q8	3404	3569	3513	3513
q9	8930	8891	8987	8891
q10	3578	3537	3516	3516
q11	597	508	508	508
q12	817	585	627	585
q13	9029	3201	3183	3183
q14	305	273	278	273
q15	516	462	458	458
q16	702	671	648	648
q17	1852	1673	1661	1661
q18	8327	7677	7879	7677
q19	1675	1509	1481	1481
q20	2075	1876	1878	1876
q21	5592	5240	5466	5240
q22	1156	1009	1007	1007
Total cold run time: 68684 ms
Total hot run time: 59521 ms

[doris-robot]

TPC-DS: Total hot run time: 197179 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bba55189ea17dda6c3d6af76e9ee7fecb580b56e, data reload: false

query1	1318	923	904	904
query2	6245	1999	1974	1974
query3	10982	4510	4351	4351
query4	65896	29386	23526	23526
query5	4973	452	446	446
query6	415	182	176	176
query7	5697	312	310	310
query8	304	231	223	223
query9	9374	2575	2574	2574
query10	472	278	275	275
query11	17975	15282	15657	15282
query12	159	101	104	101
query13	1551	444	415	415
query14	10662	6981	6877	6877
query15	216	194	187	187
query16	7183	492	530	492
query17	1093	599	609	599
query18	1837	345	329	329
query19	259	163	160	160
query20	116	112	110	110
query21	221	107	105	105
query22	4520	4487	4394	4394
query23	34288	34403	34368	34368
query24	6191	2979	2940	2940
query25	534	404	409	404
query26	647	171	169	169
query27	1975	356	357	356
query28	4409	2411	2397	2397
query29	682	462	438	438
query30	238	163	156	156
query31	977	814	828	814
query32	67	54	56	54
query33	403	297	304	297
query34	898	525	526	525
query35	845	721	734	721
query36	1081	950	979	950
query37	118	68	66	66
query38	4071	3938	4004	3938
query39	1520	1452	1459	1452
query40	213	105	97	97
query41	46	45	45	45
query42	114	103	103	103
query43	523	477	485	477
query44	1211	834	806	806
query45	181	169	168	168
query46	1163	789	739	739
query47	2049	1875	1927	1875
query48	484	393	390	390
query49	725	407	400	400
query50	844	429	433	429
query51	7451	7158	7170	7158
query52	100	88	87	87
query53	255	179	184	179
query54	561	449	468	449
query55	79	80	79	79
query56	262	241	249	241
query57	1212	1137	1099	1099
query58	207	206	208	206
query59	3098	2787	2826	2787
query60	276	250	248	248
query61	112	106	108	106
query62	778	672	672	672
query63	225	190	204	190
query64	1363	686	644	644
query65	3276	3235	3183	3183
query66	714	289	300	289
query67	15844	15547	15823	15547
query68	3665	614	566	566
query69	421	274	268	268
query70	1185	1090	1076	1076
query71	360	255	255	255
query72	6437	4067	4216	4067
query73	744	349	361	349
query74	10096	9157	9071	9071
query75	3357	2611	2665	2611
query76	1910	1104	1076	1076
query77	485	273	275	273
query78	10561	9604	9581	9581
query79	1184	604	590	590
query80	833	458	469	458
query81	507	247	239	239
query82	1228	98	87	87
query83	174	161	151	151
query84	294	79	83	79
query85	915	371	382	371
query86	334	280	288	280
query87	4405	4260	4218	4218
query88	3347	2393	2350	2350
query89	424	298	299	298
query90	1998	190	189	189
query91	178	150	147	147
query92	65	50	53	50
query93	1280	531	552	531
query94	775	308	293	293
query95	364	264	256	256
query96	607	280	285	280
query97	3340	3142	3102	3102
query98	219	202	190	190
query99	1550	1301	1279	1279
Total cold run time: 317020 ms
Total hot run time: 197179 ms

[doris-robot]

ClickBench: Total hot run time: 32.42 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bba55189ea17dda6c3d6af76e9ee7fecb580b56e, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.02	0.03
query3	0.24	0.07	0.06
query4	1.63	0.10	0.10
query5	0.54	0.51	0.51
query6	1.13	0.73	0.73
query7	0.02	0.01	0.01
query8	0.04	0.03	0.05
query9	0.56	0.50	0.50
query10	0.57	0.56	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.60	0.60
query14	2.84	2.73	2.73
query15	0.90	0.83	0.83
query16	0.37	0.38	0.39
query17	0.99	1.07	1.05
query18	0.24	0.21	0.22
query19	1.81	1.84	2.06
query20	0.01	0.01	0.01
query21	15.35	0.59	0.59
query22	3.24	2.54	2.06
query23	17.10	0.93	0.76
query24	3.42	1.28	0.96
query25	0.21	0.25	0.13
query26	0.35	0.15	0.14
query27	0.05	0.05	0.03
query28	10.01	0.50	0.45
query29	12.59	3.17	3.20
query30	0.24	0.06	0.06
query31	2.87	0.38	0.39
query32	3.23	0.47	0.46
query33	2.98	3.00	3.04
query34	16.88	4.50	4.59
query35	4.53	4.61	4.55
query36	0.69	0.48	0.48
query37	0.10	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.17	0.12	0.13
query41	0.09	0.03	0.02
query42	0.04	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 107.13 s
Total hot run time: 32.42 s

[dataroaring]

LGTM