Skip to content

Conversation

@github-actions
Copy link
Contributor

Cherry-picked from #54090

#54090)

…hen fe is abnormal



The previous operation and maintenance method force_skip_journal_ids can
only skip a specific logid. If there are many abnormal logids, it is
difficult to configure
@github-actions github-actions bot requested a review from morrySnow as a code owner August 22, 2025 13:56
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Aug 22, 2025
@hello-stephen
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32977 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b98ab256fbe901e13fada851d7bec7777a0dd2fd, data reload: false

------ Round 1 ----------------------------------
q1	17865	5589	5542	5542
q2	2022	383	280	280
q3	12236	1236	763	763
q4	10431	862	462	462
q5	9153	2399	2150	2150
q6	185	167	136	136
q7	894	739	624	624
q8	9364	1464	1180	1180
q9	5222	4945	4905	4905
q10	6786	2268	1818	1818
q11	489	286	274	274
q12	341	358	218	218
q13	17764	3637	3014	3014
q14	215	220	219	219
q15	518	475	461	461
q16	416	421	372	372
q17	622	865	377	377
q18	6977	6417	6485	6417
q19	1449	960	569	569
q20	351	346	209	209
q21	2869	2222	2014	2014
q22	1087	996	973	973
Total cold run time: 107256 ms
Total hot run time: 32977 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5608	5503	5532	5503
q2	246	328	243	243
q3	2229	2634	2329	2329
q4	1337	1785	1421	1421
q5	4414	5079	4960	4960
q6	173	165	129	129
q7	2045	1962	1789	1789
q8	2674	2800	2685	2685
q9	7316	7192	7183	7183
q10	3043	3191	2775	2775
q11	566	517	496	496
q12	697	767	615	615
q13	3444	3768	3215	3215
q14	273	294	284	284
q15	513	472	452	452
q16	437	478	456	456
q17	1234	1752	1272	1272
q18	7695	7348	7344	7344
q19	845	860	1154	860
q20	2043	2054	1921	1921
q21	5480	4936	4583	4583
q22	1085	1045	992	992
Total cold run time: 53397 ms
Total hot run time: 51507 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193643 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b98ab256fbe901e13fada851d7bec7777a0dd2fd, data reload: false

query1	949	396	400	396
query2	6225	1999	1979	1979
query3	8686	198	194	194
query4	33343	23983	23856	23856
query5	3665	610	473	473
query6	295	200	199	199
query7	4224	503	323	323
query8	306	242	260	242
query9	9409	2606	2607	2606
query10	483	325	270	270
query11	18080	15399	15218	15218
query12	137	109	101	101
query13	1554	550	418	418
query14	9016	7436	7293	7293
query15	250	193	182	182
query16	8148	657	511	511
query17	1623	802	615	615
query18	2159	428	331	331
query19	241	200	160	160
query20	122	123	121	121
query21	214	128	107	107
query22	4496	4568	4422	4422
query23	35645	34532	34477	34477
query24	7286	2722	2735	2722
query25	546	504	439	439
query26	794	275	179	179
query27	2263	480	361	361
query28	5368	2210	2188	2188
query29	676	616	463	463
query30	245	194	165	165
query31	1010	956	827	827
query32	93	63	59	59
query33	502	381	328	328
query34	762	867	549	549
query35	819	830	734	734
query36	1012	1082	978	978
query37	109	95	79	79
query38	4046	3993	3964	3964
query39	1575	1479	1471	1471
query40	212	121	106	106
query41	51	50	49	49
query42	129	106	104	104
query43	528	538	510	510
query44	1331	820	823	820
query45	189	173	172	172
query46	886	1081	666	666
query47	1949	1994	1924	1924
query48	404	440	367	367
query49	754	496	418	418
query50	692	711	427	427
query51	7295	7296	7276	7276
query52	103	104	100	100
query53	235	267	185	185
query54	549	546	490	490
query55	83	77	77	77
query56	272	279	255	255
query57	1308	1280	1224	1224
query58	230	219	227	219
query59	3137	3233	3052	3052
query60	294	291	274	274
query61	121	116	113	113
query62	795	766	702	702
query63	238	196	189	189
query64	3680	993	657	657
query65	3348	3296	3308	3296
query66	775	420	328	328
query67	16342	15799	15576	15576
query68	7725	836	528	528
query69	518	306	276	276
query70	1233	1126	1162	1126
query71	388	298	269	269
query72	5777	3716	3869	3716
query73	637	754	352	352
query74	10407	9248	9023	9023
query75	3266	3155	2670	2670
query76	3152	1191	772	772
query77	545	371	285	285
query78	10421	10572	9589	9589
query79	3129	901	603	603
query80	657	519	423	423
query81	509	254	221	221
query82	559	122	88	88
query83	155	161	145	145
query84	248	106	76	76
query85	774	353	304	304
query86	361	318	307	307
query87	4298	4381	4175	4175
query88	4757	2428	2388	2388
query89	404	335	300	300
query90	1778	190	191	190
query91	134	135	120	120
query92	65	54	51	51
query93	1298	911	538	538
query94	694	402	307	307
query95	348	281	278	278
query96	491	602	284	284
query97	3262	3241	3160	3160
query98	224	221	209	209
query99	1679	1429	1294	1294
Total cold run time: 292199 ms
Total hot run time: 193643 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.12 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b98ab256fbe901e13fada851d7bec7777a0dd2fd, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.07	0.07
query4	1.63	0.10	0.09
query5	0.51	0.51	0.52
query6	1.14	0.74	0.73
query7	0.03	0.02	0.02
query8	0.06	0.04	0.04
query9	0.54	0.48	0.49
query10	0.54	0.55	0.54
query11	0.16	0.12	0.12
query12	0.16	0.13	0.13
query13	0.61	0.60	0.58
query14	0.76	0.81	0.79
query15	0.86	0.84	0.85
query16	0.39	0.37	0.36
query17	1.01	1.01	1.04
query18	0.19	0.19	0.19
query19	1.88	1.90	1.78
query20	0.01	0.01	0.02
query21	15.39	0.92	0.65
query22	0.76	0.76	0.66
query23	14.82	1.52	0.67
query24	2.21	0.37	0.22
query25	0.14	0.08	0.09
query26	0.28	0.19	0.18
query27	0.07	0.08	0.08
query28	13.46	1.35	0.55
query29	12.66	4.08	3.36
query30	0.26	0.08	0.07
query31	2.85	0.63	0.41
query32	3.24	0.58	0.47
query33	3.05	3.08	3.06
query34	16.93	5.21	4.59
query35	4.66	4.60	4.67
query36	0.63	0.48	0.47
query37	0.20	0.16	0.16
query38	0.17	0.16	0.16
query39	0.06	0.04	0.04
query40	0.17	0.14	0.13
query41	0.09	0.05	0.06
query42	0.07	0.06	0.05
query43	0.05	0.05	0.05
Total cold run time: 103.06 s
Total hot run time: 29.12 s

@morrySnow morrySnow changed the title branch-3.1: [opt][editlog] Added the ability to skip certain editlog exceptions w… #54090 branch-3.1: [opt][editlog] Added the ability to skip certain editlog exceptions when fe is abnormal #54090 Aug 25, 2025
@morrySnow morrySnow merged commit de7183a into branch-3.1 Sep 4, 2025
23 of 24 checks passed
@morrySnow morrySnow deleted the auto-pick-54090-branch-3.1 branch September 4, 2025 01:51
@morrySnow morrySnow mentioned this pull request Sep 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants