[fix](wal) replay wal abort txn failed when execute failed #55881

mymeiyi · 2025-09-10T12:23:07Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
when replay wal failed and then abort txn, the db_id is not set, will get:

2025-09-10 15:30:00,687 DEBUG (thrift-server-pool-21|177) [FrontendServiceImpl.loadTxnRollback():1902] receive txn rollback request: TLoadTxnRollbackRequest(user:admin, passwd:, db:, tbl:, user_ip:, txnId:-1, reason:[OK], tbls:[], label:group_commit_454b486649018fd4_f8985d893061b7b9_test_wait), backend: [172.20.56.196](http://172.20.56.196/) 
2025-09-10 15:30:00,687 WARN (thrift-server-pool-21|177) [FrontendServiceImpl.loadTxnRollback():1920] failed to rollback txn, id: -1, label: group_commit_454b486649018fd4_f8985d893061b7b9_test_wait org.apache.doris.common.MetaNotFoundException: errCode = 7, detailMessage = db does not exist at org.apache.doris.service.FrontendServiceImpl.loadTxnRollbackImpl(FrontendServiceImpl.java:1964)

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-09-10T12:23:11Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

mymeiyi · 2025-09-10T12:32:51Z

run buildall

doris-robot · 2025-09-10T12:52:13Z

TPC-H: Total hot run time: 34640 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e3d7c8d17a897bf7be205e11e31030a491f84ed8, data reload: false

------ Round 1 ----------------------------------
q1	17602	5276	5103	5103
q2	1998	335	215	215
q3	10251	1303	728	728
q4	10235	1013	560	560
q5	7482	2462	2359	2359
q6	188	175	144	144
q7	939	756	636	636
q8	9326	1348	1057	1057
q9	6990	5105	5051	5051
q10	6911	2405	2001	2001
q11	494	286	282	282
q12	357	366	228	228
q13	17757	3677	3037	3037
q14	259	234	223	223
q15	537	491	503	491
q16	1014	1001	934	934
q17	593	866	357	357
q18	7460	7245	7112	7112
q19	1229	936	543	543
q20	350	352	228	228
q21	3816	2552	2357	2357
q22	1122	1045	994	994
Total cold run time: 106910 ms
Total hot run time: 34640 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5129	5071	5109	5071
q2	246	331	228	228
q3	2197	2693	2288	2288
q4	1394	1807	1361	1361
q5	4219	4528	4658	4528
q6	227	181	146	146
q7	2151	1943	1798	1798
q8	2634	2643	2663	2643
q9	7374	7349	7355	7349
q10	3146	3307	2967	2967
q11	597	503	497	497
q12	694	809	662	662
q13	3621	3954	3381	3381
q14	300	317	304	304
q15	535	490	486	486
q16	1103	1122	1124	1122
q17	1220	1594	1444	1444
q18	7826	7861	7477	7477
q19	801	789	836	789
q20	1996	2136	1923	1923
q21	5063	4420	4345	4345
q22	1081	1050	1009	1009
Total cold run time: 53554 ms
Total hot run time: 51818 ms

doris-robot · 2025-09-10T13:04:02Z

TPC-DS: Total hot run time: 189565 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e3d7c8d17a897bf7be205e11e31030a491f84ed8, data reload: false

query1	1046	451	399	399
query2	6581	1767	1688	1688
query3	6748	225	236	225
query4	26761	23519	23223	23223
query5	4846	657	520	520
query6	346	252	242	242
query7	4687	529	315	315
query8	314	267	283	267
query9	8620	2940	2961	2940
query10	479	361	306	306
query11	15868	14975	14768	14768
query12	192	130	119	119
query13	1696	575	447	447
query14	11861	9276	9352	9276
query15	252	191	174	174
query16	7734	691	538	538
query17	1638	800	632	632
query18	2051	445	337	337
query19	253	197	176	176
query20	133	127	123	123
query21	215	134	119	119
query22	4092	4137	3949	3949
query23	34253	33210	32938	32938
query24	8012	2430	2401	2401
query25	574	520	456	456
query26	1236	277	169	169
query27	2685	510	362	362
query28	4363	2293	2265	2265
query29	789	628	506	506
query30	300	225	198	198
query31	918	821	733	733
query32	94	83	86	83
query33	581	403	345	345
query34	802	868	538	538
query35	820	851	770	770
query36	959	1032	932	932
query37	144	120	99	99
query38	3513	3530	3486	3486
query39	1500	1467	1439	1439
query40	226	147	130	130
query41	68	61	59	59
query42	131	119	129	119
query43	514	502	481	481
query44	1366	889	874	874
query45	185	177	171	171
query46	854	1027	655	655
query47	1767	1821	1802	1802
query48	398	411	330	330
query49	744	528	415	415
query50	689	684	415	415
query51	3975	3990	3912	3912
query52	129	118	112	112
query53	250	271	210	210
query54	624	619	558	558
query55	98	91	91	91
query56	361	336	315	315
query57	1171	1215	1131	1131
query58	303	282	277	277
query59	2572	2588	2563	2563
query60	383	363	352	352
query61	163	153	163	153
query62	816	742	669	669
query63	241	206	205	205
query64	4383	1146	877	877
query65	4037	3985	3994	3985
query66	1091	447	355	355
query67	15725	15299	15073	15073
query68	8174	934	580	580
query69	491	337	297	297
query70	1407	1353	1402	1353
query71	588	364	330	330
query72	5801	5040	5158	5040
query73	759	659	363	363
query74	9278	9224	8928	8928
query75	4047	3284	2788	2788
query76	3710	1158	742	742
query77	790	421	340	340
query78	9665	9757	8851	8851
query79	2390	854	590	590
query80	649	582	524	524
query81	498	259	236	236
query82	493	163	204	163
query83	270	261	257	257
query84	265	108	94	94
query85	923	477	416	416
query86	391	303	313	303
query87	3732	3684	3615	3615
query88	3519	2233	2263	2233
query89	402	324	297	297
query90	1866	234	230	230
query91	167	172	138	138
query92	100	79	73	73
query93	1844	1019	662	662
query94	667	410	332	332
query95	410	420	325	325
query96	488	573	283	283
query97	2915	3003	2871	2871
query98	252	223	215	215
query99	1333	1419	1304	1304
Total cold run time: 279504 ms
Total hot run time: 189565 ms

doris-robot · 2025-09-10T13:09:31Z

ClickBench: Total hot run time: 30.05 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e3d7c8d17a897bf7be205e11e31030a491f84ed8, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.06
query3	0.25	0.09	0.08
query4	1.61	0.12	0.11
query5	0.28	0.27	0.25
query6	1.18	0.66	0.66
query7	0.03	0.03	0.03
query8	0.05	0.04	0.05
query9	0.62	0.53	0.53
query10	0.60	0.57	0.57
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.63	0.64	0.62
query14	1.07	1.01	1.05
query15	0.87	0.85	0.83
query16	0.40	0.41	0.39
query17	1.07	1.04	1.04
query18	0.21	0.20	0.20
query19	1.90	1.81	1.85
query20	0.02	0.01	0.02
query21	15.45	0.97	0.58
query22	0.80	1.14	0.81
query23	14.82	1.38	0.61
query24	6.70	1.73	0.81
query25	0.50	0.12	0.07
query26	0.56	0.17	0.14
query27	0.06	0.06	0.06
query28	10.06	0.92	0.43
query29	12.59	3.91	3.28
query30	0.29	0.12	0.11
query31	2.84	0.60	0.40
query32	3.24	0.55	0.47
query33	3.05	3.04	3.11
query34	15.93	5.49	4.88
query35	4.89	4.94	4.95
query36	0.72	0.51	0.51
query37	0.12	0.08	0.07
query38	0.06	0.05	0.05
query39	0.03	0.03	0.03
query40	0.19	0.15	0.15
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 104.33 s
Total hot run time: 30.05 s

dataroaring

LGTM

github-actions · 2025-09-10T13:14:25Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-09-10T13:14:28Z

PR approved by anyone and no changes requested.

doris-robot · 2025-09-10T13:43:38Z

BE UT Coverage Report

Increment line coverage 0.00% (0/4) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	51.96% (17301/33295)
Line Coverage	37.30% (157666/422653)
Region Coverage	31.96% (120342/376562)
Branch Coverage	33.32% (52815/158511)

liaoxin01

LGTM

hello-stephen · 2025-09-10T15:16:35Z

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/4) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.61% (23413/32694)
Line Coverage	58.51% (247153/422399)
Region Coverage	54.84% (209494/381991)
Branch Coverage	55.81% (88961/159414)

dataroaring · 2025-09-11T01:08:26Z

run buildall

doris-robot · 2025-09-11T01:40:35Z

TPC-H: Total hot run time: 34846 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 12674c93b99d9ebb304ff7dda84c708f57ed0daa, data reload: false

------ Round 1 ----------------------------------
q1	17627	5221	5095	5095
q2	2014	311	220	220
q3	10270	1289	748	748
q4	10223	1024	553	553
q5	7531	2457	2368	2368
q6	191	169	138	138
q7	934	779	636	636
q8	9350	1347	1187	1187
q9	6797	5124	5152	5124
q10	6976	2402	2015	2015
q11	488	341	296	296
q12	359	385	234	234
q13	17783	3720	3067	3067
q14	244	237	217	217
q15	565	498	484	484
q16	1033	1010	948	948
q17	611	871	365	365
q18	7417	7021	6931	6931
q19	1528	954	571	571
q20	331	341	219	219
q21	3874	3392	2430	2430
q22	1065	1015	1000	1000
Total cold run time: 107211 ms
Total hot run time: 34846 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5206	5105	5107	5105
q2	257	321	230	230
q3	2209	2723	2315	2315
q4	1395	1790	1349	1349
q5	4264	4593	4631	4593
q6	218	173	132	132
q7	2060	2021	1915	1915
q8	2632	2730	2668	2668
q9	7448	7188	7296	7188
q10	3163	3298	2958	2958
q11	590	530	499	499
q12	946	781	637	637
q13	3629	3948	3423	3423
q14	291	307	296	296
q15	525	483	486	483
q16	1089	1146	1082	1082
q17	1191	1779	1407	1407
q18	7734	7639	7475	7475
q19	878	832	863	832
q20	2061	2102	1884	1884
q21	4853	4484	4352	4352
q22	1081	1040	999	999
Total cold run time: 53720 ms
Total hot run time: 51822 ms

doris-robot · 2025-09-11T01:52:17Z

TPC-DS: Total hot run time: 189801 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 12674c93b99d9ebb304ff7dda84c708f57ed0daa, data reload: false

query1	1053	458	442	442
query2	6554	1682	1660	1660
query3	6759	228	228	228
query4	26292	23559	22888	22888
query5	4861	651	519	519
query6	334	253	254	253
query7	4687	530	327	327
query8	336	304	283	283
query9	8712	3016	2938	2938
query10	488	346	303	303
query11	15799	15127	14745	14745
query12	174	122	119	119
query13	1676	562	435	435
query14	11153	9459	9460	9459
query15	209	194	180	180
query16	7427	654	512	512
query17	1219	754	642	642
query18	2031	438	333	333
query19	207	213	176	176
query20	137	146	128	128
query21	209	133	125	125
query22	4076	4272	4066	4066
query23	34058	33096	33115	33096
query24	8121	2431	2424	2424
query25	598	555	464	464
query26	1220	287	174	174
query27	2724	516	371	371
query28	4400	2274	2288	2274
query29	846	634	493	493
query30	291	231	205	205
query31	911	817	730	730
query32	93	90	86	86
query33	579	416	359	359
query34	820	873	538	538
query35	833	839	768	768
query36	998	1027	943	943
query37	131	118	97	97
query38	3594	3570	3559	3559
query39	1480	1429	1425	1425
query40	236	142	132	132
query41	70	65	71	65
query42	136	121	128	121
query43	527	499	496	496
query44	1419	891	892	891
query45	190	183	182	182
query46	915	1066	679	679
query47	1803	1805	1744	1744
query48	389	430	336	336
query49	775	532	421	421
query50	683	720	417	417
query51	3922	3939	4037	3939
query52	120	117	107	107
query53	254	293	199	199
query54	623	617	550	550
query55	98	92	93	92
query56	354	342	350	342
query57	1193	1207	1121	1121
query58	299	301	291	291
query59	2613	2649	2568	2568
query60	361	366	351	351
query61	170	215	156	156
query62	836	754	652	652
query63	240	208	203	203
query64	4531	1149	836	836
query65	4037	4002	4001	4001
query66	1151	441	358	358
query67	15637	15067	15000	15000
query68	4910	975	606	606
query69	507	338	317	317
query70	1402	1338	1284	1284
query71	565	377	336	336
query72	5990	4972	5130	4972
query73	699	667	357	357
query74	9045	9192	8892	8892
query75	3334	3255	2735	2735
query76	3215	1181	781	781
query77	602	416	339	339
query78	9565	9799	8885	8885
query79	1564	864	581	581
query80	1640	594	528	528
query81	542	266	225	225
query82	422	170	147	147
query83	273	264	245	245
query84	261	108	93	93
query85	866	462	493	462
query86	388	312	312	312
query87	3740	3719	3666	3666
query88	2923	2242	2266	2242
query89	395	332	298	298
query90	1742	233	241	233
query91	166	182	136	136
query92	86	76	76	76
query93	1165	1022	668	668
query94	695	438	326	326
query95	415	339	333	333
query96	484	606	280	280
query97	2887	2976	2872	2872
query98	255	223	223	223
query99	1335	1425	1334	1334
Total cold run time: 271750 ms
Total hot run time: 189801 ms

doris-robot · 2025-09-11T01:57:48Z

ClickBench: Total hot run time: 30.48 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 12674c93b99d9ebb304ff7dda84c708f57ed0daa, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.06	0.07
query3	0.26	0.08	0.08
query4	1.61	0.12	0.12
query5	0.29	0.27	0.26
query6	1.17	0.68	0.65
query7	0.04	0.03	0.04
query8	0.05	0.05	0.04
query9	0.63	0.54	0.53
query10	0.59	0.58	0.57
query11	0.17	0.12	0.12
query12	0.16	0.12	0.12
query13	0.62	0.63	0.62
query14	1.02	1.04	1.03
query15	0.86	0.86	0.85
query16	0.40	0.39	0.41
query17	1.04	1.04	1.06
query18	0.21	0.20	0.20
query19	1.97	1.83	1.88
query20	0.01	0.02	0.01
query21	15.42	0.99	0.58
query22	0.80	1.31	0.82
query23	14.74	1.42	0.68
query24	6.43	2.99	0.78
query25	0.48	0.20	0.16
query26	0.61	0.16	0.13
query27	0.05	0.06	0.06
query28	10.12	0.89	0.44
query29	12.58	4.12	3.43
query30	0.28	0.12	0.11
query31	2.83	0.62	0.39
query32	3.24	0.57	0.47
query33	3.15	3.20	3.12
query34	16.15	5.47	4.87
query35	4.90	4.94	4.98
query36	0.70	0.55	0.52
query37	0.11	0.07	0.07
query38	0.07	0.05	0.05
query39	0.04	0.03	0.04
query40	0.19	0.15	0.14
query41	0.09	0.04	0.03
query42	0.04	0.04	0.03
query43	0.04	0.04	0.03
Total cold run time: 104.31 s
Total hot run time: 30.48 s

hello-stephen · 2025-09-11T02:27:44Z

BE UT Coverage Report

Increment line coverage 0.00% (0/4) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	51.97% (17303/33296)
Line Coverage	37.31% (157687/422647)
Region Coverage	31.95% (120332/376633)
Branch Coverage	33.33% (52837/158509)

hello-stephen · 2025-09-11T04:01:56Z

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/4) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	70.74% (23128/32695)
Line Coverage	57.14% (241355/422393)
Region Coverage	52.64% (201119/382062)
Branch Coverage	54.18% (86369/159412)

dataroaring

LGTM

Problem Summary: when replay wal failed and then abort txn, the db_id is not set, will get: ``` 2025-09-10 15:30:00,687 DEBUG (thrift-server-pool-21|177) [FrontendServiceImpl.loadTxnRollback():1902] receive txn rollback request: TLoadTxnRollbackRequest(user:admin, passwd:, db:, tbl:, user_ip:, txnId:-1, reason:[OK], tbls:[], label:group_commit_454b486649018fd4_f8985d893061b7b9_test_wait), backend: [172.20.56.196](http://172.20.56.196/) 2025-09-10 15:30:00,687 WARN (thrift-server-pool-21|177) [FrontendServiceImpl.loadTxnRollback():1920] failed to rollback txn, id: -1, label: group_commit_454b486649018fd4_f8985d893061b7b9_test_wait org.apache.doris.common.MetaNotFoundException: errCode = 7, detailMessage = db does not exist at org.apache.doris.service.FrontendServiceImpl.loadTxnRollbackImpl(FrontendServiceImpl.java:1964) ``` ### Release note

…#55881 (#55923) Cherry-picked from #55881 Co-authored-by: meiyi <meiyi@selectdb.com>

…#55881 (#55924) Cherry-picked from #55881 Co-authored-by: meiyi <meiyi@selectdb.com>

dataroaring approved these changes Sep 10, 2025

View reviewed changes

dataroaring added dev/3.0.x dev/3.1.x p0_b labels Sep 10, 2025

github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 10, 2025

github-actions bot added the reviewed label Sep 10, 2025

liaoxin01 approved these changes Sep 10, 2025

View reviewed changes

fix replay wal abort txn

12674c9

dataroaring force-pushed the fix-replay-wal branch from e3d7c8d to 12674c9 Compare September 11, 2025 01:08

dataroaring approved these changes Sep 11, 2025

View reviewed changes

dataroaring merged commit 2a5a109 into apache:master Sep 11, 2025
25 of 27 checks passed

github-actions bot mentioned this pull request Sep 11, 2025

branch-3.0: [fix](wal) replay wal abort txn failed when execute failed #55881 #55923

Merged

github-actions bot mentioned this pull request Sep 11, 2025

branch-3.1: [fix](wal) replay wal abort txn failed when execute failed #55881 #55924

Merged

dataroaring pushed a commit that referenced this pull request Sep 12, 2025

branch-3.0: [fix](wal) replay wal abort txn failed when execute failed …

9ba62f3

…#55881 (#55923) Cherry-picked from #55881 Co-authored-by: meiyi <meiyi@selectdb.com>

dataroaring added dev/3.0.9-merged and removed dev/3.0.x labels Sep 12, 2025

morrySnow pushed a commit that referenced this pull request Sep 12, 2025

branch-3.1: [fix](wal) replay wal abort txn failed when execute failed …

ffa8ce5

…#55881 (#55924) Cherry-picked from #55881 Co-authored-by: meiyi <meiyi@selectdb.com>

morrySnow added dev/3.1.1-merged and removed dev/3.1.x labels Sep 12, 2025

morrySnow mentioned this pull request Sep 22, 2025

Release Note 3.1.1 #56289

Closed

[fix](wal) replay wal abort txn failed when execute failed #55881

[fix](wal) replay wal abort txn failed when execute failed #55881

Uh oh!

Conversation

mymeiyi commented Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Sep 10, 2025

Uh oh!

mymeiyi commented Sep 10, 2025

Uh oh!

doris-robot commented Sep 10, 2025

Uh oh!

doris-robot commented Sep 10, 2025

Uh oh!

doris-robot commented Sep 10, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Sep 10, 2025

Uh oh!

github-actions bot commented Sep 10, 2025

Uh oh!

doris-robot commented Sep 10, 2025

BE UT Coverage Report

Uh oh!

liaoxin01 left a comment

Choose a reason for hiding this comment

Uh oh!

hello-stephen commented Sep 10, 2025

BE Regression && UT Coverage Report

Uh oh!

dataroaring commented Sep 11, 2025

Uh oh!

doris-robot commented Sep 11, 2025

Uh oh!

doris-robot commented Sep 11, 2025

Uh oh!

doris-robot commented Sep 11, 2025

Uh oh!

hello-stephen commented Sep 11, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Sep 11, 2025

BE Regression && UT Coverage Report

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

mymeiyi commented Sep 10, 2025 •

edited

Loading