Skip to content

branch-3.1: [fix](wal) replay wal abort txn failed when execute failed #55881 #55924

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 1 commit into from
Sep 12, 2025
Merged

branch-3.1: [fix](wal) replay wal abort txn failed when execute failed #55881 #55924

morrySnow merged 1 commit into from
Sep 12, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Sep 11, 2025

Cherry-picked from #55881

@mymeiyi
[fix](wal) replay wal abort txn failed when execute failed (#55881)
cb691cd 
Problem Summary:
when replay wal failed and then abort txn, the db_id is not set, will
get:
```
2025-09-10 15:30:00,687 DEBUG (thrift-server-pool-21|177) [FrontendServiceImpl.loadTxnRollback():1902] receive txn rollback request: TLoadTxnRollbackRequest(user:admin, passwd:, db:, tbl:, user_ip:, txnId:-1, reason:[OK], tbls:[], label:group_commit_454b486649018fd4_f8985d893061b7b9_test_wait), backend: [172.20.56.196](http://172.20.56.196/) 
2025-09-10 15:30:00,687 WARN (thrift-server-pool-21|177) [FrontendServiceImpl.loadTxnRollback():1920] failed to rollback txn, id: -1, label: group_commit_454b486649018fd4_f8985d893061b7b9_test_wait org.apache.doris.common.MetaNotFoundException: errCode = 7, detailMessage = db does not exist at org.apache.doris.service.FrontendServiceImpl.loadTxnRollbackImpl(FrontendServiceImpl.java:1964)
```

### Release note
@github-actions github-actions bot requested a review from morrySnow as a code owner September 11, 2025 11:03
@Thearas
Copy link
Contributor

Thearas commented Sep 11, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Sep 11, 2025
@Thearas
Copy link
Contributor

Thearas commented Sep 11, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Sep 11, 2025

TPC-H: Total hot run time: 32570 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cb691cda0df2ee72c3cf64062ae93908f2eededb, data reload: false

------ Round 1 ----------------------------------
q1	17618	5508	5434	5434
q2	2027	417	285	285
q3	12129	1237	742	742
q4	10547	874	456	456
q5	9418	2391	2107	2107
q6	180	162	133	133
q7	895	726	603	603
q8	9317	1395	1136	1136
q9	5299	4953	4908	4908
q10	6765	2302	1828	1828
q11	481	284	262	262
q12	342	360	216	216
q13	17769	3604	2994	2994
q14	222	222	207	207
q15	540	478	456	456
q16	406	434	380	380
q17	584	861	363	363
q18	6919	6470	6337	6337
q19	1204	943	518	518
q20	336	335	199	199
q21	2802	2184	2016	2016
q22	1060	1019	990	990
Total cold run time: 106860 ms
Total hot run time: 32570 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5514	5487	5474	5474
q2	232	327	240	240
q3	2260	2628	2308	2308
q4	1347	1802	1334	1334
q5	4424	5012	4918	4918
q6	170	166	134	134
q7	2028	1990	1863	1863
q8	2654	2810	2733	2733
q9	7409	7345	7312	7312
q10	3071	3337	2799	2799
q11	592	529	501	501
q12	670	779	639	639
q13	3453	3756	3158	3158
q14	280	305	282	282
q15	537	472	471	471
q16	426	475	427	427
q17	1233	1757	1272	1272
q18	7629	7354	7272	7272
q19	786	1118	1119	1118
q20	1974	2052	1887	1887
q21	5456	4994	4638	4638
q22	1108	1059	1026	1026
Total cold run time: 53253 ms
Total hot run time: 51806 ms

@doris-robot
Copy link

doris-robot commented Sep 11, 2025

TPC-DS: Total hot run time: 193530 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cb691cda0df2ee72c3cf64062ae93908f2eededb, data reload: false

query1	951	418	376	376
query2	6213	1948	1842	1842
query3	8689	208	202	202
query4	33771	24328	24027	24027
query5	4260	613	443	443
query6	302	199	189	189
query7	4202	498	331	331
query8	303	236	227	227
query9	9499	2606	2590	2590
query10	485	346	261	261
query11	17964	15768	15127	15127
query12	156	110	105	105
query13	1555	545	423	423
query14	10211	7629	7248	7248
query15	248	190	178	178
query16	8116	662	481	481
query17	1588	776	608	608
query18	2234	423	335	335
query19	226	192	175	175
query20	123	126	124	124
query21	207	132	109	109
query22	4561	4589	4440	4440
query23	35290	34794	34184	34184
query24	7344	2775	2751	2751
query25	563	495	435	435
query26	1224	290	179	179
query27	2111	480	369	369
query28	5265	2213	2192	2192
query29	808	616	472	472
query30	249	201	163	163
query31	1018	935	850	850
query32	90	63	63	63
query33	529	373	311	311
query34	744	857	525	525
query35	817	843	725	725
query36	1004	1062	954	954
query37	101	93	73	73
query38	4110	4005	3935	3935
query39	1539	1477	1472	1472
query40	222	124	110	110
query41	49	48	49	48
query42	126	109	106	106
query43	513	521	494	494
query44	1415	828	849	828
query45	189	180	175	175
query46	900	1072	690	690
query47	2031	2009	1976	1976
query48	421	433	357	357
query49	766	488	420	420
query50	685	698	457	457
query51	7308	7312	7159	7159
query52	102	106	92	92
query53	225	270	197	197
query54	547	550	496	496
query55	84	82	82	82
query56	298	270	264	264
query57	1276	1279	1239	1239
query58	234	219	222	219
query59	3065	3196	3118	3118
query60	304	284	265	265
query61	143	117	123	117
query62	802	773	731	731
query63	243	208	196	196
query64	4590	1011	627	627
query65	3385	3282	3269	3269
query66	993	415	309	309
query67	16048	15748	15529	15529
query68	7578	830	542	542
query69	488	301	273	273
query70	1249	1115	1122	1115
query71	412	291	263	263
query72	5728	3765	3676	3676
query73	648	747	348	348
query74	10634	9447	9303	9303
query75	3243	3152	2678	2678
query76	3098	1165	792	792
query77	706	358	295	295
query78	10411	10359	9637	9637
query79	3599	828	584	584
query80	748	513	444	444
query81	505	252	224	224
query82	585	122	92	92
query83	186	163	144	144
query84	283	107	85	85
query85	782	371	301	301
query86	348	316	289	289
query87	4321	4359	4269	4269
query88	4838	2539	2402	2402
query89	411	326	303	303
query90	1798	191	192	191
query91	135	140	107	107
query92	66	56	50	50
query93	1343	891	538	538
query94	688	412	311	311
query95	342	276	267	267
query96	493	611	294	294
query97	3205	3281	3132	3132
query98	233	212	196	196
query99	1566	1406	1321	1321
Total cold run time: 296304 ms
Total hot run time: 193530 ms

@doris-robot
Copy link

doris-robot commented Sep 11, 2025

ClickBench: Total hot run time: 28.43 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cb691cda0df2ee72c3cf64062ae93908f2eededb, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.24	0.07	0.06
query4	1.62	0.11	0.10
query5	0.53	0.52	0.51
query6	1.12	0.74	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.56	0.50	0.48
query10	0.55	0.55	0.56
query11	0.14	0.10	0.11
query12	0.14	0.11	0.10
query13	0.61	0.60	0.59
query14	0.77	0.82	0.78
query15	0.84	0.82	0.82
query16	0.40	0.40	0.38
query17	1.06	1.08	1.08
query18	0.24	0.22	0.23
query19	1.90	1.76	1.76
query20	0.01	0.01	0.01
query21	15.43	0.98	0.57
query22	0.74	0.79	0.68
query23	15.06	1.46	0.53
query24	3.08	0.51	0.56
query25	0.22	0.32	0.05
query26	0.23	0.14	0.13
query27	0.07	0.05	0.05
query28	13.65	1.04	0.44
query29	12.59	3.99	3.30
query30	0.25	0.09	0.07
query31	2.81	0.60	0.39
query32	3.24	0.54	0.46
query33	2.98	3.04	3.05
query34	16.56	5.18	4.52
query35	4.58	4.58	4.54
query36	0.64	0.49	0.47
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.16	0.14	0.12
query41	0.07	0.02	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.5 s
Total hot run time: 28.43 s

@doris-robot
Copy link

doris-robot commented Sep 11, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/4) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 45.53% (12759/28023)
Line Coverage 36.37% (113779/312828)
Region Coverage 33.99% (65053/191381)
Branch Coverage 31.03% (34145/110050)

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 11, 2025

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/4) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.54% (21096/27561)
Line Coverage 69.86% (217874/311857)
Region Coverage 67.83% (130332/192158)
Branch Coverage 61.35% (67832/110564)

@morrySnow morrySnow merged commit ffa8ce5 into branch-3.1 Sep 12, 2025
21 of 23 checks passed
@github-actions github-actions bot deleted the auto-pick-55881-branch-3.1 branch September 12, 2025 02:49
@morrySnow morrySnow mentioned this pull request Sep 22, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@Thearas @doris-robot @hello-stephen @morrySnow @dataroaring @mymeiyi