Skip to content

branch-3.0: [fix](wal) replay wal abort txn failed when execute failed #55881 #55923

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Sep 12, 2025
Merged

branch-3.0: [fix](wal) replay wal abort txn failed when execute failed #55881 #55923

dataroaring merged 1 commit into from
Sep 12, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Sep 11, 2025

Cherry-picked from #55881

@mymeiyi
[fix](wal) replay wal abort txn failed when execute failed (#55881)
4bc4c0f 
Problem Summary:
when replay wal failed and then abort txn, the db_id is not set, will
get:
```
2025-09-10 15:30:00,687 DEBUG (thrift-server-pool-21|177) [FrontendServiceImpl.loadTxnRollback():1902] receive txn rollback request: TLoadTxnRollbackRequest(user:admin, passwd:, db:, tbl:, user_ip:, txnId:-1, reason:[OK], tbls:[], label:group_commit_454b486649018fd4_f8985d893061b7b9_test_wait), backend: [172.20.56.196](http://172.20.56.196/) 
2025-09-10 15:30:00,687 WARN (thrift-server-pool-21|177) [FrontendServiceImpl.loadTxnRollback():1920] failed to rollback txn, id: -1, label: group_commit_454b486649018fd4_f8985d893061b7b9_test_wait org.apache.doris.common.MetaNotFoundException: errCode = 7, detailMessage = db does not exist at org.apache.doris.service.FrontendServiceImpl.loadTxnRollbackImpl(FrontendServiceImpl.java:1964)
```

### Release note
@Thearas
Copy link
Contributor

Thearas commented Sep 11, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Sep 11, 2025
@Thearas
Copy link
Contributor

Thearas commented Sep 11, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Sep 11, 2025

TPC-H: Total hot run time: 39804 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4bc4c0f044e194142fca82fd572637623f89d89b, data reload: false

------ Round 1 ----------------------------------
q1	17625	6826	6668	6668
q2	2050	175	160	160
q3	10612	1146	1213	1146
q4	10252	780	696	696
q5	7735	2812	2738	2738
q6	210	128	133	128
q7	984	616	610	610
q8	9354	1922	1985	1922
q9	6654	6412	6390	6390
q10	7032	2301	2289	2289
q11	452	264	269	264
q12	491	216	213	213
q13	17779	2992	2980	2980
q14	229	203	207	203
q15	511	473	453	453
q16	476	371	381	371
q17	979	574	642	574
q18	7229	6666	6735	6666
q19	1391	1060	1115	1060
q20	454	205	202	202
q21	3955	3205	3092	3092
q22	1118	979	996	979
Total cold run time: 107572 ms
Total hot run time: 39804 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6620	6554	6570	6554
q2	328	228	239	228
q3	2889	2917	2980	2917
q4	2009	1919	1837	1837
q5	5763	5738	5723	5723
q6	204	128	126	126
q7	2253	1788	1818	1788
q8	3435	3512	3606	3512
q9	8847	8925	8891	8891
q10	3553	3536	3545	3536
q11	586	491	512	491
q12	800	642	612	612
q13	9169	3188	3147	3147
q14	296	262	262	262
q15	488	476	465	465
q16	481	427	449	427
q17	1869	1646	1609	1609
q18	8205	8075	8203	8075
q19	1708	1599	1520	1520
q20	2090	1917	1950	1917
q21	5377	5199	5115	5115
q22	1193	1036	1009	1009
Total cold run time: 68163 ms
Total hot run time: 59761 ms

@doris-robot
Copy link

doris-robot commented Sep 11, 2025

TPC-DS: Total hot run time: 192637 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4bc4c0f044e194142fca82fd572637623f89d89b, data reload: false

query1	926	403	386	386
query2	6223	1907	1876	1876
query3	8680	206	202	202
query4	33785	23510	23728	23510
query5	3826	490	464	464
query6	309	188	190	188
query7	4201	312	317	312
query8	302	230	228	228
query9	9529	2579	2581	2579
query10	462	253	249	249
query11	18257	15282	15257	15257
query12	162	102	100	100
query13	1562	421	404	404
query14	9147	6847	7093	6847
query15	237	171	180	171
query16	7978	498	524	498
query17	1596	621	598	598
query18	2170	346	336	336
query19	245	164	167	164
query20	125	119	121	119
query21	207	112	114	112
query22	4702	4555	4412	4412
query23	34690	34696	33797	33797
query24	11292	2944	2948	2944
query25	667	396	412	396
query26	1328	172	171	171
query27	2565	351	367	351
query28	7740	2162	2191	2162
query29	880	433	426	426
query30	268	153	158	153
query31	1059	845	836	836
query32	94	55	54	54
query33	764	319	300	300
query34	917	525	514	514
query35	849	748	737	737
query36	1085	968	944	944
query37	127	66	64	64
query38	4167	3964	4029	3964
query39	1704	1454	1464	1454
query40	212	106	101	101
query41	49	48	47	47
query42	118	110	107	107
query43	546	504	479	479
query44	1316	816	808	808
query45	195	165	164	164
query46	1206	785	761	761
query47	2019	1918	1921	1918
query48	473	379	374	374
query49	938	400	392	392
query50	859	439	456	439
query51	7496	7350	7204	7204
query52	110	91	100	91
query53	265	186	190	186
query54	1235	487	474	474
query55	83	79	79	79
query56	259	261	250	250
query57	1315	1198	1185	1185
query58	221	247	211	211
query59	3234	3215	3256	3215
query60	285	256	259	256
query61	118	163	134	134
query62	894	700	685	685
query63	234	198	193	193
query64	4958	676	635	635
query65	3738	3334	3289	3289
query66	1445	310	296	296
query67	16316	15673	15634	15634
query68	4474	581	577	577
query69	431	265	265	265
query70	1196	1125	1090	1090
query71	343	260	257	257
query72	6278	4121	4155	4121
query73	784	360	357	357
query74	10409	9017	8974	8974
query75	3360	2660	2679	2660
query76	2695	1049	1114	1049
query77	404	297	289	289
query78	10487	9612	9549	9549
query79	1876	606	607	606
query80	1055	454	471	454
query81	563	221	220	220
query82	616	92	90	90
query83	239	148	148	148
query84	244	88	81	81
query85	1431	290	294	290
query86	433	310	303	303
query87	4327	4268	4249	4249
query88	3618	2418	2397	2397
query89	422	301	291	291
query90	1996	191	189	189
query91	194	149	145	145
query92	65	49	49	49
query93	2440	572	565	565
query94	904	299	304	299
query95	368	258	263	258
query96	640	288	278	278
query97	3298	3154	3162	3154
query98	224	199	192	192
query99	1463	1322	1305	1305
Total cold run time: 302241 ms
Total hot run time: 192637 ms

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 11, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/4) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 42.10% (11245/26712)
Line Coverage 32.63% (96272/295075)
Region Coverage 30.56% (55285/180889)
Branch Coverage 26.89% (27366/101780)

@doris-robot
Copy link

doris-robot commented Sep 11, 2025

ClickBench: Total hot run time: 30.23 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4bc4c0f044e194142fca82fd572637623f89d89b, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.24	0.06	0.06
query4	1.61	0.10	0.10
query5	0.52	0.52	0.52
query6	1.12	0.73	0.72
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.57	0.50	0.51
query10	0.55	0.55	0.57
query11	0.15	0.11	0.11
query12	0.14	0.11	0.10
query13	0.61	0.60	0.60
query14	0.78	0.79	0.77
query15	0.84	0.84	0.83
query16	0.39	0.38	0.40
query17	1.07	1.04	1.03
query18	0.25	0.22	0.22
query19	1.86	1.73	1.90
query20	0.01	0.02	0.02
query21	15.38	0.61	0.58
query22	2.07	2.24	1.76
query23	17.07	0.97	0.77
query24	3.11	1.13	1.29
query25	0.23	0.08	0.05
query26	0.44	0.14	0.13
query27	0.05	0.05	0.04
query28	10.11	0.47	0.49
query29	12.62	3.28	3.24
query30	0.24	0.07	0.06
query31	2.86	0.39	0.38
query32	3.26	0.45	0.45
query33	3.02	3.05	3.01
query34	16.77	4.51	4.50
query35	4.58	4.57	4.55
query36	0.66	0.48	0.47
query37	0.08	0.06	0.06
query38	0.04	0.03	0.04
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 103.77 s
Total hot run time: 30.23 s

dataroaring
dataroaring approved these changes Sep 12, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 9ba62f3 into branch-3.0 Sep 12, 2025
23 of 25 checks passed
@github-actions github-actions bot deleted the auto-pick-55881-branch-3.0 branch September 12, 2025 01:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@Thearas @doris-robot @hello-stephen @dataroaring @mymeiyi