[fix](merge-on-write) Remove pending delete bitmap check when `commit_txn()` #47136

bobhan1 · 2025-01-17T03:41:01Z

What problem does this PR solve?

Related PR: #46039

Problem Summary:

#46039 introduce an defensive check when commit_txn(), but this may influence the commit process. This PR remove this check totally to eliminate this overhead.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-01-17T03:41:06Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

bobhan1 · 2025-01-17T03:41:29Z

run buildall

bobhan1 · 2025-01-17T05:11:38Z

run buildall

cloud/src/meta-service/meta_service_txn.cpp

bobhan1 · 2025-01-17T07:20:29Z

run buildall

zhannngchen

LGTM

github-actions · 2025-01-17T07:22:49Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-01-17T07:22:51Z

PR approved by anyone and no changes requested.

bobhan1 · 2025-01-17T07:47:26Z

run buildall

zhannngchen

LGTM

github-actions · 2025-01-17T08:52:36Z

PR approved by at least one committer and no changes requested.

doris-robot · 2025-01-17T08:56:17Z

TPC-H: Total hot run time: 32378 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bcf9689de11cf57bc707fbab4d149e19476b4c2a, data reload: false

------ Round 1 ----------------------------------
q1	17707	5591	5436	5436
q2	2055	315	177	177
q3	10693	1242	739	739
q4	10235	1015	543	543
q5	7691	2450	2160	2160
q6	199	165	133	133
q7	918	779	602	602
q8	9240	1404	1175	1175
q9	5269	5023	4943	4943
q10	6982	2364	1881	1881
q11	483	269	255	255
q12	352	351	218	218
q13	17758	3663	3121	3121
q14	230	227	215	215
q15	526	473	476	473
q16	639	628	578	578
q17	599	875	336	336
q18	7161	6404	6353	6353
q19	2376	959	555	555
q20	304	316	186	186
q21	2940	2193	1992	1992
q22	367	335	307	307
Total cold run time: 104724 ms
Total hot run time: 32378 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5636	5496	5544	5496
q2	240	337	232	232
q3	2239	2686	2330	2330
q4	1475	1808	1411	1411
q5	4341	4738	4769	4738
q6	168	159	129	129
q7	2078	1945	1806	1806
q8	2684	2835	2759	2759
q9	7316	7231	7320	7231
q10	3014	3318	2727	2727
q11	570	503	486	486
q12	661	748	669	669
q13	3642	3860	3311	3311
q14	270	308	281	281
q15	500	483	461	461
q16	681	698	674	674
q17	1267	1759	1271	1271
q18	7691	7454	7519	7454
q19	807	1154	1069	1069
q20	2038	2076	1907	1907
q21	5703	5101	5048	5048
q22	587	581	583	581
Total cold run time: 53608 ms
Total hot run time: 52071 ms

doris-robot · 2025-01-17T09:08:53Z

TPC-DS: Total hot run time: 193422 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bcf9689de11cf57bc707fbab4d149e19476b4c2a, data reload: false

query1	1310	977	940	940
query2	6074	2096	2033	2033
query3	10983	4371	4467	4371
query4	60282	29028	23242	23242
query5	5554	603	447	447
query6	441	212	181	181
query7	5565	506	299	299
query8	333	247	231	231
query9	8228	2633	2578	2578
query10	448	287	242	242
query11	17696	15128	15572	15128
query12	152	114	105	105
query13	1446	538	433	433
query14	11451	6379	6568	6379
query15	221	200	191	191
query16	7282	677	464	464
query17	1087	757	602	602
query18	1861	414	331	331
query19	210	188	159	159
query20	115	110	136	110
query21	204	124	103	103
query22	4344	4402	4497	4402
query23	34337	33445	33489	33445
query24	5703	2277	2295	2277
query25	449	475	400	400
query26	640	288	157	157
query27	1613	474	341	341
query28	4328	2462	2403	2403
query29	524	575	428	428
query30	217	191	158	158
query31	932	961	828	828
query32	70	60	55	55
query33	458	384	301	301
query34	730	886	505	505
query35	819	837	773	773
query36	1026	1073	944	944
query37	122	105	77	77
query38	4360	4324	4275	4275
query39	1661	1543	1461	1461
query40	204	114	101	101
query41	51	52	49	49
query42	125	108	109	108
query43	519	530	505	505
query44	1321	846	814	814
query45	182	181	165	165
query46	890	1078	653	653
query47	1952	1903	1858	1858
query48	392	399	312	312
query49	722	506	428	428
query50	669	674	395	395
query51	6907	7053	6877	6877
query52	103	109	94	94
query53	250	259	188	188
query54	488	525	416	416
query55	86	82	81	81
query56	258	267	243	243
query57	1231	1199	1137	1137
query58	253	242	240	240
query59	3141	2954	2969	2954
query60	290	282	260	260
query61	126	124	116	116
query62	746	718	641	641
query63	219	182	184	182
query64	1266	1112	641	641
query65	3228	3164	3156	3156
query66	744	396	295	295
query67	15956	15652	15589	15589
query68	5038	770	522	522
query69	486	301	253	253
query70	1173	1114	1057	1057
query71	401	294	255	255
query72	6384	3981	3892	3892
query73	787	758	350	350
query74	10055	8934	9119	8934
query75	3264	3138	2628	2628
query76	3758	1171	760	760
query77	477	363	270	270
query78	10254	9881	9352	9352
query79	3486	805	569	569
query80	782	542	451	451
query81	515	277	242	242
query82	1153	152	122	122
query83	170	227	148	148
query84	294	94	76	76
query85	762	358	307	307
query86	406	309	305	305
query87	4387	4508	4346	4346
query88	4807	2129	2118	2118
query89	412	323	302	302
query90	1593	186	187	186
query91	148	145	110	110
query92	69	55	57	55
query93	2953	868	522	522
query94	763	411	261	261
query95	337	256	254	254
query96	499	629	288	288
query97	2843	2913	2750	2750
query98	233	194	191	191
query99	1781	1377	1249	1249
Total cold run time: 316065 ms
Total hot run time: 193422 ms

doris-robot · 2025-01-17T09:14:10Z

ClickBench: Total hot run time: 30.86 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bcf9689de11cf57bc707fbab4d149e19476b4c2a, data reload: false

query1	0.04	0.03	0.03
query2	0.09	0.04	0.04
query3	0.22	0.06	0.06
query4	1.65	0.08	0.08
query5	0.42	0.41	0.40
query6	1.17	0.66	0.66
query7	0.02	0.01	0.02
query8	0.05	0.05	0.05
query9	0.57	0.49	0.50
query10	0.57	0.56	0.56
query11	0.16	0.12	0.12
query12	0.15	0.13	0.13
query13	0.60	0.60	0.60
query14	2.72	2.73	2.73
query15	0.91	0.83	0.83
query16	0.38	0.37	0.38
query17	1.06	1.06	1.06
query18	0.19	0.19	0.18
query19	1.88	1.85	1.99
query20	0.02	0.01	0.02
query21	15.35	0.96	0.66
query22	0.78	0.77	0.69
query23	15.00	1.51	0.67
query24	2.25	0.37	0.21
query25	0.16	0.09	0.09
query26	0.28	0.19	0.18
query27	0.08	0.08	0.08
query28	13.36	1.22	0.55
query29	12.63	4.09	3.34
query30	0.25	0.09	0.06
query31	2.81	0.61	0.40
query32	3.23	0.57	0.47
query33	2.99	3.05	3.05
query34	16.57	5.12	4.52
query35	4.62	4.55	4.53
query36	0.62	0.50	0.48
query37	0.21	0.16	0.17
query38	0.16	0.15	0.15
query39	0.06	0.04	0.04
query40	0.16	0.13	0.14
query41	0.10	0.05	0.06
query42	0.07	0.05	0.05
query43	0.05	0.05	0.04
Total cold run time: 104.66 s
Total hot run time: 30.86 s

dataroaring

LGTM

…_txn()` (apache#47136) Related PR: apache#46039 Problem Summary: apache#46039 introduce an defensive check when `commit_txn()`, but this may influence the commit process. This PR remove this check totally to eliminate this overhead.

…when commit_txn() #47136 (#47283) pick #47136

…_txn()` (apache#47136) ### What problem does this PR solve? Related PR: apache#46039 Problem Summary: apache#46039 introduce an defensive check when `commit_txn()`, but this may influence the commit process. This PR remove this check totally to eliminate this overhead.

fix

0514a92

bobhan1 requested review from dataroaring, gavinchou and w41ter as code owners January 17, 2025 03:41

zhannngchen added the dev/3.0.x label Jan 17, 2025

update

09bc744

zhannngchen reviewed Jan 17, 2025

View reviewed changes

cloud/src/meta-service/meta_service_txn.cpp Show resolved Hide resolved

fix

a25141c

zhannngchen previously approved these changes Jan 17, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 17, 2025

github-actions bot added the reviewed label Jan 17, 2025

fix

bcf9689

bobhan1 dismissed zhannngchen’s stale review via bcf9689 January 17, 2025 07:46

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jan 17, 2025

zhannngchen approved these changes Jan 17, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 17, 2025

dataroaring approved these changes Jan 21, 2025

View reviewed changes

dataroaring changed the title ~~[Opt](merge-on-write) Remove pending delete bitmap check when commit_txn()~~ [fix](merge-on-write) Remove pending delete bitmap check when commit_txn() Jan 21, 2025

dataroaring merged commit a15a901 into apache:master Jan 21, 2025
25 of 27 checks passed

github-actions bot added the dev/3.0.x-conflict label Jan 21, 2025

bobhan1 mentioned this pull request Jan 21, 2025

branch-3.0: [fix](merge-on-write) Remove pending delete bitmap check when commit_txn() #47136 #47283

Merged

dataroaring pushed a commit that referenced this pull request Jan 24, 2025

branch-3.0: [fix](merge-on-write) Remove pending delete bitmap check …

84a21bd

…when commit_txn() #47136 (#47283) pick #47136

dataroaring added dev/3.0.4-merged and removed dev/3.0.x dev/3.0.x-conflict labels Jan 24, 2025

[fix](merge-on-write) Remove pending delete bitmap check when commit_txn() #47136

[fix](merge-on-write) Remove pending delete bitmap check when commit_txn() #47136

Uh oh!

Conversation

bobhan1 commented Jan 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Jan 17, 2025

Uh oh!

bobhan1 commented Jan 17, 2025

Uh oh!

bobhan1 commented Jan 17, 2025

Uh oh!

Uh oh!

bobhan1 commented Jan 17, 2025

Uh oh!

zhannngchen left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 17, 2025

Uh oh!

github-actions bot commented Jan 17, 2025

Uh oh!

bobhan1 commented Jan 17, 2025

Uh oh!

zhannngchen left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 17, 2025

Uh oh!

doris-robot commented Jan 17, 2025

Uh oh!

doris-robot commented Jan 17, 2025

Uh oh!

doris-robot commented Jan 17, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

[fix](merge-on-write) Remove pending delete bitmap check when `commit_txn()` #47136

[fix](merge-on-write) Remove pending delete bitmap check when `commit_txn()` #47136

bobhan1 commented Jan 17, 2025 •

edited

Loading