Skip to content

[fix](Export) fix the lock leak issue of Export #50088

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 1 commit into from
Apr 21, 2025
Merged

[fix](Export) fix the lock leak issue of Export #50088

morningman merged 1 commit into from
Apr 21, 2025

Conversation

@BePPPower
Copy link
Contributor

@BePPPower BePPPower commented Apr 16, 2025 •
edited
Loading

Problem Summary:

  1. We construct a new StatementContext for Export job, so we shouldn't set ConnectContext

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Apr 16, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman
Copy link
Contributor

morningman commented Apr 18, 2025

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 18, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Apr 18, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 18, 2025

PR approved by anyone and no changes requested.

dataroaring
dataroaring approved these changes Apr 18, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@doris-robot
Copy link

doris-robot commented Apr 18, 2025

TPC-H: Total hot run time: 33863 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 071391bf8d7cebd794bd2152acb4859f1f98cb02, data reload: false

------ Round 1 ----------------------------------
q1	25993	5051	4999	4999
q2	2057	290	186	186
q3	10379	1247	681	681
q4	10234	987	533	533
q5	7511	2375	2305	2305
q6	174	163	134	134
q7	903	765	597	597
q8	9312	1252	1074	1074
q9	6743	5108	5100	5100
q10	6847	2295	1887	1887
q11	482	287	270	270
q12	353	361	226	226
q13	18385	3715	3142	3142
q14	223	220	218	218
q15	534	477	494	477
q16	439	443	388	388
q17	605	841	359	359
q18	7596	7067	7188	7067
q19	1441	946	556	556
q20	331	324	233	233
q21	4103	3354	2463	2463
q22	1061	1004	968	968
Total cold run time: 115706 ms
Total hot run time: 33863 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5134	5036	5102	5036
q2	232	328	239	239
q3	2128	2594	2279	2279
q4	1431	1811	1417	1417
q5	4444	4370	4377	4370
q6	208	168	128	128
q7	1978	1875	1746	1746
q8	2576	2567	2455	2455
q9	7250	7215	7013	7013
q10	3004	3189	2794	2794
q11	594	504	486	486
q12	681	762	623	623
q13	3572	3873	3226	3226
q14	290	303	270	270
q15	526	490	478	478
q16	486	500	462	462
q17	1127	1611	1329	1329
q18	7799	7499	7576	7499
q19	849	839	1008	839
q20	1992	2036	1806	1806
q21	5103	4827	4788	4788
q22	1078	1048	1027	1027
Total cold run time: 52482 ms
Total hot run time: 50310 ms

@doris-robot
Copy link

doris-robot commented Apr 18, 2025

TPC-DS: Total hot run time: 192672 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 071391bf8d7cebd794bd2152acb4859f1f98cb02, data reload: false

query1	1409	1075	1069	1069
query2	6207	1780	1781	1780
query3	11128	4660	4517	4517
query4	53711	25579	22978	22978
query5	5202	540	459	459
query6	366	214	203	203
query7	5021	520	280	280
query8	320	260	240	240
query9	6201	2534	2541	2534
query10	414	330	256	256
query11	15099	15166	14822	14822
query12	160	112	107	107
query13	1107	530	408	408
query14	10163	6498	6407	6407
query15	195	194	169	169
query16	7101	677	520	520
query17	1093	748	609	609
query18	1539	427	330	330
query19	207	200	173	173
query20	135	122	120	120
query21	206	137	108	108
query22	4511	4629	4515	4515
query23	34158	33405	33261	33261
query24	6866	2416	2458	2416
query25	468	471	424	424
query26	716	282	151	151
query27	2344	513	337	337
query28	2997	2104	2122	2104
query29	585	561	425	425
query30	259	217	199	199
query31	837	856	795	795
query32	76	64	63	63
query33	449	367	302	302
query34	810	863	528	528
query35	808	848	790	790
query36	960	964	892	892
query37	115	105	81	81
query38	4259	4338	4097	4097
query39	1556	1460	1433	1433
query40	216	124	106	106
query41	54	53	52	52
query42	119	112	106	106
query43	505	509	462	462
query44	1404	823	825	823
query45	185	172	165	165
query46	882	1048	641	641
query47	1807	1869	1775	1775
query48	374	427	301	301
query49	716	502	419	419
query50	696	769	412	412
query51	4297	4311	4178	4178
query52	115	119	111	111
query53	252	279	192	192
query54	581	579	525	525
query55	91	87	87	87
query56	320	308	296	296
query57	1168	1203	1162	1162
query58	262	278	258	258
query59	2668	2792	2630	2630
query60	339	325	330	325
query61	124	125	156	125
query62	748	749	716	716
query63	241	211	198	198
query64	1868	1028	682	682
query65	4357	4248	4212	4212
query66	778	402	309	309
query67	15957	15686	15365	15365
query68	6899	909	512	512
query69	541	318	257	257
query70	1153	1143	1087	1087
query71	512	321	321	321
query72	5936	4850	4937	4850
query73	1392	734	348	348
query74	9326	9057	9118	9057
query75	3853	3168	2702	2702
query76	4191	1196	821	821
query77	638	384	296	296
query78	10030	10065	9325	9325
query79	2532	828	559	559
query80	598	588	429	429
query81	483	247	213	213
query82	442	133	106	106
query83	361	252	232	232
query84	296	95	82	82
query85	779	352	308	308
query86	388	299	305	299
query87	4389	4482	4319	4319
query88	3479	2225	2209	2209
query89	406	319	285	285
query90	1802	211	215	211
query91	143	138	113	113
query92	72	59	60	59
query93	2086	955	623	623
query94	669	403	302	302
query95	374	291	284	284
query96	490	593	270	270
query97	3181	3271	3148	3148
query98	228	206	199	199
query99	1421	1368	1311	1311
Total cold run time: 299972 ms
Total hot run time: 192672 ms

@doris-robot
Copy link

doris-robot commented Apr 18, 2025

ClickBench: Total hot run time: 29.58 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 071391bf8d7cebd794bd2152acb4859f1f98cb02, data reload: false

query1	0.04	0.04	0.04
query2	0.13	0.10	0.11
query3	0.25	0.20	0.19
query4	1.59	0.20	0.20
query5	0.59	0.59	0.58
query6	1.16	0.72	0.73
query7	0.03	0.02	0.02
query8	0.05	0.03	0.03
query9	0.58	0.54	0.52
query10	0.57	0.64	0.56
query11	0.15	0.11	0.11
query12	0.15	0.11	0.11
query13	0.61	0.59	0.59
query14	1.17	1.18	1.22
query15	0.86	0.85	0.86
query16	0.39	0.37	0.40
query17	1.02	1.01	1.03
query18	0.21	0.20	0.20
query19	1.88	1.80	1.80
query20	0.01	0.01	0.01
query21	15.39	0.91	0.55
query22	0.75	1.25	1.00
query23	14.67	1.36	0.60
query24	7.40	1.37	0.35
query25	0.54	0.16	0.07
query26	0.61	0.16	0.13
query27	0.05	0.05	0.05
query28	9.63	0.88	0.43
query29	12.60	4.06	3.42
query30	0.25	0.08	0.06
query31	2.84	0.57	0.39
query32	3.23	0.55	0.47
query33	3.08	3.17	3.06
query34	15.72	5.13	4.50
query35	4.60	4.52	4.51
query36	0.66	0.51	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.13	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.95 s
Total hot run time: 29.58 s

@morningman morningman merged commit e63ae90 into apache:master Apr 21, 2025
31 checks passed
github-actions bot pushed a commit that referenced this pull request Apr 21, 2025
@BePPPower
[fix](Export) fix the lock leak issue of Export (#50088)
ac80925 
Problem Summary:

1. We construct a new StatementContext for Export job, so we shouldn't
set ConnectContext
@github-actions github-actions bot mentioned this pull request Apr 21, 2025
Merged
github-actions bot pushed a commit that referenced this pull request Apr 21, 2025
@BePPPower
[fix](Export) fix the lock leak issue of Export (#50088)
445dfa0 
Problem Summary:

1. We construct a new StatementContext for Export job, so we shouldn't
set ConnectContext
@github-actions github-actions bot mentioned this pull request Apr 21, 2025
Merged
dataroaring pushed a commit that referenced this pull request Apr 22, 2025
@github-actions @BePPPower
branch-3.0: [fix](Export) fix the lock leak issue of Export #50088 (#…
328c694 
…50218)

Cherry-picked from #50088

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
yiguolei pushed a commit that referenced this pull request Apr 22, 2025
@github-actions @BePPPower
branch-2.1: [fix](Export) fix the lock leak issue of Export #50088 (#…
b9bb9e1 
…50219)

Cherry-picked from #50088

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
@yiguolei yiguolei mentioned this pull request May 13, 2025
Closed
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@BePPPower @koarz
[fix](Export) fix the lock leak issue of Export (apache#50088)
de209c9 
Problem Summary:

1. We construct a new StatementContext for Export job, so we shouldn't
set ConnectContext
@gavinchou gavinchou mentioned this pull request Jun 11, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.10-merged dev/3.0.6-merged p0_b reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@BePPPower @hello-stephen @morningman @doris-robot @dataroaring @yiguolei