Skip to content

Conversation

@mymeiyi
Copy link
Contributor

@mymeiyi mymeiyi commented May 29, 2025

What problem does this PR solve?

  1. after [fix](group commit) reduce cpu cost for group_commit get_block #49822, the group_commit_scan_operator is blocked by time dependency until timeout, this pr fix it
  2. modify some debug log
  3. print the create plan error to client

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented May 29, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@mymeiyi mymeiyi force-pushed the fix-group-commit branch from 84ef6d4 to 0502b76 Compare May 29, 2025 09:45
@mymeiyi
Copy link
Contributor Author

mymeiyi commented May 29, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33939 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0502b76f7210eb61c08867ad999249a136cdbcb1, data reload: false

------ Round 1 ----------------------------------
q1	26019	5050	5036	5036
q2	1953	277	182	182
q3	10295	1282	731	731
q4	10219	990	552	552
q5	7563	2260	2398	2260
q6	192	168	130	130
q7	890	766	602	602
q8	9292	1286	1117	1117
q9	6906	5155	5200	5155
q10	6875	2340	1906	1906
q11	484	291	282	282
q12	350	351	220	220
q13	17787	3705	3112	3112
q14	228	240	218	218
q15	568	488	485	485
q16	425	435	370	370
q17	617	854	351	351
q18	7794	7164	7080	7080
q19	1480	970	581	581
q20	344	349	219	219
q21	3898	3160	2386	2386
q22	988	973	964	964
Total cold run time: 115167 ms
Total hot run time: 33939 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5177	5131	5116	5116
q2	239	320	218	218
q3	2170	2633	2338	2338
q4	1338	1790	1322	1322
q5	4508	4372	4395	4372
q6	218	168	130	130
q7	2017	1917	1765	1765
q8	2583	2629	2551	2551
q9	7165	7382	6958	6958
q10	3039	3195	2775	2775
q11	578	523	501	501
q12	677	738	606	606
q13	3535	3891	3248	3248
q14	319	297	271	271
q15	540	473	509	473
q16	453	486	449	449
q17	1142	1551	1385	1385
q18	7809	7452	7640	7452
q19	805	774	885	774
q20	1983	2051	1833	1833
q21	4758	4537	4497	4497
q22	1107	1087	991	991
Total cold run time: 52160 ms
Total hot run time: 50025 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193377 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0502b76f7210eb61c08867ad999249a136cdbcb1, data reload: false

query1	1413	1094	1078	1078
query2	6190	1858	1871	1858
query3	11040	4472	4543	4472
query4	57027	24732	23331	23331
query5	4946	480	485	480
query6	336	224	191	191
query7	4923	502	292	292
query8	269	222	196	196
query9	5415	2651	2649	2649
query10	440	329	266	266
query11	14988	15080	14916	14916
query12	157	108	106	106
query13	1033	542	409	409
query14	10190	6431	6303	6303
query15	215	197	194	194
query16	7211	661	511	511
query17	1084	743	624	624
query18	1749	417	320	320
query19	214	204	181	181
query20	132	131	121	121
query21	209	130	108	108
query22	4369	4537	4360	4360
query23	34343	33561	33648	33561
query24	6501	2406	2497	2406
query25	468	455	403	403
query26	695	276	158	158
query27	2179	516	347	347
query28	2945	2179	2171	2171
query29	576	575	433	433
query30	277	235	195	195
query31	824	847	760	760
query32	78	68	66	66
query33	451	355	342	342
query34	779	867	544	544
query35	787	830	768	768
query36	934	1014	893	893
query37	112	118	79	79
query38	4276	4294	4194	4194
query39	1552	1449	1459	1449
query40	209	129	104	104
query41	61	59	57	57
query42	122	111	110	110
query43	522	540	497	497
query44	1372	864	850	850
query45	177	175	168	168
query46	865	1029	666	666
query47	1839	1868	1786	1786
query48	414	425	325	325
query49	632	508	413	413
query50	671	690	411	411
query51	4256	4312	4201	4201
query52	114	121	106	106
query53	231	254	190	190
query54	602	590	525	525
query55	88	86	82	82
query56	335	323	304	304
query57	1154	1190	1116	1116
query58	266	271	265	265
query59	2739	2902	2876	2876
query60	335	320	318	318
query61	121	120	121	120
query62	718	766	667	667
query63	231	188	192	188
query64	1774	999	690	690
query65	4224	4151	4138	4138
query66	715	400	313	313
query67	16123	15679	15542	15542
query68	8221	887	528	528
query69	541	301	268	268
query70	1145	1077	1083	1077
query71	496	317	312	312
query72	6119	4834	4704	4704
query73	1465	596	365	365
query74	8919	9177	8933	8933
query75	3724	3226	2675	2675
query76	4154	1190	742	742
query77	590	368	394	368
query78	10129	10078	9458	9458
query79	2748	823	579	579
query80	797	518	464	464
query81	484	262	215	215
query82	444	122	94	94
query83	378	256	225	225
query84	292	110	91	91
query85	789	355	317	317
query86	405	309	274	274
query87	4425	4416	4317	4317
query88	3404	2279	2305	2279
query89	404	306	276	276
query90	1785	211	210	210
query91	145	140	112	112
query92	77	62	55	55
query93	2282	924	579	579
query94	651	401	295	295
query95	367	282	290	282
query96	497	570	277	277
query97	2753	2747	2766	2747
query98	235	203	209	203
query99	1433	1376	1293	1293
Total cold run time: 302535 ms
Total hot run time: 193377 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.27 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0502b76f7210eb61c08867ad999249a136cdbcb1, data reload: false

query1	0.04	0.03	0.03
query2	0.12	0.11	0.11
query3	0.24	0.19	0.19
query4	1.60	0.20	0.11
query5	0.46	0.40	0.43
query6	1.16	0.66	0.65
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.60	0.52	0.52
query10	0.58	0.58	0.57
query11	0.15	0.12	0.11
query12	0.15	0.12	0.11
query13	0.61	0.61	0.60
query14	0.78	0.82	0.80
query15	0.89	0.86	0.87
query16	0.39	0.39	0.40
query17	1.02	1.03	1.03
query18	0.22	0.20	0.21
query19	2.01	1.85	1.83
query20	0.02	0.01	0.01
query21	15.39	0.88	0.54
query22	0.77	1.24	0.70
query23	14.82	1.41	0.61
query24	6.76	1.47	1.00
query25	0.51	0.18	0.15
query26	0.61	0.15	0.13
query27	0.06	0.05	0.05
query28	10.33	0.90	0.42
query29	12.54	4.02	3.31
query30	0.25	0.09	0.07
query31	2.83	0.61	0.39
query32	3.23	0.56	0.46
query33	3.05	3.05	3.15
query34	15.67	5.09	4.51
query35	4.51	4.53	4.49
query36	0.66	0.50	0.49
query37	0.08	0.06	0.06
query38	0.06	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.03	0.02	0.03
query43	0.04	0.04	0.03
Total cold run time: 103.59 s
Total hot run time: 29.27 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 29, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 6.00% (3/50) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 56.16% (15029/26759)
Line Coverage 45.08% (134153/297596)
Region Coverage 44.19% (67512/152784)
Branch Coverage 38.76% (34587/89234)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 68.00% (34/50) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.60% (20969/26343)
Line Coverage 72.77% (216551/297585)
Region Coverage 70.98% (127541/179689)
Branch Coverage 64.71% (66022/102034)

@dataroaring dataroaring merged commit 7f0397e into apache:master May 30, 2025
25 of 27 checks passed
mymeiyi added a commit to mymeiyi/doris that referenced this pull request May 30, 2025
1. after apache#49822, the
group_commit_scan_operator is blocked by time dependency until timeout,
this pr fix it
2. modify some debug log
3. print the create plan error to client
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
### What problem does this PR solve?

1. after apache#49822, the
group_commit_scan_operator is blocked by time dependency until timeout,
this pr fix it
2. modify some debug log
3. print the create plan error to client
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.6-merged p0_b reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants