Skip to content

[chore](logs)Print logs of Export split tablet IDs #55170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 1 commit into from
Sep 3, 2025
Merged

[chore](logs)Print logs of Export split tablet IDs #55170

morningman merged 1 commit into from
Sep 3, 2025

Conversation

@CalvinKirs
Copy link
Member

@CalvinKirs CalvinKirs commented Aug 22, 2025

What problem does this PR solve?

2025-08-22 12:25:54,099 DEBUG (mysql-nio-pool-0|234) [ExportJob.generateOlapTableOutfile():288] Export Job [1755836749233]: parallelism 0 tablets: [1755777688615]

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 22, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman
Copy link
Contributor

morningman commented Sep 1, 2025

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 1, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2025

PR approved by anyone and no changes requested.

wm1581066
wm1581066 approved these changes Sep 1, 2025
View reviewed changes
Copy link
Collaborator

@wm1581066 wm1581066 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@doris-robot
Copy link

doris-robot commented Sep 1, 2025

TPC-H: Total hot run time: 34227 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8991b05eb13249b3c442e7aeb7f8df63ec374276, data reload: false

------ Round 1 ----------------------------------
q1	17639	5224	5177	5177
q2	2000	308	202	202
q3	10305	1285	730	730
q4	10248	1052	541	541
q5	7536	2470	2328	2328
q6	192	172	141	141
q7	961	756	644	644
q8	9363	1370	1150	1150
q9	6878	5125	5194	5125
q10	6930	2425	1955	1955
q11	520	314	294	294
q12	372	378	243	243
q13	17789	3682	3062	3062
q14	245	242	230	230
q15	569	503	487	487
q16	435	446	393	393
q17	608	855	370	370
q18	7531	6995	7060	6995
q19	1374	966	587	587
q20	358	351	246	246
q21	3967	3255	2358	2358
q22	1027	1031	969	969
Total cold run time: 106847 ms
Total hot run time: 34227 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5250	5216	5210	5210
q2	251	331	231	231
q3	2210	2647	2256	2256
q4	1349	1789	1345	1345
q5	4196	4410	4554	4410
q6	231	178	140	140
q7	2045	2006	1960	1960
q8	2673	2717	2578	2578
q9	7303	7499	7272	7272
q10	3084	3317	2933	2933
q11	598	509	497	497
q12	693	883	639	639
q13	3633	3921	3314	3314
q14	304	340	275	275
q15	533	476	477	476
q16	465	510	475	475
q17	1172	1622	1601	1601
q18	7715	7840	7652	7652
q19	870	912	1130	912
q20	1989	2163	1874	1874
q21	4708	4353	4280	4280
q22	1088	1035	1018	1018
Total cold run time: 52360 ms
Total hot run time: 51348 ms

@doris-robot
Copy link

doris-robot commented Sep 1, 2025

TPC-DS: Total hot run time: 186872 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8991b05eb13249b3c442e7aeb7f8df63ec374276, data reload: false

query1	1090	472	405	405
query2	6560	1789	1787	1787
query3	6753	234	228	228
query4	26143	23145	23240	23145
query5	4400	663	526	526
query6	342	247	234	234
query7	4663	513	304	304
query8	317	276	254	254
query9	8874	2973	2931	2931
query10	484	364	310	310
query11	15511	15063	14851	14851
query12	183	131	138	131
query13	1696	606	441	441
query14	9060	5992	6038	5992
query15	227	235	175	175
query16	7779	638	484	484
query17	1219	770	621	621
query18	2060	438	340	340
query19	216	200	175	175
query20	180	125	135	125
query21	217	129	116	116
query22	4172	4133	4022	4022
query23	33975	32901	32999	32901
query24	8177	2428	2455	2428
query25	567	520	448	448
query26	1235	280	171	171
query27	2736	517	374	374
query28	4346	2329	2279	2279
query29	769	616	491	491
query30	299	227	197	197
query31	879	824	730	730
query32	92	79	84	79
query33	569	401	360	360
query34	823	869	524	524
query35	822	826	754	754
query36	957	1009	964	964
query37	140	117	90	90
query38	4018	4107	3988	3988
query39	1500	1421	1436	1421
query40	225	137	132	132
query41	71	62	63	62
query42	139	123	123	123
query43	547	503	482	482
query44	1398	872	877	872
query45	187	177	172	172
query46	886	1026	676	676
query47	1759	1795	1737	1737
query48	391	442	328	328
query49	797	529	451	451
query50	694	709	410	410
query51	4159	4133	4024	4024
query52	119	127	114	114
query53	263	318	208	208
query54	622	605	540	540
query55	94	95	96	95
query56	345	334	327	327
query57	1201	1217	1134	1134
query58	298	287	283	283
query59	2634	2770	2623	2623
query60	373	356	343	343
query61	167	159	168	159
query62	802	751	665	665
query63	246	200	202	200
query64	4374	1129	817	817
query65	4290	4199	4188	4188
query66	1110	435	349	349
query67	15710	15370	14953	14953
query68	8826	939	582	582
query69	499	338	295	295
query70	1260	1162	1161	1161
query71	575	354	339	339
query72	5819	5030	5258	5030
query73	791	692	365	365
query74	9129	9007	8827	8827
query75	4234	3105	2641	2641
query76	3706	1193	784	784
query77	810	418	342	342
query78	9502	9735	8844	8844
query79	3897	870	598	598
query80	810	591	525	525
query81	485	262	224	224
query82	584	153	112	112
query83	297	278	261	261
query84	305	120	97	97
query85	891	474	418	418
query86	347	316	298	298
query87	4258	4278	4151	4151
query88	2901	2240	2261	2240
query89	466	319	290	290
query90	1931	243	233	233
query91	162	172	134	134
query92	100	80	77	77
query93	2251	1013	656	656
query94	702	421	323	323
query95	423	343	344	343
query96	493	603	281	281
query97	2637	2746	2594	2594
query98	245	220	219	219
query99	1447	1448	1325	1325
Total cold run time: 278734 ms
Total hot run time: 186872 ms

@doris-robot
Copy link

doris-robot commented Sep 1, 2025

ClickBench: Total hot run time: 32.82 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8991b05eb13249b3c442e7aeb7f8df63ec374276, data reload: false

query1	0.06	0.05	0.06
query2	0.09	0.05	0.06
query3	0.25	0.08	0.08
query4	1.61	0.12	0.12
query5	0.43	0.44	0.42
query6	1.16	0.65	0.67
query7	0.03	0.03	0.03
query8	0.06	0.05	0.04
query9	0.60	0.55	0.52
query10	0.59	0.58	0.56
query11	0.17	0.12	0.11
query12	0.16	0.12	0.12
query13	0.63	0.63	0.60
query14	0.82	0.83	0.83
query15	0.87	0.85	0.83
query16	0.39	0.40	0.40
query17	1.04	1.03	1.02
query18	0.21	0.20	0.20
query19	1.90	1.82	1.87
query20	0.01	0.01	0.01
query21	15.39	1.00	0.57
query22	0.81	1.11	0.67
query23	15.00	1.40	0.59
query24	6.68	1.70	0.79
query25	0.43	0.27	0.16
query26	0.57	0.18	0.15
query27	0.08	0.05	0.05
query28	9.76	0.95	0.45
query29	12.74	3.92	3.29
query30	3.08	3.08	3.06
query31	2.83	0.60	0.39
query32	3.24	0.54	0.46
query33	3.14	3.19	3.02
query34	16.10	5.48	4.90
query35	4.97	4.93	4.94
query36	0.70	0.52	0.50
query37	0.10	0.08	0.07
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.17	0.15
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 107.15 s
Total hot run time: 32.82 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 1, 2025

FE UT Coverage Report

Increment line coverage 0.00% (0/4) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 1, 2025

FE Regression Coverage Report

Increment line coverage 50.00% (2/4) 🎉
Increment coverage report
Complete coverage report

@morningman morningman merged commit 11da05e into apache:master Sep 3, 2025
27 of 29 checks passed
github-actions bot pushed a commit that referenced this pull request Sep 3, 2025
@CalvinKirs
[chore](logs)Print logs of Export split tablet IDs (#55170)
710cead 
### What problem does this PR solve?
`2025-08-22 12:25:54,099 DEBUG (mysql-nio-pool-0|234)
[ExportJob.generateOlapTableOutfile():288] Export Job [1755836749233]:
parallelism 0 tablets: [1755777688615]`
@github-actions github-actions bot mentioned this pull request Sep 3, 2025
Merged
morrySnow pushed a commit that referenced this pull request Sep 4, 2025
@github-actions @CalvinKirs
branch-3.1: [chore](logs)Print logs of Export split tablet IDs #55170 (…
52ae520 
…#55646)

Cherry-picked from #55170

Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
wenzhenghu pushed a commit to wenzhenghu/doris that referenced this pull request Sep 8, 2025
@CalvinKirs @wenzhenghu
[chore](logs)Print logs of Export split tablet IDs (apache#55170)
710ec9c 
### What problem does this PR solve?
`2025-08-22 12:25:54,099 DEBUG (mysql-nio-pool-0|234)
[ExportJob.generateOlapTableOutfile():288] Export Job [1755836749233]:
parallelism 0 tablets: [1755777688615]`
@morrySnow morrySnow mentioned this pull request Sep 22, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

+1 more reviewer

@wm1581066 wm1581066 wm1581066 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.1-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@CalvinKirs @hello-stephen @morningman @doris-robot @wm1581066 @morrySnow