Skip to content

branch-3.0: [bugfix](k8s) using stdout to output #54129 #54171

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Aug 1, 2025
Merged

branch-3.0: [bugfix](k8s) using stdout to output #54129 #54171

dataroaring merged 1 commit into from
Aug 1, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Aug 1, 2025

Cherry-picked from #54129

@yiguolei
[bugfix](k8s) using stdout to output (#54129)
a13a030 
### What problem does this PR solve?

K8S using stdout to capture BE INFO logs. But when BE crash be will try
to output crash stack to be.out file. In the past, it could not output
to be.out file. In this feature, I format the behaviors of be log:
1. -- daemon, output log to be.info file and output crash stack to
be.out file.
2. --console,
2.1 enable_file_logger = true, output all logs to console (stdout) and
output all logs to be.info and crash stack to be.out
2.2 enable_file_logger = false, output all logs to console(stdout) and
crash stack to be.out

we will always output crash stack to logs/be.out file.
@github-actions github-actions bot requested a review from dataroaring as a code owner August 1, 2025 01:08
@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 1, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Aug 1, 2025
@dataroaring dataroaring reopened this Aug 1, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 1, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 1, 2025

TPC-H: Total hot run time: 39521 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a13a03022a8cee9b628a74b9f8666db67c5c8db5, data reload: false

------ Round 1 ----------------------------------
q1	17594	6769	6589	6589
q2	2079	171	151	151
q3	10559	1127	1196	1127
q4	10223	763	720	720
q5	7716	2849	2830	2830
q6	212	127	128	127
q7	959	613	590	590
q8	9340	1948	1969	1948
q9	6697	6414	6402	6402
q10	7011	2238	2261	2238
q11	461	256	251	251
q12	444	214	211	211
q13	17766	2991	3005	2991
q14	239	211	203	203
q15	509	485	466	466
q16	476	374	371	371
q17	964	534	615	534
q18	7276	6563	6647	6563
q19	1391	1097	994	994
q20	478	198	197	197
q21	3897	3110	3065	3065
q22	1068	953	989	953
Total cold run time: 107359 ms
Total hot run time: 39521 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6644	6623	6596	6596
q2	331	229	225	225
q3	2916	2891	2900	2891
q4	2026	1840	1833	1833
q5	5717	5732	5729	5729
q6	205	131	127	127
q7	2241	1776	1814	1776
q8	3376	3541	3542	3541
q9	8774	8936	8970	8936
q10	3585	3552	3545	3545
q11	603	501	500	500
q12	793	635	635	635
q13	9468	3206	3155	3155
q14	297	265	268	265
q15	529	463	457	457
q16	485	436	443	436
q17	1848	1656	1603	1603
q18	8325	7868	7748	7748
q19	1676	1588	1539	1539
q20	2044	1819	1831	1819
q21	5247	5018	5013	5013
q22	1118	1047	1021	1021
Total cold run time: 68248 ms
Total hot run time: 59390 ms

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 1, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/35) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 41.90% (11137/26581)
Line Coverage 32.43% (95419/294210)
Region Coverage 31.56% (49277/156121)
Branch Coverage 27.99% (25247/90196)

@doris-robot
Copy link

doris-robot commented Aug 1, 2025

TPC-DS: Total hot run time: 196265 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a13a03022a8cee9b628a74b9f8666db67c5c8db5, data reload: false

query1	1277	934	897	897
query2	6235	1849	1819	1819
query3	10906	4527	4435	4435
query4	33219	23963	23529	23529
query5	4242	489	461	461
query6	290	182	177	177
query7	4004	303	318	303
query8	287	229	222	222
query9	9337	2570	2585	2570
query10	488	255	253	253
query11	18148	15202	15072	15072
query12	154	103	100	100
query13	1557	413	413	413
query14	8650	6452	6642	6452
query15	226	178	181	178
query16	7979	496	505	496
query17	1641	610	592	592
query18	2068	321	323	321
query19	203	168	168	168
query20	127	119	117	117
query21	201	107	107	107
query22	4676	4486	4433	4433
query23	36142	34656	34539	34539
query24	12645	2897	2871	2871
query25	543	402	408	402
query26	723	164	164	164
query27	2991	349	345	345
query28	7309	2123	2180	2123
query29	665	457	441	441
query30	257	160	158	158
query31	1046	864	806	806
query32	94	55	63	55
query33	740	289	314	289
query34	951	497	488	488
query35	872	729	729	729
query36	1127	941	959	941
query37	108	64	65	64
query38	4181	3952	4019	3952
query39	1542	1469	1452	1452
query40	200	97	97	97
query41	49	46	45	45
query42	123	100	95	95
query43	516	476	468	468
query44	1264	814	811	811
query45	183	168	169	168
query46	1199	725	740	725
query47	1983	1904	1951	1904
query48	464	389	381	381
query49	925	384	407	384
query50	842	412	413	412
query51	7412	7264	7211	7211
query52	100	89	85	85
query53	257	184	186	184
query54	1121	469	468	468
query55	76	72	80	72
query56	273	242	248	242
query57	1288	1204	1200	1200
query58	226	206	205	205
query59	3164	2930	2939	2930
query60	285	259	260	259
query61	104	114	111	111
query62	876	697	666	666
query63	227	196	196	196
query64	3958	702	626	626
query65	3352	3265	3315	3265
query66	771	295	320	295
query67	16107	15474	15455	15455
query68	4486	583	563	563
query69	432	259	266	259
query70	1161	1121	1132	1121
query71	343	245	253	245
query72	6051	3932	3956	3932
query73	750	339	343	339
query74	10356	9321	9060	9060
query75	3383	2622	2683	2622
query76	2667	1114	1001	1001
query77	373	274	262	262
query78	10535	9604	9509	9509
query79	1196	601	592	592
query80	837	432	430	430
query81	536	219	220	219
query82	1266	89	85	85
query83	210	145	138	138
query84	228	74	71	71
query85	1100	368	346	346
query86	324	311	306	306
query87	4417	4292	4257	4257
query88	3593	2346	2342	2342
query89	411	296	292	292
query90	1913	182	182	182
query91	179	145	146	145
query92	57	49	49	49
query93	1138	541	547	541
query94	754	308	279	279
query95	359	258	247	247
query96	608	275	291	275
query97	3290	3214	3146	3146
query98	217	208	196	196
query99	1502	1321	1320	1320
Total cold run time: 299868 ms
Total hot run time: 196265 ms

@doris-robot
Copy link

doris-robot commented Aug 1, 2025

ClickBench: Total hot run time: 30.71 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a13a03022a8cee9b628a74b9f8666db67c5c8db5, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.04
query3	0.24	0.06	0.07
query4	1.60	0.10	0.10
query5	0.53	0.50	0.50
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.56	0.51	0.51
query10	0.54	0.55	0.56
query11	0.15	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.59	0.60
query14	0.77	0.78	0.81
query15	0.84	0.83	0.83
query16	0.37	0.39	0.39
query17	1.03	0.99	1.06
query18	0.23	0.21	0.21
query19	2.01	1.86	1.88
query20	0.02	0.02	0.01
query21	15.36	0.59	0.58
query22	2.95	2.05	1.57
query23	16.92	1.17	0.89
query24	3.41	1.65	1.48
query25	0.23	0.13	0.14
query26	0.37	0.13	0.13
query27	0.06	0.05	0.05
query28	9.12	0.53	0.45
query29	12.59	3.19	3.21
query30	0.25	0.06	0.05
query31	2.85	0.39	0.39
query32	3.24	0.46	0.45
query33	2.98	2.99	3.00
query34	17.29	4.58	4.58
query35	4.56	4.60	4.58
query36	0.68	0.50	0.48
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.16	0.13	0.12
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.04
Total cold run time: 104.3 s
Total hot run time: 30.71 s

dataroaring
dataroaring approved these changes Aug 1, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 9eca445 into branch-3.0 Aug 1, 2025
23 of 25 checks passed
@github-actions github-actions bot deleted the auto-pick-54129-branch-3.0 branch August 1, 2025 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hello-stephen @doris-robot @dataroaring @yiguolei