Skip to content

branch-3.1: [fix](load): fix incorrect progress on finished loads #55509 #55530

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 1 commit into from
Sep 4, 2025
Merged

branch-3.1: [fix](load): fix incorrect progress on finished loads #55509 #55530

morrySnow merged 1 commit into from
Sep 4, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Sep 1, 2025

Cherry-picked from #55509

@kaijchen
[fix](load): fix incorrect progress on finished loads (#55509)
cf2c873 
Finished load jobs could show less than 100% progress because the status
was not reported correctly when any file scanner had
`num_rows_load_total == 0` and `num_rows_load_filtered == 0`.

This fix ensures the status is reported correctly and finished loads
now display 100% progress.

Before:
```
| 1756660232981 | load_data_4_objects_large_high_column | FINISHED | 0.00% (0/1000)            | BROKER | ...
```

After:
```
| 1756660232984 | load_data_5_objects_large_high_column | FINISHED | 100.00% (1000/1000)       | BROKER | ...
```
@github-actions github-actions bot requested a review from morrySnow as a code owner September 1, 2025 07:46
@Thearas
Copy link
Contributor

Thearas commented Sep 1, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Sep 1, 2025
@dataroaring dataroaring reopened this Sep 1, 2025
@Thearas
Copy link
Contributor

Thearas commented Sep 1, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Sep 1, 2025

TPC-H: Total hot run time: 32594 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cf2c87353f194dc706f0c0db6570dc281dd54711, data reload: false

------ Round 1 ----------------------------------
q1	17606	5442	5490	5442
q2	2018	393	278	278
q3	11962	1253	743	743
q4	10288	863	449	449
q5	9155	2392	2145	2145
q6	195	162	133	133
q7	898	744	618	618
q8	9347	1420	1171	1171
q9	5203	5008	4886	4886
q10	6780	2278	1839	1839
q11	478	276	257	257
q12	335	358	208	208
q13	17766	3567	2980	2980
q14	221	220	224	220
q15	516	478	457	457
q16	423	430	368	368
q17	601	857	357	357
q18	6798	6354	6487	6354
q19	1206	978	562	562
q20	322	328	199	199
q21	2968	2152	1928	1928
q22	1044	1059	1000	1000
Total cold run time: 106130 ms
Total hot run time: 32594 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5547	5495	5517	5495
q2	233	327	231	231
q3	2175	2580	2324	2324
q4	1351	1725	1367	1367
q5	4400	4914	4981	4914
q6	178	168	128	128
q7	2094	1998	1825	1825
q8	2688	2842	2709	2709
q9	7402	7297	7224	7224
q10	3057	3275	2738	2738
q11	577	537	484	484
q12	680	763	623	623
q13	3403	3768	3179	3179
q14	272	302	272	272
q15	523	461	462	461
q16	429	485	437	437
q17	1239	1721	1259	1259
q18	7641	7341	7276	7276
q19	810	1037	1132	1037
q20	2007	2059	1910	1910
q21	5311	4867	4468	4468
q22	1115	1072	1012	1012
Total cold run time: 53132 ms
Total hot run time: 51373 ms

@doris-robot
Copy link

doris-robot commented Sep 1, 2025

TPC-DS: Total hot run time: 194206 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cf2c87353f194dc706f0c0db6570dc281dd54711, data reload: false

query1	959	388	385	385
query2	6225	1907	1902	1902
query3	8677	203	194	194
query4	33651	24379	23920	23920
query5	3884	599	465	465
query6	306	202	190	190
query7	4216	505	320	320
query8	308	256	252	252
query9	9518	2616	2583	2583
query10	510	317	268	268
query11	18193	15586	15233	15233
query12	160	112	102	102
query13	1552	553	447	447
query14	9349	6959	7089	6959
query15	232	185	179	179
query16	8123	659	553	553
query17	1562	802	591	591
query18	2205	439	356	356
query19	230	200	163	163
query20	126	120	118	118
query21	211	129	113	113
query22	4428	4691	4492	4492
query23	34710	34645	34603	34603
query24	7306	2765	2789	2765
query25	538	505	446	446
query26	926	300	177	177
query27	2133	493	381	381
query28	5341	2248	2225	2225
query29	769	615	461	461
query30	243	191	160	160
query31	988	957	825	825
query32	100	66	61	61
query33	511	361	329	329
query34	758	868	547	547
query35	793	830	753	753
query36	997	1069	955	955
query37	110	101	71	71
query38	4085	4048	3981	3981
query39	1491	1660	1483	1483
query40	210	116	106	106
query41	57	52	48	48
query42	123	105	108	105
query43	520	507	487	487
query44	1396	860	849	849
query45	188	179	171	171
query46	904	1065	679	679
query47	1996	1992	1968	1968
query48	432	448	353	353
query49	767	525	410	410
query50	689	707	438	438
query51	7294	7217	7263	7217
query52	105	107	95	95
query53	245	277	207	207
query54	604	565	484	484
query55	80	78	81	78
query56	283	282	258	258
query57	1265	1271	1209	1209
query58	247	221	219	219
query59	3121	3211	3132	3132
query60	295	292	260	260
query61	124	114	114	114
query62	782	753	723	723
query63	238	194	199	194
query64	4579	1005	666	666
query65	3399	3334	3318	3318
query66	1031	417	317	317
query67	16558	15788	15625	15625
query68	7520	846	566	566
query69	502	302	276	276
query70	1254	1179	1068	1068
query71	375	289	272	272
query72	5716	3767	3830	3767
query73	643	755	355	355
query74	10298	9210	9071	9071
query75	3204	3158	2703	2703
query76	3115	1175	773	773
query77	487	372	288	288
query78	10486	10508	9663	9663
query79	3417	896	605	605
query80	633	525	455	455
query81	516	255	223	223
query82	586	118	91	91
query83	164	166	148	148
query84	235	99	84	84
query85	794	407	307	307
query86	382	320	284	284
query87	4300	4321	4272	4272
query88	4953	2447	2449	2447
query89	414	339	297	297
query90	1791	190	194	190
query91	142	165	115	115
query92	64	56	55	55
query93	2063	900	559	559
query94	688	402	311	311
query95	352	286	270	270
query96	490	616	286	286
query97	3227	3249	3165	3165
query98	225	211	198	198
query99	1642	1420	1304	1304
Total cold run time: 294634 ms
Total hot run time: 194206 ms

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 1, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 45.48% (12723/27973)
Line Coverage 36.37% (113462/311931)
Region Coverage 34.00% (64935/191010)
Branch Coverage 31.05% (34092/109814)

@doris-robot
Copy link

doris-robot commented Sep 1, 2025

ClickBench: Total hot run time: 29.17 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cf2c87353f194dc706f0c0db6570dc281dd54711, data reload: false

query1	0.03	0.02	0.03
query2	0.06	0.03	0.03
query3	0.24	0.06	0.07
query4	1.63	0.11	0.10
query5	0.50	0.51	0.53
query6	1.13	0.74	0.72
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.56	0.51	0.49
query10	0.54	0.54	0.55
query11	0.14	0.10	0.10
query12	0.13	0.11	0.11
query13	0.62	0.60	0.59
query14	0.79	0.79	0.78
query15	0.84	0.83	0.82
query16	0.37	0.42	0.36
query17	1.06	1.06	1.03
query18	0.24	0.22	0.22
query19	1.96	1.88	1.81
query20	0.01	0.01	0.01
query21	15.39	0.93	0.59
query22	0.73	0.99	0.89
query23	14.70	1.47	0.61
query24	3.87	0.87	1.22
query25	0.18	0.16	0.13
query26	0.25	0.16	0.15
query27	0.05	0.05	0.04
query28	13.39	1.03	0.45
query29	12.61	3.90	3.27
query30	0.26	0.08	0.05
query31	2.84	0.60	0.38
query32	3.22	0.55	0.46
query33	3.02	3.03	3.05
query34	16.54	5.17	4.56
query35	4.57	4.62	4.57
query36	0.64	0.49	0.47
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.08	0.03	0.03
query42	0.03	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 103.64 s
Total hot run time: 29.17 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 1, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.46% (21029/27503)
Line Coverage 69.82% (217052/310880)
Region Coverage 67.80% (130002/191748)
Branch Coverage 61.40% (67728/110308)

@morrySnow morrySnow merged commit e751dc8 into branch-3.1 Sep 4, 2025
22 of 23 checks passed
@morrySnow morrySnow deleted the auto-pick-55509-branch-3.1 branch September 4, 2025 02:34
@morrySnow morrySnow mentioned this pull request Sep 22, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@Thearas @doris-robot @hello-stephen @morrySnow @dataroaring @kaijchen