Skip to content

branch-3.1: [refactor](paimon) clean unused PaimonJniScanner code and refactor PaimonSysJniScanner (#54679 #54804) #55020

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 3 commits into from
Aug 21, 2025
Merged

branch-3.1: [refactor](paimon) clean unused PaimonJniScanner code and refactor PaimonSysJniScanner (#54679 #54804) #55020

morningman merged 3 commits into from
Aug 21, 2025

Conversation

@suxiaogang223
Copy link
Contributor

@suxiaogang223 suxiaogang223 commented Aug 19, 2025 •
edited
Loading

bp:
#54679 adapt session time_zone for PaimonJniScanner
#54804 refactor PaimonSysJniScanner

@suxiaogang223
Copy link
Contributor Author

suxiaogang223 commented Aug 19, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 19, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@suxiaogang223 suxiaogang223 changed the title branch-3.1: [fix](paimon) Adapt SessionVariable time_zone for PaimonJniScanner (#54679) branch-3.1: [refactor](paimon) clean unused PaimonJniScanner code and refactor PaimonSysJniScanner (#54679 #54804) Aug 19, 2025
@suxiaogang223
Copy link
Contributor Author

suxiaogang223 commented Aug 19, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 19, 2025

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.18% (1212/1493)
Line Coverage 65.49% (21557/32916)
Region Coverage 67.10% (10867/16195)
Branch Coverage 56.66% (5723/10100)

suxiaogang223 and others added 2 commits August 20, 2025 10:02
@suxiaogang223
[fix](paimon) Adapt SessionVariable time_zone for PaimonJniScanner (a…
f7844c0 
…pache#54679)

### What problem does this PR solve?
Problem Summary:
Adapt SessionVariable `time_zone` for PaimonJniScanner to make result
correction
@suxiaogang223
[refactor](paimon) clean unused PaimonJniScanner code and refactor Pa…
ec3cbea 
…imonSysJniScanner (apache#54804)

1. Refactor `PaimonSysJniScanner` and `IcebergSysJniScanner`
2. Clean unused codes about `PaimonJniScanner`
@suxiaogang223
Copy link
Contributor Author

suxiaogang223 commented Aug 20, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 20, 2025

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.18% (1212/1493)
Line Coverage 65.52% (21567/32916)
Region Coverage 67.14% (10873/16195)
Branch Coverage 56.67% (5724/10100)

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 20, 2025

FE UT Coverage Report

Increment line coverage 0.00% (0/51) 🎉
Increment coverage report
Complete coverage report

@suxiaogang223
Copy link
Contributor Author

suxiaogang223 commented Aug 20, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 20, 2025

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.18% (1212/1493)
Line Coverage 65.59% (21589/32916)
Region Coverage 67.14% (10874/16195)
Branch Coverage 56.72% (5729/10100)

@doris-robot
Copy link

doris-robot commented Aug 20, 2025

TPC-H: Total hot run time: 32627 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c2cbc8b42f2858c04ee4f4eaa11f2e0060a33372, data reload: false

------ Round 1 ----------------------------------
q1	17876	5586	5496	5496
q2	2063	288	160	160
q3	10562	1291	720	720
q4	10209	865	445	445
q5	7986	2412	2137	2137
q6	186	164	133	133
q7	906	754	602	602
q8	9326	1466	1202	1202
q9	5248	4953	4916	4916
q10	6784	2255	1835	1835
q11	465	283	255	255
q12	328	359	211	211
q13	17780	3609	3024	3024
q14	229	245	204	204
q15	539	464	469	464
q16	411	420	371	371
q17	602	866	379	379
q18	7158	6359	6529	6359
q19	1561	960	565	565
q20	325	332	200	200
q21	2788	2158	1970	1970
q22	1053	1018	979	979
Total cold run time: 104385 ms
Total hot run time: 32627 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5611	5612	5563	5563
q2	243	341	235	235
q3	2270	2656	2330	2330
q4	1339	1823	1347	1347
q5	4382	4887	5023	4887
q6	176	162	130	130
q7	2096	1895	1805	1805
q8	2663	2769	2714	2714
q9	7270	7155	7078	7078
q10	2995	3259	2812	2812
q11	572	530	493	493
q12	665	777	592	592
q13	3352	3745	3179	3179
q14	298	322	283	283
q15	513	470	457	457
q16	464	487	434	434
q17	1235	1751	1265	1265
q18	7774	7357	7205	7205
q19	804	1105	1135	1105
q20	1993	2087	1860	1860
q21	5293	4930	4566	4566
q22	1124	1092	1054	1054
Total cold run time: 53132 ms
Total hot run time: 51394 ms

@doris-robot
Copy link

doris-robot commented Aug 20, 2025

TPC-DS: Total hot run time: 191340 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c2cbc8b42f2858c04ee4f4eaa11f2e0060a33372, data reload: false

query1	970	383	381	381
query2	6546	1918	1888	1888
query3	6705	218	217	217
query4	33986	23611	23537	23537
query5	4298	596	474	474
query6	267	188	175	175
query7	4624	509	317	317
query8	289	244	240	240
query9	9539	2589	2527	2527
query10	475	332	253	253
query11	18179	15387	15357	15357
query12	160	112	102	102
query13	1649	565	420	420
query14	9466	7235	7684	7235
query15	222	193	190	190
query16	8105	654	521	521
query17	1704	767	591	591
query18	2171	430	325	325
query19	250	193	187	187
query20	142	124	119	119
query21	206	127	111	111
query22	4550	4585	4297	4297
query23	34870	33760	33600	33600
query24	7702	2628	2670	2628
query25	487	476	419	419
query26	1224	282	174	174
query27	1999	470	342	342
query28	5149	2163	2120	2120
query29	676	592	465	465
query30	258	193	161	161
query31	980	870	837	837
query32	67	63	57	57
query33	524	380	306	306
query34	730	825	515	515
query35	758	782	731	731
query36	1004	1039	943	943
query37	110	104	72	72
query38	3933	3936	3883	3883
query39	1499	1440	1416	1416
query40	211	121	114	114
query41	60	65	65	65
query42	124	103	107	103
query43	507	508	476	476
query44	1339	810	790	790
query45	184	174	169	169
query46	872	1029	672	672
query47	1888	1906	1825	1825
query48	432	432	358	358
query49	811	509	424	424
query50	695	704	419	419
query51	7128	7222	7175	7175
query52	101	100	91	91
query53	231	264	188	188
query54	528	539	467	467
query55	81	79	77	77
query56	269	264	252	252
query57	1251	1220	1173	1173
query58	240	223	216	216
query59	2922	3104	2980	2980
query60	300	295	271	271
query61	115	119	117	117
query62	807	725	661	661
query63	238	193	197	193
query64	3943	971	650	650
query65	3316	3208	3198	3198
query66	1040	412	313	313
query67	15959	15752	15620	15620
query68	7028	826	524	524
query69	472	315	268	268
query70	1220	1122	1126	1122
query71	557	301	266	266
query72	5646	3714	3893	3714
query73	645	749	352	352
query74	10489	9106	9220	9106
query75	3238	3098	2701	2701
query76	3311	1191	775	775
query77	613	372	274	274
query78	10441	10423	9577	9577
query79	2710	909	616	616
query80	639	538	454	454
query81	503	272	218	218
query82	611	118	87	87
query83	176	168	143	143
query84	247	110	85	85
query85	781	350	295	295
query86	362	326	323	323
query87	4357	4266	4218	4218
query88	4679	2418	2365	2365
query89	401	329	299	299
query90	1907	190	187	187
query91	140	145	109	109
query92	69	57	51	51
query93	1455	921	539	539
query94	697	397	310	310
query95	343	281	260	260
query96	496	607	297	297
query97	3227	3330	3113	3113
query98	226	218	208	208
query99	1551	1423	1294	1294
Total cold run time: 291302 ms
Total hot run time: 191340 ms

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 20, 2025

FE UT Coverage Report

Increment line coverage 0.00% (0/51) 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

doris-robot commented Aug 20, 2025

ClickBench: Total hot run time: 28.31 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c2cbc8b42f2858c04ee4f4eaa11f2e0060a33372, data reload: false

query1	0.03	0.03	0.04
query2	0.06	0.04	0.03
query3	0.23	0.07	0.06
query4	1.62	0.11	0.11
query5	0.55	0.52	0.54
query6	1.13	0.73	0.73
query7	0.02	0.02	0.02
query8	0.06	0.03	0.04
query9	0.56	0.49	0.52
query10	0.55	0.54	0.56
query11	0.14	0.10	0.10
query12	0.14	0.11	0.10
query13	0.61	0.60	0.59
query14	0.80	0.80	0.78
query15	0.84	0.82	0.84
query16	0.39	0.37	0.39
query17	1.06	1.02	1.00
query18	0.23	0.24	0.23
query19	1.90	1.89	1.80
query20	0.02	0.02	0.01
query21	15.39	0.92	0.56
query22	0.73	0.74	0.65
query23	15.14	1.43	0.60
query24	3.75	0.57	1.89
query25	0.12	0.05	0.11
query26	0.43	0.15	0.13
query27	0.04	0.05	0.04
query28	13.30	0.99	0.43
query29	12.59	3.80	3.22
query30	0.26	0.10	0.06
query31	2.82	0.57	0.37
query32	3.23	0.53	0.45
query33	2.98	3.07	2.98
query34	16.65	5.20	4.55
query35	4.52	4.50	4.56
query36	0.66	0.48	0.50
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.16	0.12	0.13
query41	0.09	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.05 s
Total hot run time: 28.31 s

@doris-robot
Copy link

doris-robot commented Aug 20, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/14) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 45.49% (12724/27969)
Line Coverage 36.37% (113385/311777)
Region Coverage 34.01% (64914/190867)
Branch Coverage 31.04% (34070/109750)

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 20, 2025

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/14) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.09% (20923/27499)
Line Coverage 69.35% (215491/310720)
Region Coverage 67.27% (128896/191601)
Branch Coverage 60.90% (67134/110242)

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 20, 2025

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/14) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.43% (21018/27499)
Line Coverage 69.80% (216874/310720)
Region Coverage 67.77% (129841/191601)
Branch Coverage 61.33% (67612/110242)

@morningman morningman merged commit 8d9bc86 into apache:branch-3.1 Aug 21, 2025
20 of 22 checks passed
@suxiaogang223 suxiaogang223 deleted the adapt_time_zone_3.1 branch September 23, 2025 03:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@suxiaogang223 @hello-stephen @doris-robot @morrySnow @morningman