Skip to content

[fix](paimon) Adapt SessionVariable time_zone for PaimonJniScanner #54679

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 3 commits into from
Aug 14, 2025
Merged

[fix](paimon) Adapt SessionVariable time_zone for PaimonJniScanner #54679

morningman merged 3 commits into from
Aug 14, 2025

Conversation

@suxiaogang223
Copy link
Contributor

@suxiaogang223 suxiaogang223 commented Aug 13, 2025 •
edited by morningman
Loading

What problem does this PR solve?

Problem Summary:
Adapt SessionVariable time_zone for PaimonJniScanner to make result correction

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@suxiaogang223
Copy link
Contributor Author

suxiaogang223 commented Aug 13, 2025

run buildall

@suxiaogang223 suxiaogang223 force-pushed the support_paimon_time_zone branch from 0b8d8df to acf6020 Compare August 13, 2025 09:20
@suxiaogang223
Copy link
Contributor Author

suxiaogang223 commented Aug 13, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 13, 2025

TPC-H: Total hot run time: 33873 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit acf60206685dddacdaef5ee2fb2f184ff289a19f, data reload: false

------ Round 1 ----------------------------------
q1	17585	5223	5013	5013
q2	1916	293	179	179
q3	10302	1283	722	722
q4	10228	994	515	515
q5	7506	2449	2306	2306
q6	180	158	128	128
q7	887	728	618	618
q8	9298	1298	1052	1052
q9	6987	5214	5152	5152
q10	6896	2412	1985	1985
q11	472	278	268	268
q12	342	351	225	225
q13	17793	3656	3006	3006
q14	234	233	233	233
q15	534	479	475	475
q16	417	421	385	385
q17	580	855	358	358
q18	7325	7129	7217	7129
q19	1085	943	552	552
q20	345	325	216	216
q21	3934	3210	2373	2373
q22	1097	1035	983	983
Total cold run time: 105943 ms
Total hot run time: 33873 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5137	5121	5109	5109
q2	257	325	222	222
q3	2179	2708	2281	2281
q4	1357	1791	1306	1306
q5	4211	4432	4561	4432
q6	230	174	125	125
q7	2090	1918	1833	1833
q8	2672	2556	2585	2556
q9	7388	7222	7294	7222
q10	3071	3359	2912	2912
q11	580	520	491	491
q12	696	788	642	642
q13	3497	3840	3337	3337
q14	293	311	317	311
q15	504	497	503	497
q16	459	491	442	442
q17	1236	1596	1372	1372
q18	7927	7744	7785	7744
q19	838	820	886	820
q20	1962	1929	1812	1812
q21	4707	4302	4269	4269
q22	1100	1070	1011	1011
Total cold run time: 52391 ms
Total hot run time: 50746 ms

@doris-robot
Copy link

doris-robot commented Aug 13, 2025

TPC-DS: Total hot run time: 184989 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit acf60206685dddacdaef5ee2fb2f184ff289a19f, data reload: false

query1	991	392	391	391
query2	6518	1677	1715	1677
query3	6742	220	226	220
query4	26549	23551	23117	23117
query5	4343	597	467	467
query6	285	203	197	197
query7	4623	489	296	296
query8	274	247	211	211
query9	8608	2846	2833	2833
query10	468	314	279	279
query11	15961	15030	15262	15030
query12	161	113	113	113
query13	1667	550	431	431
query14	8978	5690	5558	5558
query15	191	194	161	161
query16	7223	627	448	448
query17	1215	688	577	577
query18	1971	405	306	306
query19	183	183	150	150
query20	132	119	115	115
query21	214	118	105	105
query22	4317	4275	4309	4275
query23	34067	33395	33310	33310
query24	8167	2327	2340	2327
query25	561	477	397	397
query26	1237	267	155	155
query27	2755	510	341	341
query28	4448	2282	2233	2233
query29	773	547	441	441
query30	285	219	197	197
query31	868	778	742	742
query32	80	73	74	73
query33	553	361	330	330
query34	783	825	509	509
query35	804	825	761	761
query36	962	985	909	909
query37	117	104	81	81
query38	3992	4065	3990	3990
query39	1503	1636	1424	1424
query40	226	122	111	111
query41	61	54	53	53
query42	119	110	109	109
query43	522	492	461	461
query44	1351	854	861	854
query45	178	170	164	164
query46	854	993	637	637
query47	1787	1842	1775	1775
query48	366	417	307	307
query49	717	475	374	374
query50	654	689	392	392
query51	4093	4165	4178	4165
query52	116	107	101	101
query53	231	258	190	190
query54	587	581	513	513
query55	89	84	80	80
query56	296	309	305	305
query57	1188	1228	1144	1144
query58	277	263	272	263
query59	2589	2729	2598	2598
query60	341	351	342	342
query61	158	122	123	122
query62	772	713	669	669
query63	223	201	185	185
query64	4364	979	681	681
query65	4297	4221	4166	4166
query66	1161	405	319	319
query67	15571	15168	15117	15117
query68	5661	911	586	586
query69	492	336	287	287
query70	1242	1114	1082	1082
query71	394	322	320	320
query72	5554	4778	4968	4778
query73	677	656	350	350
query74	8973	8979	8633	8633
query75	3108	3059	2639	2639
query76	3209	1127	732	732
query77	487	398	323	323
query78	9437	9727	8966	8966
query79	1450	801	578	578
query80	635	597	455	455
query81	475	252	222	222
query82	185	133	105	105
query83	251	250	237	237
query84	242	109	80	80
query85	751	374	348	348
query86	346	338	313	313
query87	4214	4318	4138	4138
query88	2744	2158	2152	2152
query89	373	325	290	290
query90	1757	218	215	215
query91	140	136	111	111
query92	70	73	67	67
query93	1063	982	637	637
query94	608	397	304	304
query95	384	315	321	315
query96	480	601	280	280
query97	2596	2689	2573	2573
query98	229	223	210	210
query99	1344	1398	1289	1289
Total cold run time: 266328 ms
Total hot run time: 184989 ms

@doris-robot
Copy link

doris-robot commented Aug 13, 2025

ClickBench: Total hot run time: 32.75 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit acf60206685dddacdaef5ee2fb2f184ff289a19f, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.05	0.04
query3	0.23	0.07	0.07
query4	1.62	0.11	0.11
query5	0.43	0.41	0.41
query6	1.17	0.64	0.65
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.59	0.52	0.53
query10	0.58	0.57	0.57
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.62	0.60	0.60
query14	0.80	0.92	0.84
query15	0.86	0.86	0.85
query16	0.38	0.40	0.38
query17	1.04	1.02	1.07
query18	0.22	0.20	0.19
query19	1.94	1.84	1.84
query20	0.01	0.01	0.02
query21	15.41	0.92	0.55
query22	0.79	1.19	0.72
query23	14.83	1.39	0.61
query24	6.36	1.96	1.14
query25	0.48	0.24	0.08
query26	0.54	0.16	0.14
query27	0.06	0.05	0.06
query28	9.40	0.90	0.42
query29	12.68	3.88	3.21
query30	3.06	3.00	2.93
query31	2.82	0.56	0.38
query32	3.25	0.55	0.47
query33	3.11	3.08	3.07
query34	15.81	5.45	4.82
query35	4.91	4.91	4.98
query36	0.68	0.50	0.50
query37	0.09	0.07	0.08
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.18	0.13	0.15
query41	0.07	0.02	0.02
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.66 s
Total hot run time: 32.75 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 13, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 59.04% (16627/28164)
Line Coverage 47.93% (150818/314682)
Region Coverage 36.75% (112960/307413)
Branch Coverage 39.68% (50122/126305)

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 13, 2025

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.76% (22598/27641)
Line Coverage 74.39% (234104/314689)
Region Coverage 61.68% (194717/315674)
Branch Coverage 65.74% (84050/127851)

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 13, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Aug 13, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 13, 2025

PR approved by anyone and no changes requested.

@morningman morningman merged commit 9d51a5a into apache:master Aug 14, 2025
29 of 31 checks passed
suxiaogang223 added a commit to suxiaogang223/doris that referenced this pull request Aug 19, 2025
@suxiaogang223
[fix](paimon) Adapt SessionVariable time_zone for PaimonJniScanner (a…
2d303d9 
…pache#54679)

### What problem does this PR solve?
Problem Summary:
Adapt SessionVariable `time_zone` for PaimonJniScanner to make result
correction
@suxiaogang223 suxiaogang223 mentioned this pull request Aug 19, 2025
Merged
suxiaogang223 added a commit to suxiaogang223/doris that referenced this pull request Aug 20, 2025
@suxiaogang223
[fix](paimon) Adapt SessionVariable time_zone for PaimonJniScanner (a…
f7844c0 
…pache#54679)

### What problem does this PR solve?
Problem Summary:
Adapt SessionVariable `time_zone` for PaimonJniScanner to make result
correction
morningman pushed a commit that referenced this pull request Aug 21, 2025
@suxiaogang223
branch-3.1: [refactor](paimon) clean unused PaimonJniScanner code and…
8d9bc86 
… refactor PaimonSysJniScanner (#54679 #54804)  (#55020)

bp: 
#54679  adapt session time_zone for PaimonJniScanner
#54804 refactor PaimonSysJniScanner
@suxiaogang223 suxiaogang223 deleted the support_paimon_time_zone branch September 1, 2025 02:15
suxiaogang223 added a commit to suxiaogang223/doris that referenced this pull request Sep 1, 2025
@suxiaogang223
[fix](paimon) Adapt SessionVariable time_zone for PaimonJniScanner (a…
7586d8d 
…pache#54679)

Problem Summary:
Adapt SessionVariable `time_zone` for PaimonJniScanner to make result
correction
@suxiaogang223 suxiaogang223 mentioned this pull request Sep 1, 2025
Closed
yiguolei pushed a commit to yiguolei/incubator-doris that referenced this pull request Sep 28, 2025
@suxiaogang223
branch-3.0: [fix](paimon) Adapt SessionVariable time_zone for PaimonJ…
5e6a524 
…niScanner (apache#54679) (apache#4693)

bp: apache#54679
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@CalvinKirs CalvinKirs CalvinKirs approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@suxiaogang223 @doris-robot @hello-stephen @morningman @CalvinKirs