Skip to content

[fix](hudi) fix quering hudi tables error #52548

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 1 commit into from
Jul 1, 2025
Merged

[fix](hudi) fix quering hudi tables error #52548

morningman merged 1 commit into from
Jul 1, 2025

Conversation

@suxiaogang223
Copy link
Contributor

@suxiaogang223 suxiaogang223 commented Jun 30, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #51152

Problem Summary:
Fix hudi p2 cases caused by NullPointerException

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Jun 30, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@suxiaogang223
Copy link
Contributor Author

suxiaogang223 commented Jun 30, 2025

run buildall

@suxiaogang223 suxiaogang223 changed the title [fix](hudi) fix hudi p2 cases [fix](hudi) fix quering hudi tables error Jun 30, 2025
@doris-robot
Copy link

doris-robot commented Jun 30, 2025

TPC-H: Total hot run time: 34150 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8c90082711371a05c413a2baeacc32e643a8bad0, data reload: false

------ Round 1 ----------------------------------
q1	17569	5303	5108	5108
q2	1974	289	191	191
q3	10293	1278	748	748
q4	10233	1009	531	531
q5	7529	2442	2367	2367
q6	179	159	128	128
q7	904	764	610	610
q8	9340	1300	1099	1099
q9	7398	5115	5124	5115
q10	6886	2417	1973	1973
q11	500	301	273	273
q12	358	356	216	216
q13	17778	3678	3122	3122
q14	237	226	214	214
q15	544	471	484	471
q16	431	424	378	378
q17	610	887	365	365
q18	7469	7114	7182	7114
q19	2263	968	560	560
q20	329	346	216	216
q21	3723	3152	2412	2412
q22	1053	999	939	939
Total cold run time: 107600 ms
Total hot run time: 34150 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5287	5145	5128	5128
q2	249	319	217	217
q3	2208	2677	2309	2309
q4	1435	1799	1349	1349
q5	4344	4369	4437	4369
q6	207	165	128	128
q7	2013	1904	1738	1738
q8	2658	2548	2627	2548
q9	7108	7231	7128	7128
q10	3074	3301	2817	2817
q11	565	513	516	513
q12	655	774	608	608
q13	3612	3941	3327	3327
q14	272	286	263	263
q15	517	480	475	475
q16	428	473	438	438
q17	1148	1495	1344	1344
q18	7386	7175	7121	7121
q19	816	803	900	803
q20	1917	2024	1887	1887
q21	4833	4443	4234	4234
q22	1081	1035	986	986
Total cold run time: 51813 ms
Total hot run time: 49730 ms

@doris-robot
Copy link

doris-robot commented Jun 30, 2025

TPC-DS: Total hot run time: 184110 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8c90082711371a05c413a2baeacc32e643a8bad0, data reload: false

query1	1031	390	385	385
query2	6516	1686	1686	1686
query3	6742	208	207	207
query4	26317	23723	23106	23106
query5	4406	628	444	444
query6	306	220	213	213
query7	4622	500	283	283
query8	281	216	217	216
query9	8598	2632	2650	2632
query10	481	328	273	273
query11	15625	15037	15213	15037
query12	168	103	104	103
query13	1647	523	401	401
query14	9467	5684	5655	5655
query15	215	191	183	183
query16	7656	604	441	441
query17	1201	712	583	583
query18	2019	398	325	325
query19	205	190	155	155
query20	118	116	112	112
query21	225	123	104	104
query22	4038	4052	3924	3924
query23	34018	33060	33008	33008
query24	8430	2419	2343	2343
query25	503	448	381	381
query26	815	265	145	145
query27	2732	508	355	355
query28	4272	2134	2110	2110
query29	680	554	432	432
query30	278	215	225	215
query31	920	826	786	786
query32	72	76	61	61
query33	547	350	301	301
query34	802	836	516	516
query35	797	790	757	757
query36	947	991	888	888
query37	110	117	73	73
query38	4187	4091	4079	4079
query39	1463	1413	1421	1413
query40	208	113	105	105
query41	55	56	51	51
query42	123	110	112	110
query43	491	524	484	484
query44	1299	813	818	813
query45	178	175	161	161
query46	822	1011	625	625
query47	1761	1799	1721	1721
query48	383	413	299	299
query49	696	478	376	376
query50	650	683	428	428
query51	4138	4199	4094	4094
query52	120	104	102	102
query53	227	250	178	178
query54	582	570	498	498
query55	86	82	81	81
query56	281	320	289	289
query57	1190	1165	1106	1106
query58	272	243	255	243
query59	2534	2693	2563	2563
query60	323	316	309	309
query61	126	119	122	119
query62	816	717	645	645
query63	220	180	189	180
query64	3484	1070	700	700
query65	4283	4147	4126	4126
query66	879	391	307	307
query67	15692	15417	15080	15080
query68	9423	877	530	530
query69	474	310	263	263
query70	1182	1130	1065	1065
query71	459	319	292	292
query72	5501	4769	4926	4769
query73	720	654	353	353
query74	8841	8805	8645	8645
query75	4304	3152	2699	2699
query76	3843	1178	728	728
query77	994	381	284	284
query78	10097	10208	9296	9296
query79	2164	813	570	570
query80	729	527	438	438
query81	469	291	224	224
query82	458	126	93	93
query83	274	250	246	246
query84	292	104	79	79
query85	778	356	305	305
query86	363	318	264	264
query87	4344	4438	4354	4354
query88	3346	2273	2269	2269
query89	386	310	294	294
query90	1960	204	203	203
query91	138	144	107	107
query92	72	57	53	53
query93	1974	960	580	580
query94	668	413	288	288
query95	375	297	284	284
query96	498	601	276	276
query97	2690	2726	2649	2649
query98	226	210	205	205
query99	1446	1421	1281	1281
Total cold run time: 274864 ms
Total hot run time: 184110 ms

@doris-robot
Copy link

doris-robot commented Jun 30, 2025

ClickBench: Total hot run time: 29.98 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8c90082711371a05c413a2baeacc32e643a8bad0, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.03	0.04
query3	0.25	0.09	0.07
query4	1.61	0.11	0.11
query5	0.42	0.42	0.40
query6	1.17	0.66	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.60	0.51	0.53
query10	0.57	0.57	0.55
query11	0.16	0.11	0.11
query12	0.14	0.11	0.12
query13	0.64	0.62	0.61
query14	0.79	0.82	0.83
query15	0.90	0.88	0.86
query16	0.37	0.39	0.40
query17	1.05	1.05	1.06
query18	0.22	0.22	0.21
query19	1.93	1.84	1.88
query20	0.01	0.02	0.01
query21	15.40	0.89	0.55
query22	0.77	1.23	0.76
query23	14.78	1.43	0.60
query24	6.84	1.54	1.27
query25	0.50	0.39	0.07
query26	0.47	0.16	0.13
query27	0.07	0.05	0.05
query28	9.25	0.90	0.44
query29	12.53	3.95	3.34
query30	0.25	0.09	0.07
query31	2.86	0.60	0.38
query32	3.23	0.56	0.48
query33	3.12	3.10	3.07
query34	15.98	5.37	4.76
query35	4.82	4.78	4.83
query36	0.70	0.50	0.49
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.06 s
Total hot run time: 29.98 s

@suxiaogang223
Copy link
Contributor Author

suxiaogang223 commented Jul 1, 2025

run p0

morningman
morningman approved these changes Jul 1, 2025
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 1, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 1, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 1, 2025

PR approved by anyone and no changes requested.

@morningman morningman merged commit 0a69acb into apache:master Jul 1, 2025
29 of 30 checks passed
github-actions bot pushed a commit that referenced this pull request Jul 1, 2025
@suxiaogang223
[fix](hudi) fix quering hudi tables error (#52548)
5d2f66a 
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #51152

Problem Summary:
Fix hudi p2 cases caused by NullPointerException
@github-actions github-actions bot mentioned this pull request Jul 1, 2025
Merged
morrySnow pushed a commit that referenced this pull request Jul 2, 2025
@github-actions @suxiaogang223
branch-3.1: [fix](hudi) fix quering hudi tables error #52548 (#52618)
8af21fc 
Cherry-picked from #52548

Co-authored-by: Socrates <suxiaogang223@icloud.com>
koarz pushed a commit to koarz/doris that referenced this pull request Jul 4, 2025
@suxiaogang223 @koarz
[fix](hudi) fix quering hudi tables error (apache#52548)
9cb04ed 
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: apache#51152

Problem Summary:
Fix hudi p2 cases caused by NullPointerException
koarz pushed a commit to koarz/doris that referenced this pull request Jul 4, 2025
@suxiaogang223 @koarz
[fix](hudi) fix quering hudi tables error (apache#52548)
e0b2ff2 
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: apache#51152

Problem Summary:
Fix hudi p2 cases caused by NullPointerException
koarz pushed a commit to koarz/doris that referenced this pull request Jul 4, 2025
@suxiaogang223 @koarz
[fix](hudi) fix quering hudi tables error (apache#52548)
38e771f 
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: apache#51152

Problem Summary:
Fix hudi p2 cases caused by NullPointerException
@suxiaogang223 suxiaogang223 deleted the fix_hudi branch July 10, 2025 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@CalvinKirs CalvinKirs CalvinKirs approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@suxiaogang223 @hello-stephen @doris-robot @morningman @CalvinKirs @morrySnow