Skip to content

[refactor](filecache) unify endian ops in lru dump #53969

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Aug 4, 2025
Merged

[refactor](filecache) unify endian ops in lru dump #53969

dataroaring merged 1 commit into from
Aug 4, 2025

Conversation

@freemandealer
Copy link
Contributor

@freemandealer freemandealer commented Jul 28, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@freemandealer
[refractor](filecache) unify endian ops in lru dump
ed8f989 
Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
@Thearas
Copy link
Contributor

Thearas commented Jul 28, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@freemandealer
Copy link
Contributor Author

freemandealer commented Jul 28, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 28, 2025

TPC-H: Total hot run time: 33896 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ed8f98929cc2d4c9abca5bdcdda95817e939e318, data reload: false

------ Round 1 ----------------------------------
q1	17587	5316	5246	5246
q2	1926	284	186	186
q3	10302	1389	696	696
q4	10225	973	520	520
q5	8250	2367	2281	2281
q6	223	169	135	135
q7	882	777	631	631
q8	9322	1294	1075	1075
q9	6752	5039	5108	5039
q10	6882	2364	2017	2017
q11	463	287	281	281
q12	351	382	227	227
q13	17789	3482	3020	3020
q14	247	235	227	227
q15	537	462	462	462
q16	441	453	376	376
q17	577	822	361	361
q18	7357	6999	7002	6999
q19	2183	966	539	539
q20	341	323	224	224
q21	3302	3020	2327	2327
q22	1011	1060	1027	1027
Total cold run time: 106950 ms
Total hot run time: 33896 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5385	5260	5329	5260
q2	229	312	210	210
q3	2068	2552	2218	2218
q4	1275	1696	1328	1328
q5	4144	4458	4459	4458
q6	216	177	136	136
q7	2027	1896	1754	1754
q8	2691	2544	2448	2448
q9	7178	7476	7164	7164
q10	3173	3362	2990	2990
q11	549	508	576	508
q12	689	848	635	635
q13	3722	3763	3167	3167
q14	370	303	290	290
q15	490	470	460	460
q16	648	508	442	442
q17	1185	1645	1386	1386
q18	7714	7927	7650	7650
q19	6943	958	955	955
q20	2024	2038	1905	1905
q21	15436	4841	4742	4742
q22	1047	1049	1025	1025
Total cold run time: 69203 ms
Total hot run time: 51131 ms

@doris-robot
Copy link

doris-robot commented Jul 28, 2025

TPC-DS: Total hot run time: 172150 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ed8f98929cc2d4c9abca5bdcdda95817e939e318, data reload: false

============================================
query1	1002	399	403	399
query2	6529	1883	1695	1695
query3	6750	236	228	228
query4	27060	24093	22857	22857
query5	4355	632	525	525
query6	338	240	225	225
query7	4628	527	296	296
query8	284	255	233	233
query9	8595	2966	2913	2913
query10	473	348	286	286
query11	15886	15075	15073	15073
query12	195	137	135	135
query13	1657	563	415	415
query14	8710	6062	5962	5962
query15	222	201	164	164
query16	7636	627	446	446
query17	1369	802	646	646
query18	2093	485	331	331
query19	282	216	202	202
query20	154	141	141	141
query21	221	131	117	117
query22	4029	4124	3803	3803
query23	34352	34153	34120	34120
query24	7638	2397	2423	2397
query25	555	515	450	450
query26	727	293	171	171
query27	2555	508	347	347
query28	3064	2270	2299	2270
query29	701	595	490	490
query30	293	242	203	203
query31	893	766	726	726
query32	86	77	75	75
query33	538	445	375	375
query34	831	864	519	519
query35	814	847	764	764
query36	1015	1046	955	955
query37	140	116	96	96
query38	4009	4090	3989	3989
query39	1428	1398	1382	1382
query40	235	146	130	130
query41	61	62	56	56
query42	142	125	128	125
query43	507	510	481	481
query44	1414	871	894	871
query45	204	190	187	187
query46	952	1083	701	701
query47	1842	1888	1749	1749
query48	408	429	315	315
query49	719	500	421	421
query50	675	703	424	424
query51	5511	5581	5478	5478
query52	132	136	119	119
query53	262	295	217	217
query54	658	658	579	579
query55	95	92	89	89
query56	369	360	379	360
query57	1229	1219	1144	1144
query58	339	331	330	330
query59	2690	2691	2565	2565
query60	403	404	404	404
query61	126	126	132	126
query62	797	726	650	650
query63	253	214	210	210
query64	2834	1107	774	774
query65	4224	4081	4097	4081
query66	903	465	335	335
query67	query68	18329	1050	917	917
query69	1025	286	301	286
query70	1358	1138	1165	1138
query71	730	345	335	335
query72	9177	2323	2356	2323
query73	3642	641	367	367
query74	8908	8929	8799	8799
query75	7911	3141	2688	2688
query76	8904	1228	787	787
query77	1155	424	340	340
query78	9742	10754	9454	9454
query79	15083	606	589	589
query80	1762	568	486	486
query81	553	265	233	233
query82	489	156	117	117
query83	356	290	275	275
query84	295	102	87	87
query85	900	369	342	342
query86	355	339	331	331
query87	4347	4264	4248	4248
query88	5558	2263	2292	2263
query89	499	363	317	317
query90	2597	227	234	227
query91	147	142	114	114
query92	85	72	71	71
query93	6412	969	664	664
query94	1061	404	281	281
query95	427	332	330	330
query96	517	594	288	288
query97	2731	2758	2631	2631
query98	264	241	220	220
query99	1465	1419	1260	1260
Total cold run time: 306875 ms
Total hot run time: 172150 ms

@doris-robot
Copy link

doris-robot commented Jul 28, 2025

ClickBench: Total hot run time: 33.42 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ed8f98929cc2d4c9abca5bdcdda95817e939e318, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.24	0.07	0.07
query4	1.62	0.11	0.11
query5	0.42	0.44	0.45
query6	1.16	0.69	0.68
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.57	0.47	0.48
query10	0.53	0.54	0.52
query11	0.16	0.10	0.10
query12	0.14	0.11	0.12
query13	0.63	0.66	0.65
query14	0.92	1.08	1.08
query15	0.95	0.91	0.89
query16	0.40	0.40	0.38
query17	1.04	1.07	1.10
query18	0.22	0.20	0.21
query19	1.96	1.82	1.84
query20	0.01	0.01	0.02
query21	15.39	0.83	0.53
query22	0.83	1.23	0.84
query23	14.79	1.23	0.65
query24	6.95	1.91	0.86
query25	0.49	0.20	0.21
query26	0.56	0.17	0.14
query27	0.06	0.05	0.06
query28	10.09	0.82	0.44
query29	12.61	3.84	3.32
query30	3.05	3.03	2.92
query31	2.83	0.57	0.40
query32	3.25	0.57	0.49
query33	3.02	3.23	3.25
query34	16.07	5.34	4.83
query35	4.91	4.96	4.90
query36	0.72	0.52	0.50
query37	0.09	0.08	0.08
query38	0.05	0.05	0.04
query39	0.03	0.03	0.03
query40	0.17	0.14	0.16
query41	0.09	0.03	0.03
query42	0.04	0.02	0.03
query43	0.05	0.04	0.04
Total cold run time: 107.3 s
Total hot run time: 33.42 s

@doris-robot
Copy link

doris-robot commented Jul 28, 2025

BE UT Coverage Report

Increment line coverage 100.00% (4/4) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.58% (15967/27732)
Line Coverage 46.33% (143561/309858)
Region Coverage 35.74% (108239/302821)
Branch Coverage 38.30% (47778/124759)

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 28, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (4/4) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.07% (22069/27223)
Line Coverage 73.71% (228095/309445)
Region Coverage 61.43% (190771/310570)
Branch Coverage 65.13% (82177/126167)

sollhui
sollhui reviewed Jul 28, 2025
View reviewed changes
Copy link
Contributor

@sollhui sollhui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: refractor -> refactor?

@freemandealer
Copy link
Contributor Author

freemandealer commented Jul 29, 2025

typo: refractor -> refactor?

@sollhui thanks for pointing that out! I have misspelled the word for years

@freemandealer freemandealer changed the title [refractor](filecache) unify endian ops in lru dump [refrator](filecache) unify endian ops in lru dump Jul 29, 2025
@freemandealer freemandealer changed the title [refrator](filecache) unify endian ops in lru dump [refactor](filecache) unify endian ops in lru dump Jul 29, 2025
freemandealer added a commit to freemandealer/doris that referenced this pull request Jul 29, 2025
@freemandealer
[enhancement](cloud) Persist LRU information for file cache (pick apa…
808b49e 
…che#49456 apache#53969)

to make osx happy

Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
@morrySnow morrySnow mentioned this pull request Jul 30, 2025
Merged
morrySnow pushed a commit that referenced this pull request Jul 30, 2025
@freemandealer
branch-3.1: [fix](filecache) unify endian operations and thread renam…
6953c3d 
…ing method #53140 #53969 (#54019)

picked from #53140 #53969

---------

Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
@morrySnow morrySnow assigned morrySnow and unassigned morrySnow Jul 30, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 31, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 31, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 31, 2025

PR approved by anyone and no changes requested.

dataroaring
dataroaring approved these changes Aug 4, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit d8c8c3d into apache:master Aug 4, 2025
31 of 34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gavinchou gavinchou gavinchou approved these changes

@dataroaring dataroaring dataroaring approved these changes

+1 more reviewer

@sollhui sollhui sollhui left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@freemandealer @Thearas @doris-robot @hello-stephen @gavinchou @sollhui @dataroaring @morrySnow