Skip to content

[enhancement](filecache) add wait time for other downloader in profile #57090

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 2 commits into from
Oct 17, 2025
Merged

[enhancement](filecache) add wait time for other downloader in profile #57090

dataroaring merged 2 commits into from
Oct 17, 2025

Conversation

@freemandealer
Copy link
Contributor

@freemandealer freemandealer commented Oct 17, 2025 •
edited by gavinchou
Loading

No description provided.

@freemandealer
[enhancement](filecache) add wait time for other downloader in profile
f123f20 
Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
@Thearas
Copy link
Contributor

Thearas commented Oct 17, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@freemandealer freemandealer marked this pull request as ready for review October 17, 2025 03:33
@freemandealer
Copy link
Contributor Author

freemandealer commented Oct 17, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Oct 17, 2025

BE UT Coverage Report

Increment line coverage 100.00% (5/5) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.54% (17820/33916)
Line Coverage 37.74% (161914/429017)
Region Coverage 32.23% (123621/383529)
Branch Coverage 33.61% (54221/161332)

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 17, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (5/5) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.22% (23685/33255)
Line Coverage 57.63% (247117/428775)
Region Coverage 52.68% (204672/388493)
Branch Coverage 54.55% (88490/162227)

dataroaring
dataroaring approved these changes Oct 17, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 17, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 17, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 17, 2025

PR approved by anyone and no changes requested.

gavinchou
gavinchou reviewed Oct 17, 2025
View reviewed changes
be/src/io/cache/cached_remote_file_reader.cpp
TEST_SYNC_POINT_CALLBACK("CachedRemoteFileReader::max_wait_time", &max_wait_time);
if (block_state != FileBlock::State::DOWNLOADED) {
do {
SCOPED_RAW_TIMER(&stats.remote_wait_timer);
Copy link
Contributor

@gavinchou gavinchou Oct 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems duplicated with remote_read_timer
exposing remote_read_timer to profile is enough?

@doris-robot
Copy link

doris-robot commented Oct 17, 2025

ClickBench: Total hot run time: 31.07 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 01021f10d841b1b5f25215d088d1812143688532, data reload: false

query1	0.06	0.06	0.06
query2	0.10	0.06	0.06
query3	0.25	0.09	0.08
query4	1.62	0.12	0.12
query5	0.28	0.27	0.26
query6	1.17	0.67	0.67
query7	0.04	0.04	0.03
query8	0.07	0.05	0.06
query9	0.63	0.55	0.54
query10	0.61	0.61	0.60
query11	0.17	0.12	0.14
query12	0.18	0.13	0.13
query13	0.66	0.62	0.61
query14	1.05	1.04	1.04
query15	0.90	0.87	0.88
query16	0.41	0.42	0.43
query17	1.09	1.21	1.21
query18	0.23	0.21	0.22
query19	1.99	1.91	1.85
query20	0.01	0.02	0.01
query21	15.39	0.98	0.61
query22	0.77	1.24	0.72
query23	14.77	1.45	0.63
query24	7.01	1.26	0.61
query25	0.44	0.22	0.10
query26	0.64	0.18	0.13
query27	0.07	0.07	0.06
query28	8.81	1.43	0.93
query29	12.57	4.18	3.51
query30	0.28	0.14	0.12
query31	2.87	0.63	0.40
query32	3.25	0.57	0.48
query33	3.12	3.21	3.11
query34	16.11	5.53	4.87
query35	4.97	4.93	4.96
query36	0.70	0.54	0.51
query37	0.11	0.08	0.08
query38	0.07	0.06	0.05
query39	0.04	0.03	0.03
query40	0.19	0.15	0.15
query41	0.10	0.03	0.04
query42	0.04	0.05	0.03
query43	0.05	0.04	0.04
Total cold run time: 103.89 s
Total hot run time: 31.07 s

@dataroaring dataroaring merged commit e2cb3d6 into apache:master Oct 17, 2025
32 of 34 checks passed
github-actions bot pushed a commit that referenced this pull request Oct 17, 2025
@freemandealer
[enhancement](filecache) add wait time for other downloader in profile (
3a14114 
#57090)

Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
@github-actions github-actions bot mentioned this pull request Oct 17, 2025
Merged
yiguolei pushed a commit that referenced this pull request Oct 20, 2025
@github-actions @freemandealer
branch-4.0: [enhancement](filecache) add wait time for other download…
f292135 
…er in profile #57090 (#57120)

Cherry-picked from #57090

Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
Co-authored-by: zhengyu <zhangzhengyu@selectdb.com>
freemandealer added a commit to freemandealer/doris that referenced this pull request Dec 9, 2025
@freemandealer
[enhancement](filecache) add wait time for other downloader in profile (
40f2588 
apache#57090)

Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
@freemandealer freemandealer mentioned this pull request Dec 9, 2025
Merged
morrySnow pushed a commit that referenced this pull request Dec 10, 2025
@freemandealer
branch-3.1: [enhancement](filecache) add wait time for other download…
6a63987 
…er in profile #57090 (#58870)

picked from #57090

Signed-off-by: zhengyu <zhangzhengyu@selectdb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gavinchou gavinchou gavinchou approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.x dev/3.1.4-merged dev/4.0.1-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@freemandealer @Thearas @doris-robot @hello-stephen @gavinchou @dataroaring @yiguolei @morrySnow