rafs: prefetch based on blob chunks rather than files #887
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
changweige
changed the title
anolis-bot
added
anolis_test_pass
anolis_test_fail
and removed
anolis_testing
anolis_test_pass
labels
changweige
force-pushed
the
2.1-v6-blob-prefetch
branch
from
bf02dcc
to
d36029a
Compare
changweige
changed the title
changweige
force-pushed
the
2.1-v6-blob-prefetch
branch
2 times, most recently
from
2a9ad1b
to
ba58a7d
Compare
jiangliu
reviewed
Nov 23, 2022
changweige
force-pushed
the
2.1-v6-blob-prefetch
branch
from
ba58a7d
to
08eb400
Compare
anolis-bot
added
anolis_test_pass
anolis_test_fail
and removed
anolis_testing
anolis_test_pass
labels
jiangliu
reviewed
Nov 25, 2022
| } else { | ||
| Ok(false) | ||
| } | ||
| } else { |
There was a problem hiding this comment.
md_v6.rs should only contains code for v6, so is this branch redundant?
There was a problem hiding this comment.
Seems like it's used for both v5 and v6, so it would be better to move into a common source file.
There was a problem hiding this comment.
Yes. Could we do the code movement on the master branch as I am planning to commit these changes to the master in the future?
jiangliu
reviewed
Nov 25, 2022
jiangliu
reviewed
Nov 25, 2022
Perform different policy for v5 format and v6 format as rafs v6's blobs are capable to to download chunks and decompress them all by themselves. For rafs v6, directly perform chunk based full prefetch to reduce requests to container registry and P2P cluster. Signed-off-by: Changwei Ge <gechangwei@bytedance.com>
changweige
force-pushed
the
2.1-v6-blob-prefetch
branch
from
08eb400
to
9b0a538
Compare
anolis-bot
added
anolis_testing
anolis_test_fail
and removed
anolis_test_fail
anolis_testing
labels
|
Please also open a PR for master? It should also improve master branch:) |
|
/retest |
Sure. :-) Will raise another PR to master branch |
|
@yqleng1987 , The CI test is completed, please check result:
Congratulations, your test job passed! |
||||||||||||||||||||||||
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Perform different policy for v5 format and v6 format as rafs v6's blobs are capable to download chunks and decompress them all by themselves. For rafs v6, directly perform chunk based full prefetch to reduce requests to container registry and P2P cluster.
From benchmark test results, we can see that this PR has the same performance as the master code with significantly decreased registry requests (440->219)
--- master code
--- current v2.1
--- This PR optimized
Signed-off-by: Changwei Ge gechangwei@bytedance.com