Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fscache: fix large stargz chunks processing #546

Merged
merged 2 commits into from
Jul 2, 2022

Conversation

imeoer
Copy link
Collaborator

@imeoer imeoer commented Jul 1, 2022

builder: sort stargz chunks by uncompressed_offset

Ensure that the stargz chunks in the blob meta are sorted by
uncompressed_offset and ordered by chunk index so that they
can be found quickly at runtime with a binary search.

fscache: some fixups for large stargz chunk

1. use blob_compressed_size to compute stargz's compressed gzip_size;
2. use binary search to find stargz chunk (sorted by uncompressed_offset in blob meta);

Signed-off-by: Yan Song yansong.ys@antfin.com

@hsiangkao
Copy link
Contributor

hsiangkao commented Jul 2, 2022

Tested OK with fscache backend and ghcr.io/stargz-containers/wordpress:5.9.2-esgz !

Ensure that the stargz chunks in the blob meta are sorted by
uncompressed_offset and ordered by chunk index so that they
can be found quickly at runtime with a binary search.

Signed-off-by: Yan Song <yansong.ys@antfin.com>
@yqleng1987
Copy link
Contributor

@imeoer , your pull request has been updated. A new test job will be submitted. Please wait in patience.

@yqleng1987
Copy link
Contributor

@imeoer , the test job has been submitted. Please wait in patience.

1. use blob_compressed_size to compute stargz's compressed gzip_size;
2. use binary search to find stargz chunk (sorted by uncompressed_offset in blob meta);

Signed-off-by: Yan Song <yansong.ys@antfin.com>
@yqleng1987
Copy link
Contributor

@imeoer , your pull request has been updated, but no change is detected in the code.

@imeoer imeoer changed the title [WIP] stargz: fix large chunk processing for fscache fscache: fix large stargz chunks processing Jul 2, 2022
@yqleng1987
Copy link
Contributor

@imeoer , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅SUCCESS
build-docker-image✅SUCCESS
compile-nydus✅SUCCESS
compile-ctr-remote✅SUCCESS
compile-nydus-snapshotter✅SUCCESS
start-nydus-snapshotter-config-containerd✅SUCCESS
run-container-with-nydus-image✅SUCCESS

Congratulations, your test job passed!

@hsiangkao
Copy link
Contributor

@imeoer can we highlight this feature in README.md?

@imeoer imeoer requested review from hsiangkao and jiangliu July 2, 2022 14:17
@hsiangkao
Copy link
Contributor

@imeoer can we highlight this feature in README.md?

I think they can be in another PR, let's fix the break first.

@imeoer imeoer merged commit b1be87b into dragonflyoss:master Jul 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants