Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve implementation to convert stargz TOC into RAFS fs #792

Merged
merged 5 commits into from
Oct 17, 2022

Conversation

jiangliu
Copy link
Collaborator

Improve implementation to convert stargz TOC into RAFS fs.

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26647

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26651

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus❌ FAIL

Sorry, your test job failed. Please get the details in the link.

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26654

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

@jiangliu jiangliu force-pushed the stargz-toc branch 2 times, most recently from daa2cec to cb2c89b Compare October 14, 2022 06:49
@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26667

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26668

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus❌ FAIL

Sorry, your test job failed. Please get the details in the link.

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26673

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26731

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26779

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26820

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

}

match matches.value_of("blob-data-size") {
None => bail!("no value specified for '--blob-data-size'"),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it seem must be calculated by blob_size - footer_size in snapshotter?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the toc offset from estargz image. When snapshotter tries to downloaded the TOC, it already gets the blob-data-size by parsing the eStargz footer.

@imeoer
Copy link
Collaborator

imeoer commented Oct 15, 2022

LGTM.

Add more doc for stargz related code.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Do not allocate chunk index for duplicated chunks, so we could get
a compact blob compression information array.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
RAFS v6 doesn't support inode digest, so only compute inode digest for
v5 when converting stargz.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Set default chunk size to 4M for stargz.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Enforce stricter validation when converting from stargz images.
Also generate valid compressed size for chunks reference stargz blob.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
@imeoer imeoer merged commit 796b3a6 into dragonflyoss:master Oct 17, 2022
@jiangliu jiangliu deleted the stargz-toc branch March 28, 2023 01:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants