Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance nydus-image to generate RAFS filesystem directly from tarball #795

Merged
merged 9 commits into from
Oct 22, 2022

Conversation

jiangliu
Copy link
Collaborator

@jiangliu jiangliu commented Oct 14, 2022

root@5ad838c8bf7b:/nydus# time target/debug/nydus-image create -t tar-rafs -D target/tmp tests/texture/tar/all-entry-type.tar
[2022-10-14 10:42:01.722533 +00:00] INFO [src/bin/nydus-image/main.rs:743] build successfully: BuildOutput { blobs: ["9d63c3b5b7623d1fa3dc7fd1547313b9546c6d0fbbb6773a420613b7a17995c8"], blob_size: Some(15), bootstrap_path: Some("target/tmp/ad523c15e5b7ef406796e0fda8b7cf9c9bbc149ade207c0eb4b5890ba9b7f05b") }

real 0m0.074s
user 0m0.008s
sys 0m0.015s
root@5ad838c8bf7b:/nydus# time target/debug/nydus-image create -t tar-rafs -D target/tmp --fs-version 6 tests/texture/tar/all-entry-type.tar
[2022-10-14 10:42:17.033159 +00:00] INFO [src/bin/nydus-image/main.rs:600] v6 enforces to use "aligned-chunk".
[2022-10-14 10:42:17.077944 +00:00] INFO [rafs/src/metadata/md_v6.rs:49] rafs superblock features: COMPRESS_LZ4_BLOCK | DIGESTER_BLAKE3 | EXPLICIT_UID_GID
[2022-10-14 10:42:17.078813 +00:00] INFO [src/bin/nydus-image/main.rs:743] build successfully: BuildOutput { blobs: ["1a3343b2d2b8218de2837da905f59af02460f4f46d8466c6e4399619575c3cc5"], blob_size: Some(15), bootstrap_path: Some("target/tmp/f91d8e99167a83761ab6ef8e67f536a9586f4dc62e2485ebbd2f954e4a0fbe52") }

real 0m0.081s
user 0m0.006s
sys 0m0.010s

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26728

@anolis-bot
Copy link
Collaborator

@jiangliu , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26729

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26821

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch❌ FAIL

Sorry, your test job failed. Please get the details in the link.

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26899

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26916

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/26920

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/27015

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

Abstract common code in builder as helper functions to reduce duplicated
code.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Simplify code in blob-compatct.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Implment format for ConversionType.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Avoid copy child node when apply node to tree.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Only allow single SOUECE file for the 'nydus-image create' subcommand,
there's no support of multiple SOURCE files at all.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Hardlink must be regular files, otherwise directory will be reported
as hardlink.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Enable support of --type tar-rafs to convert tar file into rafs fs.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
Enable converting tar.gz(OCIv1) file into RAFS filesystem directly.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/27191

Enable converting estargz file to RAFS filesystem. The main different
between targz and estargz is that several estargz specific files should
be filtered out.

Signed-off-by: Jiang Liu <gerry@linux.alibaba.com>
@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/27195

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
merge-target-branch✅ SUCCESS
build-docker-image✅ SUCCESS
compile-nydus✅ SUCCESS
compile-ctr-remote✅ SUCCESS
compile-nydus-snapshotter✅ SUCCESS
start-nydus-snapshotter-config-containerd✅ SUCCESS
run-container-with-nydus-image✅ SUCCESS

Congratulations, your test job passed!

Copy link
Collaborator

@imeoer imeoer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

if let Some(blob_stor) = ctx.blob_storage.clone() {
writer = Some(ArtifactWriter::new(blob_stor, ctx.inline_bootstrap)?);
} else {
return Err(anyhow!("missing configuration for target path"));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just use bail!("missing configuration for target path") (not a blocker for this PR).

@imeoer imeoer merged commit b841e21 into dragonflyoss:master Oct 22, 2022
@jiangliu jiangliu deleted the tar-rafs branch March 28, 2023 01:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants