Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nydusify: speed up conversion with tar2rafs #1073

Merged
merged 1 commit into from
Feb 15, 2023

Conversation

imeoer
Copy link
Collaborator

@imeoer imeoer commented Feb 11, 2023

Default to enable --type tar-rafs option for nydus-image, which
converting OCI tar blob stream into nydus blob directly, eliminating
the need to decompress it to a local directory first, thus greatly
accelerating the pack process.

For compatibility, we still support env NYDUS_DISABLE_TAR2RAFS=true
variable to disable the optimization.

It's an internal feature, so we just only bump acceleration-service
and nydus-snapshotter packages to the latest version.

Related PR: containerd/nydus-snapshotter#352

Signed-off-by: Yan Song imeoer@linux.alibaba.com

@imeoer imeoer requested a review from a team as a code owner February 11, 2023 13:24
@imeoer imeoer requested review from jiangliu, luodw and hsiangkao and removed request for a team February 11, 2023 13:24
@imeoer imeoer force-pushed the nydusify-speedup branch 5 times, most recently from addb4f1 to d72a557 Compare February 13, 2023 08:42
@imeoer imeoer changed the title [WIP] nydusify: speed up conversion with tar2rafs nydusify: speed up conversion with tar2rafs Feb 13, 2023
@anolis-bot
Copy link
Collaborator

@imeoer , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/50431

@anolis-bot
Copy link
Collaborator

@imeoer , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS

Congratulations, your test job passed!

@imeoer imeoer changed the title nydusify: speed up conversion with tar2rafs [WIP] nydusify: speed up conversion with tar2rafs Feb 13, 2023
@imeoer
Copy link
Collaborator Author

imeoer commented Feb 13, 2023

Maybe try --type tar-rafs and --features blob-toc first, then fallback if the builder is not supported.

Default to enable `--type tar-rafs` option for nydus-image, which
converting OCI tar blob stream into nydus blob directly, eliminating
the need to decompress it to a local directory first, thus greatly
accelerating the pack process.

For compatibility, we still support env `NYDUS_DISABLE_TAR2RAFS=true`
variable to disable the optimization.

It's an internal feature, so we just only bump acceleration-service
and nydus-snapshotter packages to the latest version.

Related PR: containerd/nydus-snapshotter#352

Signed-off-by: Yan Song <imeoer@linux.alibaba.com>
@imeoer imeoer changed the title [WIP] nydusify: speed up conversion with tar2rafs nydusify: speed up conversion with tar2rafs Feb 15, 2023
@anolis-bot
Copy link
Collaborator

@imeoer , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/50954

@anolis-bot
Copy link
Collaborator

@imeoer , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/50953

@anolis-bot
Copy link
Collaborator

@imeoer , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@imeoer , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS

Congratulations, your test job passed!

@jiangliu jiangliu merged commit f6681a4 into dragonflyoss:master Feb 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants