Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add the benchmark test for nydus #1219

Merged
merged 2 commits into from
Apr 20, 2023
Merged

feat: add the benchmark test for nydus #1219

merged 2 commits into from
Apr 20, 2023

Conversation

Desiki-high
Copy link
Member

@Desiki-high Desiki-high commented Apr 15, 2023 •
edited
Loading

To compare the container startup speed of nydus and oci, add the new feature.

misc/benchmark/*.py is the benchmark workflow python script:

  1. bench.py: bench the pull image time, create container time and start container time
  2. benchmark.py : the benchmark satrt with four mode(oci, nydus without prefetch , nydus with prefetch all, nydus with prefetch file list)
  3. convert.py: convet the image from oci to nydus by nydusify
  4. metrics.py: use the nydus metrics api to collet the files which accessd by container in start time
  5. prefetch_list.py: sort the file list by access time, maybe the more smart strategy will add in the future
  6. util.py: the utility functions used in benchmark

the other files in misc/benchmark is the config files or environment files:

  1. cni_bridge.py: used by cni plugins
  2. nydusd_config.json: nydusd config file
  3. containerd_config.toml: snapshotter config file with containerd
  4. nydus-snapshotter.service: systemd service to creat nydusd process

add four jobs in smoke.yml :

we use wondershape to limit to local net bandwith(10M in config.yml)

  1. env.IMAGE: choose the image for benchmark(if you modify this please also modiy the config.yaml)
  2. benchmark-oci: benchmark for oci
  3. benchmark-nydus-no-prefetch: benchmark for nydsu image (without prefetch)
  4. benchmark-nydus-all-prefetch: benchmark for nydsu image (prefetch /)
  5. benchmark-nydus-filelist-prefetch: benchmark for nydsu image (prefetch the file list create by prefetch_list.py )

upload the benchmark result in the artifacts

@Desiki-high Desiki-high requested review from a team as code owners April 15, 2023 08:45
@Desiki-high Desiki-high requested review from bergwolf, liubogithub and hsiangkao and removed request for a team April 15, 2023 08:45
@anolis-bot
Copy link
Collaborator

@Desiki-high , a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/67741

@anolis-bot
Copy link
Collaborator

@Desiki-high , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/67755

@anolis-bot
Copy link
Collaborator

@Desiki-high , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS
run container with rafs and compile linux✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@Desiki-high , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS
run container with rafs and compile linux✅ SUCCESS

Congratulations, your test job passed!

@jiangliu
Copy link
Collaborator

jiangliu commented Apr 17, 2023 •
edited
Loading

In addition to run it in smoke tests, tt would be great to also run the performance test as a per-week cron task too with:

  1. more container images
  2. different network bandwidth configurations.

@Desiki-high
Copy link
Member Author

In addition to run it in smoke tests, tt would be great to also run the performance test as a per-week cron task too with:

  1. more container images
  2. different network bandwidth configurations.

In smoke tests, only one container image was tested due to time constraints. If you want to support more container images and different network bandwidth configurations, this script cant't support this idea currently.

@anolis-bot
Copy link
Collaborator

@Desiki-high , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/68108

@anolis-bot
Copy link
Collaborator

@Desiki-high , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/68110

@anolis-bot
Copy link
Collaborator

@Desiki-high , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/68111

@anolis-bot
Copy link
Collaborator

@Desiki-high , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/68112

@Desiki-high Desiki-high requested a review from jiangliu April 17, 2023 02:17
@anolis-bot
Copy link
Collaborator

@Desiki-high , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS
run container with rafs and compile linux✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@Desiki-high , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/68682

@Desiki-high Desiki-high mentioned this pull request Apr 19, 2023
Closed
@anolis-bot
Copy link
Collaborator

@Desiki-high , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS
run container with rafs and compile linux✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@Desiki-high , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/68904

@anolis-bot
Copy link
Collaborator

@Desiki-high , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS
run container with rafs and compile linux✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@Desiki-high , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/68905

@anolis-bot
Copy link
Collaborator

@Desiki-high , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS
run container with rafs and compile linux✅ SUCCESS

Congratulations, your test job passed!

@Desiki-high
add the benchmark test for nydus image
8adb227 
1. add the benchmark scripts in misc/benchmark
2. add five benchmark jobs in smoke test and the benchmark-result job for show the benchmark result in the PR comment

Signed-off-by: Desiki-high <ding_yadong@foxmail.com>
@Desiki-high
change the comment content
c08e3b4 
Signed-off-by: Desiki-high <ding_yadong@foxmail.com>
@anolis-bot
Copy link
Collaborator

@Desiki-high , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/68911

imeoer
imeoer approved these changes Apr 20, 2023
View reviewed changes
Copy link
Collaborator

@imeoer imeoer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@imeoer imeoer merged commit 36d8a5b into dragonflyoss:master Apr 20, 2023
@anolis-bot
Copy link
Collaborator

@Desiki-high , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS
run container with rafs and compile linux✅ SUCCESS

Congratulations, your test job passed!

@Desiki-high Desiki-high deleted the benchmark branch April 21, 2023 05:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sctb512 sctb512 sctb512 left review comments

@imeoer imeoer imeoer approved these changes

@bergwolf bergwolf Awaiting requested review from bergwolf bergwolf is a code owner automatically assigned from dragonflyoss/nydus-maintainers

@liubogithub liubogithub Awaiting requested review from liubogithub liubogithub is a code owner automatically assigned from dragonflyoss/nydus-maintainers

@hsiangkao hsiangkao Awaiting requested review from hsiangkao hsiangkao is a code owner automatically assigned from dragonflyoss/nydus-maintainers

@jiangliu jiangliu Awaiting requested review from jiangliu

Assignees
No one assigned
Labels
anolis_test_pass
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@Desiki-high @anolis-bot @jiangliu @imeoer @adamqqqplay @sctb512