Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tracking Issue] Implement Integraion Tests #361

Open
16 of 60 tasks
YJDoc2 opened this issue Oct 4, 2021 · 80 comments
Open
16 of 60 tasks

[Tracking Issue] Implement Integraion Tests #361

YJDoc2 opened this issue Oct 4, 2021 · 80 comments
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@YJDoc2
Copy link
Collaborator

YJDoc2 commented Oct 4, 2021

As per #56 , we are porting the OCI validation tests into rust. This is the tracking issue for the same.
The folder /youki_integration_test contains the currently implemented tests which are lifecyle, creation and huge_tlb

Here is the documentation for developing the integration tests:
https://youki-dev.github.io/youki/developer/e2e/rust_oci_test.html

Please only work at one test at a time, and wait for its PR to get merged before opening another one. It helps us to have an easier way for reviewing the PRs 🙏

Default

List of all test, as per https://github.com/opencontainers/runtime-tools/tree/master/validation :

Extensions

This contains tests are not in https://github.com/opencontainers/runtime-tools/tree/master/validation , or need are not satisfactory enough.

  • cgroups v1
  • cgroups v2

Out of these, currently only huge_tlb is ported completely. lifecycle and create tests are implemented but need much improvements, and lifecycle tests should be split into individuals as per listed above.
Please read the README of youki_integration_test to get idea for how one should go about implementing the tests, and what utils etc are available.

@YJDoc2
Copy link
Collaborator Author

YJDoc2 commented Oct 4, 2021

Waiting for #360 , which moves test_framework into youki_integration_test, but other than that , ready for individual test claiming.

@YJDoc2 YJDoc2 added enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed labels Oct 4, 2021
@utam0k
Copy link
Member

utam0k commented Oct 5, 2021

@YJDoc2 That's great! It would be a good idea to add a link to the sample implementation you implemented to the description of this issue.

@utam0k utam0k pinned this issue Oct 5, 2021
@tsturzl
Copy link
Collaborator

tsturzl commented Oct 6, 2021

I'll have some free time soon and I'd love to pitch in on this effort! I think this tooling will be a force multiplier and I think it's really impactful. Are there any pain points that I could tackle that anyone can think of that might not be a good first issue? Otherwise I'm more than happy to just jump in on cgroups controllers where I'm pretty familiar.

@YJDoc2
Copy link
Collaborator Author

YJDoc2 commented Oct 8, 2021

Hey @tsturzl , sorry for late reply. Currently there are no obvious pain points that I know, and as you are already familiar with cgroups, it'd be great if you can work on those tests. Also if you take any test from the first list (which oci-tools have) please comment here, so we can keep track.

@utam0k I am a bit busy for few days, but after that I'm planning to work on either pid test or ns_itype test as currently both fail for runc and youki using oci-tests. Assuming that the issue is in test implementation, and not in the runtime, if we could support those tests, we can theoretically be feature equivalent with oci-tests.

@WormOn
Copy link

WormOn commented Oct 8, 2021

@utam0k i want to implement "config_updates_without_affect"

@utam0k
Copy link
Member

utam0k commented Oct 8, 2021

@utam0k i want to implement "config_updates_without_affect"

@WormOn
Sure. I assigned you.

@tsturzl
Copy link
Collaborator

tsturzl commented Oct 8, 2021

I'll start off with linux_cgroups_cpus, I think it will be an easy one to start off with.

@Furisto
Copy link
Collaborator

Furisto commented Oct 15, 2021

I do not want to take too many good first issues away, but I would like to implement at least one test to get familiar with the test framework. So I would take linux_cgroups_pids if that is alright.

@YJDoc2
Copy link
Collaborator Author

YJDoc2 commented Oct 16, 2021

Hey @Furisto
Sure, that'd be great!

@WormOn
Copy link

WormOn commented Oct 18, 2021

@utam0k i want to implement "config_updates_without_affect"

@WormOn Sure. I assigned you.

thx

@tsturzl
Copy link
Collaborator

tsturzl commented Nov 3, 2021

I'm nearly done with the linux_cgroups_cpus integrations tests. I'd like to take on the memory cgroups tasks. I'd really like to fix a problem that currently exists in the current oci validations with these specific tests.

@tsturzl
Copy link
Collaborator

tsturzl commented Nov 3, 2021

Also, maybe we should limit the context of this tracking issue to cgroups v1 and just generally getting this on par with oci runtime-tools? Perhaps then we can open another tracking issue to detail out what cgroups v2 tests should look like and what needs to get implemented.

@YJDoc2
Copy link
Collaborator Author

YJDoc2 commented Nov 3, 2021

Hey @tsturzl , sure you can take memory next 😄 , I'll assign it to you then.
It is a good idea that we use this only for tracking migration, and open separate one for cgroup v2, but I feel it'd be good to mention cgroups v2 here, until we have opened that issue, then we can link that here.

@YJDoc2
Copy link
Collaborator Author

YJDoc2 commented Nov 3, 2021

Hey @WormOn, how is the implementation going? Let know if you need any help in working on the test implementation 👍

@tsturzl
Copy link
Collaborator

tsturzl commented Dec 1, 2021

I got bored and started working on linux_cgroups_network, so I'll take that one also

@whereistejas
Copy link

Hi! I would like to contribute to this issue by adding a few tests, too. I have been working with rust for about 6 months, now. However, I have no exp with container runtimes, thus, would like to start with small and low priority tests.

I though I could start by adding the root_readonly_true test? Any thoughts?

@YJDoc2
Copy link
Collaborator Author

YJDoc2 commented Dec 31, 2021

Hey @whereistejas , thanks for your interest in contribution. I would suggest that rather than starting with root_readonly_test , you might start with craete. The reason behind this is that root_readonly uses a function test_inside_runtime a util function, which I am currently implementing for another test, but facing some issues. I hope to get it worked out in some days.

The create test uses the basic functions which are already implemented in the current utils, you can check the lifecycle tests to get an idea how to implement the create tests. If you need any help or are facing some issues, you can ask them here or on the youki discord.

Thanks :)

@whereistejas
Copy link

whereistejas commented Dec 31, 2021

Hi @YJDoc2, if my understanding is correct you are referring to the following create tests. I can see these tests are already implemented as part of lifecycle/container_create.rs. Please, correct me if I'm wrong.

@YJDoc2
Copy link
Collaborator Author

YJDoc2 commented Dec 31, 2021

Hey, yes that are the tests, but they were created as part of proof-of-concept of integration tests themselves, and should be replaced in sometime for their own dedicated tests as per the runtime tools.

Or if you are fine with it, you can wait till merging of #582 which implemented the test_inside_container , then you can work on root_readonly as well.

@whereistejas
Copy link

and should be replaced in sometime for their own dedicated tests as per the runtime tools.

So, we would like to have the create tests extracted/refactored out of lifecycle into their own module?

@YJDoc2
Copy link
Collaborator Author

YJDoc2 commented Jan 3, 2022

Hey @whereistejas , yes I meant that create tests should be refactored in there own module, as you are working on in #583 👍
Thank you for contributing!

@YJDoc2
Copy link
Collaborator Author

YJDoc2 commented Aug 23, 2024

Hey, so as far as I know, recent Ubuntu versions have dropped support of cgroups v1 by default, you need to do certain kernel feature enabling to enable cgroups v1 on them. I think ubuntu 20.04 has cgroups v1 enabled in them, and we use that in our ci -check #2645 . An easier way for your testing can be to create a vagrant box with this version, mount the youki dir in it, compile your things on host systems, so you don't have to install rust and build deps in the box, but run the tests in the box instead of host (so you'll need the runtime deps).

I have also opened #2890 for standardizing the test envs via vagrant boxes.

@nayuta-ai
Copy link

Hi! This is my first time with youki. I am interested in contributing to the youki project!
I would like to know if it would be possible to assign me to the linux_masked_paths.

@utam0k
Copy link
Member

utam0k commented Oct 8, 2024

Hi! This is my first time with youki. I am interested in contributing to the youki project! I would like to know if it would be possible to assign me to the linux_masked_paths.

@nayuta-ai Sure! I've assigned you. Thanks!

@sou1118
Copy link

sou1118 commented Oct 8, 2024

Hi @utam0k and @YJDoc2
I want to work on delete job. Could you assign me? This is my first OSS and Youki contribution.

@YJDoc2
Copy link
Collaborator Author

YJDoc2 commented Oct 8, 2024

Hey @sou1118 , I have assigned you the delete test, just check once if it has already been implemented or not (it is possible that we might have missed crossing it off the list). Also welcome to Youki, and OSS as whole! Hope you'll have fun and find the contributing interesting. Ping me if you need any help!

@moz-sec
Copy link

moz-sec commented Oct 31, 2024

Hi @YJDoc2 and @utam0k
I would like to be assigned to linux_cgroups_relative_network. Could you please assign it to me?
I'm looking forward to making my first OSS contribution!

@YamasouA
Copy link

@YJDoc2 @utam0k
Hi.
I'm intereseted in this Issue.
Can you assign any of the test cases to me?

@YJDoc2
Copy link
Collaborator Author

YJDoc2 commented Nov 1, 2024

@moz-sec , assigned the linux_cgroups_relative_network to you. Looking forward to you PR!

@YamasouA , I have assigned the kill test to you, please take a look. If you want some other let me know and I can re-assign.

In case either of you need any help while working on this, please feel free to tag/ping me here :)

@saku3
Copy link
Contributor

saku3 commented Nov 8, 2024

@YJDoc2 @utam0k
Hi!
I would like to be assigned to process_oom_score_adj. Could you please assign it to me?

@YJDoc2
Copy link
Collaborator Author

YJDoc2 commented Nov 8, 2024

@saku3 done! Feel free to ping me if you have any issues while working on this 👍

@sat0ken
Copy link
Contributor

sat0ken commented Nov 15, 2024

@YJDoc2
Hi ! I wanna try to linux_cgroups_devices and linux_cgroups_relative_devices. Could you please assign it to me?

@saku3
Copy link
Contributor

saku3 commented Nov 24, 2024

@YJDoc2
Hi ! I wanna try to linux_rootfs_propagation. Could you please assign it to me?

@moz-sec
Copy link

moz-sec commented Nov 25, 2024

Hi @YJDoc2
I would like to be assigned to linux_uid_mappings
Could you please assign it to me?

@kazmsk
Copy link

kazmsk commented Nov 25, 2024

Hi @YJDoc2
I would like to be assigned to process_capabilities_fail.
Could you please assign it to me?

@YJDoc2
Copy link
Collaborator Author

YJDoc2 commented Nov 25, 2024

@saku3 assigned linux_rootfs_propagation
@moz-sec , please wait till your existing PR is merged 😅 I think there are still some changes I need to review. However, I'll assign this to you once this is merged.
@kazmsk , assigned process_capabilities_fail.

Also, anyone who has an open PR of e2e testing, please rebase your PR, as we have just merged a big refactor of one of the functions in #2973 (review) . This is mostly applicable if your code is using test_inside_container and the changes would be minimal, but please rebase on main to be confirm. Thanks.

@ntkm61027
Copy link

Hi @YJDoc2
I'm new here and would like to try process_rlimits_fail. Would you mind assigning it to me?

@utam0k
Copy link
Member

utam0k commented Dec 3, 2024

@ntkm61027 I've assigned you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests