Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run e2e,integration tests against kubernetes versions and ssl stacks #924

Merged
merged 32 commits into from
Jun 19, 2022

Conversation

clux
Copy link
Member

@clux clux commented May 31, 2022

As part of Kubernetes version policies for #684.
Actual proposed policies are in kube-rs/website#19 and is mostly in line with our current advice.

This is larger overhaul for the CI situation for e2e tests to at least guarantee that the various Kubernetes versions work:

Screenshot from 2022-06-06 21-43-28

Main stuff done:

  • factored out integration tests from coverage job
  • running integration tests against [mk8sv, latest] (found one timing issue)
  • extend e2e incluster tests to run both ssl stacks
  • add e2e compile test to verify mk8sv and latest feature sets
  • extend integration tests to run against different cluster versions (but same feature set)
  • update automation to bump mk8sv and badge (just bump-k8s also bumps minimum now)
  • improve minimum verification from badges (both on MSRV and MK8SV)
  • general ci workflow improvement on naming and organisation (lint factored into its own, lowercase names)
  • update required statuses after review

k3s complication: k3s is lagging behind k8s; pre-releases available 1w later, but 1mo later no stable 1.24.. => pinning only minimum supported and running against latest => can only really test against minimum and max available.

Verification strategy:

  • run integration tests against v1_19 (but without changing cluster version)
  • compile e2e/boot.rs against both latest k8s-openapi feature and our msrv (currently v1_19 feature)

Thus the compile check avoids the error where we accidentally publish abstractions on apis that k8s-openapi have removed, and the integration tests run with a later k8s-openapi no matter what (because it's super hard to untangle that, and also not super necessary).

As part of Kubernetes version policies proposed in kube-rs/website#19
for #684

Signed-off-by: clux <sszynrae@gmail.com>
@clux clux mentioned this pull request May 31, 2022
33 tasks
@codecov-commenter

This comment was marked as off-topic.

clux added 4 commits May 31, 2022 20:41
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
@clux clux changed the title Run e2e tests against supported kubernetes versions Run e2e,integration tests against supported kubernetes versions May 31, 2022
@clux clux added the changelog-exclude changelog excluded prs label May 31, 2022
clux added 3 commits May 31, 2022 22:45
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
@clux clux marked this pull request as ready for review May 31, 2022 22:01
@clux clux linked an issue May 31, 2022 that may be closed by this pull request
clux added 10 commits June 2, 2022 17:13
Now two e2e tests, and more feature combinations.

Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
@clux clux added this to the 0.74.0 milestone Jun 5, 2022
clux added 2 commits June 5, 2022 15:08
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
clux added 4 commits June 5, 2022 15:17
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
@clux
Copy link
Member Author

clux commented Jun 5, 2022

Am fairly happy with the way this is now. It matches the policy in the upstream website PR, and we have better e2e tests to at least compile time guarantee we are not building on top of removed apis. Minor improvements here and there as outlined in edited main post. Feedback welcome.

EDIT: i guess the flaky test is still flaky. Will look at that later, but ultimately it should be unrelated to the rest of this PR.
EDIT2: Looks like the flaky test is due to discovery::oneshot::pinned_group in derived_resources_discoverable. It's possibly that this is a race condition within kubernetes apiserver. I.e. maybe waiting for established is not sufficient to try to discover the type. EDIT3: lol, yes looks like it.

clux added 5 commits June 5, 2022 21:20
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
Signed-off-by: clux <sszynrae@gmail.com>
@clux clux requested a review from nightkr June 8, 2022 21:46
kube-runtime/src/wait.rs Outdated Show resolved Hide resolved
@kazk
Copy link
Member

kazk commented Jun 13, 2022

Not sure why this is stuck.

By the way, I just enabled https://github.blog/changelog/2022-06-08-admins-can-require-sign-off-on-web-based-commits at org level, which will prevent the DCO issue when committing suggested changes.

@clux
Copy link
Member Author

clux commented Jun 14, 2022

Not sure why this is stuck.

It's not actually stuck. This PR has changed jobs + job names, so I need to change required statuses before merging (hence why the required ones are not reported).

By the way, I just enabled https://github.blog/changelog/2022-06-08-admins-can-require-sign-off-on-web-based-commits at org level, which will prevent the DCO issue when committing suggested changes.

Ah, that sounds great. Thanks!

clux added 2 commits June 14, 2022 09:03
(won't build until that is merged)

Signed-off-by: clux <sszynrae@gmail.com>
@clux clux merged commit f99140e into master Jun 19, 2022
@clux clux deleted the e2e-against-k8s branch June 19, 2022 06:21
@clux clux changed the title Run e2e,integration tests against supported kubernetes versions Run e2e,integration tests against supported kubernetes versions and more Jul 10, 2022
@clux clux changed the title Run e2e,integration tests against supported kubernetes versions and more Run e2e,integration tests against kubernetes versions and ssl stacks Jul 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog-exclude changelog excluded prs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

minimum supported k8s version
3 participants