Skip to content
This repository has been archived by the owner on Jul 11, 2023. It is now read-only.

tests: Create a test matrix per k8s distribution per test suite - report test passes/failures per dimension #3559

Closed
Tracked by #1489
ritazh opened this issue Jun 9, 2021 · 2 comments
Labels

Comments

@ritazh
Copy link
Contributor

ritazh commented Jun 9, 2021

Please describe the Improvement and/or Feature Request

As a developer, I would like to see a test matrix providing signal of different jobs testing different scenarios and their passing rate across different releases, cluster types, and cluster versions.


For example, the test matrix could look like:

pass rate job name job frequency release cluster type cluster version
95% install osm e2e latest supported releases periodic kind 1.20
95% install osm e2e head from main periodic kind 1.20
90% install osm e2e latest supported releases periodic openshift 4.4
85% upgrade osm control plane latest supported releases periodic kind 1.19
85% osm permissive mode head from main periodic kind 1.19
85% osm smi mode head from main periodic kind 1.19
85% osm + cert manager latest supported releases periodic kind 1.19
85% osm + HashiCorp Vault latest supported releases periodic kind 1.19

An example of how Kubernetes does this:
https://testgrid.k8s.io/sig-network-dualstack-azure-e2e#Summary

@shashankram shashankram added the area/tests Related to tests label Jun 11, 2021
@draychev draychev changed the title Test matrix tests: Create a test matrix per k8s distribution per test suite - report test passes/failures per dimension Jul 6, 2021
@github-actions
Copy link

This issue will be closed due to a long period of inactivity. If you would like this issue to remain open then please comment or update.

@github-actions github-actions bot added the stale label Feb 15, 2022
@github-actions
Copy link

Issue closed due to inactivity.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

3 participants