Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modularize test workflows #490

Open
ohltyler opened this issue Apr 5, 2022 · 2 comments
Open

Modularize test workflows #490

ohltyler opened this issue Apr 5, 2022 · 2 comments
Assignees
Labels
infra Changes to infrastructure, testing, CI/CD, pipelines, etc. Priority-Medium

Comments

@ohltyler
Copy link
Member

ohltyler commented Apr 5, 2022

Currently, the CI workflow contains 4 different tests:

  1. unit tests
  2. integ tests agains multi-node cluster (no security)
  3. integ tests against docker image (with security if exists, else no security)
  4. BWC tests

This poses many problems:

  • extremely long workflow times close to 1 hr
  • longer time to narrow down failures due to all of the different scenarios captured in this one workflow
  • possible duplicate/unnecessary integ test runs if no security (2 and 3 listed above)

I see a few solns:

  1. Split these into different workflows
  2. Provide reusable workflow(s) that can be called by other workflows to run tests. For example, a reusable workflow that pulls the github repos and builds them, can then be called by different workflows to kick off unit tests or integ tests, to reduce duplicate setup code. As an example, see this Dashboards functional-test PR.
@ohltyler ohltyler added untriaged infra Changes to infrastructure, testing, CI/CD, pipelines, etc. labels Apr 5, 2022
@ohltyler ohltyler self-assigned this Apr 5, 2022
@ohltyler
Copy link
Member Author

ohltyler commented Apr 5, 2022

Will wait to look into this once 2.0 builds are stable and the existing workflows are passing on main.

@amitgalitz
Copy link
Member

I made some progress on splitting up the workflows into bwc, tests against docker image (w/security) and a build workflow (unit, integ and multi node). The last build workflow could be split up more I guess but also means a lot more runners/bigger test matrix. I am not closing this issue yet, but if we are okay with not splitting the unit and integ tests than I can close it with this PR (#710)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infra Changes to infrastructure, testing, CI/CD, pipelines, etc. Priority-Medium
Projects
None yet
Development

No branches or pull requests

3 participants