Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test(cactus): remove flaky tests from main CI execution #2112

Closed
outSH opened this issue Jul 4, 2022 · 0 comments
Closed

test(cactus): remove flaky tests from main CI execution #2112

outSH opened this issue Jul 4, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@outSH
Copy link
Contributor

outSH commented Jul 4, 2022

Is your feature request related to a problem? Please describe.
The process from PR to merge takes long because of flaky tests that forces us to re-run the CI from time to time. I think that until flaky tests are fixed, they should be removed from the main CI, or better - their failures should not block the merge (maintainer can manually check the logs of relevant failing flaky tests for unexpected messages).

Describe the solution you'd like
Separate test suites for stable and flaky tests, make flaky tests optional in the CI.

I will send a PR with proposal solution very soon.
@outSH outSH added the enhancement New feature or request label Jul 4, 2022
@outSH outSH changed the title test(cactus): remove flaky tests from main CI execution. test(cactus): remove flaky tests from main CI execution Jul 4, 2022
outSH added a commit to outSH/cactus that referenced this issue Jul 4, 2022
@outSH
test(cactus): remove flaky tests from main CI execution
c34e857 
- Remove flaky tests from main CI execution.
- Add separate TAP and jest configuration for running only flaky tests.
- Add test:*:flaky scripts in the main package.json
- Add new step to github CI script to run flaky tests as optional
  (i.e. don't fail if flaky test fails)

Closes: hyperledger-cacti#2112

Related: hyperledger-cacti#2104
Related: hyperledger-cacti#1626
Related: hyperledger-cacti#2019
Related: hyperledger-cacti#1625
Related: hyperledger-cacti#1543
Related: hyperledger-cacti#1598
Related: hyperledger-cacti#1528
Related: hyperledger-cacti#1521
Related: hyperledger-cacti#1495
Related: hyperledger-cacti#1485
Related: hyperledger-cacti#1473
Related: hyperledger-cacti#1471
Related: hyperledger-cacti#1469
Related: hyperledger-cacti#1150

Signed-off-by: Michal Bajer <michal.bajer@fujitsu.com>
@outSH outSH mentioned this issue Jul 4, 2022
Closed
@outSH outSH closed this as completed Jul 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

test(cactus): remove flaky tests from main CI execution outSH/cactus
1 participant
@outSH