Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: forceExit and bail tests #1266

Merged

Conversation

TimoGlastra
Copy link
Contributor

@TimoGlastra TimoGlastra commented Feb 5, 2023

Signed-off-by: Timo Glastra timo@animo.id

This PR adds --forceExit --bail to the test script when running in CI. It won't necesarily fix the issues we have with flaky tests, but it should fix the tests keeping running even though a timeout has occurred. The --bail means it will also stop tests after one test has failed instead of continuing running.

Ideally we don't use --forceExit as it's discouraged by Jest and you should gracefully exit. However, I've been doing tests with minimal setups and I think the issue of not exiting is because of the Indy SDK doing tasks while the test times out, and that prevents from a graceful exit. I'm not sure this is the case, but this is my current feeling.

I've tried migrating to vitest over jest as it has way better logging and feels quicker / simpler (that's how I discovered the test issues have something to do with the indy-sdk). However, vite doesn't support decorators at the moment (well, esbuild doesn't) so we can't switch at the moment.

I hope these changes will at least give us more insights into which tests are causing the flakyness

@TimoGlastra TimoGlastra requested a review from a team as a code owner February 5, 2023 13:42
@TimoGlastra TimoGlastra marked this pull request as draft February 5, 2023 13:44
@TimoGlastra TimoGlastra marked this pull request as ready for review February 5, 2023 13:46
@codecov-commenter
Copy link

codecov-commenter commented Feb 5, 2023

Codecov Report

Merging #1266 (c0a2e25) into main (acdb20a) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1266   +/-   ##
=======================================
  Coverage   87.35%   87.35%           
=======================================
  Files         760      760           
  Lines       18149    18149           
  Branches     3081     3081           
=======================================
  Hits        15854    15854           
  Misses       2288     2288           
  Partials        7        7           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Signed-off-by: Timo Glastra <timo@animo.id>
@TimoGlastra TimoGlastra changed the title test ci: forceExit and bail tests Feb 5, 2023
Copy link
Contributor

@genaris genaris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well even if not ideal is a step forward. Let's see if we can avoid the need of --forceExit once we refactor core and test suites in a way that Indy SDK does not affect so much.

@TimoGlastra TimoGlastra merged commit f1e4937 into openwallet-foundation:main Feb 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants