Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(cypress): update to cypress 12 #484

Closed
wants to merge 15 commits into from
Closed

Conversation

ismay
Copy link
Contributor

@ismay ismay commented Mar 13, 2023

CI for this app is rather flaky. It was blocking the work I'm doing atm on the job queues. This PR fixes a couple things that during my tests have fixed the flakiness.

  • Update cypress to v12. For this I've resorted to testing directly against a dockerized backend. It adds about 1m50s overhead to the e2e test run, but removes a lot of complexity. Cypress 12 also allows us to use the new cy.session and a simple login helper. We can reuse the session without running too much risk of tests contaminating eachother. Version 12 of cypress also fixes an issue with tests failing due to rerenders during an assertion.
  • The cypress-cucumber-preprocessor we were using is no longer maintained. As part of the migration to cypress 12 I've updated to the fork that is the successor: @badeball/cypress-cucumber-preprocessor. The configuration it requires is a little different, but nothing noteworthy.

Env vars needed are:

  • CYPRESS_LOGIN_NAME
  • CYPRESS_LOGIN_PASSWORD
  • CYPRESS_LOGIN_SERVER

See cypress.env.json.example. CI is already configured with the necessary env vars.

This approach is a little novel, but it works well and removes a lot of maintenance overhead for our cypress setup. Plus we're testing against an actual backend so there's also some added realism. See https://dhis2.slack.com/archives/C1XTS67ME/p1678701948371199 for more context.

Running time

Mocked backend:

  • 2m53s with a mocked backend

Pre-seeded image (see here: #489):

  • 5m25s without any caching (about 2m30s more than the mocked backend)

Pre-seeded image with 4 extra runners:

  • 4m40s without any caching (about 1m50s more than the mocked backend)

Running time other methods

The test code for these runs isn't completely identical to the tests above, login for this suite is slightly slower (about 30s) due to logging in via the ui, so not a completely fair comparison. But these methods were slower overall anyway, so not that important.

Pre-seeded image, layer caching:

  • 8m13s with layer caching, empty cache
  • 6m45s with layer caching, primed cache

Dockerized backend:

  • 7m5s against a dockerized backend, without a pre-seeded db image

@ismay ismay force-pushed the update-cypress-master branch 5 times, most recently from 1618f1b to 40724b8 Compare March 22, 2023 10:55
@ismay ismay force-pushed the update-cypress-master branch 2 times, most recently from d83b0bb to fd8b877 Compare March 22, 2023 12:48
@ismay ismay mentioned this pull request Mar 28, 2023
24 tasks
@dhis2-bot
Copy link
Contributor

🎉 This issue has been resolved in version 101.6.4 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Re-query elements that are found 'detached' from the DOM
3 participants