Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(cypress): update to cypress 12 #484

Closed
wants to merge 15 commits into from
Closed

Conversation

ghost
Copy link

@ghost ghost commented Mar 13, 2023

CI for this app is rather flaky. It was blocking the work I'm doing atm on the job queues. This PR fixes a couple things that during my tests have fixed the flakiness.

  • Update cypress to v12. For this I've resorted to testing directly against a dockerized backend. It adds about 1m50s overhead to the e2e test run, but removes a lot of complexity. Cypress 12 also allows us to use the new cy.session and a simple login helper. We can reuse the session without running too much risk of tests contaminating eachother. Version 12 of cypress also fixes an issue with tests failing due to rerenders during an assertion.
  • The cypress-cucumber-preprocessor we were using is no longer maintained. As part of the migration to cypress 12 I've updated to the fork that is the successor: @badeball/cypress-cucumber-preprocessor. The configuration it requires is a little different, but nothing noteworthy.

Env vars needed are:

  • CYPRESS_LOGIN_NAME
  • CYPRESS_LOGIN_PASSWORD
  • CYPRESS_LOGIN_SERVER

See cypress.env.json.example. CI is already configured with the necessary env vars.

This approach is a little novel, but it works well and removes a lot of maintenance overhead for our cypress setup. Plus we're testing against an actual backend so there's also some added realism. See https://dhis2.slack.com/archives/C1XTS67ME/p1678701948371199 for more context.

Running time

Mocked backend:

  • 2m53s with a mocked backend

Pre-seeded image (see here: #489):

  • 5m25s without any caching (about 2m30s more than the mocked backend)

Pre-seeded image with 4 extra runners:

  • 4m40s without any caching (about 1m50s more than the mocked backend)

Running time other methods

The test code for these runs isn't completely identical to the tests above, login for this suite is slightly slower (about 30s) due to logging in via the ui, so not a completely fair comparison. But these methods were slower overall anyway, so not that important.

Pre-seeded image, layer caching:

  • 8m13s with layer caching, empty cache
  • 6m45s with layer caching, primed cache

Dockerized backend:

  • 7m5s against a dockerized backend, without a pre-seeded db image

@ghost ghost force-pushed the update-cypress-master branch from 0e59252 to edc5a3e Compare March 13, 2023 13:21
@ghost ghost force-pushed the update-cypress-master branch from edc5a3e to 4458be2 Compare March 13, 2023 13:37
@ghost ghost force-pushed the update-cypress-master branch 5 times, most recently from e829c62 to 5b71372 Compare March 21, 2023 14:52
@ghost ghost force-pushed the update-cypress-master branch 5 times, most recently from 1618f1b to 40724b8 Compare March 22, 2023 10:55
@ghost ghost force-pushed the update-cypress-master branch from 40724b8 to f3f9ffe Compare March 22, 2023 11:35
@ghost ghost force-pushed the update-cypress-master branch 2 times, most recently from d83b0bb to fd8b877 Compare March 22, 2023 12:48
@ghost ghost mentioned this pull request Mar 28, 2023
24 tasks
@ghost ghost closed this Oct 11, 2023
@ghost ghost mentioned this pull request Apr 3, 2024
@ghost ghost reopened this Apr 9, 2024
@ghost ghost mentioned this pull request Apr 9, 2024
@ghost ghost closed this in #609 Apr 15, 2024
@dhis2-bot
Copy link
Contributor

🎉 This issue has been resolved in version 101.6.4 🎉

The release is available on:

Your semantic-release bot 📦🚀

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Re-query elements that are found 'detached' from the DOM
2 participants