Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue where VRT test fails in CI when launching Chromium #9211

Closed
1 task
techanvil opened this issue Aug 20, 2024 · 1 comment
Closed
1 task

Fix issue where VRT test fails in CI when launching Chromium #9211

techanvil opened this issue Aug 20, 2024 · 1 comment
Labels
P0 High priority Type: Bug Something isn't working Type: Infrastructure Engineering infrastructure & tooling

Comments

@techanvil
Copy link
Collaborator

techanvil commented Aug 20, 2024

Bug Description

Unfortunately the experimental fix for our VRT workflow flakiness applied via #7965 doesn't seem to have improved its reliability and the workflow is still failing at a high rate during peak US/EU working hours.

VRT runs in CI are still intermittently failing before all the screenshots are captured, with the following error:

      COMMAND | Command "test" ended with an error after [542.215s]
      COMMAND | Error: Failed to launch the browser process!

Steps to reproduce

Screenshots

This error has been seen with a couple of permutations:

image

image

image

Additional Context

See this Slack thread for related discussion on the topic, including some suggested avenues for investigation:


Do not alter or remove anything below. The following sections will be managed by moderators only.

Acceptance criteria

  • VRT test runs should not fail with an error before all the screenshots are captured and the test has run to completion.

Note, additional issues should be raised if needed to follow up on separate avenues of investigation.

Implementation Brief

After substantial investigation, it seems this may be due to resource limitations when running the Docker container. See garris/BackstopJS#537 (comment)

Test Coverage

  • No changes; this addresses the ability of VRT to run until completion rather than the result

QA Brief

  • None needed; this can be reviewed during CR and iterated on if other issues persist. No plugin code is changed.

Changelog entry

  • Enhance stability of visual regression test runs.
@techanvil techanvil added Type: Bug Something isn't working P0 High priority Type: Infrastructure Engineering infrastructure & tooling labels Aug 20, 2024
@aaemnnosttv aaemnnosttv mentioned this issue Aug 21, 2024
18 tasks
@tofumatt tofumatt self-assigned this Aug 21, 2024
@tofumatt
Copy link
Collaborator

Moving directly to Approval given no QA is required here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P0 High priority Type: Bug Something isn't working Type: Infrastructure Engineering infrastructure & tooling
Projects
None yet
Development

No branches or pull requests

3 participants