DR | Fix disabled unit & e2e tests

[Mottie]

Value Statement

As a developer
I want to know that our unit & end-to-end tests are being run in CI
So that I can be sure that test coverage is maintained

Background Context

Platform has set up CI to run stress tests on e2e tests to determine if a test is flaky. Once flagged as flaky, the test is added to a "disallowed" list and the e2e test is then skipped in CI. A list of disallowed tests is maintained in the developer documentation, and periodically announced in the #vfs-all-teams Slack channel.

Starting 11/6/2023, PRs with disallowed unit & e2e tests will be blocked from being merged.

Acceptance Criteria

• Fix unit tests shown on the disallowed unit test page
• Fix e2e tests shown on the disallowed e2e test page
• All tests passing

Tasks

• Review disallowed unit tests page
• Fix unit tests or manually skip flaky unit tests
• Review disallowed e2e tests page
• Fix e2e tests

Definition of Ready

• Clear value description
• Testable acceptance criteria
• Accessibility added to acceptance criteria
• Approved designs attached
• Sample data provided where appropriate
• Estimated to fit within the sprint
• Dependencies and blockers linked

Definition of Done

• Meets acceptance criteria
• Passed E2E testing (90% coverage)
• Passed unit testing (90% coverage)
• Passed integration testing (if applicable)
• Code reviewed (internal)
• Submitted to staging
• Reviewed and approved by product and/or design

[Mottie]

• Work to get the 3 e2e tests that are on the disallowed list have been completed, but the disallowed list isn't updating. I've reached out to platform and they have pushed out some fixes. We need to wait until Oct 31 to see if the tests have been removed.

  Lets hold until the overnight scan runs Monday and let it do its updating as expected. If those tests are fixed, they should get reenabled by the process automatically

• Work to fix the flaky unit test is awaiting code review.
• Work to add Supplemental Claim keyboard-only e2e tests are in progress

[Mottie]

• The flaky unit test has been removed from the disallowed list 🎉
• The Supplemental Claim e2e test appear to continue to fail the stress tests 😿
• For the keyboard only tests, we're still waiting on this test to finish it's overnight run (going on 13+ hours now) before the disallowed list is updated

[saderagsdale]

@anniebtran taking a look at the failing tests

[saderagsdale]

@anniebtran @Mottie any update on the failing tests?

[anniebtran]

@saderagsdale the PR above has been approved and just needs to be merged, i think. not sure if it'll re-enable the flaky test but seems to be passing cleanly on the branch

[Mottie]

The PR has just been merged. I think we need to wait until the daily run (I think at 5 PM daily) before the test is removed from the disallowed list

[Mottie]

Today is the deadline, and I'm not seeing any of our apps on the disallowed e2e and unit test lists 🎉

[saderagsdale]

Woot!!! 🎉🤸😝