You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We recently had a PR fail the build check step due to a drop in test coverage, which you can see here.
In this case, the failure happened in PR that was a change to a GitHub Action YAML configuration file, which is not a part of the application code itself and therefore not something I'd expect to be the cause of seeing the coverage percentage drop below our threshold. I've noticed us running into this off and on in the last few months, and it's only going to continue to hamstring us until we take care of it.
This is the first step: let's get a clear picture of where we're at and where the biggest gains lie in making a real improvement here. We should figure out if we need to adjust our coverage configuration at all (without lowering the threshold further!), and which tests we need to add/modify/eliminate that will yield the most gains for us. We don't need to cover every single gap; let's find where the biggest ones are that would help us the most.
Implementation Sketch and Acceptance Criteria
Review the code coverage output after running tests in the API (you can look at the build check failure here or run the tests locally to see) and identify spots where code coverage is not at or near 100%
Review the configuration of the coverage report itself and see if it's including things that it probably shouldn't (e.g., test files!)
Write up findings and recommendations in this issue so that we can we can create some follow-on stories for any work that needs to happen as a follow-up
Coverage configuration changes
Writing new tests for areas that lack full coverage (I see several files that are closer to 80% - 90% coverage instead of at or near 100%; some much lower!)
Eliminating tests that aren't providing us any real value
Security Considerations
No sensitive information should be present in any of the tests or their configuration
Any fake/dummy data should not include real information
ccostino
changed the title
Investigate drop in API test coverage, which results in build check failures in PRs
Investigate drop in API test coverage, which results in build check failures in PRs, and come up with next steps
Sep 27, 2024
We recently had a PR fail the build check step due to a drop in test coverage, which you can see here.
In this case, the failure happened in PR that was a change to a GitHub Action YAML configuration file, which is not a part of the application code itself and therefore not something I'd expect to be the cause of seeing the coverage percentage drop below our threshold. I've noticed us running into this off and on in the last few months, and it's only going to continue to hamstring us until we take care of it.
This is the first step: let's get a clear picture of where we're at and where the biggest gains lie in making a real improvement here. We should figure out if we need to adjust our coverage configuration at all (without lowering the threshold further!), and which tests we need to add/modify/eliminate that will yield the most gains for us. We don't need to cover every single gap; let's find where the biggest ones are that would help us the most.
Implementation Sketch and Acceptance Criteria
Security Considerations
The text was updated successfully, but these errors were encountered: