Skip to content

Commit

Permalink
chore: code coverage configuration (aws#32079)
Browse files Browse the repository at this point in the history
Leverage [Codecov](https://about.codecov.io/) to enforce (new) code coverage requirements.

Resolves aws#31931

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
  • Loading branch information
iliapolo authored Nov 11, 2024
1 parent 5648199 commit c5e738e
Show file tree
Hide file tree
Showing 5 changed files with 106 additions and 1 deletion.
35 changes: 35 additions & 0 deletions .github/workflows/codecov.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: Codecov

on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

jobs:
collect:
name: collect
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Node
uses: actions/setup-node@v4

- name: Install dependencies
run: cd packages/aws-cdk && yarn install

- name: Build CLI
run: cd packages/aws-cdk && npx lerna run build --scope=aws-cdk

- name: Run tests
run: cd packages/aws-cdk && yarn test

- name: Upload results to Codecov
uses: codecov/codecov-action@v4
with:
directory: packages/aws-cdk/coverage
fail_ci_if_error: true
flags: suite.unit
token: ${{ secrets.CODECOV_TOKEN }}
34 changes: 34 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -764,6 +764,40 @@ You can find the dependabot config file [here](./.github/dependabot.yml).
**If you think your PR introduces a new unconventional dependency, make sure to call it
out in the description so that we can discuss the best way to manage that dependency.**

### Addressing Code Coverage Gaps

We leverage [Codecov](https://about.codecov.io/) to track code coverage of the project.
If your PR doesn't meet the coverage requirements, you'll see failing status checks, which will prevent the PR from merging.

There are two requirements we define, each are enforced both on the overall
project as well as individual packages.

1. Coverage percentage must not decrease.
2. Patch percentage must be at least 95%.

Following is an example of status checks for a PR that violates both requirements:

![](./images/codecov-violations.png)

To fix and diagnose coverage gaps in your PR, there are two options:

1. Push your changes to the PR and wait for Codecov to comment on the PR.
2. If you find option 1 too slow, you can open a local coverage report located in `<path-to-package>/coverage/index.html`

> [!NOTE]
> Coverage percentage in local reports differs slightly from the percentage you'll see on Codecov.
> This is ok, and is related to how Codecov handles function signatures (probably).
> Ultimately Codecov is the source of truth, but you can still use local reports to locate uncovered
> lines and address them.
Even though it should be rare, sometimes specific lines will be hard to cover by tests.
To disable coverage of specific lines, you can use:

```ts
/* istanbul ignore next */
console.log('This cannot be covered')
```

### Step 5: Merge

* Make sure your PR builds successfully (we have CodeBuild setup to automatically build all PRs).
Expand Down
32 changes: 32 additions & 0 deletions codecov.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# https://docs.codecov.com/docs/codecovyml-reference#coverage
coverage:
status:
project:
default:
# require the overall project coverage to never decrease
target: auto
patch:
default:
# require the overall patch coverage to be at least 95%
target: 95


# https://docs.codecov.com/docs/codecovyml-reference#comment
comment:
layout: "header, diff, flags, components"

# https://docs.codecov.com/docs/codecovyml-reference#component_management
component_management:
default_rules:
statuses:
# every component will produce its own status check that requires overall coverage doesnt decrease
- type: project
target: auto
# every component will produce its own status check that requires patch coverage of at least 95%
- type: patch
target: 95
individual_components:
- component_id: packages_aws_cdk # identifier that should not be changed
name: packages/aws-cdk # display name that can change freely
paths:
- packages/aws-cdk/**
Binary file added images/codecov-violations.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 5 additions & 1 deletion tools/@aws-cdk/cdk-build-tools/config/jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,11 @@ module.exports = {
},
},
collectCoverage: true,
coverageReporters: ['lcov', 'html', 'text-summary', ['text', { file: 'coverage.txt' }]],
coverageReporters: [
'text-summary', // for console summary
'cobertura', // for codecov. see https://docs.codecov.com/docs/code-coverage-with-javascript
'html' // for local deep dive
],
coveragePathIgnorePatterns: ['\\.generated\\.[jt]s$', '<rootDir>/test/', '.warnings.jsii.js$', '/node_modules/'],
reporters: ['default', ['jest-junit', { suiteName: 'jest tests', outputDirectory: 'coverage' }]],
/**
Expand Down

0 comments on commit c5e738e

Please sign in to comment.