Add documentation on matrix build, only upload coverage once

[jhidding]

When testing in a matrix, uploads often fail after the first one. I'm testing against different versions of Python, and I only need to upload the coverage information once, by including

if: matrix.python-version == 3.8

in the codecov job. It took me a while to figure this out, so maybe it is nice to include this in the readme.

[ibrahim0814]

Hello @jhidding, thank you for bringing this up!

Regarding your issue about how uploads often fail after the first one - could you elaborate more on this? What error do you see on the console? Uploads to Codecov shouldn't fail after the first build in the matrix, but if that's the case, perhaps we need to investigate this issue on our end because other users might be affected by it

[jhidding]

Github Actions gives a connection time-out, retries after 30 seconds end-so-on.

[ibrahim0814]

@jhidding Any updates on this particular issue? If the problem still persists, we can punch in a support ticket

[rfgamaral]

I've started using Codecov with GitHub Actions (on this test project for now) and I'm wondering what's the best practice for matrix builds!?

This particular project tests the code against 3 different Node.js version and with my current workflow I'm uploading 3 different coverage reports (but they are always the same):

This doesn't make much sense to me. Ideally, only one coverage report would be uploaded.

Now, I understand I can add an if statement to the upload step like @jhidding mentioned in the original post, but somehow that feels like a hacky solution.

What is the recommended best practice to upload the coverage report only once independently of the number of jobs due to matrix configurations in GitHub actions?

[zhuanyan-wang]

Multiple Node.js and multiple OS

Expand on what @jhidding and @rfgamaral have said. If you are testing on multiple Node.js versions and multiple OS at the same time, something like this will work:

strategy:
  matrix:
    os: [ubuntu-latest, windows-latest, macos-latest]
    node-version: [10.x, 12.x, 14.x]

runs-on: ${{ matrix.os }}

steps:
  - name: Post Coverage
    uses: codecov/codecov-action@v1
    if: matrix.os == 'ubuntu-latest' && matrix.node-version == '10.x'

This posts the coverage reports to Codecov, only when the OS is ubuntu-latest and the Node.js version is 10.x. Just change if: matrix.os == 'ubuntu-latest' && matrix.node-version == '10.x' based on your own setup.

Result

The Post Coverage is skipped when the OS or the Node.js version does not match:

Two subdirectories Default and Container have posted their reports once only (otherwise there will be 12 builds here):