[CI] coverage test is failing

[kasia-kujawa]

Component(s)

No response

Describe the issue you're reporting

coverage test is failing even for pull request with only documentation changes, e.g. https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/8613373633/job/23605721246?pr=32248
There is a problem with upload of coverage report and this seems to be related to codecov/codecov-action#1359

[crobert-1]

Thanks for filing @kkujawa-sumo!

It looks like codecov-action supports tokenless uploads for public repos in v3 of their action, but going without a token also results in rate limiting. The rate limiting is GitHub limiting codecov itself across all public repos, so the options are to use a token, or wait (without re-trying as that will make the problem worse) for codecov's own rate limiting to resolve. I think the issue you've posted shows this is a wide reaching issue going on right now for codecov, not just us.

Project maintainers could create a codecov token to resolve this, but there may be some token privacy implications with running against contrib forks. From comment:

Public repositories that rely on PRs via forks will find that they cannot effectively use Codecov if the token is stored as a GitHub secret. The scope of the Codecov token is only to confirm that the coverage uploaded comes from a specific repository, not to pull down source code or make any code changes.

For this reason, we recommend that teams with public repositories that rely on PRs via forks consider the security ramifications of making the Codecov token available as opposed to being in a secret.

[codeboten]

a while ago there was an org wide token added for codecov (open-telemetry/community#1926) i'm curious if that token's not being used as expected

[crobert-1]

Thanks for pointing that out @codeboten, looks like we just haven't updated the collector repos to use the existing variable. 👍

[crobert-1]

I implemented this in core, but we're still hitting failures. At least it doesn't make anything worse, and can keep main runs green, AFAICT.

I'm at a bit of a loss to know what else to try. I checked around a bit and see that opentelemetry-go is using the same general workflow, the only difference being they're not relying on the Wandalen/wretry.action functionality like the collector repos are. All of opentelemetry-go's coverage uploads are succeeding.

[crobert-1]

I'm going to close as fixed by #32260. The understanding is that this still may fail on PRs from forked repos, but this should be good enough for now.