How to block the pull requests which will decrease the total test coverage.

[wuhuizuo]

Enhancement

As title said, currently we need to increase the project test coverage, but how to block it in CI that we need to do.

How to do

1. We can make it failed in github commit status with codecov service feature.

• Ref 1: https://docs.codecov.com/docs/codecovyml-reference#coveragestatus
• Ref 2: https://docs.codecov.com/docs/commit-status

2. Then we block it with prow tide context policy:

• set required-if-present-contexts key and add codecov/project into it's value.

Ref

prow's tide configuration example:

tide:
  # .....

  context_options:
    orgs:
      pingcap:
        repos:
          tidb:
            skip-unknown-contexts: true
            from-branch-protection: true
            branches:
              master:
                required-if-present-contexts:
                - "codecov/project"

The configuration is in configs repo.

[wuhuizuo]

Ref:

1. CI: make codecov/project failed when coverage decreased in pull request #45557
2. config(pingcap/tidb): add codecov/project as a required context if it present ti-community-infra/configs#941