kv/client: don't resolve lock immediately after a region is initialized (#2235)

[ti-chi-bot]

This is an automated cherry-pick of #2235

What problem does this PR solve?

Fix #2188

What is changed and how it works?

Introduce a penalty mechanism

• If TiCDC keeps receiving unchanged resolved-ts, increase the penalty.
• If TiCDC receives a new resolved ts, reset the penalty to zero
• Only resolve ts if the penalty is larger than threshold

Check List

Tests

• Unit test
• Integration test

Release note

• Don't resolve lock immediately after a region is initialized.

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has not been approved.

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[amyangfei]

/merge

[amyangfei]

/merge

[codecov-commenter]

Codecov Report

❗ No coverage uploaded for pull request base (release-5.0@b6d0954). Click here to learn what that means.
The diff coverage is n/a.

❗ Current head 4be324c differs from pull request most recent head b739291. Consider uploading reports for the commit b739291 to get more accurate results

@@               Coverage Diff                @@
##             release-5.0      #2265   +/-   ##
================================================
  Coverage               ?   54.6261%           
================================================
  Files                  ?        166           
  Lines                  ?      19941           
  Branches               ?          0           
================================================
  Hits                   ?      10893           
  Misses                 ?       8003           
  Partials               ?       1045           

[amyangfei]

/merge

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: 6087a20

[amyangfei]

/merge

[ti-chi-bot]

@ti-chi-bot: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.