pkg/chdelay: implement ChannelDelayer (#5373)

[ti-chi-bot]

This is an automated cherry-pick of #5373

What problem does this PR solve?

Issue Number: ref #5326

What is changed and how it works?

• We need a mechanism to reproduce the panic in panic: checkpointTs regressed #5326.
• I added a helper utility that can delay the messages in a channel by a time interval.
• Code in this PR will not be run in production.

Check List

Tests

• Unit test

Release note

None

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has not been approved.

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[VelocityLight]

This pull request is closed because it's related version has closed automatic cherry-picking.
If it's still needed, you can reopen it or just regenerate it using bot,
see:

• https://prow.tidb.io/command-help#cherrypick
• https://book.prow.tidb.net/#/plugins/cherrypicker

You can find more details at:

• https://internals.tidb.io/t/topic/785