consumer(ticdc): update the logic of consumer. (#4129) #4470
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is an automated cherry-pick of #4129
What problem does this PR solve?
Issue Number: close #4128
Use
kafka-consumer
program to maintain the order relationship betweencommitTs
andwatermarkTs
.watermarkTs
also known asResolvedTs
in TiCDC source code.What is changed and how it works?
Modify the implementation detail of the
kakfa-consumer
program.ResolvedTs
fallback, which means receiveResolved Event
withTs
less than currentResolvedTs
, just crash the consumer.Row Changed Event
orDDL Event
withTs
not greater than theglobalResolvedTs
, just ignore these events should already be seen.After this PR, the consumer should still be running stable.
Check List
Tests
Code changes
Side effects
Related changes
Release note