Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

syncer/checkpoint(dm): update the meaning of IsOlderThanTablePoint #4289

Merged
merged 10 commits into from
Jan 21, 2022
Merged

syncer/checkpoint(dm): update the meaning of IsOlderThanTablePoint #4289

merged 10 commits into from
Jan 21, 2022

Conversation

lance6716
Copy link
Contributor

What problem does this PR solve?

Issue Number: ref #4287

What is changed and how it works?

after #3891 , we can be confident to skip event <= table checkpoint when using position based replication

Check List

Tests

  • pass original test

Code changes

  • Has exported function/method change

Release note 

`None`.

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Jan 12, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • Ehco1996
  • GMHDBJD

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added the release-note-none Denotes a PR that doesn't merit a release note. label Jan 12, 2022
@lance6716 lance6716 added the area/dm Issues or PRs related to DM. label Jan 12, 2022
@ti-chi-bot ti-chi-bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Jan 12, 2022
@lance6716
Copy link
Contributor Author

/run-dm-integration-test

1 similar comment
@lance6716
Copy link
Contributor Author

/run-dm-integration-test

Ehco1996
Ehco1996 approved these changes Jan 13, 2022
View reviewed changes
Copy link
Contributor

@Ehco1996 Ehco1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jan 13, 2022
GMHDBJD
GMHDBJD approved these changes Jan 18, 2022
View reviewed changes
Copy link
Contributor

@GMHDBJD GMHDBJD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jan 18, 2022
@lance6716
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 58201e5

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jan 18, 2022
@codecov-commenter
Copy link

codecov-commenter commented Jan 18, 2022 •
edited
Loading

Codecov Report

Merging #4289 (62664ec) into master (ab62e6b) will decrease coverage by 0.1466%.
The diff coverage is 44.7099%.

Flag Coverage Δ
cdc 59.2196% <54.6042%> (-0.3963%) ⬇️
dm 52.1278% <40.4329%> (+0.0622%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master      #4289        +/-   ##
================================================
- Coverage   55.5183%   55.3716%   -0.1467%     
================================================
  Files           495        495                
  Lines         61151      61192        +41     
================================================
- Hits          33950      33883        -67     
- Misses        23789      23916       +127     
+ Partials       3412       3393        -19

@Ehco1996
Copy link
Contributor

/run-dm-integration-test
/run-verify
/run-kafka-integration-test

@lance6716
Copy link
Contributor Author

/run-dm-integration-test

@lance6716
Copy link
Contributor Author

/hold

found a bug in CI

@ti-chi-bot ti-chi-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 18, 2022
@ti-chi-bot ti-chi-bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed status/can-merge Indicates a PR has been approved by a committer. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jan 18, 2022
@lance6716
Copy link
Contributor Author

/run-dm-integration-test

@lance6716
Copy link
Contributor Author

lance6716 commented Jan 18, 2022 •
edited
Loading

dd95314

fix a bug that save checkpoint while job is not added to queue because of wait transaction waitComplete

PTAL @Ehco1996 @GMHDBJD

@lance6716
Copy link
Contributor Author

can't understand this change, what if added2Queue = false but err2!=nil ?

it means handleJob meets error before send the job to DMLQueue or DDLQueue, which will not happen currently.

@Ehco1996
Copy link
Contributor

can't understand this change, what if added2Queue = false but err2!=nil ?

it means handleJob meets error before send the job to DMLQueue or DDLQueue, which will not happen currently.

yes, but this could not hanppen in handleRowsEvent and how this change fix the save checkpoint while job is not added to queue 😵. could you please explain more

@lance6716
Copy link
Contributor Author

lance6716 commented Jan 19, 2022 •
edited
Loading

I mean "save checkpoint while job is not added to queue" is a bug, the commit has fixed it.

when the syncer is exiting and waiting transaction waitComplete, job may not be added to queue, and in the meantime checkpoint should not be updated

@lance6716
Copy link
Contributor Author

/unhold
/merge

@ti-chi-bot ti-chi-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 20, 2022
@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: dd95314

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jan 20, 2022
@lance6716
Copy link
Contributor Author

/run-integration-test /tidb=pr/30558
/run-kafka-integration-test /tidb=pr/30558

@lance6716
Copy link
Contributor Author

/run-integration-test /tidb=pr/30558
/run-kafka-integration-test /tidb=pr/30558

@lance6716
Copy link
Contributor Author

/run-kafka-integration-test /tidb=pr/30558

@ti-chi-bot ti-chi-bot merged commit e0eeac0 into pingcap:master Jan 21, 2022
@lance6716 lance6716 deleted the refine-tablepoint branch October 13, 2022 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Ehco1996 Ehco1996 Ehco1996 approved these changes

@GMHDBJD GMHDBJD GMHDBJD approved these changes

Assignees
No one assigned
Labels
area/dm Issues or PRs related to DM. release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@lance6716 @ti-chi-bot @codecov-commenter @Ehco1996 @GMHDBJD