Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ddl: fix duplicate elementID allocation to make sure gc work for partition table #33726

Merged
merged 22 commits into from
Apr 11, 2022
Merged

ddl: fix duplicate elementID allocation to make sure gc work for partition table #33726

merged 22 commits into from
Apr 11, 2022

Conversation

wjhuang2016
Copy link
Member

@wjhuang2016 wjhuang2016 commented Apr 6, 2022 •
edited
Loading

What problem does this PR solve?

Issue Number: close #33620

Problem Summary:
We use partitionID or indexID as elementID sometimes.
When drop index for partition table, we use indexID as elementID. So we may miss some deleteRange.

What is changed and how it works?

Allocate elementID as 1, 2, 3, 4 for a DDL job.
Besides, add a comment to log so that we can debug easier.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

fix duplicate elementID allocation to make sure gc work for partition table

@wjhuang2016
done
fbef542 
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
@wjhuang2016
Merge branch 'master' of https://github.com/pingcap/tidb into fix_gc_…
8e5a2a3 
…element_id
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Apr 6, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • tangenta
  • zimulala

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. needs-cherry-pick-4.0 needs-cherry-pick-release-5.0 needs-cherry-pick-release-5.1 needs-cherry-pick-release-5.2 needs-cherry-pick-release-5.3 Type: Need cherry pick to release-5.3 needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 6, 2022
@wjhuang2016
done
0617572 
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
@wjhuang2016
Merge branch 'master' of https://github.com/pingcap/tidb into fix_gc_…
3e80908 
…element_id
@wjhuang2016
fix test
e44e53e 
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
@wjhuang2016
fix test
e6cada3 
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
@wjhuang2016
fix test
8e212cc 
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
@ti-chi-bot ti-chi-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 6, 2022
@wjhuang2016
fmt
2bfef74 
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
@wjhuang2016
fmt
ef249f3 
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
@wjhuang2016
fmt
98b445d 
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
@wjhuang2016
Merge branch 'master' of https://github.com/pingcap/tidb into fix_gc_…
262044d 
…element_id
@wjhuang2016
refine
ab3de43 
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
@wjhuang2016
fix test
52f4212 
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
@sre-bot
Copy link
Contributor

sre-bot commented Apr 7, 2022 •
edited
Loading

Code Coverage Details: https://codecov.io/github/pingcap/tidb/commit/9c7b71e43f27cdcc3a1db4a9d56dcb2d8d94ae56

bb7133
bb7133 reviewed Apr 8, 2022
View reviewed changes
ddl/delete_range.go
@@ -238,12 +239,13 @@ func (dr *delRange) doTask(ctx sessionctx.Context, r util.DelRangeTask) error {

// insertJobIntoDeleteRangeTable parses the job into delete-range arguments,
// and inserts a new record into gc_delete_range table. The primary key is
// job ID, so we ignore key conflict error.
// (job ID, element ID), so we ignore key conflict error.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the bug exists from day 1 of table partition?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's a long story.

bb7133
bb7133 reviewed Apr 8, 2022
View reviewed changes
ddl/delete_range.go Outdated Show resolved Hide resolved
tangenta
tangenta reviewed Apr 8, 2022
View reviewed changes
ddl/main_test.go Outdated Show resolved Hide resolved
@wjhuang2016
address comment
d7d726d 
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 8, 2022
@ti-chi-bot ti-chi-bot merged commit 48efcf6 into pingcap:master Apr 11, 2022
ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Apr 11, 2022
@wjhuang2016 @ti-srebot
cherry pick pingcap#33726 to release-4.0
c60534d 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot ti-srebot mentioned this pull request Apr 11, 2022
Closed
12 tasks
@ti-srebot
Copy link
Contributor

cherry pick to release-4.0 in PR #33851

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Apr 11, 2022
@wjhuang2016 @ti-srebot
cherry pick pingcap#33726 to release-5.0
ce271a4 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot ti-srebot mentioned this pull request Apr 11, 2022
Closed
12 tasks
@ti-srebot
Copy link
Contributor

cherry pick to release-5.0 in PR #33852

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Apr 11, 2022
@wjhuang2016 @ti-srebot
cherry pick pingcap#33726 to release-5.1
7ae2b0e 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot ti-srebot mentioned this pull request Apr 11, 2022
Closed
12 tasks
@ti-srebot
Copy link
Contributor

cherry pick to release-5.1 in PR #33853

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Apr 11, 2022
@wjhuang2016 @ti-srebot
cherry pick pingcap#33726 to release-5.2
d50908f 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot ti-srebot mentioned this pull request Apr 11, 2022
Closed
12 tasks
@ti-srebot
Copy link
Contributor

cherry pick to release-5.2 in PR #33854

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Apr 11, 2022
@wjhuang2016 @ti-srebot
cherry pick pingcap#33726 to release-5.3
d46039d 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot ti-srebot mentioned this pull request Apr 11, 2022
Closed
12 tasks
@ti-srebot
Copy link
Contributor

cherry pick to release-5.3 in PR #33855

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Apr 11, 2022
@wjhuang2016 @ti-srebot
cherry pick pingcap#33726 to release-5.4
fccf276 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot ti-srebot mentioned this pull request Apr 11, 2022
Closed
12 tasks
@ti-srebot
Copy link
Contributor

cherry pick to release-5.4 in PR #33856

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Apr 11, 2022
@wjhuang2016 @ti-srebot
cherry pick pingcap#33726 to release-6.0
ea0ce7c 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot ti-srebot mentioned this pull request Apr 11, 2022
Closed
12 tasks
@ti-srebot
Copy link
Contributor

cherry pick to release-6.0 in PR #33857

@tangenta tangenta mentioned this pull request Apr 11, 2022
Closed
@tiancaiamao tiancaiamao mentioned this pull request Sep 21, 2022
Merged
12 tasks
@wjhuang2016 wjhuang2016 deleted the fix_gc_element_id branch November 17, 2022 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bb7133 bb7133 bb7133 left review comments

@tangenta tangenta tangenta approved these changes

@zimulala zimulala zimulala approved these changes

Assignees
No one assigned
Labels
needs-cherry-pick-release-5.0 needs-cherry-pick-release-5.1 needs-cherry-pick-release-5.2 needs-cherry-pick-release-5.3 Type: Need cherry pick to release-5.3 needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

delete range of drop index for partition table is not correct.
7 participants
@wjhuang2016 @ti-chi-bot @sre-bot @zimulala @ti-srebot @bb7133 @tangenta