Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gc_worker: fix deadlock in physicalScanAndResolveLocks #16393

Merged
merged 6 commits into from
Apr 28, 2020

Conversation

youjiali1995
Copy link
Contributor

Signed-off-by: youjiali1995 zlwgx1023@gmail.com

What problem does this PR solve?

Issue Number: close #16304

Problem Summary:
It's possible to block on channel sending in physicalScanAndResolveLocks when error occours.

What is changed and how it works?

What's Changed:
Check ctx.Done() when sending to a channel.

How it Works:

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Integration test

Side effects

Release note

Fix the issue that GC worker blocks on channel sending forever when error occours.

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
@codecov
Copy link

codecov bot commented Apr 15, 2020

Codecov Report

Merging #16393 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #16393   +/-   ##
===========================================
  Coverage   80.3941%   80.3941%           
===========================================
  Files           507        507           
  Lines        137561     137561           
===========================================
  Hits         110591     110591           
  Misses        18304      18304           
  Partials       8666       8666           

@youjiali1995 youjiali1995 requested a review from sticnarf April 15, 2020 09:24
@sticnarf
Copy link
Contributor

LGTM. Is there any way we can confirm the issue is resolved?

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
@youjiali1995
Copy link
Contributor Author

LGTM. Is there any way we can confirm the issue is resolved?

I added a test.

@youjiali1995
Copy link
Contributor Author

@MyonKeminta PTAL

@youjiali1995
Copy link
Contributor Author

/test

@youjiali1995
Copy link
Contributor Author

/run-unit-test

@youjiali1995
Copy link
Contributor Author

/run-all-tests

Copy link
Contributor

@MyonKeminta MyonKeminta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

if err != nil {
logutil.Logger(innerCtx).Error("resolve locks failed", zap.Error(err))
logutil.Logger(ctx).Error("resolve locks failed", zap.Error(err))
errCh <- err
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think here it should check ctx too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a buffered channel.

@youjiali1995
Copy link
Contributor Author

/merge

@sre-bot sre-bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 28, 2020
@youjiali1995 youjiali1995 modified the milestones: v4.0.0-rc.1, v4.0.0-ga Apr 28, 2020
@sre-bot
Copy link
Contributor

sre-bot commented Apr 28, 2020

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Apr 28, 2020

@youjiali1995 merge failed.

@youjiali1995
Copy link
Contributor Author

/merge

@sre-bot
Copy link
Contributor

sre-bot commented Apr 28, 2020

/run-all-tests

@sre-bot sre-bot merged commit 4749940 into pingcap:master Apr 28, 2020
@sre-bot
Copy link
Contributor

sre-bot commented Apr 28, 2020

cherry pick to release-4.0 in PR #16915

@youjiali1995 youjiali1995 deleted the fix-gc-worker-block branch April 28, 2020 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/GC status/can-merge Indicates a PR has been approved by a committer. type/bugfix This PR fixes a bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GC worker blocks forever
4 participants