Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lib/worker(engine): refactor BaseWorker (Part I) #5520

Merged
merged 8 commits into from
May 24, 2022
Merged

lib/worker(engine): refactor BaseWorker (Part I) #5520

merged 8 commits into from
May 24, 2022

Conversation

liuzix
Copy link
Contributor

@liuzix liuzix commented May 23, 2022 •
edited
Loading

What problem does this PR solve?

Issue Number: ref #5521

What is changed and how it works?

  • Move MasterClient into a separate package.
  • Remove OnMasterFailOver.

Reviewer's Guide

  • Focus on engine/lib/worker/master_client.go.
  • The basic structure of MasterClient is the same as before, but with some asynchronous enhancement and minor bug fixes.
  • Please make sure that removing OnMasterFailOver does not harm the correctness of the business logic.

Check List

Tests

  • Unit test
  • Integration test
None

@ti-chi-bot
Copy link
Member

ti-chi-bot commented May 23, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • GMHDBJD
  • amyangfei

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels May 23, 2022
@liuzix liuzix changed the title [WIP] ib/worker(engine): refactor MasterClient [WIP] lib/worker(engine): refactor MasterClient May 23, 2022
@codecov-commenter
Copy link

codecov-commenter commented May 23, 2022 •
edited
Loading

Codecov Report

Merging #5520 (4627092) into master (1f844e7) will decrease coverage by 0.0629%.
The diff coverage is n/a.

Flag Coverage Δ
cdc 61.5795% <ø> (-0.0072%) ⬇️
dm 51.9482% <ø> (-0.0948%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master      #5520        +/-   ##
================================================
- Coverage   56.2521%   56.1891%   -0.0630%     
================================================
  Files           530        530                
  Lines         69880      69791        -89     
================================================
- Hits          39309      39215        -94     
+ Misses        26840      26837         -3     
- Partials       3731       3739         +8

@ti-chi-bot ti-chi-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed do-not-merge/needs-triage-completed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels May 23, 2022
@liuzix liuzix changed the title [WIP] lib/worker(engine): refactor MasterClient lib/worker(engine): refactor BaseWorker (Part I) May 24, 2022
@ti-chi-bot ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 24, 2022
@liuzix liuzix added area/engine Issues or PRs related to Dataflow Engine. status/ptal Could you please take a look? labels May 24, 2022
GMHDBJD
GMHDBJD approved these changes May 24, 2022
View reviewed changes
Copy link
Contributor

@GMHDBJD GMHDBJD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label May 24, 2022
amyangfei
amyangfei reviewed May 24, 2022
View reviewed changes
engine/lib/worker/master_client.go
sinceLastAcked < m.timeoutConfig.WorkerTimeoutDuration {

// We ignore the error here
_ = m.asyncReloadMasterInfo(context.Background())
Copy link
Contributor

@amyangfei amyangfei May 24, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it better to add a timeout, such as WorkerTimeoutDuration, in case of leaving some stale goroutines even worker has exited. Besides why removing the context parameter in this function

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides why removing the context parameter in this function

The semantics of CheckMasterTimeout does not seem to require a context. It should always be non-blocking to avoid any surprises.

engine/lib/worker/master_client.go
zap.String("master-id", m.masterID))
if !ok {
// Reloads master info asynchronously.
_ = m.asyncReloadMasterInfo(context.Background())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we also need to add a timeout here?

@liuzix liuzix requested a review from amyangfei May 24, 2022 07:32
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels May 24, 2022
@liuzix
Copy link
Contributor Author

liuzix commented May 24, 2022

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 8993880

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label May 24, 2022
@ti-chi-bot ti-chi-bot merged commit e9433f5 into pingcap:master May 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@GMHDBJD GMHDBJD GMHDBJD approved these changes

@amyangfei amyangfei amyangfei approved these changes

@maxshuang maxshuang Awaiting requested review from maxshuang

@lance6716 lance6716 Awaiting requested review from lance6716

@sleepymole sleepymole Awaiting requested review from sleepymole

Assignees
No one assigned
Labels
area/engine Issues or PRs related to Dataflow Engine. release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. status/ptal Could you please take a look?
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@liuzix @ti-chi-bot @codecov-commenter @amyangfei @GMHDBJD