Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VReplication: throttle on target tablet #7364

Merged
merged 19 commits into from
Jan 27, 2021

Conversation

shlomi-noach
Copy link
Contributor

@shlomi-noach shlomi-noach commented Jan 24, 2021

Description

With this PR Vreplication throttles on writing to target tablet. This applied to both table-copy writes and binlog events writes.

The VReplication Engine throttles by using the standard check, which assumes the backend is a shard's primary tablet, and which throttles by shard's lag.

I'm not sure if there's a scenario where the target is a replica? If so, we can certainly also add check-self to the throttler's consideration.

Related Issue(s)

Checklist

  • Should this PR be backported?
  • Tests were added or are not required
  • Documentation was added or is not required

Deployment Notes

Impacted Areas in Vitess

Components that this PR will affect:

  • Query Serving
  • VReplication
  • Cluster Management
  • Build
  • VTAdmin

Because this PR extends #7324 it only makes sense to fully review it once #7324 is merged. Having said that, check out the specific changes under go/vt/vttablet/tabletmanager/vreplication/
cc @rohit-nayak-ps

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach shlomi-noach marked this pull request as ready for review January 25, 2021 12:45
@shlomi-noach
Copy link
Contributor Author

This PR is ready; however, it does extend #7324 so it's best to first review that.

@rohit-nayak-ps
Copy link
Contributor

What do you think of extracting the throttleStatusOK/throttle functions and related lagThrottler/lastSuccessfulThrottleCheck properties which are repeated in tableGC/vstreamer/vreplication into its own "class"?

@shlomi-noach
Copy link
Contributor Author

What do you think of extracting the throttleStatusOK/throttle functions

Yes, let me look into it. One hidden but important difference in the two implementations is that source side calls CheckSelf and target side calls Check for throttling.

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Copy link
Contributor

@rohit-nayak-ps rohit-nayak-ps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Copy link
Contributor Author

@rohit-nayak-ps I will refactor this PR a bit to consolidate the calls for throttle checks; will request an additional review afterwards.

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Copy link
Contributor Author

@rohit-nayak-ps I've created a new throttler.Client which is used by:

  • vstreamer Engine
  • replicator Engine
  • tablegc

and which offers the functionality of checking the throttler and backing off with sleep as necessary.

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach shlomi-noach merged commit f8c0583 into vitessio:master Jan 27, 2021
@shlomi-noach shlomi-noach deleted the vreplicator-throttle-target branch January 27, 2021 05:17
@askdba askdba added this to the v10.0 milestone Feb 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants