-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
VReplication: throttle on target tablet #7364
VReplication: throttle on target tablet #7364
Conversation
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
This PR is ready; however, it does extend #7324 so it's best to first review that. |
What do you think of extracting the throttleStatusOK/throttle functions and related lagThrottler/lastSuccessfulThrottleCheck properties which are repeated in tableGC/vstreamer/vreplication into its own "class"? |
Yes, let me look into it. One hidden but important difference in the two implementations is that source side calls |
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@rohit-nayak-ps I will refactor this PR a bit to consolidate the calls for throttle checks; will request an additional review afterwards. |
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@rohit-nayak-ps I've created a new
and which offers the functionality of checking the throttler and backing off with sleep as necessary. |
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Description
With this PR Vreplication throttles on writing to target tablet. This applied to both table-copy writes and binlog events writes.
The VReplication Engine throttles by using the standard
check
, which assumes the backend is a shard's primary tablet, and which throttles by shard's lag.I'm not sure if there's a scenario where the target is a replica? If so, we can certainly also add
check-self
to the throttler's consideration.Related Issue(s)
Checklist
Deployment Notes
Impacted Areas in Vitess
Components that this PR will affect:
Because this PR extends #7324 it only makes sense to fully review it once #7324 is merged. Having said that, check out the specific changes under
go/vt/vttablet/tabletmanager/vreplication/
cc @rohit-nayak-ps