Tablet throttler documentation (references/features) #512
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
|
Deploy preview for vitess ready! Built with commit c3c905e |
This was referenced Sep 10, 2020
|
ping. Request for review 🙏 |
…cas exist Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
shlomi-noach
force-pushed
the
vttablet-throttle
branch
from
a62cd73
to
e699bec
Compare
Signed-off-by: Jacques Grove <aquarapid@gmail.com>
aquarapid
approved these changes
Oct 6, 2020
deepthi
reviewed
Oct 6, 2020
| Apps will use `/throttler/check` | ||
|
|
||
| - Apps may indicate their identity via `?app=<name>` param. | ||
| - Apps may further declare themselves to be _low priority_ via `?p=low` param. Managed online schema migrations (`gh-ost`, `pt-online-schema-change`) do so, as does the table purge process. |
There was a problem hiding this comment.
what is the significance of low priority?
There was a problem hiding this comment.
It's actually very significant: today, we only use the throttler fo rstrictly offline processes: schema migrations and table purging. But, in the future, an app may want to use the throttler for bulk writes, imports, etc. The app is less tolerant to delays. If the app wants to import 10,000,000 rows into a table, and a migration runs on another table, then the two will collide; they will both make slow progress, but while th emigration doesn't case, the app will care. That's why we put migrations and table purges on low priority, such that they will yield to apps with "normal" priority. At GitHub, this change made a huge impact to application behavior.
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
bnlandry
suggested changes
Oct 7, 2020
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
shlomi-noach
force-pushed
the
vttablet-throttle
branch
from
aa60685
to
9d16f21
Compare
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
shlomi-noach
force-pushed
the
vttablet-throttle
branch
from
7367f58
to
e7d6d04
Compare
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
shlomi-noach
force-pushed
the
vttablet-throttle
branch
from
bba05a8
to
4349669
Compare
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
|
ping @bnlandry I've responded to all but one review comments (see following) - does this look good now? @askdba @deepthi @sougou please see my question: #512 (comment) |
bnlandry
approved these changes
Oct 14, 2020
|
|
||
| VTTablet runs a cooperative throttling service. This service probes the shard's MySQL topology and observes replication lag on servers. This throttler is derived from GitHub's [freno](https://github.com/github/freno). | ||
|
|
||
| _Note: the Vitess documentation is transitioning from the term "Master" (with regard to MySQL replication) to "Primary". this document reflects this transition._ |
There was a problem hiding this comment.
Suggested change
| _Note: the Vitess documentation is transitioning from the term "Master" (with regard to MySQL replication) to "Primary". this document reflects this transition._ | |
| _Note: with regard to MySQL replication, the Vitess documentation is transitioning from the term 'master' to the term 'primary.' This document reflects this transition._ |
|
|
||
| The user may override the heartbeat interval by sending `-heartbeat_interval` flag to `vttablet`. | ||
|
|
||
| Thus, the aggregated interval can be off, by default, by some `500ms`. This makes it inaccurate for evaluations that require high resolution lag evaluation. This resolution is sufficient for throttling purposes. |
There was a problem hiding this comment.
Suggested change
| Thus, the aggregated interval can be off, by default, by some `500ms`. This makes it inaccurate for evaluations that require high resolution lag evaluation. This resolution is sufficient for throttling purposes. | |
| Thus, the aggregated interval can be off, by default, by about `500ms`. This makes it inaccurate for evaluations that require high resolution lag evaluation. This resolution is sufficient for throttling purposes. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Documenting vitessio/vitess#6668: the tablet throttler. background, overview, configuration, API, usage
Only merge once vitessio/vitess#6668 is merged.