Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add realtime health stats to vtctld /api/keyspace/ks/tablets API #6569

Conversation

timvaillancourt
Copy link
Contributor

@timvaillancourt timvaillancourt commented Aug 14, 2020

(A continuation of #6524, abandoned due to git rebase issues 🤞)

This PR adds realtime health stats to the vtctld /api/keyspace/<ks>/tablets/<shard> HTTP API endpoint. This extends the PR @shlomi-noach made in #4232

The tablet health metrics are needed for github.com/github/freno to understand if a vttablet is ready for probes for replication lag. In today's implementation that looks for any tablet-type == REPLICA it is possible for an unhealthy vttablet (replication is STOP SLAVE'd, mysqld crashed, etc) to continue to be probed by Freno because it is unable to detect tablet health from the current API endpoint

This PR also adds a cell and cells (CSV list) query parameter for filtering the output by Vitess Cell. Let me know if you'd prefer this to be part of the URL path vs a query param. This is useful when you're only interested in grabbing specific cells

Example GET calls:

  1. GET /api/keyspace/<ks>/tablets/?cell=cell1
  2. GET /api/keyspace/<ks>/tablets/?cells=cell1,cell2
  3. GET /api/keyspace/<ks>/tablets/<shard>?cell=cell1
  4. GET /api/keyspace/<ks>/tablets/<shard>?cells=cell1,cell2

Lastly, this PR adds unit tests for GET /api/keyspace/<ks>/tablets/

I've made this a work in progress PR for @sougou / @shlomi-noach / @deepthi to review. After we agree on the approach I can complete the unit testing, add suggestions, etc

cc @sougou / @shlomi-noach / @deepthi / @tomkrouper / @drogart

Signed-off-by: Tim Vaillancourt <timvaillancourt@github.com>
Signed-off-by: Tim Vaillancourt <timvaillancourt@github.com>
Signed-off-by: Tim Vaillancourt <timvaillancourt@github.com>
@timvaillancourt timvaillancourt force-pushed the add-tablet-stats-to-vtctld-api-keyspace branch from 4bef9cd to 2670240 Compare August 14, 2020 20:55
Signed-off-by: Tim Vaillancourt <timvaillancourt@github.com>
Copy link
Member

@deepthi deepthi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! A couple of nitpicks.

go/vt/vtctld/api.go Show resolved Hide resolved
go/vt/vtctld/api.go Show resolved Hide resolved
Signed-off-by: Tim Vaillancourt <timvaillancourt@github.com>
Copy link
Member

@deepthi deepthi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@deepthi deepthi merged commit 9696e74 into vitessio:master Aug 17, 2020
@timvaillancourt timvaillancourt deleted the add-tablet-stats-to-vtctld-api-keyspace branch August 17, 2020 23:12
@timvaillancourt
Copy link
Contributor Author

LGTM

@deepthi thanks for the review 🙇

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants