Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Track node and service counts in the state store and emit them periodically as metrics #8603

Merged
merged 9 commits into from
Sep 2, 2020
Merged

Track node and service counts in the state store and emit them periodically as metrics #8603

merged 9 commits into from
Sep 2, 2020

Commits on Sep 2, 2020

  1. Add new usage memdb table that tracks usage counts of various elements 

    We update the usage table on Commit() by using the TrackedChanges() API
of memdb.

Track memdb changes on restore so that usage data can be compiled
    @crhino
    crhino committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    04705e9 View commit details
    Browse the repository at this point in the history

  2. Report node/service usage metrics from every server 

    Using the newly provided state store methods, we periodically emit usage
metrics from the servers.

We decided to emit these metrics from all servers, not just the leader,
because that means we do not have to care about leader election flapping
causing metrics turbulence, and it seems reasonable for each server to
emit its own view of the state, even if they should always converge
rapidly.
    @crhino
    crhino committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    3feae7f View commit details
    Browse the repository at this point in the history

  3. Add WriteTxn interface and convert more functions to ReadTxn 

    We add a WriteTxn interface for use in updating the usage memdb table,
with the forward-looking prospect of incrementally converting other
functions to accept interfaces.

As well, we use the ReadTxn in new usage code, and as a side effect
convert a couple of existing functions to use that interface as well.
    @crhino
    crhino committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    69dbc92 View commit details
    Browse the repository at this point in the history

  4. Use ReadTxn interface in state store helper functions

    @crhino
    crhino committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    086a8ea View commit details
    Browse the repository at this point in the history

  5. Refactor state store usage to track unique service names 

    This commit refactors the state store usage code to track unique service
name changes on transaction commit. This means we only need to lookup
usage entries when reading the information, as opposed to iterating over
a large number of service indices.

- Take into account a service instance's name being changed
- Do not iterate through entire list of service instances, we only care
about whether there is 0, 1, or more than 1.
    @crhino
    crhino committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    d301145 View commit details
    Browse the repository at this point in the history

  6. Update godoc string for memdb wrapper functions/structs

    @crhino
    crhino committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    a3028ca View commit details
    Browse the repository at this point in the history

  7. Set metrics reporting interval to 9 seconds 

    This is below the 10 second interval that lib/telemetry.go implements as
its aggregation interval, ensuring that we always report these metrics.
    @crhino
    crhino committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    bcb586b View commit details
    Browse the repository at this point in the history

  8. docs: add new usage metrics to telemetry page

    @crhino
    crhino committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    bf50c47 View commit details
    Browse the repository at this point in the history

  9. Changelog entry for usage metrics

    @crhino
    crhino committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    40cbd5a View commit details
    Browse the repository at this point in the history