Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics collecting and reporting support #1321

Merged
merged 23 commits into from
Jun 25, 2015

Conversation

karalabe
Copy link
Member

The goal of this PR is that - similar to logs - we should be able to add metrics collection to any part of the code without fancy constructs (counter variables, exporting them through public interfaces, adding them to the APIs, consoles, RCP/IPC, etc), and have the just work, be queryable and visualizable.

To that extent, this PR currently adds support for two metric types:

  • Meters: Event counters that can provide rate averages for the past 1, 5, 15 mins, and totals.
  • Timer: Timed meters, where each event has a duration associated and the last 1024 can be queried.

To create and use a meter and/or timer:

meter := metrics.GetOrRegisterMeter("my/fancy/meter", metrics.DefaultRegistry)
meter.Mark(125) // Register 125 events (e.x. 125 bytes written).

timer := metrics.GetOrRegisterTimer("my/fancy/timer", metrics.DefaultRegistry)
timer.Update(time.Second) // Register an event that took a second (e.x. block import)

These are automatically registered into a central metrics registry and can be queried from the javascript console through debug.metrics() for a user friendly pre-formatted output:

> debug.metrics().p2p.InboundTraffic
{
  Avg01Min: '169.12K (2.82K/s)',
  Avg05Min: '1.92M (6.42K/s)',
  Avg15Min: '3.57M (3.96K/s)',
  Total: '5.83M (2.97K/s)'
}
> debug.metrics().core.BlockInsertions
{
  Avg01Min: '10 (0.17/s)',
  Avg05Min: '61 (0.20/s)',
  Avg15Min: '168 (0.19/s)',
  Maximum: '2.157261657s',
  Minimum: '2.271716ms',
  Percentiles: {
    20: '6.993756ms',
    50: '12.342836ms',
    80: '21.765944ms',
    95: '218.500479ms',
    99: '376.015984ms'
  },
  Total: '432 (0.22/s)'
}

Or in a raw format for visualization, reporting, processing, monitoring, etc. via debug.metrics(true):

> debug.metrics(true).p2p.InboundTraffic
{
  AvgRate01Min: 1599.6190029292586,
  AvgRate05Min: 5367.754506658111,
  AvgRate15Min: 3761.057607521597,
  MeanRate: 2907.3919382272857,
  Total: 5901154
}
> debug.metrics(true).core.BlockInsertions
{
  AvgRate01Min: 0.18424704676246578,
  AvgRate05Min: 0.19801030732706026,
  AvgRate15Min: 0.18758183634199713,
  MeanRate: 0.22205942695239234,
  Percentiles: {
    0: 2271716,
    1: 2595259.96,
    // ...
    98: 267196822.1600002,
    99: 362098004.77999866,
    100: 2157261657
  },
  Total: 453
}

The PR furthermore introduces a new command to geth, geth monitor, that allows the real time visualization of any of these metrics:

Sample Metrics

The monitor will plot all requested metrics on individual line charts, expanding user patterns:

  • geth monitor core/BlockInsertions/Percentiles/80: a single metric to chart
  • geth monitor core/BlockInsertions/Percentiles/25 core/BlockInsertions/Percentiles/95: two metrics, side by side
  • geth monitor core/BlockInsertions/Percentiles/25,80,95: three metrics, with a common root
  • geth monitor p2p: expand and display all metrics under the p2p collection
  • geth monitor eth/db/block,state/Reads,Writes: various metrics with various wildcards (image above)

The charts are automatically scaled below 1000, and the scaling units inserted into the chart labels. The colors of the charts are also updated depending on the scale, ranging in [blue, cyan, green, yellow, red], with blue being none scaled, and every consecutive one a 1000 multiplier.

@obscuren obscuren modified the milestone: 0.9.34 Jun 23, 2015
@karalabe karalabe changed the title Uuuuu, what's inside here? :D Metrics collecting and reporting support Jun 23, 2015
obscuren added a commit that referenced this pull request Jun 25, 2015
Metrics collecting and reporting support
@obscuren obscuren merged commit b0a5be4 into ethereum:develop Jun 25, 2015
@obscuren obscuren removed the review label Jun 25, 2015
@obscuren obscuren modified the milestone: 0.9.34 Jun 30, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants