Statistics and Diagnostics service

[otoolep]

This change introduces the new Statistics and Monitor service, and monitors the Graphite input to start. To summarize there is a new component called monitor which allow any client that implements a simple interface to register with it. The monitor service will then call back to each registered client when statistics or diagnostics are requested via a query.

In addition -- though not yet fully implemented -- if enabled (which it is by default) the statistics will be recorded in an InfluxDB system, allowing the full power of InfluxQL to be employed. More details in the associated README file.

Remaining:

• complete diagnostics support.
• actualling recording data in an InfluxDB system.
• adding stats and diags to all services, as well as the TSDB package itself.

Example stats:

> show stats
name    Alloc   Frees   HeapAlloc       HeapIdle        HeapInUse       HeapObjects     HeapReleased    HeapSys         Lookups Mallocs NumGC   NumGoroutine    PauseTotalNs    Sys             TotalAlloc
runtime 3541080 8808326 3541080         15507456        5595136         33258           0               21102592        148     8841584 157     44              80489234        26802424        524115072


name            proto   batches_tx      bytes_rx        connections_active      connections_handled     points_rx       points_tx
graphite        tcp     209             5305678         0                       1                       208338          208338

> 

[otoolep]

@jwilder @corylanou @dgnorton @toddboom

[jwilder]

What does this config control/mean?

[otoolep]

This allows me to have the Graphite input perform all processing except actually writing to the database. It is tedious to constantly recompile the code, just to check input performance, in isolation. So the idea is a developer can set this flag, and try various other configs (mostly around batching and Goroutines). There have been some complaints about network-level performance and memory usage of Graphite, and I want to be able to test just the Graphite code.

Think of like the way we sometimes processing within the system, but drop the data before writing. It's most convenient if these performance-testing-related switches are configurable. They would not be documented in the sample config however.

[jwilder]

It seems odd to have this in the config though since the use case for this is mainly for developers. Won't running influxd config print it out as the sample config? Usually things like this are enabled using an env var in other systems. e.g. SKIP_WRITES=1 influxd

[otoolep]

Hmmm, hadn't though of the sample config. OK, let me nuke it, and deal with it another way (perhaps the environment variable).

[otoolep]

Thanks for the valuable feedback @jwilder -- I have responded to some and will update the code.

[jwilder]

LGTM 👍 I think the write-skip shouldn't be in the config but up to you.

[otoolep]

Removed write-skip from config.

[otoolep]

Got the +1 from @jwilder, so will merge now.