-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #3916 from influxdb/new_stats_diags
Statistics and Diagnostics service
- Loading branch information
Showing
16 changed files
with
581 additions
and
134 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| # System Monitoring | ||
| _System Monitoring_ means all statistical and diagnostic information made availabe to the user of InfluxDB system, about the system itself. Its purpose is to assist with troubleshooting and performance analysis. | ||
|
|
||
| ## Supported Commands | ||
|
|
||
| * `SHOW STATS` | ||
| * `SHOW DIAGNOSTICS` | ||
|
|
||
| If statistical information is also written to an InfluxDB system, the data will also be queryable by the InfluxQL query language. | ||
|
|
||
| ## Statistics vs. Diagnostics | ||
| A distinction between _statistics_ and _diagnostics_ is made for the purposes of monitoring. Generally a statistical quality is something that is being counted, and for which it makes sense to store for historical analysis. Diagnostic information is not necessarily numerical, and may not make sense to store. | ||
|
|
||
| An example of statistical information would be the number of points received over UDP, or the number of queries executed. Examples of diagnostic information would be a list of current Graphite TCP connections, the version of InfluxDB, or the uptime of the process. | ||
|
|
||
| ## Design and Implementation | ||
|
|
||
| A new module named `monitor` supports all statistics and diagnostic functionality. This includes: | ||
|
|
||
| * Allowing other modules to register statistics and diagnostics information, allowing it to be accessed on demand by the `monitor` module. | ||
| * Serving the statistics and diagnostic information to the user, in response to commands such as `SHOW DIAGNOSTICS`. | ||
| * Expose standard Go runtime information such as garbage collection statistics. | ||
| * Make all collected expvar data via HTTP, for collection by 3rd-party tools. | ||
| * Writing the statistical information to an InfluxDB system, for historical analysis. This may be the same system generating the statistical information, but it does not have to be. Information is written used the Line Protocol. | ||
|
|
||
| To register with `monitor`, a module must implement the following interface: | ||
|
|
||
| ``` | ||
| type Client interface { | ||
| Statistics() (map[string]interface{}, error) | ||
| Diagnostics() (map[string]interface{}, error) | ||
| } | ||
| ``` | ||
|
|
||
| The module then calls `Register(name string, tags map[string]string, client Client)`. `name` is the Measurement name that will be associated with the statistics. `tags` will be the tags, though an empty map is acceptable. `client` is the module which implements the `Client` interface. | ||
|
|
||
| ### expvar | ||
| Statistical information is gathered by each package using [expvar](https://golang.org/pkg/expvar). Each package registers a map using its package name. | ||
|
|
||
| Due to the nature of `expvar`, statistical information is reset to its initial state when a server is restarted. | ||
|
|
||
| ## Configuration | ||
| The `monitor` module will allow the following configuration: | ||
|
|
||
| * Whether to write statistical and diagnostic information to an InfluxDB system. This is enabled by default. | ||
| * The name of the database to where this information should be written. Defaults to `_internal`. The information is written to the default retention policy for the given database. | ||
| * The name of the retention policy, along with full configuration control of the retention policy. | ||
| * The address and port of the InfluxDB system. This will default to the system generating the data. | ||
| * The rate at which this information should be written. The maximum rate will be once a second. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| package monitor | ||
|
|
||
| import ( | ||
| "time" | ||
|
|
||
| "github.com/influxdb/influxdb/toml" | ||
| ) | ||
|
|
||
| const ( | ||
| // DefaultStoreEnabled is whether the system writes gathered information in | ||
| // an InfluxDB system for historical analysis. | ||
| DefaultStoreEnabled = true | ||
|
|
||
| // DefaultStoreDatabase is the name of the database where gathered information is written | ||
| DefaultStoreDatabase = "_internal" | ||
|
|
||
| // DefaultStoreInterval is the period between storing gathered information. | ||
| DefaultStoreInterval = time.Minute | ||
|
|
||
| // DefaultStoreAddress is the destination system for gathered information. | ||
| DefaultStoreAddress = "127.0.0.1:8086" | ||
| ) | ||
|
|
||
| // Config represents the configuration for the monitor service. | ||
| type Config struct { | ||
| StoreEnabled bool `toml:"store-enabled"` | ||
| StoreDatabase string `toml:"store-database"` | ||
| StoreInterval toml.Duration `toml:"store-interval"` | ||
| StoreAddress string `toml:"store-address"` | ||
| } | ||
|
|
||
| // NewConfig returns an instance of Config with defaults. | ||
| func NewConfig() Config { | ||
| return Config{ | ||
| StoreEnabled: false, | ||
| StoreDatabase: DefaultStoreDatabase, | ||
| StoreInterval: toml.Duration(DefaultStoreInterval), | ||
| StoreAddress: DefaultStoreAddress, | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| package monitor_test | ||
|
|
||
| import ( | ||
| "testing" | ||
| "time" | ||
|
|
||
| "github.com/BurntSushi/toml" | ||
| "github.com/influxdb/influxdb/monitor" | ||
| ) | ||
|
|
||
| func TestConfig_Parse(t *testing.T) { | ||
| // Parse configuration. | ||
| var c monitor.Config | ||
| if _, err := toml.Decode(` | ||
| store-enabled=true | ||
| store-database="the_db" | ||
| store-interval="10m" | ||
| store-address="server1" | ||
| `, &c); err != nil { | ||
| t.Fatal(err) | ||
| } | ||
|
|
||
| // Validate configuration. | ||
| if !c.StoreEnabled { | ||
| t.Fatalf("unexpected store-enabled: %v", c.StoreEnabled) | ||
| } else if c.StoreDatabase != "the_db" { | ||
| t.Fatalf("unexpected store-database: %s", c.StoreDatabase) | ||
| } else if time.Duration(c.StoreInterval) != 10*time.Minute { | ||
| t.Fatalf("unexpected store-interval: %s", c.StoreInterval) | ||
| } else if c.StoreAddress != "server1" { | ||
| t.Fatalf("unexpected store-address: %s", c.StoreAddress) | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| package monitor | ||
|
|
||
| import ( | ||
| "runtime" | ||
| ) | ||
|
|
||
| // goRuntime captures Go runtime statistics and implements the monitor client interface | ||
| type goRuntime struct{} | ||
|
|
||
| // Statistics returns the statistics for the goRuntime type | ||
| func (g *goRuntime) Statistics() (map[string]interface{}, error) { | ||
| var m runtime.MemStats | ||
| runtime.ReadMemStats(&m) | ||
|
|
||
| return map[string]interface{}{ | ||
| "Alloc": int64(m.Alloc), | ||
| "TotalAlloc": int64(m.TotalAlloc), | ||
| "Sys": int64(m.Sys), | ||
| "Lookups": int64(m.Lookups), | ||
| "Mallocs": int64(m.Mallocs), | ||
| "Frees": int64(m.Frees), | ||
| "HeapAlloc": int64(m.HeapAlloc), | ||
| "HeapSys": int64(m.HeapSys), | ||
| "HeapIdle": int64(m.HeapIdle), | ||
| "HeapInUse": int64(m.HeapInuse), | ||
| "HeapReleased": int64(m.HeapReleased), | ||
| "HeapObjects": int64(m.HeapObjects), | ||
| "PauseTotalNs": int64(m.PauseTotalNs), | ||
| "NumGC": int64(m.NumGC), | ||
| "NumGoroutine": int64(runtime.NumGoroutine()), | ||
| }, nil | ||
| } | ||
|
|
||
| // Diagnostics returns the statistics for the goRuntime type | ||
| func (g *goRuntime) Diagnostics() (map[string]interface{}, error) { | ||
| return nil, nil | ||
| } |
Oops, something went wrong.