diff --git a/README.md b/README.md index b69f38d31..7370ed7c0 100644 --- a/README.md +++ b/README.md @@ -70,7 +70,7 @@ The key features of Snap are: * **Collectors** - Collectors consume telemetry data. Collectors are plugins for leveraging existing telemetry solutions (Facter, CollectD, Ohai) as well as specific plugins for consuming Intel telemetry (Node, DCM, NIC, Disk) and can reach into new architectures through additional plugins (see [Plugin Authoring below](#author-a-plugin)). Telemetry data is organized into a dynamically generated catalog of available data points. * **Processors** - Extensible workflow injection. Convert telemetry into another data model for consumption by existing systems. Allows encryption of all or part of the telemetry payload before publishing. Inject remote queries into workflow for tokens, filtering, or other external calls. Implement filtering at an agent level reducing injection load on telemetry consumer. * **Publishers** - Store telemetry into a wide array of systems. Snap decouples the collection of telemetry from the implementation of where to send it. Snap comes with a large library of publisher plugins that allow exposure to telemetry analytics systems both custom and common. This flexibility allows Snap to be valuable to open source and commercial ecosystems alike by writing a publisher for their architectures. - * **Streaming Collectors** - Streaming collectors act just like collectors, but rather than waiting for a specified duration before sending the collected data, they send it immediatly over a grpc stream to snaptel. Check out [STREAMING.md](/docs/STREAMING.md) for more details on the differences between collectors and streaming collectors. + * **Streaming Collectors** - Streaming collectors act just like collectors, but rather than waiting for a specified duration before sending the collected data, they send it immediately over a grpc stream to snaptel. Check out [STREAMING.md](/docs/STREAMING.md) for more details on the differences between collectors and streaming collectors. * **Dynamic Updates**: Snap is designed to evolve. Each scheduled workflow automatically uses the most mature plugin for that step, unless the collection is pinned to a specific version (e.g. get `/intel/psutil/load/load1/v1`). Loading a new plugin automatically upgrades running workflows in tasks. Load plugins dynamically, without a restart to the service or server. This dynamically extends the metric catalog when loaded, giving access to new measurements immediately. Swapping a newer version plugin for an old one in a safe transaction. All of these behaviors allow for simple and secure bug fixes, security patching, and improving accuracy in production. diff --git a/docs/PLUGIN_AUTHORING.md b/docs/PLUGIN_AUTHORING.md index 230d36300..8b2f03868 100644 --- a/docs/PLUGIN_AUTHORING.md +++ b/docs/PLUGIN_AUTHORING.md @@ -41,10 +41,10 @@ Before writing a new Snap plugin, please check out the [Plugin Catalog](./PLUGIN Snap supports four types of plugins: -* collector: gathering metrics +* collector: gathering metrics based on the specified interval * processor: transforming metrics * publisher: publishing metrics -* streaming collector: streaming gathered metrics +* streaming collector: gathering metrics when they are available ### Plugin Name diff --git a/docs/STREAMING.md b/docs/STREAMING.md index b7d4bacd2..fb2c606bd 100644 --- a/docs/STREAMING.md +++ b/docs/STREAMING.md @@ -1,13 +1,13 @@ # Streaming -Uses grpc streams to allow the plugin to send data immediately or after a certain period of time instead of on an interval governed by Snap. +Streaming plugins use grpc streams to allow the plugin to send data immediately or after a certain period of time instead of on an interval governed by Snap. Streaming by snap enables - * Improved performance by enabling event based data flows -* Supports runtime configuration controlling event throughput -* Buffer configurations enable dispatching events after a given duration and/or when a given event count has been reached +* Runtime configuration controlling event throughput +* Buffer configurations which dispatch events after a given duration and/or when a given event count has been reached -Currently there are two plugins that support streaming - [snap relay] (https://github.com/intelsdi-x/snap-relay) and [snap-plugin-collector-rand-streaming] (https://github.com/intelsdi-x/snap-plugin-lib-go/tree/master/examples/snap-plugin-collector-rand-streaming). +Currently there are two plugins that support streaming - [snap relay](https://github.com/intelsdi-x/snap-relay) and [snap-plugin-collector-rand-streaming](https://github.com/intelsdi-x/snap-plugin-lib-go/tree/master/examples/snap-plugin-collector-rand-streaming). # Configuration options MaxCollectDuration and MaxMetricsBuffer are two configuration options that can be set through streaming task manifest or flags. diff --git a/docs/TASKS.md b/docs/TASKS.md index 3a9035b59..baf8bdf1e 100644 --- a/docs/TASKS.md +++ b/docs/TASKS.md @@ -184,6 +184,7 @@ Snap is designed in a way where custom schedulers can easily be dropped in. If a schedule: type: "streaming" ``` +The streaming schedule doesn't support fields such as `interval` and `count`. If those fields are provided as part of the schedule, they will simply be skipped. For more details on streaming, visit [STREAMING.md](STREAMING.md) #### Max-Failures diff --git a/examples/README.md b/examples/README.md index 7e5bee350..e767e5910 100644 --- a/examples/README.md +++ b/examples/README.md @@ -33,5 +33,4 @@ For additional examples of using Snap, checkout the examples in these repositori - [snap-plugin-publisher-influxdb](https://github.com/intelsdi-x/snap-plugin-publisher-influxdb) - [snap-plugin-publisher-graphite](https://github.com/intelsdi-x/snap-plugin-publisher-graphite) - [snap-plugin-publisher-file](https://github.com/intelsdi-x/snap-plugin-publisher-file) - - [snap-relay](https://github.com/intelsdi-x/snap-relay) (streaming collector plugin) - - [snap-plugin-collector-rand-streaming](https://github.com/intelsdi-x/snap-plugin-lib-go/blob/master/examples/snap-plugin-collector-rand-streaming) (streaming collector plugin) + - [snap-relay](https://github.com/intelsdi-x/snap-relay) (streaming collector plugin to retrieve data from collectd or statsd and include them into Snap workflow) \ No newline at end of file