Restructure monitoring docs

docs/index.asciidoc

@@ -187,14 +187,13 @@ include::static/deploying.asciidoc[]
 :edit_url: https://github.com/elastic/logstash/edit/{branch}/docs/static/performance-checklist.asciidoc
 include::static/performance-checklist.asciidoc[]
 
-// Monitoring 
-:edit_url!:
-include::static/monitoring/monitoring.asciidoc[]
-
 // X-Pack Monitoring 
 :edit_url!:
 include::static/monitoring/monitoring-overview.asciidoc[]
 
+// Monitoring 
+:edit_url!:
+include::static/monitoring/monitoring.asciidoc[]
 
 // Working with Plugins
 

docs/static/monitoring/collectors-legacy.asciidoc

@@ -0,0 +1,49 @@
+[float]
+[role="xpack"]
+[[logstash-monitoring-collectors-legacy]]
+===== Collectors
+
+Collectors, as their name implies, collect things. In monitoring for Logstash, 
+collectors are just <<pipeline,Inputs>> in the same way that ordinary Logstash 
+configurations provide inputs.
+
+Like {monitoring} for {es}, each collector can create zero or more monitoring 
+documents. As it is currently implemented, each Logstash node runs two types of 
+collectors: one for node stats and one for pipeline stats.
+
+[options="header"]
+|=======================
+| Collector       | Data Types | Description
+| Node Stats      | `logstash_stats`
+| Gathers details about the running node, such as memory utilization and CPU
+usage (for example, `GET /_stats`).
+
+This runs on every Logstash node with {monitoring} enabled. One common
+failure is that Logstash directories are copied with their `path.data` directory
+included (`./data` by default), which copies the persistent UUID of the Logstash
+node along with it. As a result, it generally appears that one or more Logstash
+nodes are failing to collect monitoring data, when in fact they are all really
+misreporting as the _same_ Logstash node. Re-use `path.data` directories only 
+when upgrading Logstash, such that upgraded nodes replace the previous versions.
+| Pipeline Stats | `logstash_state`
+| Gathers details about the node's running pipelines, which powers the 
+Monitoring Pipeline UI.
+|=======================
+
+Per collection interval, which defaults to 10 seconds (`10s`), each collector is
+run. The failure of an individual collector does not impact any other collector. 
+Each collector, as an ordinary Logstash input, creates a separate Logstash event 
+in its isolated monitoring pipeline. The Logstash output then sends the data.
+
+The collection interval can be configured dynamically and you can also disable 
+data collection. For more information about the configuration options for the 
+collectors, see <<monitoring-settings-legacy>>.
+
+WARNING: Unlike {monitoring} for {es} and {kib}, there is no 
+`xpack.monitoring.collection.enabled` setting on Logstash. You must use the 
+`xpack.monitoring.enabled` setting to enable and disable data collection. 
+
+If gaps exist in the monitoring charts in {kib}, it is typically because either 
+a collector failed or the monitoring cluster did not receive the data (for 
+example, it was being restarted). In the event that a collector fails, a logged 
+error should exist on the node that attempted to perform the collection.

docs/static/monitoring/collectors.asciidoc

@@ -3,7 +3,7 @@
 [[logstash-monitoring-collectors]]
 ===== Collectors
 
-Collectors, as their name implies, collect things. In {monitoring} for Logstash, 
+Collectors, as their name implies, collect things. In monitoring for Logstash, 
 collectors are just <<pipeline,Inputs>> in the same way that ordinary Logstash 
 configurations provide inputs.
 

docs/static/monitoring/monitoring-internal-legacy.asciidoc

@@ -1,17 +1,139 @@
 [role="xpack"]
 [[monitoring-internal-collection-legacy]]
-=== Collect {ls} monitoring data using internal legacy collectors
+=== Collect {ls} monitoring data using legacy internal collectors
 ++++
-<titleabbrev>Legacy internal collection (deprecated)</titleabbrev>
+<titleabbrev>Legacy internal collection</titleabbrev>
 ++++
 
-deprecated[7.7.0]
+IMPORTANT: Use <<monitoring-internal-collection,internal collection>> or
+<<monitoring-with-metricbeat, {metricbeat} collection>> instead of legacy
+internal collection. If you are currently using legacy internal collection, you
+should migrate to either Metricbeat collection or internal collection.
 
-Starting in {ls} version 7.7, the legacy settings for internal collection are deprecated and
-will be removed in version 8.0.0. Instead of sending monitoring data to {es} exporter,
-it’s recommended that you use the <<monitoring-internal-collection,internal collection>>
-to route monitoring data directly to your monitoring indices.
+Monitoring {ls} with legacy internal collection uses these components:
 
-Or as an alternative to internal collection, use <<monitoring-with-metricbeat, {metricbeat} collection>>.
+* <<logstash-monitoring-collectors-legacy,Collectors>>
+* <<logstash-monitoring-output-legacy,Output>>
 
+These pieces live outside of the default Logstash pipeline in a dedicated monitoring
+pipeline. This configuration ensures that all data and processing has a minimal
+impact on ordinary Logstash processing. Existing Logstash features, such as the
+<<plugins-outputs-elasticsearch,`elasticsearch` output>>, can be reused to
+benefit from its retry policies. 
+
+NOTE: The `elasticsearch` output that is used for monitoring {ls} is 
+configured exclusively through settings found in `logstash.yml`. It is not 
+configured by using anything from the Logstash configurations that might also be 
+using their own separate `elasticsearch` outputs.
+
+
+The production {es} cluster should be configured to receive {ls} monitoring
+data. This configuration enables the production {es} cluster to add metadata
+(for example, its cluster UUID) to the Logstash monitoring data and then route
+it to the monitoring clusters. For more information  about typical monitoring
+architectures, see  {ref}/how-monitoring-works.html[How monitoring works] in the
+{ref}[Elasticsearch Reference]. 
+
+
+include::collectors-legacy.asciidoc[]
+include::monitoring-output-legacy.asciidoc[]
+
+
+[[configure-internal-collectors-legacy]]
+==== Configure {ls} monitoring with internal collectors
+++++
+<titleabbrev>Configure internal collection</titleabbrev>
+++++
+
+To monitor Logstash nodes:
+
+. Specify where to send monitoring data. This cluster is often referred to as
+the _production cluster_. For examples of typical monitoring architectures, see
+{ref}/how-monitoring-works.html[How monitoring works].
++
+--
+IMPORTANT: To visualize Logstash as part of the Elastic Stack (as shown in Step
+6), send metrics to your _production_ cluster. Sending metrics to a dedicated
+monitoring cluster will show the Logstash metrics under the _monitoring_ cluster.
+
+--
+
+. Verify that the `xpack.monitoring.collection.enabled` setting is `true` on the
+production cluster. If that setting is `false`, the collection of monitoring data
+is disabled in {es} and data is ignored from all other sources.
+
+. Configure your Logstash nodes to send metrics by setting the
+`xpack.monitoring.elasticsearch.hosts` in `logstash.yml`. If {security-features}
+are enabled, you also need to specify the credentials for the
+{ref}/built-in-users.html[built-in `logstash_system` user]. For more information
+about these settings, see <<monitoring-settings-legacy>>.
++
+--
+[source,yaml]
+--------------------------------------------------
+xpack.monitoring.elasticsearch.hosts: ["http://es-prod-node-1:9200", "http://es-prod-node-2:9200"] <1>
+xpack.monitoring.elasticsearch.username: "logstash_system"
+xpack.monitoring.elasticsearch.password: "changeme"
+--------------------------------------------------
+
+<1> If SSL/TLS is enabled on the production cluster, you must
+connect through HTTPS. As of v5.2.1, you can specify multiple
+Elasticsearch hosts as an array as well as specifying a single
+host as a string. If multiple URLs are specified, Logstash
+can round-robin requests to these production nodes. 
+--
+
+. If SSL/TLS is enabled on the production {es} cluster, specify the trusted
+CA certificates that will be used to verify the identity of the nodes
+in the cluster.
++
+--
+To add a CA certificate to a Logstash node's trusted certificates, you
+can specify the location of the PEM encoded certificate with the
+`certificate_authority` setting:
+
+[source,yaml]
+--------------------------------------------------
+xpack.monitoring.elasticsearch.ssl.certificate_authority: /path/to/ca.crt
+--------------------------------------------------
+
+Alternatively, you can configure trusted certificates using a truststore
+(a Java Keystore file that contains the certificates):
+
+[source,yaml]
+--------------------------------------------------
+xpack.monitoring.elasticsearch.ssl.truststore.path: /path/to/file
+xpack.monitoring.elasticsearch.ssl.truststore.password: password
+--------------------------------------------------
+
+Also, optionally, you can set up client certificate using a keystore
+(a Java Keystore file that contains the certificate):
+
+[source,yaml]
+--------------------------------------------------
+xpack.monitoring.elasticsearch.ssl.keystore.path: /path/to/file
+xpack.monitoring.elasticsearch.ssl.keystore.password: password
+--------------------------------------------------
+
+Set sniffing to `true` to enable discovery of other nodes of the {es} cluster.
+It defaults to `false`.
+
+[source,yaml]
+--------------------------------------------------
+xpack.monitoring.elasticsearch.sniffing: false
+--------------------------------------------------
+
+--
+
+. Restart your Logstash nodes.
+
+. To verify your {monitoring} configuration, point your web browser at your {kib}
+host, and select **Monitoring** from the side navigation. Metrics reported from
+your Logstash nodes should be visible in the Logstash section. When security is
+enabled, to view the monitoring dashboards you must log in to {kib} as a user
+who has the `kibana_user` and `monitoring_user` roles.
++
+image::images/monitoring-ui.png["Monitoring",link="monitoring/images/monitoring-ui.png"]
+
+include::../settings/monitoring-settings-legacy.asciidoc[]
 

docs/static/monitoring/monitoring-internal.asciidoc

@@ -1,36 +1,15 @@
 [role="xpack"]
 [[monitoring-internal-collection]]
-=== Collect {ls} monitoring data using internal collectors
+=== Use internal collectors to send monitoring data
 ++++
 <titleabbrev>Internal collection</titleabbrev>
 ++++
 
-Using internal collectors for {ls} {monitoring} these components:
-
-* <<logstash-monitoring-collectors,Collectors>>
-* <<logstash-monitoring-output,Output>>
-
-These pieces live outside of the default Logstash pipeline in a dedicated monitoring
-pipeline. This configuration ensures that all data and processing has a minimal
-impact on ordinary Logstash processing. Existing Logstash features, such as the
-<<plugins-outputs-elasticsearch,`elasticsearch` output>>, can be reused to
-benefit from its retry policies. 
-
-NOTE: The `elasticsearch` output that is used by {monitoring} for Logstash is 
-configured exclusively via settings found in `logstash.yml`. It is not 
-configured by using anything from the Logstash configurations that might also be 
-using their own separate `elasticsearch` outputs.
-
-The {es} cluster that is configured for use with {monitoring} for Logstash is 
-expected to be the production cluster. This configuration enables the production 
-{es} cluster to add metadata (for example, its cluster UUID) to the Logstash
-monitoring data then route it to the monitoring clusters. For more information 
-about typical monitoring architectures, see 
-{ref}/how-monitoring-works.html[How monitoring works] in the {ref}[Elasticsearch Reference]. 
-
-include::collectors.asciidoc[]
-include::monitoring-output.asciidoc[]
+Internal collectors send {ls} monitoring data directly to your _monitoring_ cluster.
+The benefit of internal collection is that you have fewer pieces of software to
+install and maintain. 
 
+<<monitoring-with-metricbeat, {metricbeat} collection>> is available as an alternative. 
 
 [[configure-internal-collectors]]
 ==== Configure {ls} monitoring with internal collectors
@@ -40,45 +19,35 @@ include::monitoring-output.asciidoc[]
 
 To monitor Logstash nodes:
 
-. Specify where to send monitoring data. This cluster is often referred to as
-the _production cluster_. For examples of typical monitoring architectures, see
-{ref}/how-monitoring-works.html[How monitoring works].
-+
---
-IMPORTANT: To visualize Logstash as part of the Elastic Stack (as shown in Step
-6), send metrics to your _production_ cluster. Sending metrics to a dedicated
-monitoring cluster will show the Logstash metrics under the _monitoring_ cluster.
-
---
+. Specify the location of the _monitoring cluster_. For examples of typical
+monitoring architectures, see {ref}/how-monitoring-works.html[How monitoring
+works] in the {ref}[Elasticsearch Reference].
 
 . Verify that the `monitoring.collection.enabled` setting is `true` on the
-production cluster. If that setting is `false`, the collection of monitoring data
-is disabled in {es} and data is ignored from all other sources.
+monitoring cluster. If that setting is `false`, the collection of monitoring data
+is disabled in {es}, and data is ignored from all other sources.
 
 . Configure your Logstash nodes to send metrics by setting the
-`xpack.monitoring.elasticsearch.hosts` in `logstash.yml`. If {security-features}
+`monitoring.elasticsearch.hosts` in `logstash.yml`. If {security-features}
 are enabled, you also need to specify the credentials for the
 {ref}/built-in-users.html[built-in `logstash_system` user]. For more
 information about these settings, see <<monitoring-settings>>.
 +
 --
 [source,yaml]
 --------------------------------------------------
-monitoring.elasticsearch.hosts: ["http://es-prod-node-1:9200", "http://es-prod-node-2:9200"] <1>
-monitoring.elasticsearch.username: "logstash_system" <2>
+monitoring.elasticsearch.hosts: ["http://es-prod-node-1:9200", "http://es-prod-node-2:9200"]
+monitoring.elasticsearch.username: "logstash_system"
 monitoring.elasticsearch.password: "changeme"
 --------------------------------------------------
 
-<1> If SSL/TLS is enabled on the production cluster, you must
-connect through HTTPS. As of v5.2.1, you can specify multiple
-Elasticsearch hosts as an array as well as specifying a single
-host as a string. If multiple URLs are specified, Logstash
-can round-robin requests to these production nodes.
-<2> If {security-features} are disabled on the production cluster, you can omit 
-these `username` and `password` settings. 
+If SSL/TLS is enabled on the monitoring cluster, you must connect through HTTPS.
+You can specify a single host as a string, or multiple Elasticsearch hosts as an
+array. If multiple URLs are specified, Logstash can round-robin requests to
+these monitoring nodes.
 --
 
-. If SSL/TLS is enabled on the production {es} cluster, specify the trusted
+. If SSL/TLS is enabled on the monitoring {es} cluster, specify the trusted
 CA certificates that will be used to verify the identity of the nodes
 in the cluster.
 +
@@ -122,21 +91,39 @@ monitoring.elasticsearch.sniffing: false
 
 . Restart your Logstash nodes.
 
-. To verify your {monitoring} configuration, point your web browser at your {kib}
+. To verify your monitoring configuration, point your web browser at your {kib}
 host, and select **Monitoring** from the side navigation. Metrics reported from
 your Logstash nodes should be visible in the Logstash section. When security is
-enabled, to view the monitoring dashboards you must log in to {kib} as a user
-who has the `kibana_user` and `monitoring_user` roles.
+enabled, you must log in to {kib} as a user who has the `kibana_user` and
+`monitoring_user` roles.
 +
 image::images/monitoring-ui.png["Monitoring",link="monitoring/images/monitoring-ui.png"]
 
-[float]
-[[monitoring-upgraded-logstash]]
-===== Re-enabling Logstash Monitoring After Upgrading
+include::../settings/monitoring-settings.asciidoc[]
 
-When upgrading from older versions of {xpack}, the built-in `logstash_system`
-user is disabled for security reasons. To resume monitoring,
-change the password and re-enable the logstash_system user.
 
-include::../settings/monitoring-settings.asciidoc[]
+[[internal-collector-components]]
+==== How {ls} monitoring with internal collectors works
+
+Monitoring {ls} with internal collectors uses these components:
+
+* <<logstash-monitoring-collectors,Collectors>>
+* <<logstash-monitoring-output,Output>>
 
+These pieces live outside of the default Logstash pipeline in a dedicated
+monitoring pipeline. This configuration ensures that all data and processing has
+a minimal impact on ordinary Logstash processing. Existing Logstash features,
+such as the <<plugins-outputs-elasticsearch,`elasticsearch` output>>, can be
+reused to benefit from its retry policies. 
+
+NOTE: The `elasticsearch` output for Logstash monitoring is configured
+exclusively through settings in `logstash.yml`. 
+
+The monitoring {es} cluster should be configured to receive {ls} monitoring
+data directly from {ls}. For more information about typical monitoring
+architectures, see {ref}/how-monitoring-works.html[How monitoring works] in the
+{ref}[Elasticsearch Reference]. 
+
+
+include::collectors.asciidoc[]
+include::monitoring-output.asciidoc[]

docs/static/monitoring/monitoring-mb.asciidoc

@@ -6,16 +6,19 @@
 <titleabbrev>{metricbeat} collection</titleabbrev>
 ++++
 
-You can use {metricbeat} to collect data about {ls} 
-and ship it to the monitoring cluster, rather than routing it through the 
-production cluster as described in <<monitoring-internal-collection>>.
+You can use {metricbeat} to collect data about {ls} and ship it to the
+monitoring cluster. The benefit of Metricbeat collection is that the monitoring
+agent remains active even if the {ls} instance does not. 
+
+<<monitoring-internal-collection,Internal collection>> is available as an
+alternative.
 
 //NOTE: The tagged regions are re-used in the Stack Overview.
 
 To collect and ship monitoring data:
 
 . <<disable-default,Disable default collection of monitoring metrics>>
-. <<define-cluster__uuid,Specify optionally the target `cluster_uuid`>>
+. <<define-cluster__uuid,Specify the target `cluster_uuid` (optional)>>
 . <<configure-metricbeat,Install and configure {metricbeat} to collect monitoring data>>
 
 [float]