Merge pull request elastic#4423 from dedemorton/cherrypick_docs_5.30.17

Backport: elastic#4312, elastic#4405, and elastic#4408 into 5.4

filebeat/docs/command-line.asciidoc

@@ -1,4 +1,4 @@
-[[filebeat-command-line]]
+[[command-line-options]]
 === Command Line Options
 
 The following command line option is specific to Filebeat.

filebeat/docs/getting-started.asciidoc

@@ -18,7 +18,7 @@ After installing the Elastic Stack, read the following topics to learn how to in
 * <<filebeat-starting>>
 * <<filebeat-index-pattern>>
 * <<filebeat-modules-quickstart>>
-* <<filebeat-command-line>>
+* <<command-line-options>>
 * <<directory-layout>>
 
 [[filebeat-installation]]
@@ -27,17 +27,7 @@ After installing the Elastic Stack, read the following topics to learn how to in
 Before running Filebeat, you need to install and configure the Elastic stack. See
 {libbeat}/getting-started.html[Getting Started with Beats and the Elastic Stack].
 
-To download and install Filebeat, use the commands that work with your system
-(<<deb, deb>> for Debian/Ubuntu, <<rpm, rpm>> for Redhat/Centos/Fedora, <<mac,
-mac>> for OS X, and <<win, win>> for Windows).
-
-[NOTE]
-==================================================
-If you use Apt or Yum, you can <<setup-repositories,install Filebeat from our repositories>> to update to the newest version more easily.
-
-See our https://www.elastic.co/downloads/beats/filebeat[download page] for other installation options, such as 32-bit images.
-
-==================================================
+include::../../libbeat/docs/shared-download-and-install.asciidoc[]
 
 [[deb]]
 *deb:*
@@ -96,6 +86,24 @@ tar xzvf filebeat-{version}-darwin-x86_64.tar.gz
 
 endif::[]
 
+[[docker]]
+*docker:*
+
+ifeval::["{release-state}"=="unreleased"]
+
+Version {stack-version} of {beatname_uc} has not yet been released.
+
+endif::[]
+
+ifeval::["{release-state}"!="unreleased"]
+
+["source", "shell", subs="attributes"]
+------------------------------------------------
+docker pull {dockerimage}
+------------------------------------------------
+
+endif::[]
+
 [[win]]
 *win:*
 
@@ -139,15 +147,7 @@ started, you can skip the content in this section, including the remaining
 getting started steps, and go directly to the <<filebeat-modules-quickstart>>
 page. 
 
-To configure Filebeat manually, you edit the configuration file. For rpm and deb,
-you'll find the configuration file at `/etc/filebeat/filebeat.yml`. For mac and
-win, look in the archive that you just extracted. There’s also a full example
-configuration file called `filebeat.full.yml` that shows all non-deprecated
-options.
-
-See the
-{libbeat}/config-file-format.html[Config File Format] section of the
-_Beats Platform Reference_ for more about the structure of the config file.
+include::../../libbeat/docs/shared-configuring.asciidoc[]
 
 Here is a sample of the `filebeat` section of the `filebeat.yml` file. Filebeat uses predefined
 default values for most configuration options.
@@ -223,7 +223,7 @@ include::../../libbeat/docs/shared-template-load.asciidoc[]
 Start Filebeat by issuing the appropriate command for your platform. 
 
 NOTE: If you use an init.d script to start Filebeat on deb or rpm, you can't
-specify command line flags (see <<filebeat-command-line>>). To specify flags,
+specify command line flags (see <<command-line-options>>). To specify flags,
 start Filebeat in the foreground.
 
 *deb:*
@@ -240,6 +240,13 @@ sudo /etc/init.d/filebeat start
 sudo /etc/init.d/filebeat start
 ----------------------------------------------------------------------
 
+*docker:*
+
+["source", "shell", subs="attributes"]
+----------------------------------------------------------------------
+docker run {dockerimage}
+----------------------------------------------------------------------
+
 *mac:*
 
 [source,shell]

filebeat/docs/index.asciidoc

@@ -15,6 +15,7 @@ include::../../libbeat/docs/version.asciidoc[]
 :beatname_lc: filebeat
 :beatname_uc: Filebeat
 :security: X-Pack Security
+:dockerimage: docker.elastic.co/beats/{beatname_lc}:{version}
 
 include::./overview.asciidoc[]
 
@@ -28,6 +29,8 @@ include::../../libbeat/docs/shared-directory-layout.asciidoc[]
 
 include::../../libbeat/docs/repositories.asciidoc[]
 
+include::./running-on-docker.asciidoc[]
+
 include::./upgrading.asciidoc[]
 
 include::./how-filebeat-works.asciidoc[]

filebeat/docs/migration.asciidoc

@@ -304,7 +304,7 @@ options with Logstash Forwarder, make sure that you add your options to the
 configuration file. For naming changes, see <<renamed-options>>.
 
 Filebeat does provide command line options that are common to all Beats. For more details about
-these options, see <<filebeat-command-line>>.
+these options, see <<command-line-options>>.
 
 [[renamed-options]]
 [float]

filebeat/docs/reference/configuration/filebeat-options.asciidoc

@@ -388,11 +388,11 @@ somewhat from the patterns supported by Logstash. See <<regexp-support>> for a l
 +
 [options="header"]
 |=======================
-|Setting for `negate` | Setting for `match` | Result
-|`false`              | `after`             | Consecutive lines that match the pattern are appended to the previous line that doesn't match.
-|`false`              | `before`            | Consecutive lines that match the pattern are prepended to the next line that doesn't match.
-|`true`               | `after`             | Consecutive lines that don't match the pattern are appended to the previous line that does match.
-|`true`               | `before`            | Consecutive lines that don't match the pattern are prepended to the next line that does match.
+|Setting for `negate` | Setting for `match` | Result | Example `pattern: ^b`
+|`false`              | `after`             | Consecutive lines that match the pattern are appended to the previous line that doesn't match. | image:./images/false-after-multi.png[Lines a b b c b b become "abb" and "cbb"]
+|`false`              | `before`            | Consecutive lines that match the pattern are prepended to the next line that doesn't match. | image:./images/false-before-multi.png[Lines b b a b b c become "bba" and "bbc"]
+|`true`               | `after`             | Consecutive lines that don't match the pattern are appended to the previous line that does match. | image:./images/true-after-multi.png[Lines b a c b d e become "bac" and "bde"]
+|`true`               | `before`            | Consecutive lines that don't match the pattern are prepended to the next line that does match. | image:./images/true-before-multi.png[Lines a c b d e b become "acb" and "deb"]
 |=======================
 +
 NOTE: The `after` setting is equivalent to `previous` in https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html[Logstash], and `before` is equivalent to `next`.

filebeat/docs/running-on-docker.asciidoc

@@ -0,0 +1 @@
+include::../../libbeat/docs/shared-docker.asciidoc[]

heartbeat/docs/command-line.asciidoc

@@ -1,4 +1,4 @@
-[[heartbeat-command-line]]
+[[command-line-options]]
 === Command Line Options
 
 Heartbeat does not have any Heartbeat-specific command line options.

heartbeat/docs/getting-started.asciidoc

@@ -18,7 +18,7 @@ install, configure, and run Heartbeat:
 * <<heartbeat-configuration>>
 * <<heartbeat-template>>
 * <<heartbeat-starting>>
-* <<heartbeat-command-line>>
+* <<command-line-options>>
 * <<directory-layout>>
 
 
@@ -32,17 +32,7 @@ monitor are running.
 
 //TODO: Add a separate topic that explores deployment scenarios in more detail (like installing on a sub-network where there's a firewall etc.
 
-To download and install Heartbeat, use the commands that work with your
-system (<<deb, deb>> for Debian/Ubuntu, <<rpm, rpm>> for Redhat/Centos/Fedora,
-<<mac,mac>> for OS X, and <<win, win>> for Windows).
-
-[NOTE]
-==================================================
-If you use Apt or Yum, you can <<setup-repositories,install Heartbeat from our repositories>> to update to the newest version more easily.
-
-See our https://www.elastic.co/downloads/beats/heartbeat[download page] for other installation options, such as 32-bit images.
-
-==================================================
+include::../../libbeat/docs/shared-download-and-install.asciidoc[]
 
 [[deb]]
 *deb:*
@@ -101,6 +91,25 @@ tar xzvf heartbeat-{version}-darwin-x86_64.tar.gz
 
 endif::[]
 
+
+[[docker]]
+*docker:*
+
+ifeval::["{release-state}"=="unreleased"]
+
+Version {stack-version} of {beatname_uc} has not yet been released.
+
+endif::[]
+
+ifeval::["{release-state}"!="unreleased"]
+
+["source", "shell", subs="attributes"]
+------------------------------------------------
+docker pull {dockerimage}
+------------------------------------------------
+
+endif::[]
+
 [[win]]
 *win:*
 
@@ -147,15 +156,7 @@ options, see <<heartbeat-configuration-details>>.
 [[heartbeat-configuration]]
 === Step 2: Configuring Heartbeat
 
-To configure Heartbeat, you edit the configuration file. For rpm and deb,
-you'll find the configuration file at +/etc/heartbeat/heartbeat.yml+.
-For mac and win, look in the archive that you just extracted. There’s also a
-full example configuration file called `heartbeat.full.yml` that shows all
-non-deprecated options.
-
-See the
-{libbeat}/config-file-format.html[Config File Format] section of the
-_Beats Platform Reference_ for more about the structure of the config file.
+include::../../libbeat/docs/shared-configuring.asciidoc[]
 
 Heartbeat provides monitors to check the status of hosts at set intervals.
 You configure each monitor individually. Heartbeat currently provides monitors
@@ -233,7 +234,7 @@ include::../../libbeat/docs/shared-template-load.asciidoc[]
 Start Heartbeat by issuing the appropriate command for your platform.
 
 NOTE: If you use an init.d script to start Heartbeat on deb or rpm, you can't
-specify command line flags (see <<heartbeat-command-line>>). To specify flags,
+specify command line flags (see <<command-line-options>>). To specify flags,
 start Heartbeat in the foreground.
 
 *deb:*

heartbeat/docs/index.asciidoc

@@ -15,6 +15,7 @@ include::../../libbeat/docs/version.asciidoc[]
 :beatname_lc: heartbeat
 :beatname_uc: Heartbeat
 :security: X-Pack Security
+:dockerimage: docker.elastic.co/beats/{beatname_lc}:{version}
 
 include::./overview.asciidoc[]
 
@@ -26,6 +27,8 @@ include::../../libbeat/docs/shared-directory-layout.asciidoc[]
 
 include::../../libbeat/docs/repositories.asciidoc[]
 
+include::./running-on-docker.asciidoc[]
+
 //
 //include::./upgrading.asciidoc[]
 
@@ -57,5 +60,5 @@ include::./troubleshooting.asciidoc[]
 
 include::./faq.asciidoc[]
 
-// 
+//
 //include::./heartbeat-devguide.asciidoc[]

heartbeat/docs/running-on-docker.asciidoc

@@ -0,0 +1 @@
+include::../../libbeat/docs/shared-docker.asciidoc[]

libbeat/docs/dashboards.asciidoc

@@ -16,7 +16,7 @@
 
 {beatname_uc} comes packaged with the `scripts/import_dashboards` script that you can use to import the example dashboards,
 visualizations, and searches for {beatname_uc}. The script also creates an index pattern,
-+{beatname_lc}-*+, for {beatname_uc}. 
++{beatname_lc}-*+, for {beatname_uc}.
 
 The steps in this section show how to import {beatname_uc} dashboards. You may want to import dashboards for more than
 one Beat or specify import options that aren't described here. See {libbeat}/import-dashboards.html[Importing Existing Beat Dashboards]
@@ -28,21 +28,28 @@ ifdef::allplatforms[]
 
 *deb, rpm, and mac:*
 
-From the directory where you installed {beatname_uc}, run the `import_dashboards` script. 
+From the directory where you installed {beatname_uc}, run the `import_dashboards` script.
 
 ["source","sh",subs="attributes,callouts"]
 ----------------------------------------------------------------------
 ./scripts/import_dashboards
 ----------------------------------------------------------------------
 
-On deb and rpm, the `scripts` folder is located under the home path, which is +/usr/share/{beatname_lc}/+ unless you change it.
+*docker:*
+
+["source","sh",subs="attributes"]
+----------------------------------------------------------------------
+docker run {dockerimage} ./scripts/import_dashboards
+----------------------------------------------------------------------
+
+On deb, rpm, and docker, the `scripts` folder is located under the home path, which is +/usr/share/{beatname_lc}/+ unless you change it.
 
 By default, the script assumes that you are running Elasticsearch on `127.0.0.1:9200`. Use the `-es` option
-to specify a different location. For example: 
+to specify a different location. For example:
 
 ["source","sh",subs="attributes,callouts"]
 ----------------------------------------------------------------------
-./scripts/import_dashboards -es http://192.168.33.60:9200 
+./scripts/import_dashboards -es http://192.168.33.60:9200
 ----------------------------------------------------------------------
 
 Use the `-user` option to specify the username and password to use for Elasticsearch authentication. There are a few ways to pass
@@ -51,7 +58,7 @@ in the username and password. For example:
 ["source","sh",subs="attributes,callouts"]
 -----------------------------------------------------------------------
 ./scripts/import_dashboards -es https://xyz.found.io -user user -pass password <1>
-./scripts/import_dashboards -es https://xyz.found.io -user admin -pass $(cat ~/pass-file) <2> 
+./scripts/import_dashboards -es https://xyz.found.io -user admin -pass $(cat ~/pass-file) <2>
 -----------------------------------------------------------------------
 
 <1> Specify the username and password as options.
@@ -63,7 +70,7 @@ endif::allplatforms[]
 
 Open a PowerShell prompt as an Administrator (right-click the PowerShell icon
 and select *Run As Administrator*). If you are running Windows XP, you may need
-to download and install PowerShell. 
+to download and install PowerShell.
 
 From the PowerShell prompt, change to the directory where you installed {beatname_uc}, and run the
 `import_dashboards.exe` script:
@@ -103,6 +110,6 @@ pattern is selected to see {beatname_uc} data.
 image:./images/kibana-created-indexes.png[Discover tab with index selected]
 
 To open the loaded dashboards, go to the *Dashboard* page and select the
-dashboard that you want to open. 
+dashboard that you want to open.
 
 image:./images/kibana-navigation-vis.png[Navigation widget in Kibana]

libbeat/docs/loggingconfig.asciidoc

@@ -15,8 +15,8 @@
 
 The `logging` section of the +{beatname_lc}.yml+ config file contains options
 for configuring the Beats logging output. The logging system can write logs to
-syslog or rotate log files. If logging is not explicitly configured, file output
-is used on Windows systems, and syslog output is used on Linux and OS X.
+the syslog or rotate log files. If logging is not explicitly configured, file
+output is used on Windows systems, and syslog output is used on Linux and OS X.
 
 [source,yaml]
 ------------------------------------------------------------------------------
@@ -29,29 +29,50 @@ logging.files:
   keepfiles: 7
 ------------------------------------------------------------------------------
 
-In addition to the logging system, the logging output configuration can be
-modified from the command line.
+TIP: In addition to setting logging options in the config file, you can modify
+the logging output configuration from the command line. See
+<<command-line-options>>.
 
 ==== Logging Options
 
 You can specify the following options in the `logging` section of the +{beatname_lc}.yml+ config file:
 
 ===== to_syslog
 
-If enabled, sends all logging output to syslog. The default
-value is false. 
+When true, writes all logging output to the syslog.
 
 ===== to_files
 
-Writes all logging output to files subject to file rotation. The
-default value is true.
+When true, writes all logging output to files. The log files are automatically
+rotated when the log file size limit is reached.
 
+NOTE: {beatname_uc} only creates a log file if there is logging output. For
+example, if you set the log <<level,`level`>> to `error` and there are no errors,
+there will be no log file in the directory specified for logs.
+
+[[level]]
 ===== level
 
-Minimum log level. One of debug, info, warning, error or critical. If debug is
-used, but no selectors are configured, the `*` selector will be used.
-The default log level is "info".
+Minimum log level. One of `debug`, `info`, `warning`, `error`, or `critical`.
+The default log level is `info`.
+
+`debug`:: Logs debug messages, including a detailed printout of all events
+flushed by the Beat. Also logs informational messages, warnings, errors, and
+critical errors. When the log level is `debug`, you can specify a list of
+<<selectors,`selectors`>> to display debug messages for specific components.
+If no selectors are specified, the `*` selector is used to display debug
+messages for all components.
+
+`info`:: Logs informational messages, including the number of events
+that are published. Also logs any warnings, errors, or critical errors.
+
+`warning`:: Logs warnings, errors, and critical errors.
+
+`error`:: Logs errors and critical errors.
+
+`critical`:: Logs critical errors only.
 
+[[selectors]]
 ===== selectors
 
 The list of debugging-only selector tags used by different Beats components. Use `*`