Skip to content

Commit

Permalink
Log by default to journald on systems with systemd (#8942)
Browse files Browse the repository at this point in the history
Add `-e` flag to command line when running beats from systemd so beats
logs are collected by journald by default in systems with systemd.

Add also some environment variables so command line flags can be
more easily overriden with a drop-in unit or with `systemctl edit`.

Add documentation about the use of Beats with systemd.
  • Loading branch information
jsoriano authored Dec 9, 2018
1 parent 2a9c217 commit 07d4f7a
Show file tree
Hide file tree
Showing 10 changed files with 132 additions and 1 deletion.
1 change: 1 addition & 0 deletions CHANGELOG-developer.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ The list below covers the major changes between 6.3.0 and master only.
used because the value is specified in magefile.go. {pull}7670[7670]
- Outputs must implement String. {pull}6404[6404]
- Renamed `-beat-name` CLI option used in `kibana_index_pattern.go` to `-beat` for consistency with other scripts in `dev-tools/cmd`. {pull}8615[8615]
- Systemd unit file template used on Linux packaging now includes environment variables to ease flag overriding. One of them includes the `-e` flag, making beats log to stderr by default on systemd uses. {pull}8942[8942]
- Removed dashboards and index patterns generation for Kibana 5. {pull}8927[8927]
- Move generator packages of Filebeat from `scripts/generator` to `generator`. {pull}9147[9147]

Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ https://github.com/elastic/beats/compare/v6.5.0...v7.0.0-alpha1[View commits]
*Affecting all Beats*
- Dissect syntax change, use * instead of ? when working with field reference. {issue}8054[8054]
- On systems with systemd, the Beats log is now written to journald by default rather than file. To revert this behaviour override BEAT_LOG_OPTS with an empty value. {pull}8942[8942].
- Removed dashboards and index patterns generation for Kibana 5. {pull}8927[8927]
*Auditbeat*
Expand Down
4 changes: 4 additions & 0 deletions auditbeat/docs/setting-up-running.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ This section includes additional information on how to set up and run

* <<running-on-kubernetes>>

* <<running-with-systemd>>


//MAINTAINERS: If you add a new file to this section, make sure you update the bulleted list ^^ too.

Expand All @@ -35,4 +37,6 @@ include::./running-on-docker.asciidoc[]

include::./running-on-kubernetes.asciidoc[]

include::../../libbeat/docs/shared-systemd.asciidoc[]

include::../../libbeat/docs/shared-shutdown.asciidoc[]
5 changes: 4 additions & 1 deletion dev-tools/packaging/templates/linux/systemd.unit.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@ Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/usr/share/{{.BeatName}}/bin/{{.BeatName}} -c /etc/{{.BeatName}}/{{.BeatName}}.yml -path.home /usr/share/{{.BeatName}} -path.config /etc/{{.BeatName}} -path.data /var/lib/{{.BeatName}} -path.logs /var/log/{{.BeatName}}
Environment=BEAT_LOG_OPTS="-e"
Environment=BEAT_CONFIG_OPTS="-c /etc/{{.BeatName}}/{{.BeatName}}.yml"
Environment=BEAT_PATH_OPTS="-path.home /usr/share/{{.BeatName}} -path.config /etc/{{.BeatName}} -path.data /var/lib/{{.BeatName}} -path.logs /var/log/{{.BeatName}}"
ExecStart=/usr/share/{{.BeatName}}/bin/{{.BeatName}} $BEAT_LOG_OPTS $BEAT_CONFIG_OPTS $BEAT_PATH_OPTS
Restart=always

[Install]
Expand Down
4 changes: 4 additions & 0 deletions filebeat/docs/setting-up-running.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ This section includes additional information on how to set up and run

* <<running-on-kubernetes>>

* <<running-with-systemd>>


//MAINTAINERS: If you add a new file to this section, make sure you update the bulleted list ^^ too.

Expand All @@ -35,4 +37,6 @@ include::./running-on-docker.asciidoc[]

include::./running-on-kubernetes.asciidoc[]

include::../../libbeat/docs/shared-systemd.asciidoc[]

include::../../libbeat/docs/shared-shutdown.asciidoc[]
4 changes: 4 additions & 0 deletions heartbeat/docs/setting-up-running.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ This section includes additional information on how to set up and run

* <<running-on-docker>>

* <<running-with-systemd>>

//MAINTAINERS: If you add a new file to this section, make sure you update the bulleted list ^^ too.

include::../../libbeat/docs/shared-directory-layout.asciidoc[]
Expand All @@ -30,4 +32,6 @@ include::../../libbeat/docs/command-reference.asciidoc[]

include::./running-on-docker.asciidoc[]

include::../../libbeat/docs/shared-systemd.asciidoc[]

include::../../libbeat/docs/shared-shutdown.asciidoc[]
3 changes: 3 additions & 0 deletions journalbeat/docs/setting-up-running.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ This section includes additional information on how to set up and run
* <<directory-layout>>
* <<keystore>>
* <<command-line-options>>
* <<running-with-systemd>>
* <<shutdown>>


Expand All @@ -28,4 +29,6 @@ include::../../libbeat/docs/keystore.asciidoc[]

include::../../libbeat/docs/command-reference.asciidoc[]

include::../../libbeat/docs/shared-systemd.asciidoc[]

include::../../libbeat/docs/shared-shutdown.asciidoc[]
103 changes: 103 additions & 0 deletions libbeat/docs/shared-systemd.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
[[running-with-systemd]]
=== {beatname_uc} and systemd

The DEB and RPM packages include a service unit for Linux systems with
systemd. On these systems, you can manage {beatname_uc} by using the usual
systemd commands.

==== Start and stop {beatname_uc}

Use `systemctl` to start or stop {beatname_uc}:

["source", "sh", subs="attributes"]
------------------------------------------------
systemctl start {beatname_lc}
------------------------------------------------

["source", "sh", subs="attributes"]
------------------------------------------------
systemctl stop {beatname_lc}
------------------------------------------------

By default, the {beatname_uc} service starts automatically when the system
boots. To enable or disable auto start use:

["source", "sh", subs="attributes"]
------------------------------------------------
systemctl enable {beatname_lc}
------------------------------------------------

["source", "sh", subs="attributes"]
------------------------------------------------
systemctl disable {beatname_lc}
------------------------------------------------


==== {beatname_uc} status and logs

To get the service status, use `systemctl`:

["source", "sh", subs="attributes"]
------------------------------------------------
systemctl status {beatname_lc}
------------------------------------------------

Logs are stored by default in journald. To view the Logs, use `journalctl`:

["source", "sh", subs="attributes"]
------------------------------------------------
journalctl -u {beatname_lc}.service
------------------------------------------------

NOTE: The unit file included in the packages sets the `-e` flag by default.
This flag makes {beatname_uc} log to stderr and disables other log outputs.
Systemd stores all output sent to stderr in journald.

[float]
=== Customize systemd unit for {beatname_uc}

The systemd service unit file includes environment variables that you can
override to change the default options.

[cols="<h,<,<m",options="header",]
|=======================================
| Variable | Description | Default value
| BEAT_LOG_OPTS | Log options | `-e`
| BEAT_CONFIG_OPTS | Flags for configuration file path | +-c /etc/{beatname_lc}/{beatname_lc}.yml+
| BEAT_PATH_OPTS | Other paths | +-path.home /usr/share/{beatname_lc} -path.config /etc/{beatname_lc} -path.data /var/lib/{beatname_lc} -path.logs /var/log/{beatname_lc}+
|=======================================

To override these variables, create a drop-in unit file in the
+/etc/systemd/system/{beatname_lc}.service.d+ directory.

For example a file with the following content placed in
+/etc/systemd/system/{beatname_lc}.service.d/debug.conf+
would override `BEAT_LOG_OPTS` to enable debug for Elasticsearch output.

["source", "systemd", subs="attributes"]
------------------------------------------------
[Service]
Environment=BEAT_LOG_OPTS="-e -d elasticsearch"
------------------------------------------------

To use settings from the {beatname_uc} file, empty the environment
variable. For example:

["source", "systemd", subs="attributes"]
------------------------------------------------
[Service]
Environment=BEAT_LOG_OPTS=""
------------------------------------------------

To apply your changes, reload the systemd configuration and restart
the service:

["source", "sh", subs="attributes"]
------------------------------------------------
systemctl daemon-reload
systemctl restart {beatname_lc}
------------------------------------------------

NOTE: It is recommended that you use a configuration management tool to
include drop-in unit files. If you need to add a drop-in manually, use
+systemctl edit {beatname_lc}.service+.
4 changes: 4 additions & 0 deletions metricbeat/docs/setting-up-running.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ This section includes additional information on how to set up and run

* <<running-on-kubernetes>>

* <<running-with-systemd>>

//MAINTAINERS: If you add a new file to this section, make sure you update the bulleted list ^^ too.

include::../../libbeat/docs/shared-directory-layout.asciidoc[]
Expand All @@ -34,4 +36,6 @@ include::./running-on-docker.asciidoc[]

include::./running-on-kubernetes.asciidoc[]

include::../../libbeat/docs/shared-systemd.asciidoc[]

include::../../libbeat/docs/shared-shutdown.asciidoc[]
4 changes: 4 additions & 0 deletions packetbeat/docs/setting-up-running.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ This section includes additional information on how to set up and run

* <<running-on-docker>>

* <<running-with-systemd>>

//MAINTAINERS: If you add a new file to this section, make sure you update the bulleted list ^^ too.

include::../../libbeat/docs/shared-directory-layout.asciidoc[]
Expand All @@ -30,4 +32,6 @@ include::../../libbeat/docs/command-reference.asciidoc[]

include::./running-on-docker.asciidoc[]

include::../../libbeat/docs/shared-systemd.asciidoc[]

include::../../libbeat/docs/shared-shutdown.asciidoc[]

0 comments on commit 07d4f7a

Please sign in to comment.