Skip to content

Commit

Permalink
Added breaking changes doc for 6.0
Browse files Browse the repository at this point in the history
Part of elastic#4737, but there are some TODOs left.
  • Loading branch information
Tudor Golubenco committed Aug 10, 2017
1 parent af3ecf9 commit 5062edd
Showing 1 changed file with 134 additions and 1 deletion.
135 changes: 134 additions & 1 deletion libbeat/docs/breaking.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,148 @@
== Breaking changes

As a general rule, we strive to keep backwards compatibility between minor
versions (e.g. 5.x to 5.y) so you can upgrade without any configuration file
versions (e.g. 6.x to 6.y) so you can upgrade without any configuration file
changes, but there are breaking changes between major versions (e.g. 5.x to
6.y).

* <<breaking-changes-6.0>>
* <<breaking-changes-5.0>>

We also list the breaking changes at the top of the <<release-notes,release notes>>
for each version.

[[breaking-changes-6.0]]
=== Breaking changes in 6.0

This section discusses the main changes that you should be aware of if you
upgrade the Beats from version 5.x to 6.x. Please also review the relevant
Breaking Changes sections of the <<release-notes,release notes>>.

// TODO: better link to the consolidated release notes for 6.0.0.


[[breaking-changes-spooler-removed]]
==== Filebeat spooler removed

Version 6.0 comes with a new architecture for the internal pipeline of all
Beats. This architecture refactoring is mostly internal, but one of the more
visible effects is that the Spooler component of Filebeat is removed. The
functionality of the Spooler was similar to the one of the publisher queue from
libbeat (the code shared by all Beats), and the presence of multiple queues
made the performance tuning of Filebeat more complex than it needed to be.

As a result, the following options are removed:

- `spool_size`
- `publish_async`
- `idle_timeout`
- `queue_size`
- `bulk_queue_size`

The first three are specific to Filebeat, while `queue_size` and
`bulk_queue_size` exist in all Beats. If any of these options is set, Filebeat
6.0 will refuse to start.

Instead of the settings above, the `queue.mem` settings are introduced. If you
had to tune the `spool_size` or the `queue_size` before, you might want to tune
the `queue.mem.events` when upgrading. However, it is best to leave the rest of
the `queue.mem` settings to their default values, as they are appropriate for
all loads.

The `publish_async` option (which was deprecated since 5.3) is removed because
the new pipeline already works asynchronously by default.

// TODO: for the above new settings, link to their configuration settings.


[[breaking-changes-types]]
==== Filebeat prospector type and document type changes

The `document_type` setting, from the prospector configuration, was removed
because the `_type` concept is being removed from Elasticsearch. Instead of the
`document_type` setting, you can use a custom field.

This has led also to the rename of the `input_type` configuration setting to
`type`. This change is backwards compatible because the old setting still
works. However, the `input_type` output field was renamed to `prospector_type`.

[[breaking-changes-default-config]]
==== Filebeat default prospector disabled in the configuration file

The default startup behaviour (based on the included sample configuration) of
Filebeat was to read all the files matching the `/var/log/*.log` pattern.
Starting with version 6.0, Filebeat doesn't read any files in its default
configuration. However, you can easily enable the `system` module, for example
with a CLI flag:

["source","sh",subs="attributes"]
----
filebeat --modules=system
----

[[breaking-changes-import-dashboards]]
==== Changes for importing the Kibana dashboards

The `import_dashboards` program, used to load the Kibana dashboards in previous
versions of Beats, is replaced by the `setup` command. For example, the
following command:

["source","sh",subs="attributes"]
----
./scripts/import_dashboards -user elastic -pass changeme
----

Can be replaced with:

["source","sh",subs="attributes"]
----
./filebeat setup -E "output.elasticsearch.username=elastic" -E "output.elasticsearch.password=changeme"
----

Note that the `-E` flags are only required if the Elasticsearch output is not
already configured in the configuration file.

Besides the change in the commands, it's important to note that loading the
Kibana dashboards works differently in the 6.0 version of the stack. Prior to
6.0, the dashboards were inserted directly in the `.kibana` Elasticsearch
index. Starting with 6.0, the Beats use a Kibana server API. This means that
the Beat that loads the dashboards needs direct access to Kibana and that the
Kibana URL needs to be set. The option to set the Kibana URL is
`setup.kibana.host`, which you can set in the configuration file or via the
`-E` CLI flag:


["source","sh",subs="attributes"]
----
./filebeat setup -E "setup.kibana.host=http://kibana-host:5601"
----

The default value for the Kibana host is `localhost:5601`.

[[breaking-changes-filters]]
==== Metricbeat filters renamed to processors

The "local" processors, which are configured at the module level, used to be
called `filters` in Metricbeat, but were offering similar functionality with
the global `processors`. A notable difference between the two was that the
filters accessed fields relatively to the metricset (for example,
`mount_point`), while the processors referred to fields by their fully
qualified name (for example, `system.filesystem.mount_point`).

Starting with version 6.0, the `filters` are renamed to `processors` and they
can access the fields only by using the fully qualified names.

[[breaking-changes-cgo]]
==== Binaries are dynamically compiled against libc

Prior to 6.0, Metricbeat and Packetbeat were compiled using
https://golang.org/cmd/cgo/[Cgo], while Filebeat, Winlogbeat, and Heartbeat
were compiled using the pure Go compiler. One of the side-effects of compiling
with Cgo is that libc is dynamically compiled. Starting with 6.0, all the Beats
are compiled using Cgo and therefore dynamically compiled against libc. This
can reduce the portability of the binaries, but none of the supported platforms
is affected.

[[breaking-changes-5.0]]
=== Breaking changes in 5.0

Expand Down

0 comments on commit 5062edd

Please sign in to comment.