Change config defaults

[dedemorton]

Changed rotate_every_kb default because it was incorrectly documented.

[tsg]

Ah, there's a confusion here. We've changed bulk_max_size' default for the Logstash output, where this setting is not documented at all (no idea why). Might be worth for me or @urso (hint) to come up with a longer explanation since this is critical for performance.

[dedemorton]

Whoops. Monica mentioned that you changed the default, and since the setting only appears in one place, I assumed I was changing the correct setting. :-) I'll revert this value to 50 and wait for you to submit a description.

[urso]

So we've got multiple output plugins:

• Elasticsearch
• logstash
• console
• file

The default bulk_max_size is used for all output plugins but elasticsearch, which sets the default to 50.

This options sets maximum number of events that can be combined internally into batches and will be publishable by the output plugins =>

• if beat tries to send single events, the events are collected into batches

• if beat tries to publish large batch of event (bigger bulk_max_size), the batch will be split.

  Bigger batch sizes can improve performance due to ammortizing per event sending overhead. On the other hand to big batch sizes can increase processing time such that queues in logstash/elasticsearch can not be processed -> APIs return errors, connections get killed or publish requests time out. This increases latency and lowers throughput for indexing events.

[dedemorton]

@urso I've created a new doc issue #586 to track the doc changes for bulk_max_size so these comments don't get lost under the outdated diff....