Add close and clean harvesting #70
Conversation
|
|
||
| # When enabling this option, a file handler is closed immediately in case a file can't be found | ||
| # any more. In case the file shows up again later, harvesting will continue at the last known position | ||
| # after scan_frequency. | ||
| #close_removed: true | ||
| <%- if @close_removed -%> | ||
| close_removed: <%= @close_removed %> |
There was a problem hiding this comment.
If close_removed is false then this will be excluded, defaulting to true.
|
|
||
| # Removes the state for file which cannot be found on disk anymore immediately | ||
| #clean_removed: true | ||
| <%- if @clean_removed -%> | ||
| clean_removed: <%= @clean_removed %> |
There was a problem hiding this comment.
The logic for setting this will work the same way that close_removed works, so we should probably improve that a bit here.
|
@pcfens: you were right. Now we are validating that some of those vars are booleans and adding the value into the template without conditional. Moreover puppet will print a notify if the prospector configuration does not make sense (IMHO it makes more sense a notify than a fail because is not puppet who has to stop the execution, I think filebeat is who has to do that). Thoughts? |
|
|
||
| $prospector_template = $filebeat::real_version ? { | ||
| '1' => 'prospector1.yml.erb', | ||
| default => 'prospector5.yml.erb', | ||
| } | ||
|
|
||
| if $close_removed == false and $clean_removed == true { | ||
| notify('If you set close_removed to false, does not make sense to set clean_removed to true. Review the official Filebeat documentation') |
There was a problem hiding this comment.
I'm not sure how much commentary to add about configuration issues - there are a number of conflicting configuration parameters that can be set (e.g. ignore_older has to be greater than close_inactive).
Checking and warning for every conflict would be difficult to maintain, so I'm inclined to leave them out entirely.
There was a problem hiding this comment.
Fair enough. Let's skip these kind of conflicts
This PR allow us to close and clean harvesting parameters within the prospectors. This feature is for filebeat 5.
By defult I kept the filebeat values.