filebeat unable to parse yaml for processors created by sensu-puppet

[cabecada]

with the latest puppet-filebeat (that installs filebeat 5.x)
commit: adc8045

   class { 'filebeat':
     outputs => {
       'logstash' => {
         'bulk_max_size' => 1024,
         'hosts' => [
           'localhost:5044',
         ],
       },
       'file'     => {
         'path' => '/tmp',
         'filename' => 'filebeat',
         'rotate_every_kb' => 10240,
         'number_of_files' => 2,
       },
     },
     shipper => {
       refresh_topology_freq => 10,
       topology_expire => 15,
       queue_size => 1000,
     },
     logging => {
       to_files => true,
       level => 'debug',
       files => {
         rotateeverybytes => 10485760,
         keepfiles => 7,
       }
     },
     prospectors => {
       'sensu-events-to-logstash' => {
         doc_type => 'json',
         paths    => [
           '/var/log/sensu/sensu-client.log',
         ],
         fields   => {
           payload => 'payload',
           check   => 'check',
         },
         json => {
           message_key     => "message",
           keys_under_root => true,
           add_error_key   => true
         },
         tags     => [
           'tag1',
           'tag2',
           'tag3',
         ]
       }
     },
     processors => [
       { 'name' => 'drop_event', 'when' => { 'or' => [{'equals' => {'http.code' => 200}}, {'equals' => {'http.code' => 300}}] } }
     ]
   }

## puppet agent -t 
Notice: /Stage[main]/Filebeat::Config/File[filebeat.yml]/content:
--- /etc/filebeat/filebeat.yml	2017-01-23 18:26:48.813100221 +0000
+++ /tmp/puppet-file20170124-9115-1o6lngu	2017-01-24 06:29:24.090281278 +0000
@@ -50,8 +50,14 @@
 #           http.code: 200
 #
 processors:
-  - include_fields:
-      fields: ["cpu"]
+  - drop_event:
+      when:
+        ---    ####THIS MAKES THE YAML INVALID?
+          or:
+            - equals:
+                http.code: "200"
+            - equals:
+                http.code: "300"

 #================================ Outputs =====================================


Info: Computing checksum on file /etc/filebeat/filebeat.yml
Info: FileBucket got a duplicate file {md5}4a63a6d27ce4272e1e407aaa6d662d09
Info: /Stage[main]/Filebeat::Config/File[filebeat.yml]: Filebucketed /etc/filebeat/filebeat.yml to puppet with sum 4a63a6d27ce4272e1e407aaa6d662d09
Debug: Executing '/usr/share/filebeat/bin/filebeat -N -configtest -c /etc/filebeat/filebeat.yml20170124-9115-roa5zd'
Error: Execution of '/usr/share/filebeat/bin/filebeat -N -configtest -c /etc/filebeat/filebeat.yml20170124-9115-roa5zd' returned 1: Exiting: error loading config file: yaml: line 55: mapping values are not allowed in this context
Error: /Stage[main]/Filebeat::Config/File[filebeat.yml]/content: change from {md5}4a63a6d27ce4272e1e407aaa6d662d09 to {md5}14888eb836dc71b293df5a107d9d0c75 failed: Execution of '/usr/share/filebeat/bin/filebeat -N -configtest -c /etc/filebeat/filebeat.yml20170124-9115-roa5zd' returned 1: Exiting: error loading config file: yaml: line 55: mapping values are not allowed in this context
Notice: /Stage[main]/Filebeat::Service/Service[filebeat]: Dependency File[filebeat.yml] has failures: true
Warning: /Stage[main]/Filebeat::Service/Service[filebeat]: Skipping because of failed dependencies
Notice: /Stage[main]/Filebeat/Anchor[filebeat::end]: Dependency File[filebeat.yml] has failures: true
Warning: /Stage[main]/Filebeat/Anchor[filebeat::end]: Skipping because of failed dependencies

also the yaml cannot be parsed by std languages also

root@ubuntu1404-2:~# cat /tmp/foo
processors:
  - include_fields:
      fields: ["cpu"]
  - drop_event:
      when:
        ---
          or:
            - equals:
                http.code: "200"
            - equals:
                http.code: "300"
root@ubuntu1404-2:~# ruby -le 'require "yaml"; p YAML.load(File.read("/tmp/foo"))'
/usr/lib/ruby/2.0.0/psych.rb:205:in `parse': (<unknown>): mapping values are not allowed in this context at line 7 column 13 (Psych::SyntaxError)
	from /usr/lib/ruby/2.0.0/psych.rb:205:in `parse_stream'
	from /usr/lib/ruby/2.0.0/psych.rb:153:in `parse'
	from /usr/lib/ruby/2.0.0/psych.rb:129:in `load'
	from -e:1:in `<main>'

[cabecada]

root@ubuntu1404-2:~# ruby --version
ruby 2.0.0p645 (2015-04-13 revision 50299) [x86_64-linux-gnu]

[pcfens]

What version of puppet are you using?

[cabecada]

i guess the --- was not getting deleted correctly.

root@master:/etc/puppet/modules/filebeat# git diff
diff --git a/templates/filebeat5.yml.erb b/templates/filebeat5.yml.erb
index 0bbcded..34dac8b 100644
--- a/templates/filebeat5.yml.erb
+++ b/templates/filebeat5.yml.erb
@@ -2,10 +2,9 @@
   def yaml_indent(conds)
     return_val = []
     tmp_val = conds.to_yaml.split("\n")
-    tmp_val.delete('---')
-
+    tmp_val.delete_if { |x| x.include?("--") }
     tmp_val.each do |val|
-      return_val << "        " + val
+      return_val << "      " + val
     end

     return_val.join("\n")

##now running puppet agent -t in filebeat.yml

processors:
  - drop_event:
      when:
        or:
          - equals:
              http.code: "200"
          - equals:
              http.code: "300"

[cabecada]

root@ubuntu1404-2:~# puppet --version
3.8.6

[pcfens]

Are you using the latest version of the module? A similar issue was fixed in #71

[cabecada]

root@master:/etc/puppet/modules/filebeat# git log --oneline  --since="30 days ago"
adc8045 Remove Ruby 1.9.3/Puppet 3.8 travis tests
e858f2d Add a check to see if processors.params is emtpy before using it.
4cb659d Increment version to v0.10.2
5325483 add close_older and force_close_files within prospector v5 (#77)
1aa78c0 Update rubocopy to match voxpupuli's latest modulesync
2482954 Increment version number
e9f1dd0 Assume that windows paths will have spaces in them (Fixes #75)
a07192a support harvesting symlinks (#74)
ca733c5 Fix typo in changelog diff links
930d8a1 Merge in changes from a previous release
dea9678 Increment version number to v0.10.0
4917f24 Add JSON decoding to the changelog
7fb3d09 Add support for json decoding (filebeat >= 5.0) (#72)
3f9a714 Increment version to 0.9.2
abf7f99 Add close and clean harvesting  (#70)
50409d5 Increment version number to 0.9.1
b553228 Fixing yaml spacing and README tutorials for params.  <--------
77b4a4f Increment version to 0.9.0
1b18504 Don't install development packages when testing
7b86bd1 Lint fixes
6287d6d Clean up tests some more
e5ebb2e Improve testing and attempt to meet voxpupuli's style standards
a38244c Update changelog
b6b8386 Add a note about the registry_path to the readme
25054a2 Update download URL for windows to latest
b636539 Align params.pp file
a6d371e Update the changelog to reflect changes over last few days
194158f Validate config files before restarting services
61cbcce All filebeat templates should include the major version number in them
a12f7ed Fix a regression in filebeat_version that caused warnings before filebeat was installed
1b6b4ef Fix the filebeat_version fact in windows
b0674fb Add Support for Filebeat Processors
2d3d240 Merge pull request #68 from plarivee/add_tags_to_prospector
077be4a Adding support for tags in prospector
root@master:/etc/puppet/modules/filebeat# git branch
* master

[pcfens]

It looks to be a puppet 3 vs. puppet 4 issue since I'm only able to re-create it when I use an older version.

Does the code in your earlier comment fix the issue?

[cabecada]

#81

[cabecada]

https://tickets.puppetlabs.com/browse/PUP-6810 was the issue i was referring to.

[pcfens]

Can you give the latest code in the master branch a try? I've modified the template that was causing problems and I think things are working now. If it works well for you I'll put a new release up on the forge.

[pcfens]

I'm pretty sure this was fixed in v0.10.3, so I'm going to go ahead and close this. If you're still having issues let me know and I can re-open this one.

Starting in v0.11 processors will need to be defined slightly differently than now (this is just a temporary fix).

[cabecada]

sorry i am in between a freeze, hence could not test it as there were other priorities. i'll test it mid next week and let you know.
Thanks for your help.