Filebeat Azure - Add missing var definitions in the manifest files

[narph]

Should fix #16270.

Due to new checks on the configuration file keys in v7.6.0, when running:

filebeat setup --module azure

users get:

Exiting: Error getting config for fileset azure/activitylogs: Error interpreting the template of the input: template: text:2:22: executing "text" at <.connection_string>: map has no entry for key "connection_string"

due to missing to define some of the vars in the manifest file.
This PR does this.

[kaiyan-sheng]

nit: extra line here.

[exekias]

Thank you for fixing this so quickly! I think we should have a system test to make sure this doesn’t happen again. This would probably caught it, right? https://github.com/elastic/beats/blob/master/filebeat/tests/system/test_setup.py, probably we can load it also from x-pack Filebeat.

Please update the description, even if the original issue has lots of details, it’s nice to have a good summary here explaining the change.

[kaiyan-sheng]

@narph Under configs in each azure fileset like https://github.com/elastic/beats/blob/master/x-pack/filebeat/module/azure/activitylogs/config/azure-eventhub.yml, the parameters that are not required (like connection_string, storage_account) needs to be checked to see if they are defined first. For example,

{{ if .connection_string }}
connection_string: {{ .connection_string }}
{{ end }}

Please see Andrew's comment in a similar PR: #15656 (comment)

[andrewkroh]

@exekias I figured that the existing module tests would have caught this case. But what I realized is that because we only use the file input in tests that the separate config template azure-eventhub.yml is never exercised in tests. So we'd need a test that minimally instantiates the module with that input. Just enough to verify that it can start.

Or maybe the the fileset package can expose an easy to use way to evaluate a module's config templates. Then you could run assertions on the generated config output based on various input settings.

[exekias]

SGTM, let's get this in for the next release and work on the testing part later

[narph]

@narph Under configs in each azure fileset like https://github.com/elastic/beats/blob/master/x-pack/filebeat/module/azure/activitylogs/config/azure-eventhub.yml, the parameters that are not required (like connection_string, storage_account) needs to be checked to see if they are defined first. For example,

{{ if .connection_string }}
connection_string: {{ .connection_string }}
{{ end }}

Please see Andrew's comment in a similar PR: #15656 (comment)

@kaiyan-sheng , all the 3 params are required in the azure module for each fileset in this case. Two of these options might not be mandatory in the azure-eventhub input in the future so I would not add the {{ if }} in order for the template to fail.

[narph]

jenkins test this

[narph]

jenkins test this