Add guidelines about the use of add_cloudfoundry_metadata

[jsoriano]

add_cloudfoundry_metadata can be used to enrich events with Cloud Foundry metadata obtained from the APIs using application ids as input.

On recent versions of Cloud Foundry, this information is already included in events coming from loggregator. In these cases the information is collected from there. To reduce the number of calls to the APIs, add_cloudfoundry_metadata does nothing when this information if this is already present in events (see #22150).

There are cases where the information in the events is not updated. This can happen for example when the application changes its name as part of an upgrade, for some unknown reason, loggregator continues including the old data. This seems to be an issue in Cloud Foundry. In these cases it may be interesting to use the API for the enrichment.

Current workaround to force add_cloudfoundry_metadata to fill this info from the APIs is to drop some of the fields included in the events, with a config like this one:

processors:
  - drop_fields:
      fields: ["cloudfoundry.app.name"]
      ignore_missing: true
  - add_cloudfoundry_metadata:
      ...

• Add recommendation about the use of the rolling strategy when updating deployed applications. Metadata in envelopes is properly updated if done this way.
• Add some guidelines about when to force the use add_cloudfoundry_metadata.
• Consider adding an explicit setting to add_cloudfoundry_metadata to overwrite these metadata fields to make it more deterministic without depending on the drop_fields workaround.

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!