[alerting event log] deal with migrating event log mappings over releases

[pmuellr]

Each future Kibana release is likely to add additional ECS fields to the event log schema/mapping, and we'll have to deal with updating the mappings stored in the index template when a new release is detected.

Obviously we'll have to take care to not downgrade the mappings if a customer happens to run a new version and then later runs an old version, against the same ES. We're likely to see this in practice at development time, as we change the mappings during development of a release.

One saving grace is that ECS generally never removes fields, only adds them. We'll want the event log to be similar.

Currently the mappings that we're generating use dynamic: "strict" for object properties, but to be more flexible and forgiving, we probably want to use false instead of 'strict'.
In that case, fields in a document that aren't in the mappings are ignored, instead of causing an error.

see: https://github.com/elastic/kibana/blob/master/x-pack/plugins/event_log/generated/mappings.json

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[pmuellr]

I'm still a bit worried with the PR that fixes this, that we'll have problems for Kibana devs when we merge a new event mapping to master. The new mapping will only get picked up when the cached ES in the .es directory gets updated - before that, writes to the eventLog may not work if the event written doesn't match the old mappings.

I think we can fix this by not using "strict" in our object mappings, but "ignore". That way you could write "new" incompatible events to an "old" event log, but of course "new" fields wouldn't be "stored/indexed" (in the ES sense, I think they'd still be in the _source).

[mikecote]

I discussed this with @pmuellr offline and we've decided to postpone doing anything for now until it becomes a problem. I've opened #58518 to track the issue and we can prioritize it separately.