Ensure Kibana Apps gracefully handle index pattern field changes and removal

[mattkime]

Mappings and field lists change and we need to make sure that Kibana handles this gracefully. Field lists are expected to change more frequently with the introduction of runtime fields. Runtime fields may be removed or their types may change, either of which may be disruptive to code that assumes their presence is unchanged.

We should ensure that when a field changes or is missing that the UI remains in a workable state and that the user has a path to completing their task. This is purposefully vague because of the diversity of kibana apps. It intends to be a low bar, "good enough". We do not need to design experiences around this circumstance for the purposes of this issue.

Please note how affected UIs handle changes to runtime fields.

UPDATE: We also need to verify alerting use cases

Kibana App

Owner: @timroes
Investigation: Complete

Lens: Before a visualization renders, field type and existence is checked and an explanatory error message is shown for fields used in aggregations. In the editor the user can fix the visualization (or the runtime field) to make it work again. This is only visible once a user accesses the visualization (either on a dashboard or in the Lens editor), not in the visualization listing. No special check for usage in KQL filters (Elasticsearch simply won't return any data, no error indication)

Rest visualizations: #93921

App Arch

Owner: @mattkime
Investigation: Complete
Notes: CSV reports will have a blank column if a field is missing. Filter bar - If a filter exist that references a removed field, the filter remains functional and can be removed by the user.

Alerting #93501

Owner: @mikecote
Investigation: Not started
Notes

Maps

Owner: @thomasneirynck
Investigation: see #93481
Alerts investigation: Not started

Logs & Metrics

Owner: @weltenwort
Investigation: Not yet started
Alerts investigation: Not started
Notes In process of adopting index patterns

ML #93571

Owner: @peteharverson
Investigation: Complete - see #93571
Alerts investigation: Complete - see #93571
Notes In general ML jobs and Transforms copy over the definitions of the runtime fields to the runtime_mappings of the job / transform config, and will continue to use the definition stored in the job / transform, rather than the one stored in the index pattern.

APM (#94288)

Owner: @sqren
Alerts investigation: Complete
Notes

Stack Monitoring

Owner: @chrisronline
Alerts investigation: done
Notes No index pattern usage

Not Applicable
Telemetry
Core UI
Ingest
Elasticsearch UI
Enterprise search

[pmuellr]

For alerting, we'll be looking at the "stack" alerts that the alerting team provides, but other teams also provide solution-specific alerts, so I think they need to "own" this issue for their alerts.

Specifically, I guess our thinking is that these alerts could run into issues potentially if runtime fields are being accessed by the alerts. Eg, what happens if I create an index threshold alert, which checks a field agg vs a threshold value, that field is a runtime field that exists when the alert is created, but is later changed/deleted.

I don't believe there is any alerting framework enhancement we would provide to handle this ATM, presumably every alert type is responsible for handling errors their queries are making using the existing alerting framework facilities.

So this ends up pulling in the following areas.

As an additional thing for existing "owners" listed above to look at:

• Maps (kind of - their alert IS a "stack" alert (which the alerting team "owns", but the maps team really does all the work)
• Logs & Metrics
• ML

And then these areas current listed as "not applicable" - they are now applicable :-)

• Security
• APM
• Stack Monitoring

[mattkime]

@pmuellr Thanks! Updated the description to reflect your advice.

[elasticmachine]

Pinging @elastic/kibana-app-services (Team:AppServices)

[ymao1]

Alerting investigation complete. Focused on how the UI and stack rule execution was affected by field mapping changes/deletions due to runtime fields.

Created these issues as a result of the investigation:

• [Alerting] Editing stack rules should provide warning if fields have unexpected mapping #95523 - Provide warning when editing stack rules if mapping has been changed/removed
• [Alerting] Do we need to validate field mappings before rule execution? #95520 - Discuss validating field mappings before rule execution
• [Alerting] Log more descriptive error messages when runtime field mappings are updated to be incompatible with original query #95516 - Log better error messages when rule execution fails due to changed mappings

[thomasneirynck]

wrt Maps-app: cf #93481

tl;dr:

• Runtime fields add through the index-pattern UX do not work in Maps (they do work when added to the mapping through the Elasticsearch-API). I will put up a PR with a fix. [Maps] Support query-time runtime fields #95701
• When runtime fields or changed or removed, an error will be shown in the LayerTOC-UX. This is similar to how any other index-mapping change is handled.