[Monitoring] Adapt UI code to read from metricbeat-* indices

[chrisronline]

Relates to elastic/beats#19747

In an effort to ensure a smooth migration path to the upcoming Agent, we want to adapt all queries in the UI to handle data living in .monitoring-* indices, or metricbeat-* indices. The Beats team is helping us in adapting each stack module to ship directly into metricbeat-* and we need to handle the UI changes to ensure that the UI will work properly.

Work items

• Update mappings to ensure we only map the fields used by the Stack Monitoring UI -> [Monitoring] Apply minimal mappings to es_archive monitoring data #76008
• Adapt UI code to read ES node_stats from metricbeat-* -> [Monitoring] Read from metricbeat-* for ES node_stats #76015
• Adapt UI code to read ES index_stats from metricbeat-* -> [Monitoring] Ensure Elasticsearch parts of Stack Monitoring work with metricbeat-* indices #90838
• Adapt UI code to read ES indices_stats from metricbeat-* -> [Monitoring] Ensure Elasticsearch parts of Stack Monitoring work with metricbeat-* indices #90838
• Adapt UI code to read ES job_stats from metricbeat-* -> [Monitoring] Ensure Elasticsearch parts of Stack Monitoring work with metricbeat-* indices #90838
• Adapt UI code to read ES shard from metricbeat-* -> [Monitoring] Ensure Elasticsearch parts of Stack Monitoring work with metricbeat-* indices #90838
• Adapt UI code to read ES ccr_stats from metricbeat-* -> [Monitoring] Ensure Elasticsearch parts of Stack Monitoring work with metricbeat-* indices #90838
• Adapt UI code to read Kibana kibana_stats from metricbeat-*
• Adapt UI code to read Logstash logstash_stats from metricbeat-*
• Adapt UI code to read Logstash logstash_settings from metricbeat-*
• Adapt UI code to read Beats beats_stats from metricbeat-*
• Adapt UI code to read Beats beats_state from metricbeat-*
• Ensure telemetry code reads from metricbeat-*
• Ensure security concerns are handled [Monitoring] R&D security changes necessary to migrate to reading from MB indices #90660

[elasticmachine]

Pinging @elastic/stack-monitoring (Team:Monitoring)

[chrisronline]

I have some updates here.

I have written code that will generate the minimal set of mappings necessary for each stack product. I checked my work by modifying all existing es_archive mapping files to use these new mappings, then ensuring all tests pass.

Elasticsearch: https://gist.github.com/chrisronline/a002d91b51f0fdb1beb1aa2af31718b3
Kibana: https://gist.github.com/chrisronline/abcb27a2cca09fd542ef7b042984f005
Beats: https://gist.github.com/chrisronline/abbbbaf723f21c53a2aaaacc9a44000c
Logstash: https://gist.github.com/chrisronline/23922534b97b3a001d227d3e772cdb16

Using the above, we should be able to add the minimal mappings to metricbeat-*.

[chrisronline]

I've updated the above mappings to also account for existing stack monitoring cluster alerts and telemetry code too. There were a few additions as a result