[metricbeat:elasticsearch module] Missing jvm.mem.pools metrics from node.stats

[przemek-grzedzielski]

I found that metricbeat v 8.13.4 is not ingesting some of the elasticsearch.node.stats.jvm.mem.* metrics, in particular: elasticsearch.node.stats.jvm.mem.pools.old.max.bytes and elasticsearch.node.stats.jvm.mem.pools.old.used.bytes.

Note that these (and other missing metrics) are listed in the Metricbeat 8.x documentation. They're also present in the mappings:

beats/metricbeat/module/elasticsearch/node_stats/_meta/fields.yml

Line 179 in 46fd21c

- name: used.bytes

beats/metricbeat/module/elasticsearch/node_stats/_meta/fields.yml

Line 164 in 46fd21c

- name: max.bytes

They're also exposed via Elasticsearch APIs as per the node_stats API documentation (also tested that this is the case).

@nkvoll traced the changes back to this commit or this Pull Request.

It seems that these fields have been removed inadvertently and the ask is that they be brought back since a number of services depend on them.

---

For confirmed bugs, please report:

• Version: 8.13.4
• Operating System: Ubuntu 22.04

[consulthys]

@przemek-grzedzielski The actual PR where the elasticsearch.node.stats.jvm.mem.pools.* fields were actually removed is #23901 but the PR doesn't contain any reason why.

My hunch is that those fields have been removed because they were nowhere used in the Stack Monitoring UI, which looking at the SM UI source code seems to confirm this.

Maybe @pickypg has an idea or some background context since he was involved at that time.