new openmetrics types are not supported for prometheus module

[newly12]

Describe the enhancement:

support new openmetrics types

Describe a specific use case for the enhancement or feature:

take prometheus java library as example, since 0.10.0 there are new openmetrics types supported, as metricbeat sends the header to accept openmetrics format by default, the java metrics exporter responds with new types. i.e.

    164 # TYPE jvm info
    165 # HELP jvm VM version info
    166 jvm_info{runtime="OpenJDK Runtime Environment",vendor="Oracle Corporation",version="1.8.0_282-b08"} 1.0

vs response for Accept: text/plain header

    152 # HELP jvm_info VM version info
    153 # TYPE jvm_info gauge
    154 jvm_info{runtime="OpenJDK Runtime Environment",vendor="Oracle Corporation",version="1.8.0_282-b08",} 1.0

this leads to metrics scrapping/parsing failure.

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[newly12]

quick thought, is it reasonable to set default Accept header as text/plain until the underlying parser updated to support new openmetrics types?

[exekias]

This is a good point, we should look into updating our clients to support new types. Thanks for opening!

About updating the headers, that sounds like a reasonable workaround. I wonder if it has other unexpected drawbacks, like breaking compatibility with other clients

[newly12]

I wonder if it has other unexpected drawbacks, like breaking compatibility with other clients

fair point, do you think anyone should be involved in this discussion?

I am not java expert, found this repo(thanks to open source) and did some tests, I guess text/plain is compatible in both old version and new version for java client.

• update one of them

$ git --no-pager diff client_java/pom.xml 
diff --git a/client_java/pom.xml b/client_java/pom.xml
index a2b51f5..1cac7b5 100644
--- a/client_java/pom.xml
+++ b/client_java/pom.xml
@@ -12,7 +12,7 @@
 
   <properties>
     <version.thorntail>2.3.0.Final</version.thorntail>
-    <version.client_java>0.8.1</version.client_java>
+    <version.client_java>0.10.0</version.client_java>
     <maven.compiler.source>1.8</maven.compiler.source>
     <maven.compiler.target>1.8</maven.compiler.target>
     <failOnMissingWebXml>false</failOnMissingWebXml>

• run ./build.sh and then start containers

docker run --rm -d -p 8081:8080 --network prometheus --name client_java client_java
docker run --rm -d -p 8082:8080 --network prometheus --name mp_metrics mp_metrics
docker run --rm -d -p 8083:8080 --network prometheus --name micrometer micrometer

• check metrics

$ for p in 808{1..3};do echo "===$p==="; curl localhost:$p/metrics -H "Accept: text/plain" -s | promtool check metrics > /dev/null 2>&1; echo $?; done 
===8081===
0
===8082===
3
===8083===
0

$ for p in 808{1..3};do echo "===$p==="; curl localhost:$p/metrics -H "Accept: application/openmetrics-text" -s | promtool check metrics > /dev/null 2>&1; echo $?; done 
===8081===
1
===8082===
1
===8083===
0

As per promtool, exit code 1 is for error, while 3 for problem/warning.