JMX module / input

[exekias]

Metricbeat Module / Dataset release checklist

We already have a Jolokia module, but that requires users to install the jolokia agent to collect metrics from Java applications. Having a native JMX input would remove that barrier and simplify the architecture.

It would be interesting to explore having a similar config layout to the one we have in Jolokia, that could potentially allow us to reuse existing lightweight modules just by changing the input.

We can leverage #18323 to implement this

Modules

For a metricset to go GA, the following criterias should be met:

• Supported versions are documented
• Supported operating systems are documented (if applicable)
• Integration tests exist
• System tests exist
• Automated checks that all fields are documented
• Documentation
• Fields follow ECS and naming conventions
• Dashboards exists (if applicable)
• Kibana Home Tutorial (if applicable)
  • Open PR against Kibana repo with tutorial. Examples can be found here.

Metricbeat module

• Example data.json exists and an automated way to generate it exists (go test -data)
• Test environment in Docker exist for integration tests

[elasticmachine]

Pinging @elastic/integrations-platforms (Team:Platforms)

[exekias]

cc @jsoriano

[jsoriano]

We already have a Jolokia module, but that requires users to install the jolokia agent to collect metrics from Java applications. Having a native JMX input would remove that barrier and simplify the architecture.

Any JMX-based monitoring solution for Java applications is going to require some configuration in the application to be monitored. Basically one of these two things:

1. Install an agent in the JVM to export the metrics.
2. Allow remote JMX connections (what I think is allowed by default for local connections).

Both options are already supported by Jolokia and Metricbeat, Jolokia can be deployed as agent (option 1), or as standalone JMX proxy (option 2). Metricbeat supports querying Jolokia in both cases (proxy since #6475).

We can leverage #18323 to implement this

If we implement #18323 then we can start Jolokia in proxy mode, and continue using the Jolokia module.

Another option is that the Elastic Agent could start Jolokia: if a Jolokia configuration is installed in an Elastic Agent, then it ensures that Jolokia and Metricbeat are running and configures Metricbeat to use Jolokia as proxy. Then we wouldn't need Metricbeat to run anything. A Jolokia proxy can be used to monitor many targets, so only one would be required per host.

I wonder if we really need another module.

[exekias]

If we implement #18323 then we can start Jolokia in proxy mode, and continue using the Jolokia module.

That definitely sounds interesting, I can think of 2 questions we need to answer in order to follow this approach:

• Can we redistribute jolokia with our binaries (license wise)?
• How heavy is it in terms of disk usage & CPU/Mem. I understand jolokia has multiple purposes, so it may have a cost

The alternative to this would be writing a small java app to do this JMX to stdout extraction and ship it with Metricbeat

[jsoriano]

• Can we redistribute jolokia with our binaries (license wise)?

Jolokia is licensed with the Apache License, so I guess we could distribute it.

• How heavy is it in terms of disk usage & CPU/Mem. I understand jolokia has multiple purposes, so it may have a cost

In terms of disk usage it is quite lightweight, the war agent that must be used for proxy mode takes less than 350KB, but it requires a servlet container if it is executed standalone. netty or jetty take about 20MB. If we prepare our own package we could maybe ship it with something like undertow that is much lighter.

In terms of CPU/Mem, the problem can be in the parsing of HTTP+JSON responses, that may be heavier than other protocols or formats. On the other side, it is also a protocol proven to work for this use case, and it has support for bulk requests, specially thought for performance of monitoring solutions.

We also have to take into account the weight of the JVM, if Jolokia is deployed as agent, then this weight is near to zero, if it is deployed as proxy it can monitory several applications with a single JVM, and it could use the JVM installed in the machine for the rest of applications. If we want to distribute also our own java runtime, then the weight goes up. (This paragraph applies to any other solution 🙂 )

The alternative to this would be writing a small java app to do this JMX to stdout extraction and ship it with Metricbeat

We also have to evaluate the effort of doing it from scratch. Though a middle ground approach could be to reuse the parts of Jolokia that fetch the metrics and implement only a non-HTTP exporter. Not sure if it worths it, we'd have to do some benchmarks to see if we reach a point where it works better than HTTP. If it works maybe we can contribute back a different exporter to Jolokia.

There is also a prometheus exporter (Apache License) for JMX metrics, that supports being run independently, but with limitations.

There is also the option of jmxfetch (BSD license), that collects metrics, and seems to send them directly to a cloud endpoint. We could maybe add support for other outputs, so it can send the data to Metricbeat instead. Though there maybe other problems, documentation is close to zero, so not sure how it is configured, and for example it mentions to have a limit of 350 metrics, and I am not sure if one jmxfetch instance can collect metrics from many Java applications, while Jolokia in proxy mode can do it.

Oh, and extra ball, I have just found that the Elastic APM agent also can run as an standalone application, and it can collect some JVM metrics, maybe it can be extended to collect custom metrics.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.