Unify metric collection between JMX Metrics Gatherer and JMX Metrics Insight #9765
Labels
Comments
mackjmr
added
enhancement
4 tasks
bjrara
pushed a commit
to aws-observability/aws-otel-java-instrumentation
that referenced
this issue
Oct 10, 2024
*Description of changes:* ### 1. rules/*.yaml The [JMX Metric Insight](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/jmx-metrics/javaagent/README.md) and the [JMX Metric Gatherer](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/jmx-metrics) have different pre-defined target systems (activemq, tomcat, etc.) and even different metrics within each shared target system (open-telemetry/opentelemetry-java-instrumentation#9765). This is an effort to unify the metrics being generated by deferring to the Gatherer's definition files. By placing it in the same path as the JMX Metric Insight resources, the rule files can be overwritten and referred to [directly](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/jmx-metrics/javaagent/src/main/java/io/opentelemetry/instrumentation/javaagent/jmx/JmxMetricInsightInstaller.java#L55). ``` > gradlew build > jar tf otelagent/build/libs/aws-opentelemetry-agent-1.33.0-SNAPSHOT.jar | grep jmx inst/META-INF/io/opentelemetry/instrumentation/io.opentelemetry.jmx-metrics.properties inst/io/opentelemetry/javaagent/shaded/instrumentation/javaagent/jmx/ inst/io/opentelemetry/javaagent/shaded/instrumentation/javaagent/jmx/JmxMetricInsightInstaller.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/ inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/ inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/AttributeInfo.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/BeanAttributeExtractor.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/BeanFinder$1.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/BeanFinder.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/BeanGroup.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/DetectionStatus.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/JmxMetricInsight.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricAttribute.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricAttributeExtractor.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricConfiguration.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricDef.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricExtractor.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricInfo$Type.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricInfo.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricRegistrar$1.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricRegistrar.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/yaml/ inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/yaml/JmxConfig.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/yaml/JmxRule.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/yaml/Metric.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/yaml/MetricStructure.classdata inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/yaml/RuleParser.classdata inst/jmx/ inst/jmx/rules/ inst/jmx/rules/activemq.yaml inst/jmx/rules/hadoop.yaml inst/jmx/rules/jetty.yaml inst/jmx/rules/kafka-broker.yaml inst/jmx/rules/tomcat.yaml inst/jmx/rules/wildfly.yaml inst/jmx/ inst/jmx/rules/ inst/jmx/rules/jvm.yaml inst/jmx/rules/kafka-consumer.yaml inst/jmx/rules/kafka-producer.yaml inst/jmx/rules/kafka.yaml inst/jmx/rules/tomcat.yaml inst/jmx/view.yaml ``` This makes it easier for users to specify the target systems via environment variables. ``` OTEL_JMX_TARGET_SYSTEM: jvm, tomcat ``` This results in ``` petclinic [otel.javaagent 2024-06-14 20:27:36:575 +0000] [main] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.yaml.RuleParser - jvm: found 5 metric rules petclinic [otel.javaagent 2024-06-14 20:27:36:607 +0000] [main] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.yaml.RuleParser - tomcat: found 6 metric rules petclinic [otel.javaagent 2024-06-14 20:27:37:731 +0000] [pool-8-thread-1] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.engine.MetricRegistrar - Created Gauge for jvm.classes.loaded petclinic [otel.javaagent 2024-06-14 20:27:37:738 +0000] [pool-8-thread-1] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.engine.MetricRegistrar - Created Counter for jvm.gc.collections.count petclinic [otel.javaagent 2024-06-14 20:27:37:739 +0000] [pool-8-thread-1] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.engine.MetricRegistrar - Created Counter for jvm.gc.collections.elapsed petclinic [otel.javaagent 2024-06-14 20:27:37:740 +0000] [pool-8-thread-1] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.engine.MetricRegistrar - Created Gauge for jvm.memory.heap petclinic [otel.javaagent 2024-06-14 20:27:37:740 +0000] [pool-8-thread-1] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.engine.MetricRegistrar - Created Gauge for jvm.memory.nonheap petclinic [otel.javaagent 2024-06-14 20:27:37:745 +0000] [pool-8-thread-1] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.engine.MetricRegistrar - Created Gauge for jvm.memory.pool.usage petclinic [otel.javaagent 2024-06-14 20:27:37:745 +0000] [pool-8-thread-1] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.engine.MetricRegistrar - Created Gauge for jvm.threads.count ``` By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
JMX Metrics Insight was initially built as a replacement to JMX Metric Gatherer for the following reasons (#6131):
The end goal of collecting JMX data is the same, only the approach changes. However, today these options cannot be used as replacements, as the two options collect metrics from different target systems, and and collect different metrics within target systems.
Describe the solution you'd like
Unify metrics collected between the JMX Metrics Insight and JMX Metric Gatherer, meaning:
Keeping both in sync manually is not a good solution, as this requires high maintaining efforts and is error prone. Creating a shared metric package to define target systems and metrics to collect will solve this issue. This will make updating metrics in one place apply to both solutions, and ensure that users have a consistent experience based on the option they choose.
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: