Skip to content

Commit

Permalink
Add configuration files for JMX support. (#817)
Browse files Browse the repository at this point in the history
*Description of changes:* 

### 1. rules/*.yaml
The [JMX Metric
Insight](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/jmx-metrics/javaagent/README.md)
and the [JMX Metric
Gatherer](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/jmx-metrics)
have different pre-defined target systems (activemq, tomcat, etc.) and
even different metrics within each shared target system
(open-telemetry/opentelemetry-java-instrumentation#9765).
This is an effort to unify the metrics being generated by deferring to
the Gatherer's definition files.

By placing it in the same path as the JMX Metric Insight resources, the
rule files can be overwritten and referred to
[directly](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/jmx-metrics/javaagent/src/main/java/io/opentelemetry/instrumentation/javaagent/jmx/JmxMetricInsightInstaller.java#L55).
```
> gradlew build
> jar tf otelagent/build/libs/aws-opentelemetry-agent-1.33.0-SNAPSHOT.jar | grep jmx
inst/META-INF/io/opentelemetry/instrumentation/io.opentelemetry.jmx-metrics.properties
inst/io/opentelemetry/javaagent/shaded/instrumentation/javaagent/jmx/
inst/io/opentelemetry/javaagent/shaded/instrumentation/javaagent/jmx/JmxMetricInsightInstaller.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/AttributeInfo.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/BeanAttributeExtractor.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/BeanFinder$1.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/BeanFinder.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/BeanGroup.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/DetectionStatus.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/JmxMetricInsight.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricAttribute.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricAttributeExtractor.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricConfiguration.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricDef.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricExtractor.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricInfo$Type.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricInfo.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricRegistrar$1.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/engine/MetricRegistrar.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/yaml/
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/yaml/JmxConfig.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/yaml/JmxRule.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/yaml/Metric.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/yaml/MetricStructure.classdata
inst/io/opentelemetry/javaagent/shaded/instrumentation/jmx/yaml/RuleParser.classdata
inst/jmx/
inst/jmx/rules/
inst/jmx/rules/activemq.yaml
inst/jmx/rules/hadoop.yaml
inst/jmx/rules/jetty.yaml
inst/jmx/rules/kafka-broker.yaml
inst/jmx/rules/tomcat.yaml
inst/jmx/rules/wildfly.yaml
inst/jmx/
inst/jmx/rules/
inst/jmx/rules/jvm.yaml
inst/jmx/rules/kafka-consumer.yaml
inst/jmx/rules/kafka-producer.yaml
inst/jmx/rules/kafka.yaml
inst/jmx/rules/tomcat.yaml
inst/jmx/view.yaml
```
This makes it easier for users to specify the target systems via
environment variables.
```
OTEL_JMX_TARGET_SYSTEM: jvm, tomcat
```

This results in
```
petclinic [otel.javaagent 2024-06-14 20:27:36:575 +0000] [main] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.yaml.RuleParser - jvm: found 5 metric rules
petclinic [otel.javaagent 2024-06-14 20:27:36:607 +0000] [main] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.yaml.RuleParser - tomcat: found 6 metric rules
petclinic [otel.javaagent 2024-06-14 20:27:37:731 +0000] [pool-8-thread-1] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.engine.MetricRegistrar - Created Gauge for jvm.classes.loaded
petclinic [otel.javaagent 2024-06-14 20:27:37:738 +0000] [pool-8-thread-1] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.engine.MetricRegistrar - Created Counter for jvm.gc.collections.count
petclinic [otel.javaagent 2024-06-14 20:27:37:739 +0000] [pool-8-thread-1] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.engine.MetricRegistrar - Created Counter for jvm.gc.collections.elapsed
petclinic [otel.javaagent 2024-06-14 20:27:37:740 +0000] [pool-8-thread-1] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.engine.MetricRegistrar - Created Gauge for jvm.memory.heap
petclinic [otel.javaagent 2024-06-14 20:27:37:740 +0000] [pool-8-thread-1] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.engine.MetricRegistrar - Created Gauge for jvm.memory.nonheap
petclinic [otel.javaagent 2024-06-14 20:27:37:745 +0000] [pool-8-thread-1] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.engine.MetricRegistrar - Created Gauge for jvm.memory.pool.usage
petclinic [otel.javaagent 2024-06-14 20:27:37:745 +0000] [pool-8-thread-1] INFO io.opentelemetry.javaagent.shaded.instrumentation.jmx.engine.MetricRegistrar - Created Gauge for jvm.threads.count
```


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
  • Loading branch information
jefchien authored Oct 10, 2024
1 parent 4077ee0 commit 51098f9
Show file tree
Hide file tree
Showing 5 changed files with 510 additions and 5 deletions.
37 changes: 32 additions & 5 deletions instrumentation/jmx-metrics/src/main/resources/jmx/rules/jvm.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ rules:
LoadedClassCount:
metric: loaded
desc: Number of loaded classes
- bean: java.lang:type=GarbageCollector,*
- bean: java.lang:type=GarbageCollector,name=*
prefix: jvm.gc.collections.
type: counter
metricAttribute:
Expand All @@ -27,13 +27,31 @@ rules:
prefix: jvm.memory.
type: gauge
mapping:
HeapMemoryUsage.init:
metric: heap.init
desc: The initial amount of memory that the JVM requests from the operating system for the heap
HeapMemoryUsage.used:
metric: heap.used
desc: The current heap usage
HeapMemoryUsage.committed:
metric: heap.committed
desc: The amount of memory that is guaranteed to be available for the heap
HeapMemoryUsage.max:
metric: heap.max
desc: The maximum amount of memory can be used for the heap
NonHeapMemoryUsage.init:
metric: nonheap.init
desc: The initial amount of memory that the JVM requests from the operating system for non-heap purposes
NonHeapMemoryUsage.used:
metric: nonheap.used
desc: The current non-heap usage
- bean: java.lang:type=MemoryPool,*
desc: The current non-heap memory usage
NonHeapMemoryUsage.committed:
metric: nonheap.committed
desc: The amount of memory that is guaranteed to be available for non-heap purposes
NonHeapMemoryUsage.max:
metric: nonheap.max
desc: The maximum amount of memory can be used for non-heap purposes
- bean: java.lang:type=MemoryPool,name=*
unit: by
prefix: jvm.memory.pool.
type: gauge
Expand All @@ -43,9 +61,18 @@ rules:
CollectionUsage.used:
metric: used_after_last_gc
desc: Memory used after the most recent gc event
Usage.init:
metric: init
desc: The initial amount of memory that the JVM requests from the operating system for the memory pool
Usage.used:
metric: used
desc: Current memory pool used
desc: The current memory pool memory usage
Usage.committed:
metric: committed
desc: The amount of memory that is guaranteed to be available for the memory pool
Usage.max:
metric: max
desc: The maximum amount of memory can be used for the memory pool
- bean: java.lang:type=Threading
unit: "1"
prefix: jvm.threads.
Expand All @@ -54,7 +81,7 @@ rules:
ThreadCount:
metric: count
desc: Number of threads
- bean: java.lang:type=OperatingSystem,*
- bean: java.lang:type=OperatingSystem
prefix: jvm.cpu.
type: gauge
mapping:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
rules:
- bean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id=*
metricAttribute:
client-id: param(client-id)
mapping:
fetch-rate:
metric: kafka.consumer.fetch-rate
type: gauge
desc: The number of fetch requests for all topics per second
unit: "1"
records-lag-max:
metric: kafka.consumer.records-lag-max
type: gauge
desc: Number of messages the consumer lags behind the producer
unit: "1"
bytes-consumed-rate:
metric: kafka.consumer.total.bytes-consumed-rate
type: gauge
desc: The average number of bytes consumed for all topics per second
unit: by
fetch-size-avg:
metric: kafka.consumer.total.fetch-size-avg
type: gauge
desc: The average number of bytes fetched per request for all topics
unit: by
records-consumed-rate:
metric: kafka.consumer.total.records-consumed-rate
type: gauge
desc: The average number of records consumed for all topics per second
unit: "1"
- bean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id=*,topic=*
metricAttribute:
client-id: param(client-id)
topic: param(topic)
mapping:
bytes-consumed-rate:
metric: kafka.consumer.bytes-consumed-rate
type: gauge
desc: The average number of bytes consumed per second
unit: by
fetch-size-avg:
metric: kafka.consumer.fetch-size-avg
type: gauge
desc: The average number of bytes fetched per request
unit: by
records-consumed-rate:
metric: kafka.consumer.records-consumed-rate
type: gauge
desc: The average number of records consumed per second
unit: "1"
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
rules:
- bean: kafka.producer:type=producer-metrics,client-id=*
metricAttribute:
client-id: param(client-id)
mapping:
io-wait-time-ns-avg:
metric: kafka.producer.io-wait-time-ns-avg
type: gauge
desc: The average length of time the I/O thread spent waiting for a socket ready for reads or writes
unit: ns
outgoing-byte-rate:
metric: kafka.producer.outgoing-byte-rate
type: gauge
desc: The average number of outgoing bytes sent per second to all servers
unit: by
request-latency-avg:
metric: kafka.producer.request-latency-avg
type: gauge
desc: The average request latency
unit: ms
request-rate:
metric: kafka-producer.request-rate
type: gauge
desc: The average number of requests sent per second
unit: "1"
response-rate:
metric: kafka.producer.response-rate
type: gauge
desc: Responses received per second
unit: "1"
- bean: kafka.producer:type=producer-topic-metrics,client-id=*,topic=*
metricAttribute:
client-id: param(client-id)
topic: param(topic)
mapping:
byte-rate:
metric: kafka.producer.byte-rate
type: gauge
desc: The average number of bytes sent per second for a topic
unit: by
compression-rate:
metric: kafka.producer.compression-rate
type: gauge
desc: The average compression rate of record batches for a topic
unit: "1"
record-error-rate:
metric: kafka.producer.record-error-rate
type: gauge
desc: The average per-second number of record sends that resulted in errors for a topic
unit: "1"
record-retry-rate:
metric: kafka.producer.record-retry-rate
type: gauge
desc: The average per-second number of retried record sends for a topic
unit: "1"
record-send-rate:
metric: kafka.producer.record-send-rate
type: gauge
desc: The average number of records sent per second for a topic
unit: "1"
Loading

0 comments on commit 51098f9

Please sign in to comment.