Fix exemplars being added to gauge metrics in the prometheus exporter #5912

trthomps · 2024-10-22T17:33:46Z

Prometheus Gauge metrics don't support exemplars and while addGaugeMetric() doesn't add them, addSumMetric() will if the metric is monotonic. This causes the prometheus client to throw an error:

* error collecting metric Desc{fqName: "http_server_request_body_size_bytes", help: "Measures size of RPC request messages (uncompressed).", constLabels: {}, variableLabels: {net_protocol_name,net_protocol_version,http_method,http_route,http_scheme,net_host_name,net_host_port,otel_scope_name,otel_scope_version}}: cannot inject exemplar into Gauge, Summary or Untyped

linux-foundation-easycla · 2024-10-22T17:33:51Z

The committers listed above are authorized under a signed CLA.

✅ login: trthomps / name: Travis Thompson (7f366df, ab6a89e, b9cdef3, 88cd690, b664503, 0e82181, fdc2cb8, 11b88a7, 2d16850, 4caf61e, e841bbb)
✅ login: dmathieu / name: Damien Mathieu (3700b79)

dashpole · 2024-10-22T17:52:35Z

Would be good to have a test

codecov · 2024-10-22T17:53:57Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.6%. Comparing base (30c4a9a) to head (3700b79).
Report is 1 commits behind head on main.

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #5912   +/-   ##
=====================================
  Coverage   84.6%   84.6%           
=====================================
  Files        272     272           
  Lines      22837   22839    +2     
=====================================
+ Hits       19322   19325    +3     
+ Misses      3171    3170    -1     
  Partials     344     344

see 2 files with indirect coverage changes

trthomps · 2024-10-22T23:08:12Z

Would be good to have a test

@dashpole Added a test, let me know if you know of a better way to test this, I did step through the debugger and exemplars are getting added to the DataPoint.

CHANGELOG.md

exporters/prometheus/exporter.go

exporters/prometheus/exporter_test.go

Co-authored-by: Damien Mathieu <42@dmathieu.com>

Co-authored-by: David Ashpole <dashpole@google.com>

Co-authored-by: Damien Mathieu <42@dmathieu.com>

trthomps · 2024-10-23T17:58:16Z

Thanks so much for working with me on this @dashpole.

exporters/prometheus/exporter_test.go

### Added - Add `go.opentelemetry.io/otel/sdk/metric/exemplar.AlwaysOffFilter`, which can be used to disable exemplar recording. (#5850) - Add `go.opentelemetry.io/otel/sdk/metric.WithExemplarFilter`, which can be used to configure the exemplar filter used by the metrics SDK. (#5850) - Add `ExemplarReservoirProviderSelector` and `DefaultExemplarReservoirProviderSelector` to `go.opentelemetry.io/otel/sdk/metric`, which defines the exemplar reservoir to use based on the aggregation of the metric. (#5861) - Add `ExemplarReservoirProviderSelector` to `go.opentelemetry.io/otel/sdk/metric.Stream` to allow using views to configure the exemplar reservoir to use for a metric. (#5861) - Add `ReservoirProvider`, `HistogramReservoirProvider` and `FixedSizeReservoirProvider` to `go.opentelemetry.io/otel/sdk/metric/exemplar` to make it convenient to use providers of Reservoirs. (#5861) - The `go.opentelemetry.io/otel/semconv/v1.27.0` package. The package contains semantic conventions from the `v1.27.0` version of the OpenTelemetry Semantic Conventions. (#5894) - Add `Attributes attribute.Set` field to `Scope` in `go.opentelemetry.io/otel/sdk/instrumentation`. (#5903) - Add `Attributes attribute.Set` field to `ScopeRecords` in `go.opentelemetry.io/otel/log/logtest`. (#5927) - `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc` adds instrumentation scope attributes. (#5934) - `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp` adds instrumentation scope attributes. (#5934) - `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc` adds instrumentation scope attributes. (#5935) - `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp` adds instrumentation scope attributes. (#5935) - `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc` adds instrumentation scope attributes. (#5933) - `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp` adds instrumentation scope attributes. (#5933) - `go.opentelemetry.io/otel/exporters/prometheus` adds instrumentation scope attributes in `otel_scope_info` metric as labels. (#5932) ### Changed - Support scope attributes and make them as identifying for `Tracer` in `go.opentelemetry.io/otel` and `go.opentelemetry.io/otel/sdk/trace`. (#5924) - Support scope attributes and make them as identifying for `Meter` in `go.opentelemetry.io/otel` and `go.opentelemetry.io/otel/sdk/metric`. (#5926) - Support scope attributes and make them as identifying for `Logger` in `go.opentelemetry.io/otel` and `go.opentelemetry.io/otel/sdk/log`. (#5925) - Make schema URL and scope attributes as identifying for `Tracer` in `go.opentelemetry.io/otel/bridge/opentracing`. (#5931) - Clear unneeded slice elements to allow GC to collect the objects in `go.opentelemetry.io/otel/sdk/metric` and `go.opentelemetry.io/otel/sdk/trace`. (#5804) ### Fixed - Global MeterProvider registration unwraps global instrument Observers, the undocumented Unwrap() methods are now private. (#5881) - `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc` now keeps the metadata already present in the context when `WithHeaders` is used. (#5892) - `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc` now keeps the metadata already present in the context when `WithHeaders` is used. (#5911) - `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc` now keeps the metadata already present in the context when `WithHeaders` is used. (#5915) - Fix `go.opentelemetry.io/otel/exporters/prometheus` trying to add exemplars to Gauge metrics, which is unsupported. (#5912) - Fix `WithEndpointURL` to always use a secure connection when an https URL is passed in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`. (#5944) - Fix `WithEndpointURL` to always use a secure connection when an https URL is passed in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`. (#5944) - Fix `WithEndpointURL` to always use a secure connection when an https URL is passed in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`. (#5944) - Fix `WithEndpointURL` to always use a secure connection when an https URL is passed in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`. (#5944) - Fix incorrect metrics generated from callbacks when multiple readers are used in `go.opentelemetry.io/otel/sdk/metric`. (#5900) ### Removed - Remove all examples under `go.opentelemetry.io/otel/example` as they are moved to [Contrib repository](https://github.com/open-telemetry/opentelemetry-go-contrib/tree/main/examples). (#5930)

trthomps requested review from MrAlias, XSAM, dashpole, pellared and dmathieu as code owners October 22, 2024 17:33

trthomps marked this pull request as draft October 22, 2024 17:34

trthomps added 2 commits October 22, 2024 10:38

Fix exemplars being added to gauge metrics in the prometheus exporter

b664503

update changelog with PR id

7f366df

trthomps force-pushed the fix-prometheus-gauge-exemplars branch from e713527 to 7f366df Compare October 22, 2024 17:38

Add tests

2d16850

trthomps force-pushed the fix-prometheus-gauge-exemplars branch from 45c3a10 to 2d16850 Compare October 22, 2024 20:27

trthomps marked this pull request as ready for review October 22, 2024 20:27

Merge branch 'main' into fix-prometheus-gauge-exemplars

11b88a7

Merge branch 'main' into fix-prometheus-gauge-exemplars

fdc2cb8

dmathieu reviewed Oct 23, 2024

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

exporters/prometheus/exporter.go Outdated Show resolved Hide resolved

exporters/prometheus/exporter_test.go Outdated Show resolved Hide resolved

dashpole reviewed Oct 23, 2024

View reviewed changes

exporters/prometheus/exporter_test.go Outdated Show resolved Hide resolved

trthomps and others added 4 commits October 23, 2024 10:54

Update exporters/prometheus/exporter.go

0e82181

Co-authored-by: Damien Mathieu <42@dmathieu.com>

Update exporters/prometheus/exporter_test.go

88cd690

Co-authored-by: Damien Mathieu <42@dmathieu.com>

Update exporters/prometheus/exporter_test.go

b9cdef3

Co-authored-by: David Ashpole <dashpole@google.com>

Update CHANGELOG.md

e841bbb

Co-authored-by: Damien Mathieu <42@dmathieu.com>

dashpole reviewed Oct 23, 2024

View reviewed changes

exporters/prometheus/exporter_test.go Show resolved Hide resolved

dashpole reviewed Oct 23, 2024

View reviewed changes

exporters/prometheus/exporter_test.go Outdated Show resolved Hide resolved

dashpole reviewed Oct 23, 2024

View reviewed changes

exporters/prometheus/exporter_test.go Outdated Show resolved Hide resolved

trthomps added 2 commits October 23, 2024 11:45

don't use trace sdk, rename test data file

ab6a89e

Merge branch 'main' into fix-prometheus-gauge-exemplars

4caf61e

dashpole approved these changes Oct 23, 2024

View reviewed changes

dmathieu approved these changes Oct 24, 2024

View reviewed changes

Merge branch 'main' into fix-prometheus-gauge-exemplars

3700b79

dmathieu merged commit 664a075 into open-telemetry:main Oct 24, 2024
32 checks passed

trthomps deleted the fix-prometheus-gauge-exemplars branch October 24, 2024 16:30

MrAlias added this to the v1.32.0 milestone Nov 7, 2024

pellared mentioned this pull request Nov 8, 2024

Release v1.32.0/v0.54.0/v0.8.0/v0.0.11 #5960

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix exemplars being added to gauge metrics in the prometheus exporter #5912

Fix exemplars being added to gauge metrics in the prometheus exporter #5912

trthomps commented Oct 22, 2024

linux-foundation-easycla bot commented Oct 22, 2024 •

edited

Loading

dashpole commented Oct 22, 2024

codecov bot commented Oct 22, 2024 •

edited

Loading

trthomps commented Oct 22, 2024

trthomps commented Oct 23, 2024

Fix exemplars being added to gauge metrics in the prometheus exporter #5912

Fix exemplars being added to gauge metrics in the prometheus exporter #5912

Conversation

trthomps commented Oct 22, 2024

linux-foundation-easycla bot commented Oct 22, 2024 • edited Loading

dashpole commented Oct 22, 2024

codecov bot commented Oct 22, 2024 • edited Loading

Codecov Report

trthomps commented Oct 22, 2024

trthomps commented Oct 23, 2024

linux-foundation-easycla bot commented Oct 22, 2024 •

edited

Loading

codecov bot commented Oct 22, 2024 •

edited

Loading