.NET 6 Metrics - How to configure buckets for histograms?

[hannahchan]

Hi,

I'm currently experimenting with collecting metrics using the .NET 6 Metric APIs and exporting them via OTLP to an OpenTelemetry Collector. I can't seem to find any documentation on how to configure bucket sizes for histograms. Is anyone able to share their understanding here on how they work?

Side Note: I've notice that in the AspNetCoreInstrumentation, the buckets sizes for http_server_duration_bucket are 0, 5, 10, 25, 50, 75, 100, 250, 500, 1000 and +Inf. Is there any good reason why these bucket sizes have been chosen and is it possible to change them?

[tomkerkhove]

You can add a view and define the bucket size, for example:

 var configuredMetricProvider = Sdk.CreateMeterProviderBuilder()
                .AddMeter("Sample.*")
                .AddOtlpExporter(options => options.Endpoint = new Uri(collectorUri))
                .AddView("*", new ExplicitBucketHistogramConfiguration { Boundaries = histogramBuckets })
                .Build();

You cannot do this per bucket, unfortunately. See #2755

[cijothomas]

The default is coming from the OTel spec. The bounds are fully configurable with Views API.
See the doc: https://github.com/open-telemetry/opentelemetry-dotnet/tree/main/docs/metrics/customizing-the-sdk#specify-custom-boundaries-for-histogram

[hannahchan]

Thanks. This is the documentation I needed to see. I wasn't aware of views. Is this an OpenTelemetry concept or is it limited to this library?

Reading the linked documentation above, it looks like views could potentially solve another problem I have in another project. Unfortunately that other project isn't .NET. Can some of the responsibilities views has be offloaded to the OpenTelemetry Collector?

Other than histograms, I'm also interested in the ability to filter and drop tags. The goal is to send low cardinality metrics to a monitoring tool like Prometheus and high cardinality metrics to a data lake for offline analysis in a tool like PySpark without instrumenting twice.

[cijothomas]

https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#view Almost everything implemented in this repo is following the Otel spec. Apart from few language specific features, most other languages offer same/similar feature.

You can setup multiple exporters, but unfortunately the Views are tied to MeterProvider, so every exporter gets same View configuration.

[hannahchan]

I'm wondering if anyone else has my use case regarding exporting low/high cardinality metrics at the same time to different destinations.

Where would be the best place to have this discussion?

[cijothomas]

A new discussion for the very specific question/ask in this repo would be the right place.

You can instrument once, but setup 2 MeterProviders, 1 with low cardinality tags paired with Prometheus, another with high cardinality tags paired with xyz exporter.
Alternatively, send all tags from the SDK to OTLP, and have the further aggregation/dropping/etc in Collector.