Prometheus: Expand UseOpenTelemetryPrometheusScrapingEndpoint extension

[CodeBlanch]

Fixes #2776
Alternative to #2969

Changes

This PR seeks to make the Prometheus middleware registration more flexible.

• You can now pass the path to control when the pipeline will branch.
• You can now pass a predicate to control when the pipeline will branch. For example, custom port.
• You can now pass a callback to configure the branched pipeline. For example, to add authentication.
• Previously if you configured the ScrapeEndpointPath as part of the AddPrometheusExporter call (a la options.AddPrometheusExporter(o => o.ScrapeEndpointPath = "/custom/path")) the middleware would not see that, it will now.

Public API Changes

public static class PrometheusExporterApplicationBuilderExtensions
{
-     public static IApplicationBuilder UseOpenTelemetryPrometheusScrapingEndpoint(this IApplicationBuilder app, MeterProvider meterProvider = null)
+     public static IApplicationBuilder UseOpenTelemetryPrometheusScrapingEndpoint(this IApplicationBuilder app)
+     public static IApplicationBuilder UseOpenTelemetryPrometheusScrapingEndpoint(this IApplicationBuilder app, string path)
+     public static IApplicationBuilder UseOpenTelemetryPrometheusScrapingEndpoint(this IApplicationBuilder app, Func<HttpContext, bool> predicate)
+     public static IApplicationBuilder UseOpenTelemetryPrometheusScrapingEndpoint(this IApplicationBuilder app, MeterProvider meterProvider, Func<HttpContext, bool> predicate, string path, Action<IApplicationBuilder> configureBranchedPipeline)
}

TODOs

• Appropriate CHANGELOG.md updated for non-trivial changes
• Changes in public API reviewed
• Unit tests

[CodeBlanch]

@tillig For your use case (#2969) now you should be able to do...

app.UseOpenTelemetryPrometheusScrapingEndpoint(predicate: ctx => ctx.Connection.LocalPort == internalPort)

[tillig]

So... the challenge with something like this is that it feels really weird to have some of the connection/endpoint settings in options but others specified here.

You also end up with the potential shoot-yourself-in-the-foot setup of:

app.UseOpenTelemetryPrometheusScrapingEndpoint(
  predicate: ctx =>
    ctx.Connection.LocalPort == internalPort && ctx.Request.Path == "/not-the-right-path");

...which then means, again, you run into a weird sort of double-location where you can control the endpoint.

I actually mentioned this option in my original PR comment a bit because the only way you can trust that the predicate is in control is to also ignore the endpoint configuration in the options.

[CodeBlanch]

@tillig What I think we should do is provide an easy way to get up and running that would satisfy the most common use case. The predicate would be the advanced usage.

To add Prometheus in the default ASP.NET Core .NET 6 template we currently have this:

using OpenTelemetry.Metrics;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
builder.Services.AddRazorPages();
builder.Services.AddOpenTelemetryMetrics(o => o.AddPrometheusExporter());

var app = builder.Build();

// Configure the HTTP request pipeline.
if (!app.Environment.IsDevelopment())
{
    app.UseExceptionHandler("/Error");
    // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
    app.UseHsts();
}

app.UseHttpsRedirection();
app.UseStaticFiles();

app.UseRouting();

app.UseAuthorization();

app.MapRazorPages();

app.UseOpenTelemetryPrometheusScrapingEndpoint();

app.Run();

Not too bad! I think for "getting started" that is very nice and concise. One of the reasons why we have ScrapeEndpointPath setting on PrometheusExporterOptions is because we need a safe default to use. Basically, we don't want to force users to do anything if they are fine with the defaults.

I'm guessing the next most common thing users will want to do is change the default path. Right now they can do:

builder.Services.Configure<PrometheusExporterOptions>(o => o.ScrapeEndpointPath = "/admin/metrics");

Or to bind it to configuration:

services.Configure<PrometheusExporterOptions>(this.Configuration.GetSection("Prometheus"));

That second case, where we bind to configuration, is highly compelling! I know folks doing environment variable injection into containers like:

ASPNETCORE_PROMETHEUS__SCRAPEENDPOINTPATH: /internal/metrics

If we don't have the options object and path property, we lose that.

I just updated this PR so you can also now do the path configuration as a one-liner like this:

app.UseOpenTelemetryPrometheusScrapingEndpoint(configure: o => o.ScrapeEndpointPath = "/otel")

All that is left is the super advanced case. Changing ports. Or looking for custom headers. Perhaps having allow-lists by IP address, stuff like that. We could allow the predicate to take full control of the decision making? Something like this...

        public static IApplicationBuilder UseOpenTelemetryPrometheusScrapingEndpoint(
            this IApplicationBuilder app,
            MeterProvider meterProvider = null,
            Func<HttpContext, bool> predicate = null,
            Action<PrometheusExporterOptions> configure = null,
            Action<IApplicationBuilder> configureBranchedPipeline = null)
        {
            if (predicate == null)
            {
                var options = app.ApplicationServices.GetOptions<PrometheusExporterOptions>();

                configure?.Invoke(options);

                string path = options.ScrapeEndpointPath ?? PrometheusExporterOptions.DefaultScrapeEndpointPath;
                if (!path.StartsWith("/"))
                {
                    path = $"/{path}";
                }

                predicate = context => context.Request.Path == path;
            }

            return app.MapWhen(
                predicate,
                builder =>
                {
                    configureBranchedPipeline?.Invoke(builder);
                    builder.UseMiddleware<PrometheusExporterMiddleware>(meterProvider ?? app.ApplicationServices.GetRequiredService<MeterProvider>());
                });
        }

I think some users might have the opposite reaction to that and wonder why ScrapeEndpointPath isn't used in that case, but I'm not opposed to it.

[codecov]

Codecov Report

Merging #3029 (50895b5) into main (33125ac) will increase coverage by 0.04%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #3029      +/-   ##
==========================================
+ Coverage   84.78%   84.82%   +0.04%     
==========================================
  Files         259      259              
  Lines        9294     9313      +19     
==========================================
+ Hits         7880     7900      +20     
+ Misses       1414     1413       -1     

Impacted Files	Coverage Δ
.../PrometheusExporterApplicationBuilderExtensions.cs	100.00% <100.00%> (+14.28%)	⬆️

[tillig]

I think the simple case looks easy but you're probably incurring a lot of longer term pain by not separating the concerns of the routing and the exporter configuration up front. For example, with that configureBranchedPipeline lambda you're assuming that everything needs to run before the Prometheus stuff is attached and nothing runs after. That's the exact problem we had with the Autofac OWIN middleware thing I mentioned in the issue.

You could still get that nice, clean startup case if the path was part of the UseOpenTelemetryPrometheusScrapingEndpoint and not in the options at all, and you'd be able to support the advanced cases without the confusion.

Like you could have

public static IApplicationBuilder UseOpenTelemetryPrometheusScrapingEndpoint(
  this IApplicationBuilder app,
  MeterProvider meterProvider = null,
  string path = "/metrics")
{
  // Do app.Map here, no need for options.
}

public static IApplicationBuilder UseOpenTelemetryPrometheusScrapingEndpoint(
  this IApplicationBuilder app,
  Func<HttpContext, bool> predicate,
  MeterProvider meterProvider = null)
{
  // Do app.MapWhen with the required predicate here, again no need for options.
}

But I suppose as long as I can do what I need to do I'm good. I will admit I'll likely end up creating something like this to enforce the separation of concerns on our end, like a bridge set of options.

public class SimplePrometheusOptions
{
  public int ScrapeResponseCacheDurationMilliseconds { get; set; }
}

public class PrometheusConfigurator : IConfigureOptions<PrometheusExporterOptions>
{
  private IOptions<SimplePrometheusOptions> _simple;

  public PrometheusConfigurator(IOptions<SimplePrometheusOptions> simple)
  {
    this._simple = simple;
  }

  public void Configure(PrometheusExporterOptions options)
  {
    options.StartHttpListener = false;
    options.ScrapeResponseCacheDurationMilliseconds = this._simple.ScrapeResponseCacheDurationMilliseconds;;
  }
}

public static class ServiceCollectionExtensions
{
  public static OptionsBuilder<SimplePrometheusOptions> AddPrometheusOptions(this IServiceCollection services)
  {
    // Add the default options
    services.AddOptions<PrometheusExporterOptions>();
    // Registering the configurator will automatically run when IOptions<PrometheusExporterOptions> is resolved
    services.AddTransient<IConfigureOptions<PrometheusExporterOptions>, PrometheusConfigurator>();
    // Allow more configuration
    return services.AddOptions<SimplePrometheusOptions>();
  }
}

The idea there is I will mask the invalid options from the consumer.

Re: configuration, since environment variables are already part of configuration, I'm also having to write the ability to specify defaults in JSON with overrides using the standard environment variable names. Doing a bind from JSON/config to an object model isn't so easy to figure out if overrides are happening in the environment, and it makes things hard if you're using something like a central config service that defines things using the standard SDK keys.

public class JaegerExporterConfigurator : IConfigureOptions<JaegerExporterOptions>
{
    private readonly IConfiguration _configuration;

    public JaegerExporterConfigurator(IConfiguration configuration)
    {
        this._configuration = configuration ?? throw new ArgumentNullException(nameof(configuration));
    }

    public void Configure(JaegerExporterOptions options)
    {
        if (options == null)
        {
            throw new ArgumentNullException(nameof(options));
        }

        // config.TryGetValue<T> is an extension method that checks to see if a given section exists
        // and only binds the corresponding value if it does. Config can have JSON, Environment, etc.
        // all in one big hierarchical set. Means we can also use things like Azure App Configuration
        // to centrally set endpoints for tracing.
        //
        // I also didn't find any public constants for the environment variables defined in the SDK so
        // I had to add that.
        //
        // All of this is effectively the same as what's in the JaegerExporterOptions constructor but
        // with no ties to config.
        if (this._configuration.TryGetValue<string>(OpenTelemetryEnvironmentVariable.JaegerExporterProtocol, out var protocol))
        {
            // I moved this out to a separate class for testability and because it's internal, so I got to copy/paste it.
            options.Protocol = JaegerExportProtocolMap.Map(protocol);
        }

        if (this._configuration.TryGetValue<string>(OpenTelemetryEnvironmentVariable.JaegerExporterAgentHost, out var agentHost))
        {
            options.AgentHost = agentHost;
        }

        if (this._configuration.TryGetValue<int>(OpenTelemetryEnvironmentVariable.JaegerExporterAgentPort, out var agentPort))
        {
            options.AgentPort = agentPort;
        }

        if (this._configuration.TryGetValue<Uri>(OpenTelemetryEnvironmentVariable.JaegerExporterEndpoint, out var endpoint))
        {
            if (!endpoint.IsAbsoluteUri)
            {
                throw new InvalidOperationException($"Jaeger endpoint must be expressed as an absolute URI. Value '{endpoint}' is not absolute.");
            }

            options.Endpoint = endpoint;
        }
    }
}

Anyway, that's a tangent. On the Prometheus middleware... like I said, if I can do what I need to do, I'll be good, and I'll hide the extra stuff as needed.

[CodeBlanch]

@tillig Thanks for all the help on this!

For example, with that configureBranchedPipeline lambda you're assuming that everything needs to run before the Prometheus stuff is attached and nothing runs after.

Something I did consider. Looking at the PrometheusExporterMiddleware it automatically sends response headers, writes out the body, and never calls the "next"/inner delegate. Practically speaking, you can't really do anything useful after the middleware so I decided that the "before" behavior would be fine. I can think of lots of things (potentially) we might need to register before: authentication, compression, logging, redirecting, error handling, etc. I can't think of anything after. Anything come to mind for you?

• RE: Having multiple extensions

  I like this. I just added overloads to hopefully make the common cases very easy. I had to do a bunch because we have an analyzer enforcing that there aren't multiple methods defined with optional parameters because that breaks backwards compatibility.

  Let me know if you think this will alleviate your need to wrap things.

• RE: Splitting out the options

  On the fence on this one. Having a single options class is nice for the binding from config scenario. Also not sure what will happen with the listener. If it becomes its own assembly, that will basically drive simplifying the options. I think table this for now?

• RE: Jaeger options with environment variables

  Interested in this. Are you able to ping me on Slack? Would like to just pick your brain and understand the challenge a bit more.

[tillig]

I think with this new set of overloads I'll be able to do what I need to do. It still seems like a lot of work to go and own/support long-term to so you can avoid providing a simple app.UseTheMiddleware() sort of extension or making the middleware public, but this would unblock me so I'm good.

You might want to consider having that final extension that takes all the parameters be something that doesn't use default parameter values and instead offer an explicit 0-parameter extension. You could get into a situation later where you need to add more extensions but have a weird "ambiguous match" problem due to something being able to match against the new overload and the one with a bunch of optionals.

Re:

Having a single options class is nice for the binding from config scenario.

and

Jaeger options with environment variables

These are kind of tied together.

I think that the "bind options from config" use case is really uninteresting. Actually, let me be more specific:

• Reading options from config is interesting.
• Binding options from config is not interesting.

There's a big difference here. I'll start in reverse order.

Why isn't binding interesting?

Binding from configuration (so we're talking about the same thing) is when you have a configuration structure (which could be expressed in INI, environment variables, JSON, XML, or any of the other supported configuration formats - not just JSON) where you can call config.Get<T>() and have the configuration mechanism map configuration values and parse them into a strongly typed object. This can be very convenient, but in a library it introduces several problems not only for the developer but for the library.

The key takeaway (TLDR) here is that the configuration format is a contract provided by the library, so it is likely better to explicitly determine that contract than it is to tie it to an object from code.

First, as we're talking about potentially splitting the options into something where Prometheus might be two different sets of options - one for the exporter, one for the endpoint information (either middleware or standalone listener) - changing that in code is pretty easy and the compiler will catch issues, no problem. I take an upgrade of the library, recompile, good to go. Making sure I get that changed in config is not as easy to detect. Especially if I was using the configuration binding thing and not configuring anything, I may not get any sort of compiler error at all, but suddenly my Prometheus listener isn't on the place I configured anymore... because an object model changed. You can't change any internals such that the same config settings are read/managed by different components, it's now tied together forever.

Next, if any config property is an IEnumerable<T> or an array of some nature, that's actually really hard to deal with properly in config overrides and binding. You may want to represent lists of strings, for example, as a "comma,delimited,list" rather than an array in JSON. The reason is that, when they're represented in the normalized configuration format, ordinal collections like arrays get flattened to have numeric keys.

Let's say you have a config setup like:

{
  "endpoints": ["first", "second"]
}

Don't forget everything in JSON also needs to be able to be represented in INI, Environment Variables, etc. So what this really looks like is:

{
  "endpoints:0": "first",
  "endpoints:1": "second"
}

That way you can specify an environment variable override: ENDPOINTS__0=override.

That makes any sort of enumerable property (like PrometheusExporterOptions.HttpListenerPrefixes kind of hard to override if someone provides a default in appsettings.json and wants to override it in the environment or in appsettings.development.json. If there are two endpoints defined, you need to know that, you need to know which order they're in, and then you have to override the right one. Oh, and you can't really "unconfigure" one, like if you have the base settings with two endpoints but production only needs one or something like that. Arrays aren't arrays in config, they're ordinal collections. But it can be super easy to forget that if you're only working in JSON and assuming databinding.

The part that ties into the Jaeger thing is the big one for me, which comes to...

Why is reading from config interesting?

It's easiest to explain this in a specific use case.

In the OpenTelemetry SDK, there's a set of well-known environment variables that configure things like the Jaeger exporter or OpenTelemetry exporter. (I know you already know that, but for the sake of completeness I'm mentioning it anyway.)

Let's say you want to configure the Jaeger agent host for a service. Per the SDK, I should set OTEL_EXPORTER_JAEGER_AGENT_HOST to that value.

I want to provide a default value for that which ships with my application, as part of appsettings.json. I want to optionally override that with environment variables or even command line args. That's easy to do because the default WebApplicationBuilder adds the JSON then adds environment variables and command line arguments in a nice hierarchical override.

Ideally what I could do is provide an appsettings.json file like this:

{
  "OTEL_EXPORTER_JAEGER_AGENT_HOST": "dev-agent"
}

and then I could easily override that with an environment variable at deploy:

OTEL_EXPORTER_JAEGER_AGENT_HOST=prod-agent

That's pretty easy to visualize and there's only one config setting that has the agent in it. I could even do something in testing like:

dotnet run OTEL_EXPORTER_JAEGER_AGENT_HOST=localhost

...because the command line config provider works like that.

Everything lines up. It's all the same name, whether it's JSON, INI, Environment Variables, command line args, whatever.

However, that's not really how it works today. That stuff doesn't line up. Instead, there's both this object binding thing, where I could configure it like:

{
  "Jaeger": {
    "AgentHost": "localhost",
  }
}

...and the environment variable. But the environment variable doesn't override config.

If I want to override it, I can try to use the environment variable OTEL_EXPORTER_JAEGER_AGENT_HOST, but the constructor of the JaegerExporterOptions is where the environment gets read, which will happen before binding to config... so config actually overrides environment, not vice-versa. If I want to override it, I can't use the SDK environment variable, I instead need to specify JAEGER__AGENTHOST. None of that lines up, nor is that override mechanism obvious.

Thus, reading from config is super interesting. If done right, you can define a config contract that allows you to:

• Separate config format from your object model to have an explicit contract that will withstand code refactoring.
• Provide a predictable hierarchical override model to allow maximum flexibility in deployment solutions.

A great example of this in .NET Core is the Logging system. The configuration format was thought through such that they can refactor the logging internals, the stuff that parses the configuration, and change things as needed; but it's also easy to define and override without dealing in ordinal collections.

Now, granted, it means more work to read and parse configuration. That does suck. But it's probably worth it for the reasons mentioned above.

It also means the configuration mechanism is not as pretty as having a nice hierarchical object in JSON like there is now. However, the OpenTelemetry SDK sort of gave you the config contract you're stuck with, so... there's that.

Plus, there's a use case where folks may have a standard template for deploying microservices of all types - .NET, Node, Ruby, whatever - and that template might provide the environment variable for the agent host. That should "just work" but if someone in .NET accidentally checks in a JSON file for testing that has Jaeger__AgentHost defined in there, it breaks the contract that the SDK defines. Not good.

You could do both...

If you really wanted, you could probably have the object binding and the environment variables thing, but you'd have to make sure the environment variables (or configuration that uses the SDK environment variable names) takes precedence over the object binding every time. And that could be weird because you might have OTEL_EXPORTER_JAEGER_AGENT_HOST defined in appsettings.json and Jaeger__AgentHost in environment... but the appsettings.json would have to win out because it's the SDK config contract.

---

Anyway, I hope that helps clarify why I've got this IOptions<T> stuff in here. Configuration, in general, will have environment variables in it; but if it doesn't (or if there isn't any config passed in) you could always build a simple one internally, like

var config = new ConfigurationBuilder().AddEnvironmentVariables().Build();

It doesn't hurt that, by doing it that way, you get the added side benefit of all your config reading mechanisms being totally testable without having to actually change the environment variables on the process.

[CodeBlanch]

@tillig I'm not opposed to making the middleware public. My feeling is that having extensions for the common cases is nice for users and sort of guides them to success. Want to get some feedback from the other maintainers.

You might want to consider having that final extension that takes all the parameters be something that doesn't use default parameter values and instead offer an explicit 0-parameter extension. You could get into a situation later where you need to add more extensions but have a weird "ambiguous match" problem due to something being able to match against the new overload and the one with a bunch of optionals.

Makes sense, I'll change it.

RE: Options

Thanks for the feedback! That all makes sense. I have run into the enumerable binding thing in IConfiguration and yes, it is very annoying. Part of the challenge in this repo is the way we are supporting .NET Framework. The exporters aren't IOptions native, they kind of patch it in if it is available at runtime. I'm going to circle back to this and see if there is anything we can do to smooth it out. Would you imagine the OTel standard environment variables should always apply last? I don't think the spec is explicit about that, but it seems reasonable to me.

[tillig]

My thoughts on what should apply when:

• I should be able to specify the OTEL_ values in configuration.
• If I want the OTEL_ stuff to be used from the environment, I can always put it in configuration.
• For people that don't provide configuration, you could still read it from config using new ConfigurationBuilder().AddEnvironmentVariables().Build() so the parsing for config is unified.
• The order in which things are applied should be up to me based on how I build my configuration.

On that latter one, I may build config like:

new ConfigurationBuilder()
  .AddJsonFile("appsettings.json")
  .AddJsonFile($"appsettings.{env}.json", optional: true)
  .AddEnvironmentVariables()
  .AddAzureAppConfiguration()
  .Build()

... and the precedence order is controlled by me. For the vast, vast majority of folks using the default WebApplicationBuilder or whatever, the order is already

• Base JSON
• Environment specific JSON
• Environment variables
• Command line args

You wouldn't have to worry about precedence. It'd be taken care of for you.

For ease of use, you COULD offer more factory-related methods to build things instead of having it part of the constructor.

// Do what is in the constructor right now, except maybe use the config mechanism instead.
var opt = JaegerExporterOptions.FromEnvironment();

// Do the reading like I showed in IOptions<T>.
var opt = JaegerExporterOptions.FromConfiguration(config);

And for convenience, you could have an IConfigureOptions<JaegerExporterOptions> implementation that simply calls into the JaegerExporterOptions.FromConfiguration method. Folks can use it or not, whatever.

I think that does affect the IServiceCollection registrations, though - someone really needs to be doing services.AddOptions<JaegerExporterOptions>() to register the default, and likely it should be required that the dev registers configuration with the app so IConfiguration can be resolved. If I'm not mistaken, this may already be happening automatically but I can't remember.

[CodeBlanch]

(Converted to a draft. I'll switch it back to live PR when I get tests added.)

[CodeBlanch]

Did some testing with this offline. I was able to configure the middleware on a custom port with a different auth scheme (JWT) from the rest of the app (cookies). I think the extensions provide a good level of configurability.

@alanwest @cijothomas Ready for review on this.

Set up for my testing:

using Microsoft.AspNetCore.Authentication;
using Microsoft.AspNetCore.Authentication.Cookies;
using Microsoft.AspNetCore.Authentication.JwtBearer;
using OpenTelemetry.Metrics;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddRazorPages();

builder.Services.AddOpenTelemetryMetrics(builder =>
    builder
        .AddAspNetCoreInstrumentation()
        .AddPrometheusExporter());

builder.Services
    .AddAuthentication(CookieAuthenticationDefaults.AuthenticationScheme) // Default policy always uses cookies.
    .AddCookie() // Add cookie auth scheme for normal calls.
    .AddJwtBearer(); // Add JWT auth scheme for internal calls.

var app = builder.Build();

if (!app.Environment.IsDevelopment())
{
    app.UseExceptionHandler("/Error");
}

app.UseOpenTelemetryPrometheusScrapingEndpoint(
    meterProvider: null,
    predicate: context => context.Request.Path == "/internal/metrics" && context.Connection.LocalPort == 5067,
    path: null,
    configureBranchedPipeline: app =>
    {
        // This could be made into a middleware itself, just doing inline for demo purposes.
        app.Use(async (httpcontext, next) =>
        {
            // Execute auth pipeline for JWT scheme.
            var result = await httpcontext.AuthenticateAsync(JwtBearerDefaults.AuthenticationScheme).ConfigureAwait(false);

            // Require JWT auth before allowing Prometheus.
            if (!result.Succeeded || result.Principal?.Identity?.IsAuthenticated != true)
            {
                httpcontext.Response.StatusCode = 401;
                return;
            }

            await next().ConfigureAwait(false); // Flow through to Prometheus.
        });
    });

app.UseStaticFiles();

app.UseAuthentication();

app.UseRouting();

app.UseAuthorization(); // Allow [Authorize] on razor pages which will use cookies.

app.MapRazorPages();

app.Run();

[github-actions]

This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or Pushing will instruct the bot to automatically remove the label. This bot runs once per day.

[github-actions]

Closed as inactive. Feel free to reopen if this PR is still being worked on.

[CodeBlanch]

Ping @cijothomas @alanwest

[alanwest]

Seems unlikely that someone would set this to null and expect the path to result in /metrics. Should we fail initialization in this case?

[alanwest]

Oh, is this the case where options are read from configuration - like a file, environment, etc? If so, does null in this context effectively mean the user didn't specify it and that the default would be expected?

[CodeBlanch]

@alanwest What this test is verifying is that if the user does mess up and specify null (either from code or through config binding) for the endpoint path we fallback to the default path. I could switch it to a throw (fast-fail) on startup for that case. Might be a better experience for users that way 🤔

[alanwest]

I think I'd kinda prefer separate overloads... that is, one that takes a predicate and another that take a path.

I imagine you were trying to keep the overload permutations to a minimum, but passing in either null or a value that is ignored feels funny.

Side question... I've pondered when if ever we should consider adopting nullable reference types.

[cijothomas]

mm... we can explore this further and decide if we want to add more overloads, to make usage looks less awkward.

[CodeBlanch]

I imagine you were trying to keep the overload permutations to a minimum, but passing in either null or a value that is ignored feels funny.

For the overload with everything specified I tried to capture all the behaviors in the XML comments but ya, don't totally love this. Went through a few iterations of different numbers of extensions, different optional parameters, etc. Nothing felt totally 100% solid.

Side question... I've pondered when if ever we should consider adopting nullable reference types.

We should yes! I tried it once in a branch, lots and lots of effort to pull it off 😭 You can turn on nullable analysis per file, so that part is nice for rolling it out over time. But what I couldn't figure out was how to tell public api analyzer how to respect the per-file mode. It seemed like an all or nothing thing where we would have to do it all in one shot per project. Painful that way!

[cijothomas]

we fallback to ScrapeEndpointPath, if path is not given, right?

[CodeBlanch]

• In the case of the extension which accepts path, you are required to specify it:

public static IApplicationBuilder UseOpenTelemetryPrometheusScrapingEndpoint(this IApplicationBuilder app, string path)
{
   Guard.ThrowIfNull(path);

I figured if user was calling the path extension, without path, they probably made a mistake!

• In the case of the extension where you can specify everything, there is fallback logic:

            if (predicate == null) // Respect predicate if set
            {
                if (path == null) // Respect path if set
                {
                    var options = app.ApplicationServices.GetOptions<PrometheusExporterOptions>();

                    // Respect options if set otherwise fallback to default
                    path = options.ScrapeEndpointPath ?? PrometheusExporterOptions.DefaultScrapeEndpointPath;
                }

[cijothomas]

Sorry for the delay in reviews!

LGTM.
Couple of non-blocking comments. Suggest merging to unblock progress, and follow up as needed.

[cijothomas]

@tillig Could you confirm if this unblocks your scenarios?

[tillig]

@cijothomas Confirmed - I think I can work with this. Thanks for the efforts here!

[cijothomas]

@CodeBlanch Please fix conflict, and we can merge this.

(We can make this part of the 1.3.0.alpha.1 release, as 1.2.0 will not include Prometheus)

[CodeBlanch]

@cijothomas Merge conflict resolved.

[CodeBlanch]

@tillig We did it! 🤣

[tillig]

WOOOO 🎉 🍾

[tillig]

Thanks again for this one, I have it integrated and it's working great!