Add support for "middleware extensions"

[jpkrohling]

Is your feature request related to a problem? Please describe.

When developing a custom auth extension, @rlankfo mentioned he needs access to the request URI to determine the proper auth parameters (tenant, for instance). Instead of just adding the possibility of specifying the interceptors to use as part of the auth package and using the experience gathered over the past couple of years since implementing the auth capabilities, I think it might be helpful to add support for middleware extensions instead: middleware extensions would have the possibility of intercepting the incoming requests (HTTP and gRPC) and potentially doing things that are beyond auth (and possibly advanced auth scenarios, like @rlankfo's). Another example of a middleware extension would be @MovieStoreGuy's open-telemetry/opentelemetry-collector-contrib#6908.

Depending on how we progress with this idea, server auth extensions could be implemented as a specialization of a middleware extension.

Describe the solution you'd like
I had a proposal for "authorizers" as part of @MovieStoreGuy's rate limiting extension proposal, and I would change that a bit, so that it reads middleware instead:

extensions:
  oidc:

  ratelimiter:
    qps: 1000
  blocklist:
    source: https://example.com/blocklist.txt

receivers:
  otlp:
    protocols:
      grpc:
        auth:
          authenticator: oidc
        middlewares:
        - blocklist
        - ratelimiter


processors:

exporters:
  logging:

service:
  extensions: [oidc, ratelimiter]
  pipelines:
    traces:
      receivers: [otlp]
      processors: []
      exporters: [logging]

Describe alternatives you've considered
Solving @rlankfo's problem could also be done by adding the notion of Interceptors to the server auth extension interface, with the current interptors (http.Handler, gRPC's Unary and Stream interceptors) as default implementations, allowing auth extensions to override when needed via options (WithInterceptor in addition to the current WithAuthenticate). Another possible solution to the original problem would be to add request metadata (URL, for instance) to the client's context, which is made available later on to auth extensions.

[codeboten]

Capturing some thoughts from the discussion in the SIG call on March 29:

• middleware/interceptors should not have access to the telemetry data. this is to prevent processing from being moved into these types of components
• what happens to existing auth configuration? does it get deprecated in favour of this new config? does it remain separate?

the role and capabilities for middleware needs to be well defined, what it has access to, what its purpose is

[jpkrohling]

Other thoughts:

• we should call it interceptors, not middlewares
• we are having client and server interceptors
• header setter extension would be the first client interceptor (cc @kovrus)
• the plan is to create a PR with the initial proposal for server (first PR) and client (second PR) interceptors, then see how auth can use these new primitives, and finally see which helpers can be built so that people don't have to implement all functions (like, "authorizers", where people would implement a function that receives context and metadata, and decides whether a request can continue or not)

[dyanneo]

Hi - I got here while searching for how to implement HTTP Strict Transport Security. My organization requires this setting.
It seems simple enough if you control how the HTTP ResponseWriter's Headers are configured. We don't - we import the majority of what we need from your code bases. We do use some extensions today, and a "header_injector" sounded like a good idea.

How soon might there be work put into a header_injector extension?

Do you have recommendations on how to implement HTTP Strict Transport Security?
Thanks!

[mx-psi]

From the Collector SIG meeting on 2024-12-11, we can start with an extension/middleware module (name to be bikeshedded) since this can be easily worked as an independent module without affecting 1.0 work. Adding to confighttp/configgrpc is a different beast, but we can start with the interface design

[jpkrohling]

@dyanneo, to answer your question directly: I can't think of a way to achieve that right now, which is why I brought this to the SIG call today. We decided to move forward with this feature, but there is no ETA at the moment.

[dyanneo]

@jpkrohling Thank you!!
If it helps scope the work, we're also looking for:

• X-Frame-Options header to DENY on all HTTP responses
• Validate all HTTP responses does not leak information

So, the no ETA - is that for middleware or the confighttp components?

[jpkrohling]

So, the no ETA - is that for middleware or the confighttp components?

Both: I'll start with the middleware part (interfaces and one or two implementations), but they can't be used until we have all plugged in as part of confighttp/configgrpc. This is currently relatively high on my queue, so while we don't have an ETA (it's end of the year, after all), I think we might be able to get it done "soon".

Would you able to help me design and develop this?