[OtlpExporter] Adding Tracing and Metrics with custom endpoints messes up the settings

[sharpSteff]

Bug Report

OpenTelemetry.Exporter.Console 1.4.0-rc.1
OpenTelemetry.Exporter.OpenTelemetryProtocol 1.4.0-rc.1
OpenTelemetry.Extensions.Hosting 1.4.0-rc.1
OpenTelemetry.Instrumentation.AspNetCore 1.0.0-rc9.10
OpenTelemetry.Instrumentation.Http 1.0.0-rc9.10
OpenTelemetry.Instrumentation.Runtime 1.1.0-beta.2

net6.0`

Symptom

When addind Tracing and Metrics via the builder API. traceOptions and metricsOptions are the same object.
Therefore its not possible to set different endpoints for metrics and traces, which is essential when not using the default value ("http://localhost:4318").

var olotEndpoint = new Uri(Configuration.GetValue<string>("Otlp:Endpoint"));
            var traceEndpoint = AppendPathIfNotPresent(olotEndpoint, "v1/traces");
            var metricsEndpoint = AppendPathIfNotPresent(olotEndpoint, "v1/metrics");
            services.AddOpenTelemetry()
                .ConfigureResource(x => x.AddService(
                    serviceName: Configuration.GetValue<string>("ServiceName"),
                    serviceVersion: Assembly.GetExecutingAssembly().GetName().Version?.ToString() ?? "unknown",
                    serviceInstanceId: Environment.MachineName))
                .WithTracing(traceBuilder => traceBuilder
                    .SetSampler(new AlwaysOnSampler())
                    .AddHttpClientInstrumentation()
                    .AddAspNetCoreInstrumentation()
                    .AddOtlpExporter(traceOptions =>
                    {
                        traceOptions.Protocol = OtlpExportProtocol.HttpProtobuf;
                        traceOptions.HttpClientFactory = CreateClient;

                        // Use IConfiguration directly for Otlp exporter endpoint option.
                        traceOptions.Endpoint = traceEndpoint;
                    }))
                .WithMetrics(builder => builder
                    .AddRuntimeInstrumentation()
                    .AddHttpClientInstrumentation()
                    .AddAspNetCoreInstrumentation()
                    .AddOtlpExporter(metricsOptions =>
                    {
                        metricsOptions .Protocol = OtlpExportProtocol.HttpProtobuf;
                        metricsOptions .HttpClientFactory = CreateClient;

                        // Use IConfiguration directly for Otlp exporter endpoint option.
                        metricsOptions .Endpoint = metricsEndpoint;
                    }))
                .StartWithHost();

What is the expected behavior?

define separate endpoint in traceOptions and metricsOptions

What is the actual behavior?
defining separate endpoints viatraceOptions and metricsOptions is not possible.
In consequence the TraceOptions get overwritten => bad request for traces:

Request:
Method: POST, RequestUri: 'http://localhost:4318/v1/metrics', Version: 1.1, Content: OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ExportClient.OtlpHttpTraceExportClient+ExportRequestContent, Headers:
{
  Content-Type: application/x-protobuf
}
Response:
StatusCode: 400, ReasonPhrase: 'Bad Request', Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:
{
  Date: Mon, 26 Dec 2022 12:43:48 GMT
  Content-Type: application/x-protobuf
  Content-Length: 29
}

Reproduce

Create a fresh Asp.NET Core and configure the services with the code above.
Set a breakpoint at metricsOptions .Protocol = OtlpExportProtocol.HttpProtobuf; and investigate the Properties.
They are already set.

To test this issue, modify the builder part in ServiceProviderHttpClientFactoryInvoked in test/OpenTelemetry.Exporter.OpenTelemetryProtocol.Tests/OtlpTraceExporterTests.cs

to

services.AddOpenTelemetry()
                .WithTracing(builder => builder
                    .AddOtlpExporter(o =>
                    {
                        o.Protocol = OtlpExportProtocol.HttpProtobuf;
                        o.Endpoint = new Uri("http://localhost:3417/v1/traces");
                    }))
                .WithMetrics(builder => builder
                    .AddOtlpExporter(o =>
                    {
                        o.Protocol = OtlpExportProtocol.HttpProtobuf;
                        o.Endpoint = new Uri("http://localhost:3417/v1/metrics");
                    }));

set a breakpoint in the second AddOtlpExporter and see that the options have the trace values set

We will close this issue if:

• The repro project you share with us is complex. We can't investigate custom
  projects, so don't point us to such, please.
• If we can not reproduce the behavior you're reporting.

Additional Context

Workaround

Passing the name parameter into AddOtlpExporter fixes the issue:

services.AddOpenTelemetry()
               .WithTracing(builder => builder
                   .AddOtlpExporter("trace" , o =>
                   {
                       o.Protocol = OtlpExportProtocol.HttpProtobuf;
                       o.Endpoint = new Uri("http://localhost:3417/v1/traces");
                   }))
               .WithMetrics(builder => builder
                   .AddOtlpExporter("metrics", o =>
                   {
                       o.Protocol = OtlpExportProtocol.HttpProtobuf;
                       o.Endpoint = new Uri("http://localhost:3417/v1/metrics");
                   }));

[utpilla]

@sharpSteff This fix is now available in 1.4.0-rc2 version of the OTLP Exporter.

[sharpSteff]

@utpilla Great. Thanks so much!