Spammed by 'read-result-set-header' traces in application insights

[Tobbeman]

Is your feature request related to a problem? Please describe.
Disclaimer; Still learning how to use OpenTelemetry

We recently added OpenTelemetry to our application as well as MySqlConnector as source
Sadly, we are spammed by traces with the message "read-result-set-header".

These seem to be created here

As a result our application insights are filled:

Note that all the "traces" are the message in question.

I have not found anyway to filter these, though I might simply be overlooking something

Describe the solution you'd like
These traces add little value as far as I can tell, I would like to be able to turn them off.
Perhaps we can add a option to do so?

I'm happy to take a crack at it.

Note: If it is possible to filter these, that would also help me. Currently our filter looks like this and I am unable to change the Events associated with the activity ( read only ):

internal class FilterProcessor : BaseProcessor<Activity>
{
    public override void OnStart(Activity activity)
    {
        FilterPomeloMySqlConnector(activity);
    }

    void FilterPomeloMySqlConnector(Activity activity) 
    {
        if (activity.Source.Name != "MySqlConnector") {
            return;
        }

        if (activity.DisplayName == "Open")
        {
            activity.IsAllDataRequested = false;
        }
    }
}

[bgrainger]

How are you feeding the Activity events into Application Insights?

"Trace" is supposed to be used for logs; see the data model documentation: https://learn.microsoft.com/en-us/azure/azure-monitor/app/data-model-complete

MySqlConnector is calling Activity.AddEvent for this telemetry. That seems like it would be better modeled as an "Event" in Application Insights (or just dropped). I would look into how and why your code is converting events into traces.

[vanbukin]

I see something similar.

MySqlConnector 2.4.0
.NET SDK 8.0.404
.NET Runtime 8.0.11
OpenTelemetry 1.10.0
Jaeger 2.0.0

[Tobbeman]

@bgrainger

How are you feeding the Activity events into Application Insights?

"Trace" is supposed to be used for logs; see the data model documentation: https://learn.microsoft.com/en-us/azure/azure-monitor/app/data-model-complete

MySqlConnector is calling Activity.AddEvent for this telemetry. That seems like it would be better modeled as an "Event" in Application Insights (or just dropped). I would look into how and why your code is converting events into traces.

That makes sense to me, but as far as I can tell I don't do anything special

 builder.Services.AddOpenTelemetry()
            .UseAzureMonitor(options =>
            {
                options.ConnectionString = config.ConnectionString;
                options.SamplingRatio = config.SamplingRate;
            })
            // TL: Enable telemetry for Pomelo. Pomelo uses MySqlConnector: https://mysqlconnector.net/diagnostics/tracing/
            // Do not be fooled by MySql.Data
            .WithTracing(b => b.AddSource("MySqlConnector")
            );

That said, this could very much be a configuration issue outside of MySqlConnector scope

Using
Azure.Monitor.OpenTelemetry.AspNetCore 1.0.0
.NET SDK 6.0.400
.NET Runtime 6.0.8

[bgrainger]

I only dug in a little, but this could ultimately be coming from .UseAzureMonitor via https://github.com/Azure/azure-sdk-for-net/blob/e986729b7f6d1367ceba27e8dd31a685d9ad65a8/sdk/monitor/Azure.Monitor.OpenTelemetry.Exporter/src/Internals/TraceHelper.cs#L250.

[Tobbeman]

@bgrainger Indeed

As I mentioned I'm happy to make a PR that allow it to be optional.
But if maintainers don't think that makes sense ( which is reasonable ) I'll look elsewhere.

[bgrainger]

Note: If it is possible to filter these, that would also help me.

https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/docs/trace/extending-the-sdk/README.md#filtering-processor says to "toggle the Activity.Recorded flag." Have you tried that?

[Tobbeman]

@bgrainger

Sadly the filtering seems to apply to the entire scope of the activity rather than only the events. Shame the events are read only.

I tried to use a TelemetryFilter for application insights directly but looks like the opentelemetry implementation use another client
https://github.com/Azure/azure-sdk-for-net/blob/e986729b7f6d1367ceba27e8dd31a685d9ad65a8/sdk/monitor/Azure.Monitor.OpenTelemetry.Exporter/src/Internals/AzureMonitorTransmitter.cs

In a similar case they implemented at the core, not sure the project is comparable though
open-telemetry/opentelemetry-dotnet-contrib#416

[bgrainger]

Npgsql recently added the ability to suppress their activity events: npgsql/npgsql#5853.

[Tobbeman]

Made a basic PR, It would be very nice if we could configure this in some way