[BUG] unable to filter log records exported to Azure Monitor

[cataggar]

Library name and version

Azure.Monitor.OpenTelemetry.Exporter 1.0.0-beta.4

Describe the bug

I want to prevent logs from being sent to Azure.Monitor. Based on the filtering processor example, I created:

open System.Diagnostics
open OpenTelemetry
open OpenTelemetry.Logs

/// Remove Heartbeat from Host.Results
type HeartbeatLogFilter() =
    inherit BaseProcessor<LogRecord>()
    override _.OnEnd log =
        if log.CategoryName = "Host.Results" then
            let fullName =
                log.StateValues |> Seq.tryPick(fun kv ->
                    if kv.Key = "FullName" then
                        Some kv.Value
                    else
                        None
                )
            if fullName = Some "Heartbeat" then
                log.TraceFlags <- log.TraceFlags &&& ~~~ ActivityTraceFlags.Recorded

It removes ActivityTraceFlags.Recorded from the LogRecord.TraceFlags.

Expected behavior

I was expecting the log records with ActivityTraceFlags.Recorded removed to not be exported to Azure Monitor.

Actual behavior

The log records still end up in Azure Monitor.

Reproduction Steps

Please see the Microsoft internal pull request.

Environment

No response

[azure-sdk]

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Storage:0.116750255,Service Bus:0.11560577,Azure.Identity:0.08867687'

[ghost]

Thank you for your feedback. This has been routed to the support team for assistance.

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @cijothomas, @reyang, @rajkumar-rangaraj, @TimothyMothra, @vishweshbankwar.

Issue Details

---

Library name and version

Azure.Monitor.OpenTelemetry.Exporter 1.0.0-beta.4

Describe the bug

I want to prevent logs from being sent to Azure.Monitor. Based on the filtering processor example, I created:

open System.Diagnostics
open OpenTelemetry
open OpenTelemetry.Logs

/// Remove Heartbeat from Host.Results
type HeartbeatLogFilter() =
    inherit BaseProcessor<LogRecord>()
    override _.OnEnd log =
        if log.CategoryName = "Host.Results" then
            let fullName =
                log.StateValues |> Seq.tryPick(fun kv ->
                    if kv.Key = "FullName" then
                        Some kv.Value
                    else
                        None
                )
            if fullName = Some "Heartbeat" then
                log.TraceFlags <- log.TraceFlags &&& ~~~ ActivityTraceFlags.Recorded

It removes ActivityTraceFlags.Recorded from the LogRecord.TraceFlags.

Expected behavior

I was expecting the log records with ActivityTraceFlags.Recorded removed to not be exported to Azure Monitor.

Actual behavior

The log records still end up in Azure Monitor.

Reproduction Steps

Please see the Microsoft internal pull request.

Environment

No response

Author:	cataggar
Assignees:	-
Labels:	Service Attention, Client, customer-reported, question, needs-team-attention, Monitor - Exporter
Milestone:	-

[navba-MSFT]

Adding Service team to look into this.

[cijothomas]

https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/docs/trace/extending-the-sdk/README.md#filtering-processor is only applicable for Traces, not logs.

OpenTelemetry Logging is based on ILogger, and the filtering capabilities provided by ILogger itself can be used to filter logs.
https://github.com/open-telemetry/opentelemetry-dotnet/tree/main/docs/logs/customizing-the-sdk#log-filtering

[cijothomas]

We can close this. This is not an issue/questions specific to AzureMonitor.

[cataggar]

Thank you @cijothomas 🥇

[cataggar]

Unfortunately, the ILoggerBuilder filters are limited to filtering by log.CategoryName and log.LogLevel only. I need to filter based on log.StateValues. Can the filtering processor be made applicable for logs too please?

[vishweshbankwar]

@cataggar There is no recommended way to filter out logs based on ActivityTraceFlags. Since this is not Azure Monitor specific question, Could you please open an issue here https://github.com/open-telemetry/opentelemetry-dotnet.

[vishweshbankwar]

Closing this as the issue is related to OTel Sdk.

[cataggar]

I opened open-telemetry/opentelemetry-dotnet#3918.

[cataggar]

You can write own FilteringProcessor wrapping the exporting processor, but that'll only work if the Exporter being target has public ctor.
open-telemetry/opentelemetry-dotnet#3918 (comment)

Can class AzureMonitorLogExporter please be made public so that it can be wrapped in a FilteringProcessor? It is currently internal:

azure-sdk-for-net/sdk/monitor/Azure.Monitor.OpenTelemetry.Exporter/src/AzureMonitorLogExporter.cs

Lines 15 to 24 in c3fc45c

	internal class AzureMonitorLogExporter : BaseExporter<LogRecord>
	{
	private readonly ITransmitter _transmitter;
	private readonly string _instrumentationKey;
	private readonly ResourceParser _resourceParser;
	private readonly AzureMonitorPersistentStorage _persistentStorage;
	public AzureMonitorLogExporter(AzureMonitorExporterOptions options, TokenCredential credential = null) : this(new AzureMonitorTransmitter(options, credential))
	{
	}

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @cijothomas, @reyang, @rajkumar-rangaraj, @TimothyMothra, @vishweshbankwar.

Issue Details

---

Library name and version

Azure.Monitor.OpenTelemetry.Exporter 1.0.0-beta.4

Describe the bug

I want to prevent logs from being sent to Azure.Monitor. Based on the filtering processor example, I created:

open System.Diagnostics
open OpenTelemetry
open OpenTelemetry.Logs

/// Remove Heartbeat from Host.Results
type HeartbeatLogFilter() =
    inherit BaseProcessor<LogRecord>()
    override _.OnEnd log =
        if log.CategoryName = "Host.Results" then
            let fullName =
                log.StateValues |> Seq.tryPick(fun kv ->
                    if kv.Key = "FullName" then
                        Some kv.Value
                    else
                        None
                )
            if fullName = Some "Heartbeat" then
                log.TraceFlags <- log.TraceFlags &&& ~~~ ActivityTraceFlags.Recorded

It removes ActivityTraceFlags.Recorded from the LogRecord.TraceFlags.

Expected behavior

I was expecting the log records with ActivityTraceFlags.Recorded removed to not be exported to Azure Monitor.

Actual behavior

The log records still end up in Azure Monitor.

Reproduction Steps

Please see the Microsoft internal pull request.

Environment

No response

Author:	cataggar
Assignees:	-
Labels:	Service Attention, Client, customer-reported, question, needs-team-attention, Monitor - Exporter
Milestone:	-

[navba-MSFT]

Adding Service team to answer the above question:

#32276 (comment)

[cijothomas]

You can write own FilteringProcessor wrapping the exporting processor, but that'll only work if the Exporter being target has public ctor.
open-telemetry/opentelemetry-dotnet#3918 (comment)

Can class AzureMonitorLogExporter please be made public so that it can be wrapped in a FilteringProcessor? It is currently internal:

azure-sdk-for-net/sdk/monitor/Azure.Monitor.OpenTelemetry.Exporter/src/AzureMonitorLogExporter.cs

Lines 15 to 24 in c3fc45c

	internal class AzureMonitorLogExporter : BaseExporter<LogRecord>
	{
	private readonly ITransmitter _transmitter;
	private readonly string _instrumentationKey;
	private readonly ResourceParser _resourceParser;
	private readonly AzureMonitorPersistentStorage _persistentStorage;
	public AzureMonitorLogExporter(AzureMonitorExporterOptions options, TokenCredential credential = null) : this(new AzureMonitorTransmitter(options, credential))
	{
	}

@rajkumar-rangaraj Did we discuss/decide about making the Exporter public?

[zyofeng]

At least in beta 3 release this was still public, alternatively it would be good to provide some kind of filtering logic in AzureMonitorExporterOptions,

something like
public Func<Activity, bool> Filter { get; set; }

[cataggar]

It looks like AzureMonitorLogExporter is still internal. Can it be made public?

[cataggar]

#35568 will make it public.

[cataggar]

I am hesitant to close those this. I was able to find a workaround for the logs created in this case. Hopefully, better ILogger filtering support will be possible generically for .NET. dotnet/runtime#82465