Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for OTEL_BSP_EXPORT_* environmental variables #2219

Merged
merged 19 commits into from
Sep 3, 2021
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions src/OpenTelemetry/BatchExportProcessorOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,57 @@
// limitations under the License.
// </copyright>

using System;
using System.Security;
using OpenTelemetry.Internal;

namespace OpenTelemetry
{
public class BatchExportProcessorOptions<T>
where T : class
{
internal const string MaxQueueSizeEnvVarKey = "OTEL_BSP_MAX_QUEUE_SIZE";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are specific to batch span processors, we need some logic to make sure it is not affecting other scenario (e.g. metrics).

Copy link
Member Author

@pellared pellared Aug 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any proposal on how to approach this?

I think that the root problem is that the .NET SDK tries to use common (premature?) abstractions to implement multiple OTel specification components. E.g. we have a common exporter base class for both metrics and traces exporter. I am aware it reduces some code duplication but it tightly couples the API and it may be hard to maintain backward compatibility going this way.

Can we use BatchExportProcessorOptions for tracing (to not break SemVer) and create a BatchMetricsExportProcessorOptions for metrics SDK?

Moreover, I think we should consider separating the metrics and traces SDKs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we did something like...

        internal const string SpanMaxQueueSizeEnvVarKey = "OTEL_BSP_MAX_QUEUE_SIZE";

        internal const string SpanMaxExportBatchSizeEnvVarKey = "OTEL_BSP_MAX_EXPORT_BATCH_SIZE";

        internal const string SpanExporterTimeoutEnvVarKey = "OTEL_BSP_EXPORT_TIMEOUT";

        internal const string SpanScheduledDelayEnvVarKey = "OTEL_BSP_SCHEDULE_DELAY";

        public BatchExportProcessorOptions()
        {
            if (typeof(T) == typeof(Activity))
            {
               LoadSpanEnvVars();
            }
        }

        private void LoadSpanEnvVars()
        {
            int value;

            if (TryLoadEnvVarInt(SpanExporterTimeoutEnvVarKey, out value))
            {
                this.ExporterTimeoutMilliseconds = value;
            }

            if (TryLoadEnvVarInt(SpanMaxExportBatchSizeEnvVarKey, out value))
            {
                this.MaxExportBatchSize = value;
            }

            if (TryLoadEnvVarInt(SpanMaxQueueSizeEnvVarKey, out value))
            {
                this.MaxQueueSize = value;
            }

            if (TryLoadEnvVarInt(SpanScheduledDelayEnvVarKey, out value))
            {
                this.ScheduledDelayMilliseconds = value;
            }
        }

I'm not really a fan of duck typing in generics like this but trying to introduce classes into the hierarchy now will probably be a breaking change.

Copy link
Member Author

@pellared pellared Aug 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CodeBlanch thanks 👍

but trying to introduce classes into the hierarchy now will probably be a breaking change

That is why I suggest changing the metrics SDK as AFAIK it od not stable yet. And leave the tracing SDK as it is.

Your proposal would work and probably it is the most pragmatic idea for this PR. But maybe it is worth taking a step back and make more refactoring in separate PR(s).

Copy link
Member Author

@pellared pellared Aug 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CodeBlanch
I noticed that OtlpMetricsExporter uses OtlpExporterOptions which has a BatchExportProcessorOptions<Activity>.

Personally, I would prefer to redesign the OtlpMetricsExporter. The main repository's readme says that metrics are experimental so probably it is acceptable. Then we could redesign OtlpMetricsExporter to not have BatchExportProcessorOptions as it is not used anyway. I think we could refactor OtlpMetricsExporter as a separate issue+PR.

Regarding this PR I see the following options:

  1. We could just improve the BatchExportProcessorOptions docs and say that this option is designed for the tracing SDK.

  2. Instead of "duck typing in generics" we could also possibly create a subclass BatchSpanExportProcessorOptions and use it like this:

    public BatchExportProcessorOptions<Activity> BatchExportProcessorOptions { get; set; } = new BatchSpanExportProcessorOptions();

    not sure how much of a breaking change - I guess 99% of the existing code should simply work. Personally, I find such "common property grouping" as a code smell and possible premature abstraction. Will the exporters handle all of the possible options?

  3. Use the "duck typing in generics" proposed here

Shouldn't the tracing and metrics packages be distributed separately to avoid confusion? How the user would distinguish between stable and unstable API?

@reyang PTAL as well

Copy link
Member Author

@pellared pellared Aug 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After the SIG meeting, I have an impression that this would be the preferred way forward:

  1. Create an issue that OtlpMetricsExporter should not reference BatchExportProcessorOptions<Activity>. Done here: OtlpMetricsExporter should not reference BatchExportProcessorOptions<Activity> #2246
  2. Create a BatchSpanExportProcessorOptions as a subclass of BatchExportProcessorOptions<Activity>. Addressed here: 092ddea


internal const string MaxExportBatchSizeEnvVarKey = "OTEL_BSP_MAX_EXPORT_BATCH_SIZE";

internal const string ExporterTimeoutEnvVarKey = "OTEL_BSP_EXPORT_TIMEOUT";

internal const string ScheduledDelayEnvVarKey = "OTEL_BSP_SCHEDULE_DELAY";

public BatchExportProcessorOptions()
{
try
{
int value;

if (TryLoadEnvVarInt(BatchExportProcessorOptions<T>.ExporterTimeoutEnvVarKey, out value))
{
this.ExporterTimeoutMilliseconds = value;
}

if (TryLoadEnvVarInt(BatchExportProcessorOptions<T>.MaxExportBatchSizeEnvVarKey, out value))
{
this.MaxExportBatchSize = value;
}

if (TryLoadEnvVarInt(BatchExportProcessorOptions<T>.MaxQueueSizeEnvVarKey, out value))
{
this.MaxQueueSize = value;
}

if (TryLoadEnvVarInt(BatchExportProcessorOptions<T>.ScheduledDelayEnvVarKey, out value))
{
this.ScheduledDelayMilliseconds = value;
}
}
catch (SecurityException ex)
{
// The caller does not have the required permission to
// retrieve the value of an environment variable from the current process.
OpenTelemetrySdkEventSource.Log.MissingPermissionsToReadEnvironmentVariable(ex);
reyang marked this conversation as resolved.
Show resolved Hide resolved
}
}

/// <summary>
/// Gets or sets the maximum queue size. The queue drops the data if the maximum size is reached. The default value is 2048.
/// </summary>
Expand All @@ -38,5 +84,24 @@ public class BatchExportProcessorOptions<T>
/// Gets or sets the maximum batch size of every export. It must be smaller or equal to MaxQueueLength. The default value is 512.
/// </summary>
public int MaxExportBatchSize { get; set; } = BatchExportProcessor<T>.DefaultMaxExportBatchSize;

private static bool TryLoadEnvVarInt(string envVarKey, out int result)
reyang marked this conversation as resolved.
Show resolved Hide resolved
{
result = 0;
string value = Environment.GetEnvironmentVariable(envVarKey);
if (string.IsNullOrEmpty(value))
{
return false;
}

if (!int.TryParse(value, out var parsedValue))
{
OpenTelemetrySdkEventSource.Log.FailedToParseEnvironmentVariable(envVarKey, value);
return false;
}

result = parsedValue;
return true;
}
}
}
7 changes: 7 additions & 0 deletions src/OpenTelemetry/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

## Unreleased

* The `BatchExportProcessorOptions` defaults can be overridden using
`OTEL_BSP_SCHEDULE_DELAY`, `OTEL_BSP_EXPORT_TIMEOUT`,
`OTEL_BSP_MAX_QUEUE_SIZE`, `OTEL_BSP_MAX_EXPORT_BATCH_SIZE`
envionmental variables as defined in the
[specification](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.5.0/specification/sdk-environment-variables.md#batch-span-processor).
([#2219](https://github.com/open-telemetry/opentelemetry-dotnet/pull/2219))

* Removes upper constraint for Microsoft.Extensions.Logging
dependencies. ([#2179](https://github.com/open-telemetry/opentelemetry-dotnet/pull/2179))

Expand Down
22 changes: 22 additions & 0 deletions src/OpenTelemetry/Internal/OpenTelemetrySdkEventSource.cs
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
#endif
using System.Diagnostics;
using System.Diagnostics.Tracing;
using System.Security;

namespace OpenTelemetry.Internal
{
Expand Down Expand Up @@ -125,6 +126,15 @@ public void TracerProviderException(string evnt, Exception ex)
}
}

[NonEvent]
public void MissingPermissionsToReadEnvironmentVariable(SecurityException ex)
{
if (this.IsEnabled(EventLevel.Warning, EventKeywords.All))
{
this.MissingPermissionsToReadEnvironmentVariable(ex.ToInvariantString());
}
}

[Event(1, Message = "Span processor queue size reached maximum. Throttling spans.", Level = EventLevel.Warning)]
public void SpanProcessorQueueIsExhausted()
{
Expand Down Expand Up @@ -287,6 +297,18 @@ public void TracerProviderException(string evnt, string ex)
this.WriteEvent(28, evnt, ex);
}

[Event(29, Message = "Failed to parse environment variable: '{0}', value: '{1}'.", Level = EventLevel.Warning)]
public void FailedToParseEnvironmentVariable(string name, string value)
{
this.WriteEvent(29, name, value);
}

[Event(30, Message = "Missing permissions to read environment variable: '{0}'", Level = EventLevel.Warning)]
public void MissingPermissionsToReadEnvironmentVariable(string exception)
{
this.WriteEvent(30, exception);
}

#if DEBUG
public class OpenTelemetryEventListener : EventListener
{
Expand Down
14 changes: 14 additions & 0 deletions src/OpenTelemetry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,10 +200,24 @@ purposes, the SDK provides the following built-in processors:
* [BatchExportProcessor&lt;T&gt;](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#batching-processor)
: This is an exporting processor which batches the telemetry before sending to
the configured exporter.

The following environment variables can be used to override the default
values of the `BatchExportProcessorOptions<T>`.

<!-- markdownlint-disable MD013 -->
| Environment variable | `BatchExportProcessorOptions<T>` property |
| -------------------------------- | ---------------------------------------------- |
| `OTEL_BSP_SCHEDULE_DELAY` | `ScheduledDelayMilliseconds` |
| `OTEL_BSP_EXPORT_TIMEOUT` | `ExporterTimeoutMilliseconds` |
| `OTEL_BSP_MAX_QUEUE_SIZE` | `MaxQueueSize` |
| `OTEL_BSP_MAX_EXPORT_BATCH_SIZE` | `MaxExportBatchSizeEnvVarKey` |
<!-- markdownlint-enable MD013 -->

* [CompositeProcessor&lt;T&gt;](../../src/OpenTelemetry/CompositeProcessor.cs)
: This is a processor which can be composed from multiple processors. This is
typically used to construct multiple processing pipelines, each ending with
its own exporter.

* [SimpleExportProcessor&lt;T&gt;](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#simple-processor)
: This is an exporting processor which passes telemetry to the configured
exporter without any batching.
Expand Down
103 changes: 103 additions & 0 deletions test/OpenTelemetry.Tests/Trace/BatchExportProcessorOptionsTest.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
// <copyright file="BatchExportProcessorOptionsTest.cs" company="OpenTelemetry Authors">
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// </copyright>

using System;
using Xunit;

using Options = OpenTelemetry.BatchExportProcessorOptions<object>;

namespace OpenTelemetry.Trace.Tests
{
public class BatchExportProcessorOptionsTest : IDisposable
{
public BatchExportProcessorOptionsTest()
{
this.ClearEnvVars();
}

public void Dispose()
{
this.ClearEnvVars();
}

[Fact]
public void BatchExportProcessorOptions_Defaults()
{
var options = new Options();

Assert.Equal(30000, options.ExporterTimeoutMilliseconds);
Assert.Equal(512, options.MaxExportBatchSize);
Assert.Equal(2048, options.MaxQueueSize);
Assert.Equal(5000, options.ScheduledDelayMilliseconds);
}

[Fact]
public void BatchExportProcessorOptions_EnvironmentVariableOverride()
{
Environment.SetEnvironmentVariable(Options.ExporterTimeoutEnvVarKey, "1");
Environment.SetEnvironmentVariable(Options.MaxExportBatchSizeEnvVarKey, "2");
Environment.SetEnvironmentVariable(Options.MaxQueueSizeEnvVarKey, "3");
Environment.SetEnvironmentVariable(Options.ScheduledDelayEnvVarKey, "4");

var options = new BatchExportProcessorOptions<object>();

Assert.Equal(1, options.ExporterTimeoutMilliseconds);
Assert.Equal(2, options.MaxExportBatchSize);
Assert.Equal(3, options.MaxQueueSize);
Assert.Equal(4, options.ScheduledDelayMilliseconds);
}

[Fact]
public void BatchExportProcessorOptions_InvalidPortEnvironmentVariableOverride()
{
Environment.SetEnvironmentVariable(BatchExportProcessorOptions<object>.ExporterTimeoutEnvVarKey, "invalid");

var options = new Options();

Assert.Equal(30000, options.ExporterTimeoutMilliseconds); // use default
}

[Fact]
public void BatchExportProcessorOptions_SetterOverridesEnvironmentVariable()
{
Environment.SetEnvironmentVariable(BatchExportProcessorOptions<object>.ExporterTimeoutEnvVarKey, "123");

var options = new Options
{
ExporterTimeoutMilliseconds = 89000,
};

Assert.Equal(89000, options.ExporterTimeoutMilliseconds);
}

[Fact]
public void BatchExportProcessorOptions_EnvironmentVariableNames()
{
Assert.Equal("OTEL_BSP_EXPORT_TIMEOUT", Options.ExporterTimeoutEnvVarKey);
Assert.Equal("OTEL_BSP_MAX_EXPORT_BATCH_SIZE", Options.MaxExportBatchSizeEnvVarKey);
Assert.Equal("OTEL_BSP_MAX_QUEUE_SIZE", Options.MaxQueueSizeEnvVarKey);
Assert.Equal("OTEL_BSP_SCHEDULE_DELAY", Options.ScheduledDelayEnvVarKey);
}

private void ClearEnvVars()
{
Environment.SetEnvironmentVariable(Options.ExporterTimeoutEnvVarKey, null);
Environment.SetEnvironmentVariable(Options.MaxExportBatchSizeEnvVarKey, null);
Environment.SetEnvironmentVariable(Options.MaxQueueSizeEnvVarKey, null);
Environment.SetEnvironmentVariable(Options.ScheduledDelayEnvVarKey, null);
}
}
}