Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add new ResourceDetector #6250

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
/*
* Copyright The OpenTelemetry Authors
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.sdk.autoconfigure.spi.internal;

import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.sdk.autoconfigure.spi.ConfigProperties;
import io.opentelemetry.sdk.autoconfigure.spi.Ordered;
import java.util.Optional;
import java.util.function.Function;

public interface ResourceDetector<D> extends Ordered {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really don't see the current ResourceProvider design as broken, and can't tell how this would alleviate confusion.

Here's a summary of the toolkit for managing resources today:

  • ResourceProvider SPI allows you provide your own resource, which is merged with resource provided by other providers
  • The environment variables OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES contribute to the environment resource provider, which runs last. These environment variables can be customized by AutoConfigurationCustomizer#addPropertiesSupplier, AutoConfigurationCustomizer#addPropertiesCustomizer.
  • ResourceProvider can optionally be ordered, so you can indicate priority relative to others
  • ResourceProvider can optionally be conditional, so you can decide whether or not to provide a resource based on the resource computed up till that point in time
  • All resource providers are enabled by default, and you can set a deny list with otel.java.disabled.resource.providers
  • All resource providers are enabled by default, but you can optionally change the semantic to an allow list by setting otel.java.enabled.resource.providers
  • After all resource providers have been applied, you can optionally filter out individual attribute keys with otel.experimental.resource.disabled.keys
  • After all resource providers have been applied and optional filter applied, you can customize the resource further with AutoConfigurationCustomizer#addResourceCustomizer
  • After the resource has been completely resolved and applied to Sdk{Signal}ProviderBuilder, you can further customize it with AutoConfigurationCustomizer#add{Signal}ProviderCustomizer via Sdk{Signal}ProviderBuilder's setResource and addResource methods.

That's quite a long and comprehensive list, with interactions which are hard to reason through for a casual user. I'm very reluctant to add additional complexity unless there is a very strong reason.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I've learned, there's one idiomatic way how to write resource providers, which is easy to get wrong.
Do the idea is to take complexity away, not to add it.
At the same time, this is not a topic that is blocking any of my current tasks.

/** Read the data for the resource attributes. */
Optional<D> readData(ConfigProperties config);

/** Registers the attributes that this resource detector can provide. */
void registerAttributes(Builder<D> builder);

/** Greater order means lower priority. The default order is 0. */
@Override
default int order() {
return 0;
}

/** Returns the name of this resource detector. */
String name();

/**
* Returns whether this resource detector is enabled by default. If not, it will only be used if
* explicitly enabled in the configuration.
*/
default boolean defaultEnabled() {
return true;
}

/** A builder for registering attributes that a resource detector can provide. */
interface Builder<D> {
/**
* Adds an attribute to the resource.
*
* @param key the attribute key
* @param getter a function that returns the value of the attribute from the data that is read
* by {@link ResourceDetector#readData(ConfigProperties)}
* @return this builder
* @param <T> the type of the attribute
*/
<T> Builder<D> add(AttributeKey<T> key, Function<D, Optional<T>> getter);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ void defaultMethodsDelegate() {
expectedMap.put("bear", "growl");

Map<String, String> map = makeTestProps();
ConfigProperties properties = DefaultConfigProperties.create(map);
ConfigProperties properties = DefaultConfigProperties.createFromMap(map);
assertThat(properties.getBoolean("test.boolean", false)).isTrue();
assertThat(properties.getString("test.string", "nah")).isEqualTo("str");
assertThat(properties.getDouble("test.double", 65.535)).isEqualTo(5.4);
Expand All @@ -244,7 +244,7 @@ void defaultMethodsDelegate() {

@Test
void defaultMethodsFallBack() {
ConfigProperties properties = DefaultConfigProperties.create(emptyMap());
ConfigProperties properties = DefaultConfigProperties.createFromMap(emptyMap());
assertThat(properties.getBoolean("foo", true)).isTrue();
assertThat(properties.getString("foo", "bar")).isEqualTo("bar");
assertThat(properties.getDouble("foo", 65.535)).isEqualTo(65.535);
Expand All @@ -255,7 +255,7 @@ void defaultMethodsFallBack() {

@Test
void defaultCollectionTypes() {
ConfigProperties properties = DefaultConfigProperties.create(emptyMap());
ConfigProperties properties = DefaultConfigProperties.createFromMap(emptyMap());
assertThat(properties.getList("foo", Arrays.asList("1", "2", "3")))
.containsExactly("1", "2", "3");
assertThat(properties.getList("foo")).isEmpty();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,28 @@
import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.common.AttributesBuilder;
import io.opentelemetry.sdk.autoconfigure.internal.ResourceDetectorReader;
import io.opentelemetry.sdk.autoconfigure.internal.SpiHelper;
import io.opentelemetry.sdk.autoconfigure.spi.ConfigProperties;
import io.opentelemetry.sdk.autoconfigure.spi.ConfigurationException;
import io.opentelemetry.sdk.autoconfigure.spi.Ordered;
import io.opentelemetry.sdk.autoconfigure.spi.ResourceProvider;
import io.opentelemetry.sdk.autoconfigure.spi.internal.ConditionalResourceProvider;
import io.opentelemetry.sdk.autoconfigure.spi.internal.DefaultConfigProperties;
import io.opentelemetry.sdk.autoconfigure.spi.internal.ResourceDetector;
import io.opentelemetry.sdk.resources.Resource;
import io.opentelemetry.sdk.resources.ResourceBuilder;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.nio.charset.StandardCharsets;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.function.BiFunction;
import javax.annotation.Nullable;

/**
* Auto-configuration for the OpenTelemetry {@link Resource}.
Expand Down Expand Up @@ -81,6 +87,7 @@ public static Resource createEnvironmentResource(ConfigProperties config) {
return Resource.create(resourceAttributes.build());
}

@SuppressWarnings({"rawtypes", "unchecked"})
static Resource configureResource(
ConfigProperties config,
SpiHelper spiHelper,
Expand All @@ -91,26 +98,70 @@ static Resource configureResource(
new HashSet<>(config.getList("otel.java.enabled.resource.providers"));
Set<String> disabledProviders =
new HashSet<>(config.getList("otel.java.disabled.resource.providers"));
for (ResourceProvider resourceProvider : spiHelper.loadOrdered(ResourceProvider.class)) {
if (!enabledProviders.isEmpty()
&& !enabledProviders.contains(resourceProvider.getClass().getName())) {
continue;
List<Ordered> providers = (List) spiHelper.load(ResourceProvider.class);
providers.addAll(spiHelper.load(ResourceDetector.class));
providers.sort(Comparator.comparingInt(Ordered::order));
for (Ordered ordered : providers) {
if (ordered instanceof ResourceProvider) {
ResourceProvider provider = (ResourceProvider) ordered;

if (!isEnabled(
provider.getClass().getName(),
enabledProviders,
disabledProviders,
/* defaultEnabled= */ true,
/* explicitEnabled= */ null)) {
continue;
}
if (provider instanceof ConditionalResourceProvider
&& !((ConditionalResourceProvider) provider).shouldApply(config, result)) {
continue;
}
result = result.merge(provider.createResource(config));
} else {
ResourceDetector<Object> detector = (ResourceDetector<Object>) ordered;

Boolean explictEnabled =
config.getBoolean(String.format("otel.resource.provider.%s.enabled", detector.name()));
if (!isEnabled(
detector.getClass().getName(),
enabledProviders,
disabledProviders,
detector.defaultEnabled(),
explictEnabled)) {
continue;
}

ResourceDetectorReader<Object> reader = new ResourceDetectorReader<>(detector);
if (reader.shouldApply(config, result)) {
result = result.merge(reader.createResource(config, result));
}
}
if (disabledProviders.contains(resourceProvider.getClass().getName())) {
continue;
}
if (resourceProvider instanceof ConditionalResourceProvider
&& !((ConditionalResourceProvider) resourceProvider).shouldApply(config, result)) {
continue;
}
result = result.merge(resourceProvider.createResource(config));
}

result = filterAttributes(result, config);

return resourceCustomizer.apply(result, config);
}

static boolean isEnabled(
String className,
Set<String> enabledProviders,
Set<String> disabledProviders,
boolean defaultEnabled,
@Nullable Boolean explicitEnabled) {
if (explicitEnabled != null) {
return explicitEnabled;
}
if (!enabledProviders.isEmpty() && !enabledProviders.contains(className)) {
return false;
}
if (disabledProviders.contains(className)) {
return false;
}
return defaultEnabled;
}

// visible for testing
static Resource filterAttributes(Resource resource, ConfigProperties configProperties) {
Set<String> disabledKeys = new HashSet<>(configProperties.getList(DISABLED_ATTRIBUTE_KEYS));
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
/*
* Copyright The OpenTelemetry Authors
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.sdk.autoconfigure.internal;

import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.common.AttributesBuilder;
import io.opentelemetry.sdk.autoconfigure.spi.ConfigProperties;
import io.opentelemetry.sdk.autoconfigure.spi.internal.ResourceDetector;
import io.opentelemetry.sdk.resources.Resource;
import java.util.HashMap;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.function.Function;

@SuppressWarnings({"unchecked", "rawtypes"})
public final class ResourceDetectorReader<D> {

private final ResourceDetector<D> resourceDetector;

public class AttributeBuilder implements ResourceDetector.Builder<D> {

private AttributeBuilder() {}

@Override
public <T> AttributeBuilder add(AttributeKey<T> key, Function<D, Optional<T>> getter) {
attributeGetters.put((AttributeKey) key, Objects.requireNonNull((Function) getter));
return this;
}
}

private final Map<AttributeKey<Object>, Function<D, Optional<?>>> attributeGetters =
new HashMap<>();

public ResourceDetectorReader(ResourceDetector<D> resourceDetector) {
this.resourceDetector = resourceDetector;
resourceDetector.registerAttributes(new AttributeBuilder());
}

public boolean shouldApply(ConfigProperties config, Resource existing) {
Map<String, String> resourceAttributes = getResourceAttributes(config);
return attributeGetters.keySet().stream()
.allMatch(key -> shouldUpdate(config, existing, key, resourceAttributes));
}

public Resource createResource(ConfigProperties config, Resource existing) {
return resourceDetector
.readData(config)
.map(
data -> {
Map<String, String> resourceAttributes = getResourceAttributes(config);
AttributesBuilder builder = Attributes.builder();
attributeGetters.entrySet().stream()
.filter(e -> shouldUpdate(config, existing, e.getKey(), resourceAttributes))
.forEach(
e ->
e.getValue()
.apply(data)
.ifPresent(value -> putAttribute(builder, e.getKey(), value)));
return Resource.create(builder.build());
})
.orElse(Resource.empty());
}

private static <T> void putAttribute(AttributesBuilder builder, AttributeKey<T> key, T value) {
builder.put(key, value);
}

private static Map<String, String> getResourceAttributes(ConfigProperties config) {
return config.getMap("otel.resource.attributes");
}

private static boolean shouldUpdate(
ConfigProperties config,
Resource existing,
AttributeKey<?> key,
Map<String, String> resourceAttributes) {
if (resourceAttributes.containsKey(key.getKey())) {
return false;
}

Object value = existing.getAttribute(key);

if (key.getKey().equals("service.name")) {
return config.getString("otel.service.name") == null && "unknown_service:java".equals(value);
}

return value == null;
}
}
Loading
Loading