diff --git a/boolean-conversion-proposal.md b/boolean-conversion-proposal.md new file mode 100644 index 00000000000..0f8fdd56b8f --- /dev/null +++ b/boolean-conversion-proposal.md @@ -0,0 +1,139 @@ +# Proposal: Standardize Configuration Type Conversion Behavior + +## Summary + +We propose standardizing configuration type conversion behavior across all data types in the next major release, specifically changing how invalid boolean values are handled to match the behavior of other numeric types. + +## Current Problem + +### Inconsistent Behavior Between Data Types + +Currently, our configuration system exhibits inconsistent behavior when handling invalid values: + +**Numeric Types (Integer, Float, Double, Long):** +```java +// Environment: DD_SOME_INT=invalid_number +int value = configProvider.getInteger("some.int", 42); +// Result: 42 (default value) ✅ +``` + +**Boolean Type:** +```java +// Environment: DD_SOME_BOOL=invalid_boolean +boolean value = configProvider.getBoolean("some.bool", true); +// Result: false (hardcoded fallback) ❌ +``` + +### Why This Is Problematic + +1. **Unexpected Behavior**: Users expect invalid values to fall back to their explicitly provided defaults +2. **Silent Failures**: Invalid boolean configurations silently become `false`, making debugging difficult +3. **API Inconsistency**: Different data types behave differently for the same logical error condition +4. **Code Complexity**: Requires custom exception handling and special-case logic + +## Current Implementation (Temporary Workaround) + +To maintain backward compatibility while fixing our test suite, we implemented a temporary solution: + +```java +// ConfigConverter.java +static class InvalidBooleanValueException extends IllegalArgumentException { + // Custom exception for backward compatibility +} + +public static Boolean booleanValueOf(String value) { + // ... validation logic ... + throw new InvalidBooleanValueException("Invalid boolean value: " + value); +} + +// ConfigProvider.java +catch (ConfigConverter.InvalidBooleanValueException ex) { + // Special case: return false instead of default for backward compatibility + return (T) Boolean.FALSE; +} +``` + +This approach works but adds complexity and maintains the inconsistent behavior. + +## Proposed Solution + +### For Next Major Release: Standardize All Type Conversions + +1. **Remove Custom Boolean Logic**: Eliminate `InvalidBooleanValueException` and special handling +2. **Consistent Exception Handling**: All invalid type conversions throw `IllegalArgumentException` +3. **Consistent Fallback Behavior**: All types fall back to user-provided defaults + +### After the Change + +```java +// All types will behave consistently: +int intValue = configProvider.getInteger("key", 42); // invalid → 42 +boolean boolValue = configProvider.getBoolean("key", true); // invalid → true +float floatValue = configProvider.getFloat("key", 3.14f); // invalid → 3.14f +``` + +## Implementation Plan + +### Phase 1: Next Major Release +- Remove `InvalidBooleanValueException` class +- Update `ConfigConverter.booleanValueOf()` to throw `IllegalArgumentException` +- Remove special boolean handling from `ConfigProvider.get()` +- Update documentation and migration guide + +### Phase 2: Communication +- Release notes highlighting the breaking change +- Update configuration documentation with examples +- Provide migration guidance for affected users + +## Breaking Change Impact + +### Who Is Affected +- Users who rely on invalid boolean values defaulting to `false` +- Applications that depend on the current behavior for error handling + +### Migration Path +Users can adapt by: +1. **Fixing Invalid Configurations**: Update configurations to use valid boolean values +2. **Adjusting Defaults**: If they want `false` as fallback, explicitly set `false` as the default +3. **Adding Validation**: Implement application-level validation if needed + +### Example Migration +```java +// Before (relied on invalid → false) +boolean feature = configProvider.getBoolean("feature.enabled", true); + +// After (explicit about wanting false for invalid values) +boolean feature = configProvider.getBoolean("feature.enabled", false); +``` + +## Benefits + +### For Users +- **Predictable Behavior**: Invalid values consistently fall back to provided defaults +- **Better Debugging**: Clear error signals when configurations are invalid +- **Explicit Intent**: Default values clearly express intended fallback behavior + +### For Maintainers +- **Simplified Codebase**: Remove custom exception types and special-case logic +- **Consistent Testing**: All type conversions can be tested with the same patterns +- **Reduced Complexity**: Fewer edge cases and branches to maintain + +## Recommendation + +We recommend implementing this change in the next major release (e.g., v2.0) because: + +1. **Improves User Experience**: More predictable and debuggable configuration behavior +2. **Reduces Technical Debt**: Eliminates custom workarounds and special cases +3. **Aligns with Principle of Least Surprise**: Consistent behavior across all data types +4. **Proper Breaking Change Window**: Major release is the appropriate time for behavior changes + +## Questions for Discussion + +1. Do we agree this inconsistency is worth fixing with a breaking change? +2. Should we provide any additional migration tooling or validation? +3. Are there other configuration behaviors we should standardize at the same time? +4. What timeline works best for communicating this change to users? + +--- + +**Next Steps**: If approved, we'll create implementation issues and begin updating documentation for the next major release cycle. diff --git a/internal-api/src/main/java/datadog/trace/api/ConfigCollector.java b/internal-api/src/main/java/datadog/trace/api/ConfigCollector.java index f4cfda6877b..6e4ebc0969c 100644 --- a/internal-api/src/main/java/datadog/trace/api/ConfigCollector.java +++ b/internal-api/src/main/java/datadog/trace/api/ConfigCollector.java @@ -27,6 +27,11 @@ public void put(String key, Object value, ConfigOrigin origin) { collected.put(key, setting); } + public void put(String key, Object value, ConfigOrigin origin, String configId) { + ConfigSetting setting = ConfigSetting.of(key, value, origin, configId); + collected.put(key, setting); + } + public void putAll(Map keysAndValues, ConfigOrigin origin) { // attempt merge+replace to avoid collector seeing partial update Map merged = diff --git a/internal-api/src/main/java/datadog/trace/api/ConfigSetting.java b/internal-api/src/main/java/datadog/trace/api/ConfigSetting.java index b688d5b477d..9b94b444a02 100644 --- a/internal-api/src/main/java/datadog/trace/api/ConfigSetting.java +++ b/internal-api/src/main/java/datadog/trace/api/ConfigSetting.java @@ -11,19 +11,26 @@ public final class ConfigSetting { public final String key; public final Object value; public final ConfigOrigin origin; + /** The config ID associated with this setting, or {@code null} if not applicable. */ + public final String configId; private static final Set CONFIG_FILTER_LIST = new HashSet<>( Arrays.asList("DD_API_KEY", "dd.api-key", "dd.profiling.api-key", "dd.profiling.apikey")); public static ConfigSetting of(String key, Object value, ConfigOrigin origin) { - return new ConfigSetting(key, value, origin); + return new ConfigSetting(key, value, origin, null); } - private ConfigSetting(String key, Object value, ConfigOrigin origin) { + public static ConfigSetting of(String key, Object value, ConfigOrigin origin, String configId) { + return new ConfigSetting(key, value, origin, configId); + } + + private ConfigSetting(String key, Object value, ConfigOrigin origin, String configId) { this.key = key; this.value = CONFIG_FILTER_LIST.contains(key) ? "" : value; this.origin = origin; + this.configId = configId; } public String normalizedKey() { @@ -99,12 +106,15 @@ public boolean equals(Object o) { if (this == o) return true; if (o == null || getClass() != o.getClass()) return false; ConfigSetting that = (ConfigSetting) o; - return key.equals(that.key) && Objects.equals(value, that.value) && origin == that.origin; + return key.equals(that.key) + && Objects.equals(value, that.value) + && origin == that.origin + && Objects.equals(configId, that.configId); } @Override public int hashCode() { - return Objects.hash(key, value, origin); + return Objects.hash(key, value, origin, configId); } @Override @@ -117,6 +127,8 @@ public String toString() { + stringValue() + ", origin=" + origin + + ", configId=" + + configId + '}'; } } diff --git a/internal-api/src/main/java/datadog/trace/bootstrap/config/provider/ConfigProvider.java b/internal-api/src/main/java/datadog/trace/bootstrap/config/provider/ConfigProvider.java index f7b890f653e..f93dd3c7d2c 100644 --- a/internal-api/src/main/java/datadog/trace/bootstrap/config/provider/ConfigProvider.java +++ b/internal-api/src/main/java/datadog/trace/bootstrap/config/provider/ConfigProvider.java @@ -79,7 +79,7 @@ public String getString(String key, String defaultValue, String... aliases) { String value = source.get(key, aliases); if (value != null) { if (collectConfig) { - ConfigCollector.get().put(key, value, source.origin()); + ConfigCollector.get().put(key, value, source.origin(), getConfigIdFromSource(source)); } return value; } @@ -198,7 +198,8 @@ private T get(String key, T defaultValue, Class type, String... aliases) T value = ConfigConverter.valueOf(sourceValue, type); if (value != null) { if (collectConfig) { - ConfigCollector.get().put(key, sourceValue, source.origin()); + ConfigCollector.get() + .put(key, sourceValue, source.origin(), getConfigIdFromSource(source)); } return value; } @@ -247,6 +248,7 @@ public List getSpacedList(String key) { public Map getMergedMap(String key, String... aliases) { Map merged = new HashMap<>(); ConfigOrigin origin = ConfigOrigin.DEFAULT; + String configId = null; // System properties take precedence over env // prior art: // https://docs.spring.io/spring-boot/docs/1.5.6.RELEASE/reference/html/boot-features-external-config.html @@ -255,6 +257,7 @@ public Map getMergedMap(String key, String... aliases) { String value = sources[i].get(key, aliases); Map parsedMap = ConfigConverter.parseMap(value, key); if (!parsedMap.isEmpty()) { + configId = getConfigIdFromSource(sources[i]); if (origin != ConfigOrigin.DEFAULT) { // if we already have a non-default origin, the value is calculated from multiple sources origin = ConfigOrigin.CALCULATED; @@ -265,7 +268,7 @@ public Map getMergedMap(String key, String... aliases) { merged.putAll(parsedMap); } if (collectConfig) { - ConfigCollector.get().put(key, merged, origin); + ConfigCollector.get().put(key, merged, origin, configId); } return merged; } @@ -273,6 +276,7 @@ public Map getMergedMap(String key, String... aliases) { public Map getMergedTagsMap(String key, String... aliases) { Map merged = new HashMap<>(); ConfigOrigin origin = ConfigOrigin.DEFAULT; + String configId = null; // System properties take precedence over env // prior art: // https://docs.spring.io/spring-boot/docs/1.5.6.RELEASE/reference/html/boot-features-external-config.html @@ -282,6 +286,7 @@ public Map getMergedTagsMap(String key, String... aliases) { Map parsedMap = ConfigConverter.parseTraceTagsMap(value, ':', Arrays.asList(',', ' ')); if (!parsedMap.isEmpty()) { + configId = getConfigIdFromSource(sources[i]); if (origin != ConfigOrigin.DEFAULT) { // if we already have a non-default origin, the value is calculated from multiple sources origin = ConfigOrigin.CALCULATED; @@ -292,7 +297,7 @@ public Map getMergedTagsMap(String key, String... aliases) { merged.putAll(parsedMap); } if (collectConfig) { - ConfigCollector.get().put(key, merged, origin); + ConfigCollector.get().put(key, merged, origin, configId); } return merged; } @@ -300,6 +305,7 @@ public Map getMergedTagsMap(String key, String... aliases) { public Map getOrderedMap(String key) { LinkedHashMap merged = new LinkedHashMap<>(); ConfigOrigin origin = ConfigOrigin.DEFAULT; + String configId = null; // System properties take precedence over env // prior art: // https://docs.spring.io/spring-boot/docs/1.5.6.RELEASE/reference/html/boot-features-external-config.html @@ -308,6 +314,7 @@ public Map getOrderedMap(String key) { String value = sources[i].get(key); Map parsedMap = ConfigConverter.parseOrderedMap(value, key); if (!parsedMap.isEmpty()) { + configId = getConfigIdFromSource(sources[i]); if (origin != ConfigOrigin.DEFAULT) { // if we already have a non-default origin, the value is calculated from multiple sources origin = ConfigOrigin.CALCULATED; @@ -318,7 +325,7 @@ public Map getOrderedMap(String key) { merged.putAll(parsedMap); } if (collectConfig) { - ConfigCollector.get().put(key, merged, origin); + ConfigCollector.get().put(key, merged, origin, configId); } return merged; } @@ -327,6 +334,7 @@ public Map getMergedMapWithOptionalMappings( String defaultPrefix, boolean lowercaseKeys, String... keys) { Map merged = new HashMap<>(); ConfigOrigin origin = ConfigOrigin.DEFAULT; + String configId = null; // System properties take precedence over env // prior art: // https://docs.spring.io/spring-boot/docs/1.5.6.RELEASE/reference/html/boot-features-external-config.html @@ -337,6 +345,7 @@ public Map getMergedMapWithOptionalMappings( Map parsedMap = ConfigConverter.parseMapWithOptionalMappings(value, key, defaultPrefix, lowercaseKeys); if (!parsedMap.isEmpty()) { + configId = getConfigIdFromSource(sources[i]); if (origin != ConfigOrigin.DEFAULT) { // if we already have a non-default origin, the value is calculated from multiple // sources @@ -348,7 +357,7 @@ public Map getMergedMapWithOptionalMappings( merged.putAll(parsedMap); } if (collectConfig) { - ConfigCollector.get().put(key, merged, origin); + ConfigCollector.get().put(key, merged, origin, configId); } } return merged; @@ -520,6 +529,13 @@ private static Properties loadConfigurationFile(ConfigProvider configProvider) { return properties; } + private static String getConfigIdFromSource(Source source) { + if (source instanceof StableConfigSource) { + return ((StableConfigSource) source).getConfigId(); + } + return null; + } + public abstract static class Source { public final String get(String key, String... aliases) { String value = get(key); diff --git a/internal-api/src/main/java/datadog/trace/bootstrap/config/provider/stableconfig/StableConfig.java b/internal-api/src/main/java/datadog/trace/bootstrap/config/provider/stableconfig/StableConfig.java index f5b09d0d579..472b8e845bb 100644 --- a/internal-api/src/main/java/datadog/trace/bootstrap/config/provider/stableconfig/StableConfig.java +++ b/internal-api/src/main/java/datadog/trace/bootstrap/config/provider/stableconfig/StableConfig.java @@ -17,7 +17,7 @@ public final class StableConfig { public StableConfig(Object yaml) { Map map = (Map) yaml; - this.configId = String.valueOf(map.get("config_id")); + this.configId = map.get("config_id") == null ? null : String.valueOf(map.get("config_id")); this.apmConfigurationDefault = unmodifiableMap( (Map) map.getOrDefault("apm_configuration_default", emptyMap())); diff --git a/internal-api/src/test/groovy/datadog/trace/api/ConfigCollectorTest.groovy b/internal-api/src/test/groovy/datadog/trace/api/ConfigCollectorTest.groovy index fd16d9b56bf..5e49e44d84b 100644 --- a/internal-api/src/test/groovy/datadog/trace/api/ConfigCollectorTest.groovy +++ b/internal-api/src/test/groovy/datadog/trace/api/ConfigCollectorTest.groovy @@ -9,6 +9,7 @@ import datadog.trace.api.config.TraceInstrumentationConfig import datadog.trace.api.config.TracerConfig import datadog.trace.api.iast.telemetry.Verbosity import datadog.trace.api.naming.SpanNaming +import datadog.trace.bootstrap.config.provider.ConfigProvider import datadog.trace.test.util.DDSpecification import datadog.trace.util.Strings @@ -228,4 +229,24 @@ class ConfigCollectorTest extends DDSpecification { "logs.injection.enabled" | "false" "trace.sample.rate" | "0.3" } + + def "config id is null for non-StableConfigSource"() { + setup: + def key = "test.key" + def value = "test-value" + injectSysConfig(key, value) + + when: + // Trigger config collection by getting a value + ConfigProvider.getInstance().getString(key) + def settings = ConfigCollector.get().collect() + + then: + // Verify the config was collected but without a config ID + def setting = settings.get(key) + setting != null + setting.configId == null + setting.value == value + setting.origin == ConfigOrigin.JVM_PROP + } } diff --git a/internal-api/src/test/groovy/datadog/trace/bootstrap/config/provider/StableConfigSourceTest.groovy b/internal-api/src/test/groovy/datadog/trace/bootstrap/config/provider/StableConfigSourceTest.groovy index 4dee86f5ae7..866158edad0 100644 --- a/internal-api/src/test/groovy/datadog/trace/bootstrap/config/provider/StableConfigSourceTest.groovy +++ b/internal-api/src/test/groovy/datadog/trace/bootstrap/config/provider/StableConfigSourceTest.groovy @@ -1,5 +1,7 @@ package datadog.trace.bootstrap.config.provider +import datadog.trace.api.ConfigCollector + import static java.util.Collections.singletonMap import datadog.trace.api.ConfigOrigin @@ -251,7 +253,42 @@ class StableConfigSourceTest extends DDSpecification { } } - // Use this if you want to explicitly write/test configId + def "test config id exists in ConfigCollector when using StableConfigSource"() { + given: + Path filePath = Files.createTempFile("testFile_", ".yaml") + String expectedConfigId = "123" + + // Create YAML content with config_id and some configuration + def yamlContent = """ +config_id: ${expectedConfigId} +apm_configuration_default: + DD_SERVICE: test-service + DD_ENV: test-env +""" + Files.write(filePath, yamlContent.getBytes()) + + // Clear any existing collected config + ConfigCollector.get().collect().clear() + + when: + // Create StableConfigSource and ConfigProvider + StableConfigSource stableConfigSource = new StableConfigSource(filePath.toString(), ConfigOrigin.LOCAL_STABLE_CONFIG) + ConfigProvider configProvider = new ConfigProvider(stableConfigSource) + + // Trigger config collection by getting a value + configProvider.getString("SERVICE", "default-service") + + then: + def collectedConfigs = ConfigCollector.get().collect() + def serviceSetting = collectedConfigs.get("SERVICE") + serviceSetting.configId == expectedConfigId + serviceSetting.value == "test-service" + serviceSetting.origin == ConfigOrigin.LOCAL_STABLE_CONFIG + + cleanup: + Files.delete(filePath) + } + def writeFileRaw(Path filePath, String configId, String data) { data = configId + "\n" + data StandardOpenOption[] openOpts = [StandardOpenOption.WRITE] as StandardOpenOption[] diff --git a/telemetry/src/main/java/datadog/telemetry/TelemetryRequestBody.java b/telemetry/src/main/java/datadog/telemetry/TelemetryRequestBody.java index 62fac7d9afe..f84a4331fd7 100644 --- a/telemetry/src/main/java/datadog/telemetry/TelemetryRequestBody.java +++ b/telemetry/src/main/java/datadog/telemetry/TelemetryRequestBody.java @@ -230,6 +230,9 @@ public void writeConfiguration(ConfigSetting configSetting) throws IOException { bodyWriter.name("value").value(configSetting.stringValue()); bodyWriter.setSerializeNulls(false); bodyWriter.name("origin").value(configSetting.origin.value); + if (configSetting.configId != null) { + bodyWriter.name("config_id").value(configSetting.configId); + } bodyWriter.endObject(); } diff --git a/telemetry/src/test/groovy/datadog/telemetry/EventSourceTest.groovy b/telemetry/src/test/groovy/datadog/telemetry/EventSourceTest.groovy index 5a492c18171..5da3f0a3ea1 100644 --- a/telemetry/src/test/groovy/datadog/telemetry/EventSourceTest.groovy +++ b/telemetry/src/test/groovy/datadog/telemetry/EventSourceTest.groovy @@ -56,7 +56,7 @@ class EventSourceTest extends DDSpecification{ where: eventType | eventQueueName | eventInstance - "Config Change" | "configChangeQueue" | new ConfigSetting("key", "value", ConfigOrigin.ENV) + "Config Change" | "configChangeQueue" | ConfigSetting.of("key", "value", ConfigOrigin.ENV) "Integration" | "integrationQueue" | new Integration("name", true) "Dependency" | "dependencyQueue" | new Dependency("name", "version", "type", null) "Metric" | "metricQueue" | new Metric()