Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix IConfiguration adapter #365

Merged
merged 8 commits into from
Aug 30, 2023
Merged

Fix IConfiguration adapter #365

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
ae33ce7
Fix IConfiguration adapter case sensitivity
Arkatufus Aug 28, 2023
25a5ae9
Add normalization flag and update documentation
Arkatufus Aug 29, 2023
fa430a7
Merge branch 'dev' into Fix_IConfiguration_adapter_2
Arkatufus Aug 30, 2023
8395eb9
Update documentation
Arkatufus Aug 30, 2023
b54efc5
Merge branch 'Fix_IConfiguration_adapter_2' of github.com:Arkatufus/A…
Arkatufus Aug 30, 2023
743fdff
Update API Approval list
Arkatufus Aug 30, 2023
06fe091
Fix typo
Arkatufus Aug 30, 2023
1aea836
Fix unit test
Arkatufus Aug 30, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 113 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -312,8 +312,9 @@ The `dependencyResolver.Props<MySingletonDiActor>()` call will leverage the `Act
## IConfiguration To HOCON Adapter

The `AddHocon` extension method can convert `Microsoft.Extensions.Configuration` `IConfiguration` into HOCON `Config` instance and adds it to the ActorSystem being configured.
* All variable name are automatically converted to lower case.
* All "." (period) in the `IConfiguration` key will be treated as a HOCON object key separator
* Unlike `IConfiguration`, all HOCON key names are **case sensitive**.
* **Unless enclosed inside double quotes**, all "." (period) in the `IConfiguration` key will be treated as a HOCON object key separator
* `IConfiguration` **does not support object composition**, if you declare the same key multiple times inside multiple configuration providers (JSON/environment variables/etc), **only the last one declared will take effect**.
* For environment variable configuration provider:
* "__" (double underline) will be converted to "." (period).
* "_" (single underline) will be converted to "-" (dash).
Expand All @@ -340,16 +341,33 @@ __Example:__
}
}
}
```
```

Environment variables:

```powershell
AKKA__ACTOR__TELEMETRY__ENABLED=true
AKKA__ACTOR__TELEMETRY__ENABLE=true
AKKA__CLUSTER__SEED_NODES__0=akka.tcp//mySystem@localhost:4055
AKKA__CLUSTER__SEED_NODES__1=akka.tcp//mySystem@localhost:4056
AKKA__CLUSTER__SEED_NODE_TIMEOUT=00:00:05
```
```

Note the integer parseable key inside the seed-nodes configuration, seed-nodes will be parsed as an array. These environment variables will be parsed as HOCON settings:

```hocon
akka {
actor {
telemetry.enabled: on
}
cluster {
seed-nodes: [
"akka.tcp//mySystem@localhost:4055",
"akka.tcp//mySystem@localhost:4056"
]
seed-node-timeout: 5s
}
}
```

Example code:

Expand Down Expand Up @@ -392,6 +410,96 @@ var host = new HostBuilder()
});
```

### Advanced Usage
Arkatufus marked this conversation as resolved.
Show resolved Hide resolved

This advanced usage of the `IConfiguration` adapter is solely used for edge cases where HOCON key capitalization needs to be preserved, such as declaring serialization binding. Note that when you're using this feature, none of the keys are normalized, you will have to write all of your keys in a HOCON compatible way.

`appsettings.json`:

```json
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
}
},
"AllowedHosts": "*",
"akka": {
"\"Key.With.Dots\"": "Key Value",
"cluster": {
"roles": [ "front-end", "back-end" ],
"min-nr-of-members": 3,
"log-info": true
}
}
}
```

Note that "Key.With.Dots" needs to be inside escaped double quotes, this is a HOCON requirement that preserves the "." (period) inside HOCON property names.

Environment variables:

```powershell
PS C:/> [Environment]::SetEnvironmentVariable('akka__actor__telemetry__enabled', 'true')
PS C:/> [Environment]::SetEnvironmentVariable('akka__actor__serialization_bindings__"System.Object"', 'hyperion')
PS C:/> [Environment]::SetEnvironmentVariable('akka__cluster__seed_nodes__0', 'akka.tcp//mySystem@localhost:4055')
PS C:/> [Environment]::SetEnvironmentVariable('akka__cluster__seed_nodes__1', 'akka.tcp//mySystem@localhost:4056')
PS C:/> [Environment]::SetEnvironmentVariable('akka__cluster__seed_node_timeout', '00:00:05')
```

Note that:
1. All of the environment variable names are in lower case, except "System.Object" where it needs to preserve name capitalization.
2. To set serialization binding via environment variable, you have to use "." (period) instead of "__" (double underscore), this might be problematic for some shell scripts and there is no way of getting around this.

Example code:

```csharp
/*
Both appsettings.json and environment variables are combined
into HOCON configuration:

akka {
"Key.With.Dots": Key Value
actor {
telemetry.enabled: on
serialization-bindings {
"System.Object" = hyperion
}
}
cluster {
roles: [ "front-end", "back-end" ]
seed-nodes: [
"akka.tcp//mySystem@localhost:4055",
"akka.tcp//mySystem@localhost:4056"
]
min-nr-of-members: 3
seed-node-timeout: 5s
log-info: true
}
}
*/
var host = new HostBuilder()
.ConfigureHostConfiguration(builder =>
{
// Setup IConfiguration to load from appsettings.json and
// environment variables
builder
.AddJsonFile("appsettings.json")
.AddEnvironmentVariables();
})
.ConfigureServices((context, services) =>
{
services.AddAkka("mySystem", (builder, provider) =>
{
// convert IConfiguration to HOCON
var akkaConfig = context.Configuration.GetSection("akka");
// Note the last method argument is set to false
builder.AddHocon(akkaConfig, HoconAddMode.Prepend, false);
});
});
```

[Back to top](#akkahosting)

<a id="microsoftextensionslogging-integration"></a>
Expand Down
141 changes: 122 additions & 19 deletions src/Akka.Hosting.Tests/Configuration/ConfigurationHoconAdapterTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using Akka.Configuration;
using Akka.Hosting.Configuration;
Expand All @@ -23,6 +24,9 @@ public class ConfigurationHoconAdapterTest
{
private const string ConfigSource = @"{
""akka"": {
""actor.serialization_bindings"" : {
""\""System.Int32\"""": ""json""
},
""cluster"": {
""roles"": [ ""front-end"", ""back-end"" ],
""role"" : {
Expand All @@ -44,7 +48,7 @@ public class ConfigurationHoconAdapterTest
""test4"": 4
}";

private readonly Config _config;
private readonly IConfigurationRoot _root;

public ConfigurationHoconAdapterTest()
{
Expand All @@ -55,32 +59,130 @@ public ConfigurationHoconAdapterTest()
Environment.SetEnvironmentVariable("AKKA__TEST_VALUE_2__22", "TWO");
Environment.SetEnvironmentVariable("AKKA__TEST_VALUE_2__1", "ONE");

Environment.SetEnvironmentVariable("akka__test_value_3__a", "a value");
Environment.SetEnvironmentVariable("akka__test_value_3__b", "b value");
Environment.SetEnvironmentVariable("akka__test_value_3__c__d", "d");
Environment.SetEnvironmentVariable("akka__test_value_4__0", "zero");
Environment.SetEnvironmentVariable("akka__test_value_4__22", "two");
Environment.SetEnvironmentVariable("akka__test_value_4__1", "one");
Environment.SetEnvironmentVariable("akka__actor__serialization_bindings2__\"System.Object\"", "hyperion");

using var stream = new MemoryStream(Encoding.UTF8.GetBytes(ConfigSource));
var configuration = new ConfigurationBuilder()
_root = new ConfigurationBuilder()
.AddJsonStream(stream)
.AddEnvironmentVariables()
.Build();
_config = configuration.ToHocon();
}

#region Adapter unit tests

[Fact(DisplayName = "Adaptor should read environment variable sourced configuration correctly")]
[Fact(DisplayName = "Normalized adaptor should read environment variable sourced configuration correctly")]
public void EnvironmentVariableTest()
{
_config.GetString("akka.test-value-1.a").Should().Be("A VALUE");
_config.GetString("akka.test-value-1.b").Should().Be("B VALUE");
_config.GetString("akka.test-value-1.c.d").Should().Be("D");
var array = _config.GetStringList("akka.test-value-2");
var config = _root.ToHocon();
// should be normalized
config.GetString("akka.test-value-1.a").Should().Be("A VALUE");
config.GetString("AKKA.TEST-VALUE-1.A").Should().BeNull();

// should be normalized
config.GetString("akka.test-value-1.b").Should().Be("B VALUE");
config.GetString("AKKA.TEST-VALUE-1.B").Should().BeNull();

// should be normalized
config.GetString("akka.test-value-1.c.d").Should().Be("D");
config.GetString("AKKA.TEST-VALUE-1.C.D").Should().BeNull();

// should be normalized
var array = config.GetStringList("akka.test-value-2");
array[0].Should().Be("ZERO");
array[1].Should().Be("ONE");
array[2].Should().Be("TWO");
config.GetStringList("AKKA.TEST-VALUE-2").Should().BeEmpty();

// proper cased environment vars should read just fine
config.GetString("akka.test-value-3.a").Should().Be("a value");
config.GetString("akka.test-value-3.b").Should().Be("b value");
config.GetString("akka.test-value-3.c.d").Should().Be("d");
array = config.GetStringList("akka.test-value-4");
array[0].Should().Be("zero");
array[1].Should().Be("one");
array[2].Should().Be("two");

// edge case should also be normalized and not usable
var bindings = config.GetConfig("akka.actor.serialization-bindings2").AsEnumerable()
.ToDictionary(kvp => kvp.Key, kvp => kvp.Value);
bindings.ContainsKey("System.Object").Should().BeFalse();
bindings.ContainsKey("system.object").Should().BeTrue();
bindings["system.object"].GetString().Should().Be("hyperion");
}

[Fact(DisplayName = "Non-normalized adaptor should read environment variable sourced configuration correctly")]
public void EnvironmentVariableCaseSensitiveTest()
{
var config = _root.ToHocon(false);

// should not be normalized
config.GetString("akka.TEST-VALUE-1.A").Should().Be("A VALUE");
config.GetString("akka.test-value-1.a").Should().BeNull();

// should not be normalized
config.GetString("akka.TEST-VALUE-1.B").Should().Be("B VALUE");
config.GetString("akka.test-value-1.b").Should().BeNull();

// should not be normalized
config.GetString("akka.TEST-VALUE-1.C.D").Should().Be("D");
config.GetString("akka.test-value-1.c.d").Should().BeNull();

// should not be normalized
config.GetStringList("akka.test-value-2").Should().BeEmpty();
var array = config.GetStringList("akka.TEST-VALUE-2");
array[0].Should().Be("ZERO");
array[1].Should().Be("ONE");
array[2].Should().Be("TWO");

// proper cased environment vars should read just fine
config.GetString("akka.test-value-3.a").Should().Be("a value");
config.GetString("akka.test-value-3.b").Should().Be("b value");
config.GetString("akka.test-value-3.c.d").Should().Be("d");
array = config.GetStringList("akka.test-value-4");
array[0].Should().Be("zero");
array[1].Should().Be("one");
array[2].Should().Be("two");

// edge case should not be normalized and usable
var bindings = config.GetConfig("akka.actor.serialization-bindings2").AsEnumerable()
.ToDictionary(kvp => kvp.Key, kvp => kvp.Value);
bindings.ContainsKey("System.Object").Should().BeTrue();
bindings["System.Object"].GetString().Should().Be("hyperion");
}

[Fact(DisplayName = "Non-normalized Adaptor should read quote enclosed key inside JSON settings correctly")]
public void NonNormalizedJsonQuotedKeyTest()
{
var config = _root.ToHocon(false);
var bindings = config.GetConfig("akka.actor.serialization-bindings").AsEnumerable()
.ToDictionary(kvp => kvp.Key, kvp => kvp.Value);
bindings.ContainsKey("System.Int32").Should().BeTrue();
bindings["System.Int32"].GetString().Should().Be("json");
}

[Fact(DisplayName = "Normalized Adaptor should read quote enclosed key inside JSON settings incorrectly")]
public void NormalizedJsonQuotedKeyTest()
{
var config = _root.ToHocon();
var bindings = config.GetConfig("akka.actor.serialization-bindings").AsEnumerable()
.ToDictionary(kvp => kvp.Key, kvp => kvp.Value);

bindings.ContainsKey("System.Int32").Should().BeFalse();
bindings.ContainsKey("system.int32").Should().BeTrue();
bindings["system.int32"].GetString().Should().Be("json");
}

[Fact(DisplayName = "Adaptor should expand keys")]
public void EncodedKeyTest()
{
var test2 = _config.GetConfig("test2");
var config = _root.ToHocon();
var test2 = config.GetConfig("test2");
test2.Should().NotBeNull();
test2.GetBoolean("a").Should().BeTrue();
test2.GetTimeSpan("b.c").Should().Be(2.Seconds());
Expand All @@ -91,18 +193,19 @@ public void EncodedKeyTest()
[Fact(DisplayName = "Adaptor should convert correctly")]
public void ArrayTest()
{
_config.GetString("test1").Should().Be("test1 content");
_config.GetInt("test3").Should().Be(3);
_config.GetInt("test4").Should().Be(4);
var config = _root.ToHocon();
config.GetString("test1").Should().Be("test1 content");
config.GetInt("test3").Should().Be(3);
config.GetInt("test4").Should().Be(4);

_config.GetStringList("akka.cluster.roles").Should().BeEquivalentTo("front-end", "back-end");
_config.GetInt("akka.cluster.role.back-end").Should().Be(5);
_config.GetString("akka.cluster.app-version").Should().Be("1.0.0");
_config.GetInt("akka.cluster.min-nr-of-members").Should().Be(99);
_config.GetStringList("akka.cluster.seed-nodes").Should()
config.GetStringList("akka.cluster.roles").Should().BeEquivalentTo("front-end", "back-end");
config.GetInt("akka.cluster.role.back-end").Should().Be(5);
config.GetString("akka.cluster.app-version").Should().Be("1.0.0");
config.GetInt("akka.cluster.min-nr-of-members").Should().Be(99);
config.GetStringList("akka.cluster.seed-nodes").Should()
.BeEquivalentTo("akka.tcp://system@somewhere.com:9999");
_config.GetBoolean("akka.cluster.log-info").Should().BeFalse();
_config.GetBoolean("akka.cluster.log-info-verbose").Should().BeTrue();
config.GetBoolean("akka.cluster.log-info").Should().BeFalse();
config.GetBoolean("akka.cluster.log-info-verbose").Should().BeTrue();
}

#endregion
Expand Down
6 changes: 4 additions & 2 deletions src/Akka.Hosting/AkkaHostingExtensions.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,13 +153,15 @@ public static AkkaConfigurationBuilder AddHocon(this AkkaConfigurationBuilder bu
/// <param name="builder">The builder instance being configured.</param>
/// <param name="configuration">The <see cref="IConfiguration"/> instance to be converted to HOCON <see cref="Config"/>.</param>
/// <param name="addMode">The <see cref="HoconAddMode"/> - defaults to appending this HOCON as a fallback.</param>
/// <param name="normalizeKeys"></param>
/// <returns>The same <see cref="AkkaConfigurationBuilder"/> instance originally passed in.</returns>
public static AkkaConfigurationBuilder AddHocon(
this AkkaConfigurationBuilder builder,
IConfiguration configuration,
HoconAddMode addMode)
HoconAddMode addMode,
bool normalizeKeys = true)
{
return builder.AddHoconConfiguration(configuration.ToHocon(), addMode);
return builder.AddHoconConfiguration(configuration.ToHocon(normalizeKeys), addMode);
}

/// <summary>
Expand Down
Loading
Loading