Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 36 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Alternatively, add the following line to your `.csproj` file:
| **InputDirectory**<br>*(Required)* | The directory containing the files that need to be aggregated. | | |
| **InputType** | Specifies the format of the input files. Refer to the [File Types](#file-types) table below for the corresponding file extensions that will be searched for. | `Json`, `Arm`, `Yaml` | `Yaml` |
| **AddSourceProperty** | Adds a `source` property to each object in the output, specifying the filename from which the object originated. | `true`, `false` | `false` |
| **AdditionalProperties** | A set of custom top-level properties to include in the final output. Use `ItemGroup` syntax to define key-value pairs. | | |
| **AdditionalProperties** | A set of custom top-level properties to include in the final output. Use `ItemGroup` syntax to define key-value pairs. See [below](#additional-properties) for usage details. | | |
| **IsQuietMode** | When true, only warning and error logs are generated by the task, suppressing standard informational output. | `true`, `false` | `false` |

## File Types
Expand All @@ -60,18 +60,12 @@ In your `.csproj` file, use the task to aggregate YAML files and output them in
<Project Sdk="Microsoft.NET.Sdk">

<Target Name="AggregateConfigs" BeforeTargets="PrepareForBuild">
<ItemGroup>
<AdditionalProperty Include="ResourceGroup=TestRG" />
<AdditionalProperty Include="Environment=Production" />
</ItemGroup>

<AggregateConfig
InputDirectory="Configs"
OutputFile="$(MSBuildProjectDirectory)\out\output.json"
AddSourceProperty="true"
InputType="Yaml"
OutputType="Json"
AdditionalProperties="@(AdditionalProperty)" />
OutputType="Json" />
</Target>

</Project>
Expand All @@ -81,7 +75,6 @@ In this example:
- The `Configs` directory contains the YAML files to be aggregated.
- The output will be generated as `out/output.json`.
- The `AddSourceProperty` flag adds the source file name to each configuration entry.
- The `AdditionalProperties` are injected into the top-level of the output as custom metadata.

### ARM Template Parameters Output Example

Expand All @@ -91,16 +84,10 @@ You can also generate Azure ARM template parameters. Here's how to modify the co
<Project Sdk="Microsoft.NET.Sdk">

<Target Name="AggregateConfigsForARM" BeforeTargets="PrepareForBuild">
<ItemGroup>
<AdditionalProperty Include="ResourceGroup=TestRG" />
<AdditionalProperty Include="Environment=Production" />
</ItemGroup>

<AggregateConfig
InputDirectory="Configs"
OutputFile="$(MSBuildProjectDirectory)\out\output.parameters.json"
OutputType="Arm"
AdditionalProperties="@(AdditionalProperty)" />
OutputType="Arm" />
</Target>

</Project>
Expand All @@ -110,15 +97,40 @@ You can also generate Azure ARM template parameters. Here's how to modify the co

You can also output the aggregated configuration back into YAML format:

```xml
<Project Sdk="Microsoft.NET.Sdk">

<Target Name="AggregateConfigsToYAML" BeforeTargets="PrepareForBuild">
<AggregateConfig
InputDirectory="Configs"
OutputFile="$(MSBuildProjectDirectory)\out\output.yaml"
OutputType="Yaml" />
</Target>

</Project>
```

### Additional Properties

At build time, you can inject additional properties into the top-level of your output configuration as key-value pairs. Conditionals and variables are supported.

In this example, two additional properties (`ResourceGroup` and `Environment`) are defined and will be included in the YAML output's top-level structure. This allows for dynamic property injection at build time.

```xml
<Project Sdk="Microsoft.NET.Sdk">

<Target Name="AggregateConfigsToYAML" BeforeTargets="PrepareForBuild">
<ItemGroup>
<AdditionalProperty Include="ResourceGroup=TestRG" />
<AdditionalProperty Include="Environment=Production" />
<!-- Define additional properties as key-value pairs -->
<AdditionalProperty Include="ResourceGroup">
<Value>TestRG</Value>
</AdditionalProperty>
<AdditionalProperty Include="Environment">
<Value>Production</Value>
</AdditionalProperty>
</ItemGroup>

<!-- Aggregate configuration into a YAML file -->
<AggregateConfig
InputDirectory="Configs"
OutputFile="$(MSBuildProjectDirectory)\out\output.yaml"
Expand All @@ -129,6 +141,11 @@ You can also output the aggregated configuration back into YAML format:
</Project>
```

#### Explanation:
- **Additional Properties:** The `AdditionalProperty` items store key-value pairs (`ResourceGroup=TestRG` and `Environment=Production`). The key is set in the `Include` attribute, and the value is defined in a nested `<Value>` element.
- **ItemGroup:** Groups the additional properties, which will later be referenced in the task as `@(AdditionalProperty)`.
- **AggregateConfig Task:** This task collects the configurations from the `Configs` directory and aggregates them into a YAML output file. The `AdditionalProperties` item group is passed to the task, ensuring that the properties are injected into the top-level of the output.

### Embedding Output Files as Resources

You can embed the output files (such as the generated JSON) as resources in the assembly. This allows them to be accessed from within your code as embedded resources.
Expand All @@ -137,16 +154,10 @@ You can embed the output files (such as the generated JSON) as resources in the
<Project Sdk="Microsoft.NET.Sdk">

<Target Name="AggregateConfigs" BeforeTargets="PrepareForBuild">
<ItemGroup>
<AdditionalProperty Include="ResourceGroup=TestRG" />
<AdditionalProperty Include="Environment=Production" />
</ItemGroup>

<AggregateConfig
InputDirectory="Configs"
OutputFile="$(MSBuildProjectDirectory)\out\output.json"
OutputType="Json"
AdditionalProperties="@(AdditionalProperty)" />
OutputType="Json" />

<!-- Embed output.json as a resource in the assembly -->
<ItemGroup>
Expand Down
9 changes: 5 additions & 4 deletions src/Task/AggregateConfig.cs
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
using AggregateConfigBuildTask.FileHandlers;
using Microsoft.Build.Framework;
using System;
using System;
using System.IO;
using System.Reflection;
using System.Runtime.CompilerServices;
using AggregateConfigBuildTask.FileHandlers;
using Microsoft.Build.Framework;
using Task = Microsoft.Build.Utilities.Task;

[assembly: InternalsVisibleTo("AggregateConfig.Tests.UnitTests")]
Expand Down Expand Up @@ -49,7 +49,7 @@ public class AggregateConfig : Task
/// <summary>
/// An array of additional properties that can be included in the output. These are user-specified key-value pairs.
/// </summary>
public string[] AdditionalProperties { get; set; }
public ITaskItem[] AdditionalProperties { get; set; }

/// <summary>
/// Gets or sets whether quiet mode is enabled. When enabled, the logger will suppress non-critical messages.
Expand Down Expand Up @@ -91,6 +91,7 @@ public override bool Execute()
{
EmitHeader();

InputDirectory = Path.GetFullPath(InputDirectory);
OutputFile = Path.GetFullPath(OutputFile);

if (!Enum.TryParse(OutputType, true, out FileType outputType) ||
Expand Down
22 changes: 18 additions & 4 deletions src/Task/JsonHelper.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
using System.IO;
using System.Text.Json;
using System.Threading.Tasks;
using Microsoft.Build.Framework;

namespace AggregateConfigBuildTask
{
Expand Down Expand Up @@ -63,12 +64,13 @@ public static object ConvertKeysToString(object data)
}

/// <summary>
/// Parses an array of key-value pairs provided as strings in the format "key=value".
/// Parses an array of key-value pairs from an <see cref="ITaskItem"/> array.
/// Prioritizes the 'Value' metadata when provided. Falls back to legacy format "key=value" if no 'Value' is present.
/// Supports escaping of the '=' sign using '\='.
/// </summary>
/// <param name="properties">An array of key-value pairs in the form "key=value".</param>
/// <param name="properties">An array of key-value pairs in the form of ITaskItem[] or legacy format strings.</param>
/// <returns>A dictionary containing the parsed key-value pairs.</returns>
public static Dictionary<string, string> ParseAdditionalProperties(string[] properties)
public static Dictionary<string, string> ParseAdditionalProperties(ITaskItem[] properties)
{
var additionalPropertiesDict = new Dictionary<string, string>();
const string unicodeEscape = "\u001F";
Expand All @@ -78,8 +80,19 @@ public static Dictionary<string, string> ParseAdditionalProperties(string[] prop
{
foreach (var property in properties)
{
var sanitizedProperty = property.Replace(@"\=", unicodeEscape);
// Check if the new 'Value' metadata is present
string key = property.ItemSpec;
string value = property.GetMetadata("Value");

if (!string.IsNullOrEmpty(value))
{
// Use the new method if the 'Value' metadata is provided
additionalPropertiesDict[key] = value;
continue;
}

// Fallback to legacy parsing if no 'Value' metadata is provided and key contains '='
var sanitizedProperty = key.Replace(@"\=", unicodeEscape);
var keyValue = sanitizedProperty.Split(split, 2);

if (keyValue.Length == 2)
Expand All @@ -88,6 +101,7 @@ public static Dictionary<string, string> ParseAdditionalProperties(string[] prop
}
}
}

return additionalPropertiesDict;
}

Expand Down
33 changes: 33 additions & 0 deletions src/UnitTests/PropertyExtensions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
using System.Collections.Generic;
using System.Linq;
using Microsoft.Build.Framework;
using Microsoft.Build.Utilities;

namespace AggregateConfigBuildTask.Tests.Unit
{
internal static class PropertyExtensions
{
/// <summary>
/// Creates an array of TaskItems from a dictionary of key-value pairs.
/// Supports both legacy format (key=value in ItemSpec) and new format (key in ItemSpec, value in metadata).
/// </summary>
/// <param name="properties">Dictionary of key-value pairs to be converted to TaskItems.</param>
/// <param name="legacyAdditionalProperties">If true, uses the legacy format for all items; otherwise, uses new format.</param>
/// <returns>An array of TaskItems.</returns>
public static ITaskItem[] CreateTaskItems(this IDictionary<string, string> properties, bool legacyAdditionalProperties)
{
return properties.Select((q) => {
if (legacyAdditionalProperties)
{
// Legacy format: "Key=Value" in ItemSpec
return new TaskItem($"{q.Key}={q.Value}");
}

// New format: Key in ItemSpec, Value as metadata
var taskItem = new TaskItem(q.Key);
taskItem.SetMetadata("Value", q.Value);
return taskItem;
}).ToArray();
}
}
}
57 changes: 43 additions & 14 deletions src/UnitTests/TaskTestBase.cs
Original file line number Diff line number Diff line change
@@ -1,22 +1,24 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.Build.Framework;
using Moq;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace AggregateConfigBuildTask.Tests.Unit
{
public class TaskTestBase
public abstract class TaskTestBase
{
private string testPath;
private readonly StringComparison comparison = StringComparison.OrdinalIgnoreCase;

private Mock<ITaskLogger> mockLogger;
internal IFileSystem virtualFileSystem;

public TestContext TestContext { get; set; }

public void TestInitialize(bool isWindowsMode, string testPath)
{
this.testPath = testPath;
Expand All @@ -25,6 +27,17 @@ public void TestInitialize(bool isWindowsMode, string testPath)
this.virtualFileSystem.CreateDirectory(testPath);
}

[TestCleanup]
public void Cleanup()
{
foreach (var invocation in mockLogger.Invocations)
{
var methodName = invocation.Method.Name;
var arguments = string.Join(", ", invocation.Arguments);
TestContext.WriteLine($"Logger call: {methodName}({arguments})");
}
}

[TestMethod]
[Description("Test that YAML files are merged into correct JSON output.")]
public void ShouldGenerateJsonOutput()
Expand Down Expand Up @@ -176,7 +189,9 @@ public void ShouldAddSourcePropertyMultipleFiles()

[TestMethod]
[Description("Test that additional properties are correctly added to the top level in JSON output.")]
public void ShouldIncludeAdditionalPropertiesInJson()
[DataRow(true, DisplayName = "Legacy properties")]
[DataRow(false, DisplayName = "Modern properties")]
public void ShouldIncludeAdditionalPropertiesInJson(bool useLegacyAdditionalProperties)
{
// Arrange: Prepare sample YAML data.
virtualFileSystem.WriteAllText($"{testPath}\\file1.yml", @"
Expand All @@ -194,7 +209,7 @@ public void ShouldIncludeAdditionalPropertiesInJson()
{
{ "Group", "TestRG" },
{ "Environment\\=Key", "Prod\\=West" }
}.Select(q => $"{q.Key}={q.Value}").ToArray(),
}.CreateTaskItems(useLegacyAdditionalProperties),
BuildEngine = Mock.Of<IBuildEngine>()
};

Expand All @@ -206,12 +221,22 @@ public void ShouldIncludeAdditionalPropertiesInJson()
string output = virtualFileSystem.ReadAllText($"{testPath}\\output.json");
var json = JsonConvert.DeserializeObject<Dictionary<string, object>>(output);
Assert.AreEqual("TestRG", json["Group"]);
Assert.AreEqual("Prod=West", json["Environment=Key"]);

if (useLegacyAdditionalProperties)
{
Assert.AreEqual("Prod=West", json["Environment=Key"]);
}
else
{
Assert.AreEqual("Prod\\=West", json["Environment\\=Key"]);
}
}

[TestMethod]
[Description("Test that additional properties are correctly added to the ARM parameters output.")]
public void ShouldIncludeAdditionalPropertiesInArmParameters()
[DataRow(true, DisplayName = "Legacy properties")]
[DataRow(false, DisplayName = "Modern properties")]
public void ShouldIncludeAdditionalPropertiesInArmParameters(bool useLegacyAdditionalProperties)
{
// Arrange: Prepare sample YAML data.
virtualFileSystem.WriteAllText($"{testPath}\\file1.yml", @"
Expand All @@ -229,7 +254,7 @@ public void ShouldIncludeAdditionalPropertiesInArmParameters()
{
{ "Group", "TestRG" },
{ "Environment", "Prod" }
}.Select(q => $"{q.Key}={q.Value}").ToArray(),
}.CreateTaskItems(useLegacyAdditionalProperties),
BuildEngine = Mock.Of<IBuildEngine>()
};

Expand Down Expand Up @@ -327,7 +352,9 @@ public void ShouldCorrectlyParseBooleanValues()

[TestMethod]
[Description("Test that additional properties are correctly added to the ARM parameters output from JSON input.")]
public void ShouldIncludeAdditionalPropertiesInJsonInput()
[DataRow(true, DisplayName = "Legacy properties")]
[DataRow(false, DisplayName = "Modern properties")]
public void ShouldIncludeAdditionalPropertiesInJsonInput(bool useLegacyAdditionalProperties)
{
// Arrange: Prepare sample JSON data.
virtualFileSystem.WriteAllText($"{testPath}\\file1.json", """
Expand All @@ -353,7 +380,7 @@ public void ShouldIncludeAdditionalPropertiesInJsonInput()
{
{ "Group", "TestRG" },
{ "Environment", "Prod" }
}.Select(q => $"{q.Key}={q.Value}").ToArray(),
}.CreateTaskItems(useLegacyAdditionalProperties),
BuildEngine = Mock.Of<IBuildEngine>()
};

Expand All @@ -375,7 +402,9 @@ public void ShouldIncludeAdditionalPropertiesInJsonInput()

[TestMethod]
[Description("Test that ARM parameters are correctly processed and additional properties are included in the output.")]
public void ShouldIncludeAdditionalPropertiesInArmParameterFile()
[DataRow(true, DisplayName = "Legacy properties")]
[DataRow(false, DisplayName = "Modern properties")]
public void ShouldIncludeAdditionalPropertiesInArmParameterFile(bool useLegacyAdditionalProperties)
{
// Arrange: Prepare ARM template parameter file data in 'file1.parameters.json'.
virtualFileSystem.WriteAllText($"{testPath}\\file1.parameters.json", """
Expand Down Expand Up @@ -406,7 +435,7 @@ public void ShouldIncludeAdditionalPropertiesInArmParameterFile()
{
{ "Group", "TestRG" },
{ "Environment", "'Prod'" }
}.Select(q => $"{q.Key}={q.Value}").ToArray(),
}.CreateTaskItems(useLegacyAdditionalProperties),
BuildEngine = Mock.Of<IBuildEngine>()
};

Expand Down
Loading