Skip to content

Commit

Permalink
Quote numeric map keys
Browse files Browse the repository at this point in the history
  • Loading branch information
fireflycons committed Oct 23, 2021
1 parent 0ae8302 commit 36a460a
Show file tree
Hide file tree
Showing 8 changed files with 366 additions and 8 deletions.
4 changes: 3 additions & 1 deletion docfx/documentation/gory-details.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,9 @@ Therefore the [Deserialize](xref:Firefly.CloudFormationParser.TemplateObjects.Te

# Solving Serialization

This was a little more straight forward than deserialization. In this case we can use implementations of [IYamlTypeConverter](https://github.com/aaubry/YamlDotNet/blob/master/YamlDotNet/Serialization/IYamlTypeConverter.cs) and override the `WriteYaml` method. Having said that, it was still necessary to find a way to serialize deep nests of objects that may be properties of the intrinsic currently being serialized. Now YamlDotNet provides [IValueSerializer](https://github.com/aaubry/YamlDotNet/blob/master/YamlDotNet/Serialization/IValueSerializer.cs) which is somewhat analogous to the `Func` argument passed to node deserializers, however it is not handily passed to you as a method argument on `WriteYaml`. Therefore it is necessary to create a default `ValueSerializer` and pass it to each `IYamlTypeConverter` implementation prior to the serialization run. Thus, when serializing the arguments of an intrinsic, where they are not a scalar or another intrinsic, they can be passed to the value serializer and the recursion of the object graph continues.
This was a little more straight forward than deserialization. In this case we can use implementations of [IYamlTypeConverter](https://github.com/aaubry/YamlDotNet/blob/master/YamlDotNet/Serialization/IYamlTypeConverter.cs) and override the `WriteYaml` method to write out intrinsics. Having said that, it was still necessary to find a way to serialize deep nests of objects that may be properties of the intrinsic currently being serialized. Now YamlDotNet provides [IValueSerializer](https://github.com/aaubry/YamlDotNet/blob/master/YamlDotNet/Serialization/IValueSerializer.cs) which is somewhat analogous to the `Func` argument passed to node deserializers, however it is not handily passed to you as a method argument on `WriteYaml`. Therefore it is necessary to create a default `ValueSerializer` and pass it to each `IYamlTypeConverter` implementation prior to the serialization run. Thus, when serializing the arguments of an intrinsic, where they are not a scalar or another intrinsic, they can be passed to the value serializer and the recursion of the object graph continues.

One bugbear in all of this was the issue of numeric keys in YAML. In a CloudFormation Template, you may want a `Mappings` section that has map keys based on AWS Account IDs. CloudFormation insists that such keys are quoted. YamlDotNet always outputs simple keys as unquoted and currently has no mechanism to change this so if you try to deploy your re-serialized template, CloudFormation borks. Therefore I had to resort to reflection to get at internal properties of the `Emitter` class to determine if a key is being emitted such that I could quote it. I raised [this issue](https://github.com/aaubry/YamlDotNet/issues/644).

# Example Syntax

Expand Down
4 changes: 2 additions & 2 deletions src/Firefly.CloudFormationParser/IResource.cs
Original file line number Diff line number Diff line change
Expand Up @@ -160,15 +160,15 @@ public interface IResource : ITemplateObject
string? Version { get; set; }

/// <summary>
/// Gets a resource property value. <see href="https://fireflycons.github.io/Firefly.CloudFormationParser/documentation/resource-props.html">Property Manipulation</see> in the documentation.
/// Gets a resource property value. See <see href="https://fireflycons.github.io/Firefly.CloudFormationParser/documentation/resource-props.html">Property Manipulation</see> in the documentation.
/// </summary>
/// <param name="propertyPath">The property path.</param>
/// <returns>The value of the property; else <c>null</c> if the property path was not found.</returns>
object? GetResourcePropertyValue(string propertyPath);

/// <summary>
/// <para>
/// Updates a property of this resource. <see href="https://fireflycons.github.io/Firefly.CloudFormationParser/documentation/resource-props.html">Property Manipulation</see> in the documentation.
/// Updates a property of this resource. See <see href="https://fireflycons.github.io/Firefly.CloudFormationParser/documentation/resource-props.html">Property Manipulation</see> in the documentation.
/// </para>
/// <para>
/// You would want to do this if you were implementing the functionality of <c>aws cloudformation package</c>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
namespace Firefly.CloudFormationParser.Serialization.Serializers
{
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Reflection;

using YamlDotNet.Core;
using YamlDotNet.Core.Events;
using YamlDotNet.Serialization;
using YamlDotNet.Serialization.EventEmitters;
using YamlDotNet.Serialization.Schemas;

internal class CloudFormationTypeAssigningEventEmitter : ChainedEventEmitter
{
private static readonly FieldInfo EventsField = typeof(Emitter).GetField(
"events",
BindingFlags.Instance | BindingFlags.NonPublic);

private static readonly FieldInfo StateField = typeof(Emitter).GetField(
"state",
BindingFlags.Instance | BindingFlags.NonPublic);

private static readonly MethodInfo CheckSimpleKeyMethod = typeof(Emitter).GetMethod(
"CheckSimpleKey",
BindingFlags.Instance | BindingFlags.NonPublic);

private enum EmitterState
{
StreamStart,
StreamEnd,
FirstDocumentStart,
DocumentStart,
DocumentContent,
DocumentEnd,
FlowSequenceFirstItem,
FlowSequenceItem,
FlowMappingFirstKey,
FlowMappingKey,
FlowMappingSimpleValue,
FlowMappingValue,
BlockSequenceFirstItem,
BlockSequenceItem,
BlockMappingFirstKey,
BlockMappingKey,
BlockMappingSimpleValue,
BlockMappingValue
}

/// <summary>
/// Initializes a new instance of the <see cref="CloudFormationTypeAssigningEventEmitter"/> class.
/// </summary>
/// <param name="nextEmitter">The next emitter.</param>
public CloudFormationTypeAssigningEventEmitter(IEventEmitter nextEmitter)
: base(nextEmitter)
{
}

/// <summary>
/// Emits the specified event information.
/// </summary>
/// <param name="eventInfo">The event information.</param>
/// <param name="emitter">The emitter.</param>
/// <exception cref="System.NotSupportedException">TypeCode.{typeCode} is not supported.</exception>
public override void Emit(ScalarEventInfo eventInfo, IEmitter emitter)
{
var suggestedStyle = ScalarStyle.Plain;

var value = eventInfo.Source.Value;

if (value == null)
{
eventInfo.Tag = JsonSchema.Tags.Null;
eventInfo.RenderedValue = string.Empty;
}
else
{
var typeCode = Type.GetTypeCode(eventInfo.Source.Type);
switch (typeCode)
{
case TypeCode.Boolean:
eventInfo.Tag = JsonSchema.Tags.Bool;
eventInfo.RenderedValue = YamlFormatter.FormatBoolean(value);
break;

case TypeCode.Byte:
case TypeCode.Int16:
case TypeCode.Int32:
case TypeCode.Int64:
case TypeCode.SByte:
case TypeCode.UInt16:
case TypeCode.UInt32:
case TypeCode.UInt64:
eventInfo.Tag = JsonSchema.Tags.Int;
eventInfo.RenderedValue = YamlFormatter.FormatNumber(value);
break;

case TypeCode.Single:
eventInfo.Tag = JsonSchema.Tags.Float;
eventInfo.RenderedValue = YamlFormatter.FormatNumber((float)value);
break;

case TypeCode.Double:
eventInfo.Tag = JsonSchema.Tags.Float;
eventInfo.RenderedValue = YamlFormatter.FormatNumber((double)value);
break;

case TypeCode.Decimal:
eventInfo.Tag = JsonSchema.Tags.Float;
eventInfo.RenderedValue = YamlFormatter.FormatNumber(value);
break;

case TypeCode.String:
case TypeCode.Char:
eventInfo.Tag = FailsafeSchema.Tags.Str;
eventInfo.RenderedValue = value.ToString()!;

// Use dirty hack to determine if a key is being emitted.
if (emitter.IsEmittingKey() && eventInfo.RenderedValue.All(char.IsDigit))
{
// Emit numeric keys quoted.
suggestedStyle = ScalarStyle.SingleQuoted;
}
else
{
suggestedStyle = ScalarStyle.Any;
}

break;

case TypeCode.DateTime:
eventInfo.Tag = DefaultSchema.Tags.Timestamp;
eventInfo.RenderedValue = YamlFormatter.FormatDateTime(value);
break;

case TypeCode.Empty:
eventInfo.Tag = JsonSchema.Tags.Null;
eventInfo.RenderedValue = string.Empty;
break;

default:
if (eventInfo.Source.Type == typeof(TimeSpan))
{
eventInfo.RenderedValue = YamlFormatter.FormatTimeSpan(value);
break;
}

throw new NotSupportedException($"TypeCode.{typeCode} is not supported.");
}
}

eventInfo.IsPlainImplicit = true;
if (eventInfo.Style == ScalarStyle.Any)
{
eventInfo.Style = suggestedStyle;
}

base.Emit(eventInfo, emitter);
}

private static class YamlFormatter
{
private static readonly NumberFormatInfo NumberFormat = new NumberFormatInfo
{
CurrencyDecimalSeparator = ".",
CurrencyGroupSeparator = "_",
CurrencyGroupSizes = new[] { 3 },
CurrencySymbol = string.Empty,
CurrencyDecimalDigits = 99,
NumberDecimalSeparator = ".",
NumberGroupSeparator = "_",
NumberGroupSizes = new[] { 3 },
NumberDecimalDigits = 99,
NaNSymbol = ".nan",
PositiveInfinitySymbol = ".inf",
NegativeInfinitySymbol = "-.inf"
};

public static string FormatBoolean(object boolean)
{
return boolean.Equals(true) ? "true" : "false";
}

public static string FormatDateTime(object dateTime)
{
return ((DateTime)dateTime).ToString("o", CultureInfo.InvariantCulture);
}

public static string FormatNumber(object number)
{
return Convert.ToString(number, NumberFormat)!;
}

public static string FormatNumber(double number)
{
return number.ToString("G17", NumberFormat);
}

public static string FormatNumber(float number)
{
return number.ToString("G17", NumberFormat);
}

public static string FormatTimeSpan(object timeSpan)
{
return ((TimeSpan)timeSpan).ToString();
}
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
namespace Firefly.CloudFormationParser.Serialization.Serializers
{
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;

using YamlDotNet.Core;
using YamlDotNet.Core.Events;

/// <summary>
/// Extensions for <c>YamlDoNet</c> emitter interface
/// </summary>
/// <remarks>
/// Dirty hack awaiting a better solution for <see href="https://github.com/aaubry/YamlDotNet/issues/644"/>
/// </remarks>
internal static class EmitterExtensions
{
/// <summary>
/// Accesses the private events field on the emitter
/// </summary>
private static readonly FieldInfo EventsField = typeof(Emitter).GetField(
"events",
BindingFlags.Instance | BindingFlags.NonPublic);

/// <summary>
/// Accesses the private state field on the emitter
/// </summary>
private static readonly FieldInfo StateField = typeof(Emitter).GetField(
"state",
BindingFlags.Instance | BindingFlags.NonPublic);

/// <summary>
/// Determines whether the emitter is about to emit a key.
/// </summary>
/// <param name="emitter">The emitter.</param>
/// <returns>
/// <c>true</c> if the next token emitted will be a key; else <c>false</c>.
/// </returns>
/// <exception cref="System.ArgumentNullException">emitter is <c>null</c></exception>
public static bool IsEmittingKey(this IEmitter? emitter)
{
if (emitter == null)
{
throw new ArgumentNullException(nameof(emitter));
}

var state = GetState(emitter);

if (state == EmitterState.BlockMappingFirstKey || state == EmitterState.BlockMappingKey)
{
// State indicates it will emit a key next.
return true;
}

// If the next event in the queue is a mapping start, then a key will be emitted next.
return GetEventQueue(emitter).FirstOrDefault()?.GetType() == typeof(MappingStart);
}

/// <summary>
/// Gets the event queue.
/// </summary>
/// <param name="emitter">The emitter.</param>
/// <returns>Event queue</returns>
private static Queue<ParsingEvent> GetEventQueue(IEmitter emitter)
{
return (Queue<ParsingEvent>)EventsField.GetValue(emitter);
}

/// <summary>
/// Gets the state.
/// </summary>
/// <param name="emitter">The emitter.</param>
/// <returns>Current state.</returns>
private static EmitterState GetState(IEmitter emitter)
{
return (EmitterState)StateField.GetValue(emitter);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
namespace Firefly.CloudFormationParser.Serialization.Serializers
{
internal enum EmitterState
{
StreamStart,

StreamEnd,

FirstDocumentStart,

DocumentStart,

DocumentContent,

DocumentEnd,

FlowSequenceFirstItem,

FlowSequenceItem,

FlowMappingFirstKey,

FlowMappingKey,

FlowMappingSimpleValue,

FlowMappingValue,

BlockSequenceFirstItem,

BlockSequenceItem,

BlockMappingFirstKey,

BlockMappingKey,

BlockMappingSimpleValue,

BlockMappingValue
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

using YamlDotNet.Core;
using YamlDotNet.Serialization;
using YamlDotNet.Serialization.EventEmitters;

/// <summary>
/// Serialize/Deserialize a CloudFormation Template
Expand Down Expand Up @@ -102,7 +103,9 @@ public static string Serialize(ITemplate template)
};

var serializerBuilder =
new SerializerBuilder().WithEmissionPhaseObjectGraphVisitor(
new SerializerBuilder()
.WithEventEmitter(inner => new CloudFormationTypeAssigningEventEmitter(inner), loc => loc.InsteadOf<TypeAssigningEventEmitter>())
.WithEmissionPhaseObjectGraphVisitor(
args => new SkipNullObjectGraphVisitor(args.InnerVisitor)); // Don't emit empty template sections

// Register the type converters
Expand Down
Loading

0 comments on commit 36a460a

Please sign in to comment.