Allow comments

[philippe-lavoie]

I'm updating a node with YamlMapping inside a potentially large Yaml file that has comments. I quickly realized that the library is not handling this correctly right now even for a simple read/write test.

        [Fact]
        public void HandlingComments()
        {
            var yamlContent = @"# Name your package! Package names should contain only lowercase characters
name: 'test_package'
target-path: target  # directory which will store compiled SQL files
models:
  test_package:
      # Applies to all files under models/example/
      example:
          materialized: view";

            var input = new StringReader(yamlContent);
            var yaml = new YamlStream();
            yaml.Load(input);
            var document = (YamlMappingNode)yaml.Documents[0].RootNode;

            var newYaml = new YamlDocument(document);
            var yamlStream = new YamlStream(newYaml);
            var buffer = new StringBuilder();
            using (var writer = new StringWriter(buffer))
            {
                yamlStream.Save(writer, assignAnchors: false);
                var yamlText = writer.ToString();
                Assert.Equal(yamlContent, yamlText);
            }
        }

I understand the standard is vague on comments, however in practice comments are very useful. I'm not asking for full serialiazation / de-serializaton support, but a basic read/update/write flow leveraging the YamlMappingNode directly should work.

Is it possible ?

[aaubry]

It is certainly possible to do that, but it will require a change in YamlDotNet. It is possible to request that comments are returned by the Parser, but the RepresentationModel does not have a way to store them.

I don't which would be the best design for this feature. Two options come to my mind:

1. Add a YamlComment type, that could appear inside a YamlDocument as any other YamlNode.
2. Add a Comments property to YamlNode that would capture all comments associated with a node.

I feel that the second option would be cleaner because it wouldn't subvert the YamlNode type hierarchy with something that is not specified as a node on the YAML spec.
Do you think that this would work for you ?

[smaillet]

I think this is dangerous feature as it is a shift in core functionality of YamlDotNet as an object serializer/Deserializer. Comments should have no place is such operations. YamlDotNet isn't a generalized YAML parser. While it could be built on top of one, there is generally a lot more functionality required of a general parser than is needed for serialize/deserialize and that could increase code bloat. If a generalized parsing is desired, I'd suggest it should be done as a distinct project that this one then consumes.

[aaubry]

That's incorrect, @smaillet . YamlDotNet consists of multiple parts. You should read the documentation to get a better understanding of the library. There's a parser / emitter that can parse or emit any valid YAML, a representation model that provides a higher-level representation of the YAML stream and again supports the whole set of YAML features. Finally, there's the serialization / deserialization components that build on the parser and emitter to do their job.

[cyrildurand]

Any news on this ?

Add a Comments property to YamlNode that would capture all comments associated with a node.

Is that idea still valid ?

How can we help on this ? I have not look at the source code yet, do you have a rough estimate on how long it would take to implement this ?

[flobernd]

Is there any news regarding this topic? It's a big showstopper for my current project 😞

[lonix1]

From here:

public class Test
{
    [YamlMember(Description = "A comment")]
    public int Value { get; set; }
}

Maybe good enough for some peoples' use cases.

[ravensorb]

It is certainly possible to do that, but it will require a change in YamlDotNet. It is possible to request that comments are returned by the Parser, but the RepresentationModel does not have a way to store them.

I don't which would be the best design for this feature. Two options come to my mind:

1. Add a YamlComment type, that could appear inside a YamlDocument as any other YamlNode.
2. Add a Comments property to YamlNode that would capture all comments associated with a node.

I feel that the second option would be cleaner because it wouldn't subvert the YamlNode type hierarchy with something that is not specified as a node on the YAML spec. Do you think that this would work for you ?

Give than comments can appear as part of a line or as line on their own (and may also be multi-line) it might make sense to go with option 1 as it would allow more flexibility (including positional notation)

[jurgen178]

Yes, option 1, adding a YamlComment type, would be perfect.
Right now, I workaround this by reading all comments to a comments table (dictionary) using the key and line number as the dictionary key. For my project, I don't have leading comment lines in the yaml. All comments are in the same line as the element. But adding support for leading comment lines for this workaround is easy. Just add another regex to capture the comment-only line and store it for the next line.

# Comment table.
$comments = @{}

# Update the comments table.
[int]$line = 1
[System.IO.File]::ReadLines($filePath) | % {

    # key: "value" # comment
    if ($_ -match "^\s*(?<key>\w+)\s*:\s*(""|').+\1\s+#\s*(?<comment>.+)$") {

        [string]$key = "$line#$($matches["key"])"
        [string]$comment = $matches["comment"]
        $comments.Add($key, $comment)
    }

    $line++
}

Get the matching line comment for a node:

    # Get the comment from the comment table.
    [int]$line = $node.Start.Line
    [string]$commentKey = "$line#$($key)"
    if($comments.ContainsKey($commentKey))
    {
        $instruction = $comments[$commentKey]
    }

[bryanbcook]

For me, I'm looking at this library as a mechanism to load, manipulate and save the document within the scope of an application. I understand that comments aren't part of the specification, but it's important that we're not losing data as part of the process.

Do we have a sense of what it would take to satisfy option 1, where we introduce a YamlComment as a YamlNode?

[bryanbcook]

Leaving this note here for others who were in my situation. While it would be useful to have YamlDotNet include comments in the RepresentationalModel, I'd likely have a similar problem with whitespace being stripped out. As I'm only interested in changing scalar values, I realized my scenario could be handled using basic string manipulation augmented with the start and end positions of the nodes that I want to change.

// break up my content into lines
string[] lines = content.Split( Environment.NewLine );

// load the content into the RepresentationalModel
var stream = new YamlStream();
stream.Load(new StringReader(content));
var root = (YamlMappingNode)stream.Documents[0].RootNode;

YamlScalarNode node = // logic to fetch interesting node

// determine position within content to modify
int line = (int)node.Start.Line - 1;
int columnValueStartPosition = (int)node.Start.Column - 1;
int columnValueEndPosition = (int)node.End.Column - 1;

// update the content
var contentPrefix = lines[line].Substring(0, columnValueStartPosition);
var contentSuffix = lines[line].Substring(columnValueEndPosition);
lines[line] = contentPrefix + "new value" + contentSuffix

// re-assemble content
content = string.Join( Environment.NewLine, lines);

[john-alessi]

I don't which would be the best design for this feature. Two options come to my mind:

1. Add a YamlComment type, that could appear inside a YamlDocument as any other YamlNode.
2. Add a Comments property to YamlNode that would capture all comments associated with a node.

I addressed this with a (messy) workaround similar to option 2 which involves wrapping types of interest in a generic IYamlConvertible that keeps track of neighboring / inline comments.

I'm leaning towards option 2 for the ability to explicitly associate comments with a specific YamlNode; if non-inline comments are treated as their own YamlNode as in option 1 then a lot of information is lost if things are re-serialized in a different order.

The only thing I'm not sure how to address is the ambiguity of some non-inline comments, for example in a snippet like this:

ambiguousMapping:
  A: 32
  # DO NOT CHANGE
  B: 16

it's not clear which value that comment is referring to / should be attached to (the yaml spec doesn't take a stance on this). I took an approach of always assuming that this kind of comment should be attached to the node below; personally I wouldn't mind this as the default behavior but obviously it would still produce undesired results if the comment in the snippet were actually intended to refer to A: 32.

If there are any thoughts on how to handle the above scenario I'd love to take a shot at implementing this.