Support Comment Features

[nezda]

It would be useful if Feature.ALLOW_YAML_COMMENTS were supported (probably Feature.ALLOW_COMMENTS for consistency sake).

[cowtowncoder]

I have been thinking about that. So, I am guessing that lines that start with '#' would be considered comments, but not ones that had '#' somewhere later on?
Also, probably no need to allow leading white space.

[nezda]

I would expect from wherever they are to the end of the line which does imply a little row edge whitespace trimming.

[cowtowncoder]

Hmmh. This would be somewhat problematic from implementation perspective, for false matches -- is there actually usage examples of comments after data, as opposed to comment-only rows?

[nezda]

Its something I've done (and seen) to (temporarily) omit columns and comment on a particular row's content, but I don't have a publicly available example/precedent.

[cowtowncoder]

Ok; I have hoped to find sort of emerging standard, that is why I am asking. It would be nice to follow existing practices if there were some.

Now: I think I am open to allowing comments anywhere; and perhaps it would be possible to allow more reliable handling by ensuring that comments can occur anywhere, but NOT inside quoted values; and to allow escaping (optional, default with backslash) hash sign.

Plus it is even possible to have additional setting ("comments only for beginning of line") if there are further concerns.

So I think this makes sense.

One final additional thought: in addition to existing feature (good point on that btw, I hadn't realized its relevance here!), could also add a flag in CsvSchema as an alternative. Not sure if it'd confuse things more or not, but it's worth mentioning.
Actually: doing this could allow custom comment char -- default of '#' (if enabled), but could even use soemthing else; some formats use semicolon (";") for example/

[cowtowncoder]

To start off things, CsvSchema now has boolean "allow comments" flag. I did not add ability to use different comment character, assuming that use of # makes most sense to start.

[cowtowncoder]

After reading documentation I could find, I further limited this so that only '#' as the first character of a line counts as a comment; this may be relaxed in future if it turns out necessary.
Also verified that leading '#' will lead to values being quoted on writing.

[cowtowncoder]

Implemented, will be in 2.5.0.

[nezda]

I really appreciate your work on this, though I can't ditch my hand rolled solution without the comments-anywhere feature as I have a raft of existing data. Should I open a separate feature request issue?

[cowtowncoder]

Yes, please open a separate issue.

Adding handling in beginning of a column
(i.e. at beginning of line, or following column separator) is relatively easy; I actually started this way, but since almost all examples I saw assumed comment-starting-from-first-char, I chose a bit more conservative initial implementation.