Snippet inclusion removes some valid lines containing # characters

[agemooij]

Hi

The @@snip feature appears to remove lines from snippets if they contain certain patterns related to # characters. It does this silently, without any warnings so users might not initially notice that their example snippets have been altered.

This is obviously related to support for nested snippets since it only apply to certain patterns of text that strongly resemble nested snippet markers.

Below is the test I've used so far to determine which lines will get filtered out of the snippet and which won't:

// Filtered out:
// 1 line with bare hash mark followed by a character: #a
// 2 line with bare hash mark followed by a character: abc#def
// 3 line with bare link with anchor: http://some.domain.com/foo#bar
// 4 line with markdown link with anchor: [IBM Cloud Object Storage](s3.md#bluemix-cloud-object-storage-with-s3-api )
//
// Not filtered out:
// 1 line with bare hash mark: #
// 2 line with quoted hash mark: "#"
// 3 line with quoted hash mark followed by a character: "#a"
// 4 line with quoted hash mark surrounded by characters: "abc#def"
// 5 line with bracket-quoted hash mark surrounded by characters: [abc#def]
// 6 line with bare link: http://some.domain.com/foo/bar
// 7 line with quoted link: http://some.domain.com/foo/bar
// 8 line with quoted link with anchor: "http://some.domain.com/foo#bar"
// 9 line with markdown link: [IBM Cloud Object Storage](s3.md)
// 10 line with quoted markdown link: "[IBM Cloud Object Storage](s3.md)"
// 11 line with quoted markdown link with anchor: "[IBM Cloud Object Storage](s3.md#bluemix-cloud-object-storage-with-s3-api)"
// 12 line with markdown link with anchor ending in bracket: [IBM Cloud Object Storage](s3.md#bluemix-cloud-object-storage-with-s3-api)

I'm not sure whether this can be entirely fixed but I think it is certainly counter-intuitive for most users that their links will be removed so perhaps the filter can at least try to detect links and URLs to prevent those from being filtered out?

/cc @olofwalker

[pvlugter]

Yes, this is for nested snippets. Originally from a3000b8. Main use case is having full examples that wrap nested snippets but exclude the snippet marker comments.

A couple of approaches so that we don't match the fragment part of links:

Only allow comment markers and whitespace before the snippet marker. We could automatically determine // comments for scala and java sources, and allow it to be configurable. I think this was the approach we used for the sphinx version of this for the older akka docs. Can also have an overall flag for whether or not to exclude nested snippet markers. The old akka doc snippets could also exclude nested snippet sections altogether, with an option to insert a new comment instead.

Or we could have a special exclude marker. So that only if the line includes something like ###exclude-from-snippets will it be excluded. No chance of excluding the wrong things, but also makes it more verbose, where a nested snippet would need something like: // #nested-snippet ###exclude-from-snippets.

[jroper]

A workaround, if not a fix, has been provided in #285, by allowing you to do this:

@@snip[example.scala](example.scala) { #example filterLabels=false }