Comments are removed in quote do expressions

[krux02]

For some reasons in quote do: the comment statements are removed and replaced by empty nodes. This breaks a test case in my ast pattern matching library:
https://github.com/krux02/ast-pattern-matching/blob/eb0743e4837f7200edf23cc6174b6eabd1217b28/tests/test1.nim#L326

Example

import macros

dumpTree:
  ## comment 1
  echo "abc"
  ## comment 2
  ## still comment 2

macro foobar(): untyped =
  result = quote do:
    ## comment 1
    echo "abc"
    ## comment 2
    ## still comment 2

  echo result.treeRepr

foobar()

Current Output

StmtList
  CommentStmt "comment 1"
  Command
    Ident "echo"
    StrLit "abc"
  CommentStmt "comment 2\nstill comment 2"
StmtList
  Empty
  Command
    Sym "echo"
    StrLit "abc"
  Empty

Expected Output

StmtList
  CommentStmt "comment 1"
  Command
    Ident "echo"
    StrLit "abc"
  CommentStmt "comment 2\nstill comment 2"
StmtList
  CommentStmt "comment 1"
  Command
    Ident "echo"
    StrLit "abc"
  CommentStmt "comment 2\nstill comment 2"

Additional Information

$ nim -v
Nim Compiler Version 0.19.9 [Linux: amd64]
Compiled at 2019-01-23
Copyright (c) 2006-2018 by Andreas Rumpf

git hash: e962be8981ff6ef09625b3cc89e0c0aa1f07b35a
active boot switches: -d:release

[timotheecour]

top-level (or block level) nkCommentStmt are transformed to nkEmpty in evalTemplateAux (as fix for #9432), all other comments (ie, inside a declarative, eg proc definition, including blocks inside it) will be preserved.

so a potential fix (for both getAst and quote) is as follows:

• add a nfKeepComments in TNodeFlag
• set that flag in proc semExpandToAst(c: PContext, n: PNode): PNode =:

result.flags.incl nfKeepComments

• change if c.isDeclarative: also check for nfKeepComments on top-level node inside evalTemplateAux

the downside is this adds a flag just for that purpose, but this should work (alternatively, this can be avoided by adding a dummy node as an extra sons to the call inside getAst and then checking for that, but that seems hacky)

(i can't think of a simpler way, because evaluation is deferred, so that semExpandToAst exits before we enter evalTemplateAux )

note

another example showing what this issue affects: tests/astspec/tastspec.nim
b85f45b#diff-8d7f8d4fd7137c40202a1a2d39aab5bbR30