Behavior change in v1.3.0 regarding Symbol in a QuoteNode

[bliang26]

I have just upgraded to v1.3.0 and I noticed a behavior change regarding QuoteNode(). Here is a MWE:

julia> Expr(:(.), 
           Expr(:(.), :Base, QuoteNode(Symbol("Enums"))),
           QuoteNode(Symbol("@enum")))
:(Base.Enums.var"@enum")

And here is what I used to get in v1.2.0:

julia> Expr(:(.), 
           Expr(:(.), :Base, QuoteNode(Symbol("Enums"))),
           QuoteNode(Symbol("@enum")))
:(Base.Enums.@enum)

which is the expected behavior.

My code heavily relies on this to automatically generate custom enum types. Please help. Thanks.

[bliang26]

Here is an even simpler example:

julia> :(Base.Enums.@enum)
:(#= REPL[12]:1 =# Base.Enums.var"@enum")

So, where is that var coming from?

[JeffBezanson]

This is a bug since the output does not parse:

julia> :(Base.Enums.@enum a b c)
:(#= REPL[8]:1 =# Base.Enums.var"@enum" a b c)

We should not use var in that context.

But, this should only affect expression printing. Why are you printing expressions instead of using macros or eval?

[bliang26]

I have some code that can parse XSD files and define corresponding data structures in Julia. Doing this on the fly is not necessary since (1) the XSD files don't often change; (2) would cause annoying warnings saying incremental compilation may be broken. So I just print the generated expressions to a .jl file and call it done.

I just did some debugging and it is indeed a print issue:

1|debug> n
In #show_sym#371(allow_macroname, , io, sym) at show.jl:990
 992      elseif allow_macroname && (sym_str = string(sym); startswith(sym_str, '@'))
 993          print(io, '@')
 994          show_sym(io, sym_str[2:end])
 995      else
>996          print(io, "var", repr(string(sym)))
 997      end
 998  end

Is there a special reason for printing var here?

[JeffBezanson]

Yes; var is needed if the dot expression occurs outside a macrocall. a.@b by itself parses as a macrocall, so Expr(:(.), :a, Symbol("@b")) needs to be printed as a.var"@b". I'm working on a fix.

[JeffBezanson]

To clarify, the exact output in the OP:

julia> Expr(:(.), 
           Expr(:(.), :Base, QuoteNode(Symbol("Enums"))),
           QuoteNode(Symbol("@enum")))
:(Base.Enums.var"@enum")

is in fact correct. If you want to invoke the macro, you need to wrap that in a :macrocall expression.

[bliang26]

@JeffBezanson , thanks for your quick response. But I would argue that printing that var is causing some inconvenience. What I want to do here is to decouple the code generation and code evaluation. So that I can generate the code only once, save it to a file, and then use it as many times as needed. However, if I print

Base.Enums.var"@enum" A begin
          a
          b
      end

to a .jl file, it is not going to run. In v1.2 it didn't print var so it worked just fine. Going forward in v1.3, is print still the right way to save the generated expressions to a file? Or should I use something else?

[JeffBezanson]

The printing with var allows the output to be parsed back to the same expression. Note the following difference:

julia> dump(:(Base.Enums.@enum))
Expr
  head: Symbol macrocall
  args: Array{Any}((2,))
    1: Expr
      head: Symbol .
      args: Array{Any}((2,))
        1: Expr
          head: Symbol .
          args: Array{Any}((2,))
            1: Symbol Base
            2: QuoteNode
              value: Symbol Enums
        2: QuoteNode
          value: Symbol @enum
    2: LineNumberNode
      line: Int64 1
      file: Symbol REPL[4]

julia> (:(Base.Enums.@enum)).args[1]
:(Base.Enums.var"@enum")

Base.Enums.@enum by itself parses as a macro call, not a dot expression. If you have just the dot expression and no macro call, we need to print it differently to distinguish them.

The expression you want to print is a macro call. So, it should use the :macrocall head and everything will work (with my PR merged).

But, part of the point of metaprogramming with structured expression objects is to avoid going through text. If you want to generate text, why not just print the strings you want and avoid Exprs entirely?

[StefanKarpinski]

But, part of the point of metaprogramming with structured expression objects is to avoid going through text. If you want to generate text, why not just print the strings you want and avoid Exprs entirely?

Or conversely, if you want to use expressions, why save them as text? Just save them as expressions using serialize and then load them back as expressions.

[bliang26]

@JeffBezanson: I really appreciate Expr() since it provides a structured way to generate code instead of playing with strings. But I would also appreciate having the option of decoupling code generation and code evaluation. @StefanKarpinski : serialize does look like a better way for my case. Thanks for the suggestion.

So, for those who really need to print expressions into text, it looks like a special treatment is needed for :macrocall just to be safe. Here is an example that I found in Clang.jl:

https://github.com/JuliaInterop/Clang.jl/blob/7f9da947fe7238b2cdd4c332e6a85b6a6c1a7d0f/src/expr_unit.jl#L40

[KristofferC]

That example seems fine, I think. It is printing julia code based on the expression but it doesn't rely on how the expression itself is printed.

[JeffBezanson]

No, no special treatment is needed for :macrocall. On my branch:

julia> ex = Expr(:macrocall, Expr(:(.), :a, QuoteNode(Symbol("@b"))), nothing, :A,
                 Expr(:block, :a, :b));

julia> print(ex)
a.@b A begin
        a
        b
    end

As you can see, it printed perfectly.

Using #34081 (or wait until it is on master), I recommend just experimenting with variations of these forms and hopefully it will make sense.

[StefanKarpinski]

@bliang26, IIUC, the issue is that the expression you're printing is not a macro call.

[bliang26]

@JeffBezanson, great, #34081 is very helpful. I'll wait for it.