New "access to undefined reference" (from optimization?)

[andrewcooke]

I am getting an "access to undefined reference" in code that used to work, and that I believe is valid. It takes a very specific combination of details to trigger the problem, so I think it is related to optimization.

As far as I can tell from Travis (assuming they update 0.3 regularly) this behaviour started in the last day or two.

andrew@laptop:~/project/CRC> julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" to list help topics
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.3.0-prerelease+2697 (2014-04-22 02:46 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 9dc5e3f* (0 days old master)
|__/                   |  x86_64-suse-linux

julia> type Single{A}
           table::Vector{A}
           Single() = new()
       end

julia> 

julia> function crc(tables=Single)
           make_tables(tables{Uint8}())
       end
crc (generic function with 2 methods)

julia> 

julia> function make_tables(tables::Single)
           tables.table = Array(Uint8, 256)
           tables.table[1] = 0x0
       end
make_tables (generic function with 1 method)

julia> 

julia> crc()
ERROR: access to undefined reference
 in crc at none:2

I believe this is valid code - undefined members of a type seem to be supported in general, as long as they are created before reading.

Note that much of the "mess" above is necessary in order to trigger this. Below I'll add some minimally modified versions that work to illustrate this:

The type parameterization is needed:

julia> type Single
           table::Vector{Uint8}
           Single() = new()
       end

julia> 

julia> function crc(tables=Single)
           make_tables(tables())
       end
crc (generic function with 2 methods)

julia> 

julia> function make_tables(tables::Single)
           tables.table = Array(Uint8, 256)
           tables.table[1] = 0x0
       end
make_tables (generic function with 1 method)

julia> 

julia> crc()
0x00

The default argument is needed:

julia> type Single{A}
           table::Vector{A}
           Single() = new()
       end

julia> 

julia> function crc(tables)
           make_tables(tables{Uint8}())
       end
crc (generic function with 1 method)

julia> 

julia> function make_tables(tables::Single)
           tables.table = Array(Uint8, 256)
           tables.table[1] = 0x0
       end
make_tables (generic function with 1 method)

julia> 

julia> crc(Single)
0x00

The separate routine to set the value is needed:

julia> type Single{A}
           table::Vector{A}
           Single() = new()
       end

julia> 

julia> function crc(tables=Single)
           t = tables{Uint8}()
           t.table = Array(Uint8, 256)
           t.table[1] = 0x0
       end
crc (generic function with 2 methods)

julia> 

julia> crc()
0x00

Finally, the example to cut+paste:

type Single{A}
    table::Vector{A}
    Single() = new()
end

function crc(tables=Single)
    make_tables(tables{Uint8}())
end

function make_tables(tables::Single)
    tables.table = Array(Uint8, 256)
    tables.table[1] = 0x0
end

crc()

[jiahao]

Can you run git bisect to identify which commit caused the error?

[andrewcooke]

We have a winner (with a suitably suspicious commit message):

e805fd6b505bf8a086b3d0b0b4dd798a1870c176 is the first bad commit
commit e805fd6b505bf8a086b3d0b0b4dd798a1870c176
Author: Jameson Nash <vtjnash@gmail.com>
Date:   Fri Apr 18 22:00:51 2014 -0400

    don't inline more than a reasonable amount -- our inliner was good enough now that it could generate enormous 1-liners. fix #6566. close #6569

:040000 040000 93604e2e7aef769824b010be4928d16a1f5bab12 048a15206717d84f4f9bad3c9cab1e8e76bca8a4 M      base

@vtjnash :o)

[Full disclosure - this is the first time I have used git bisect, so I could have screwed up, but it seemed simple enough. Also, I'm thinking maybe the problem is deeper and that patch only exposes it by restricting an optimization that hides the issue?]

[vtjnash]

the problem is related to #6566 (comment)

[vtjnash]

the different examples are just triggering different inlining thresholds, the first example manages to hit a sweet-spot

the problem is that the inliner considers new() to be something that it can copy, so you end up with two instances. (Expr(:new) should probably only be effect_free if the first argument is an immutable type)

[andrewcooke]

that seems like a reasonable explanation and solution.

i've worked around this in my own code (they are now immutable) so there's no pressure from me, but the idea that Julia is quietly duplicating mutable instances sounds a little worrying.

[JeffBezanson]

Compiler bugs are indeed scary. But they can be fixed!

[vtjnash]

See pull request #6605