GC error (probable corruption) when list comprehending a custom iterator

[arnfaldur]

I am consistently getting the error I pasted below when trying to collect the elements produced by this iterator implementation. I'm not sure if I'm misusing the iterator interface but I doubt that my abuse should crash julia.

SubBoard = BitArray{1}
newSubBoard() = falses(18)
Base.start(::SubBoard) = newSubBoard()
function Base.next(b::SubBoard, state)
    old = copy(state)
    for i = 1:9
        if !state[i] && !state[i+9]
            state[i] = true
            break
        elseif !state[i+9]
            state[i] = false
            state[i+9] = true
            break
        elseif !state[i]
            state[i+9] = false
        end
    end
    return (old, state)
end
Base.done(b::SubBoard, state) = state == vcat(falses(9), trues(9))

Error:

julia> a = [e for e in newSubBoard()]
GC error (probable corruption) :
Allocations: 5360140 (Pool: 5358760; Big: 1380); GC: 8
Base.BitArray{1}(chunks=Array{UInt64, 1}[0x0000000000005195], len=18, dims=(140414452088528,))

signal (6): Aborted
while loading no file, in expression starting on line 0
gsignal at /usr/lib/libc.so.6 (unknown line)
abort at /usr/lib/libc.so.6 (unknown line)
unknown function (ip: 0x7fb4d0676a76)
unknown function (ip: 0x7fb4d06770d4)
unknown function (ip: 0x7fb4d0677054)
unknown function (ip: 0x7fb4d067b500)
jl_gc_collect at /usr/lib/libjulia.so.0.6 (unknown line)
jl_gc_pool_alloc at /usr/lib/libjulia.so.0.6 (unknown line)
jl_gc_alloc at /usr/lib/libjulia.so.0.6 (unknown line)
jl_alloc_array_1d at /usr/lib/libjulia.so.0.6 (unknown line)
Type at ./bitarray.jl:20
collect_to! at ./array.jl:507
collect at ./array.jl:476
unknown function (ip: 0x7fb4a4be4ac2)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7fb4d064805f)
unknown function (ip: 0x7fb4d064718e)
unknown function (ip: 0x7fb4d0648868)
unknown function (ip: 0x7fb4d0648fe8)
unknown function (ip: 0x7fb4d065f4e5)
jl_toplevel_eval_in at /usr/lib/libjulia.so.0.6 (unknown line)
eval at ./boot.jl:235
unknown function (ip: 0x7fb4ca45496f)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
eval_user_input at ./REPL.jl:66
unknown function (ip: 0x7fb4ca4d5fef)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
macro expansion at ./REPL.jl:97 [inlined]
#1 at ./event.jl:73
unknown function (ip: 0x7fb4a4bc62af)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7fb4d064d3d9)
unknown function (ip: 0xffffffffffffffff)
Allocations: 5360140 (Pool: 5358760; Big: 1380); GC: 8
zsh: abort (core dumped)  julia

I also got this segfault doing the exact same thing:

signal (11): Segmentation fault
while loading no file, in expression starting on line 0
jl_gc_pool_alloc at /usr/lib/libjulia.so.0.6 (unknown line)
jl_gc_alloc at /usr/lib/libjulia.so.0.6 (unknown line)
jl_alloc_array_1d at /usr/lib/libjulia.so.0.6 (unknown line)
Type at ./inference.jl:179
unknown function (ip: 0x7fc5c659eee5)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
Type at ./inference.jl:300
typeinf_edge at ./inference.jl:2534
unknown function (ip: 0x7fc5c65ac6da)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
abstract_call_gf_by_type at ./inference.jl:1420
unknown function (ip: 0x7fc5c65aa676)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
abstract_call at ./inference.jl:1897
unknown function (ip: 0x7fc5c65a681e)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
abstract_eval_call at ./inference.jl:1927
abstract_eval at ./inference.jl:1950
unknown function (ip: 0x7fc5c65a1836)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
typeinf_work at ./inference.jl:2722
typeinf at ./inference.jl:2787
typeinf_edge at ./inference.jl:2535
unknown function (ip: 0x7fc5c65ac6da)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
abstract_call_gf_by_type at ./inference.jl:1420
unknown function (ip: 0x7fc5c65aa676)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
abstract_call at ./inference.jl:1897
unknown function (ip: 0x7fc5c65a681e)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
abstract_eval_call at ./inference.jl:1927
abstract_eval at ./inference.jl:1950
unknown function (ip: 0x7fc5c65a1836)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
typeinf_work at ./inference.jl:2722
typeinf at ./inference.jl:2787
typeinf_edge at ./inference.jl:2535
unknown function (ip: 0x7fc5c65ac6da)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
abstract_call_gf_by_type at ./inference.jl:1420
unknown function (ip: 0x7fc5c65aa676)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
abstract_call at ./inference.jl:1897
unknown function (ip: 0x7fc5c65a681e)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
abstract_eval_call at ./inference.jl:1927
abstract_eval at ./inference.jl:1950
abstract_interpret at ./inference.jl:2084
unknown function (ip: 0x7fc5c65a1e86)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
typeinf_work at ./inference.jl:2669
typeinf at ./inference.jl:2787
typeinf_edge at ./inference.jl:2535
unknown function (ip: 0x7fc5c65ac6da)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
abstract_call_gf_by_type at ./inference.jl:1420
unknown function (ip: 0x7fc5c65aa676)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
abstract_call at ./inference.jl:1897
unknown function (ip: 0x7fc5c65a681e)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
abstract_eval_call at ./inference.jl:1927
abstract_eval at ./inference.jl:1950
unknown function (ip: 0x7fc5c65a1836)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
typeinf_work at ./inference.jl:2722
typeinf at ./inference.jl:2787
typeinf_frame at ./inference.jl:2504
typeinf_code at ./inference.jl:2583
unknown function (ip: 0x7fc5c65bd11d)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
typeinf_ext at ./inference.jl:2622
unknown function (ip: 0x7fc5c659c452)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
jl_apply_with_saved_exception_state at /usr/lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7fc5cc8c209d)
unknown function (ip: 0x7fc5cc8c4c1d)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
do_respond at ./REPL.jl:646
unknown function (ip: 0x7fc59ee75bb1)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x7fc5cc8db05f)
unknown function (ip: 0x7fc5cc8da18e)
unknown function (ip: 0x7fc5cc8db7c9)
unknown function (ip: 0x7fc5cc8dbb4c)
unknown function (ip: 0x7fc5cc8f24bf)
jl_toplevel_eval_in at /usr/lib/libjulia.so.0.6 (unknown line)
eval at ./boot.jl:235
unknown function (ip: 0x7fc5c66e796f)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
run_interface at ./LineEdit.jl:1583
unknown function (ip: 0x7fc5c675f59f)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
run_frontend at ./REPL.jl:945
run_repl at ./REPL.jl:180
unknown function (ip: 0x7fc59ee57282)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
_start at ./client.jl:413
unknown function (ip: 0x7fc5c6739998)
jl_apply_generic at /usr/lib/libjulia.so.0.6 (unknown line)
unknown function (ip: 0x564f45f35c4b)
unknown function (ip: 0x564f45f35652)
__libc_start_main at /usr/lib/libc.so.6 (unknown line)
unknown function (ip: 0x564f45f35709)
Allocations: 5230655 (Pool: 5229294; Big: 1361); GC: 7
zsh: segmentation fault (core dumped)  julia

[maleadt]

On 000f243:

julia> a = [e for e in newSubBoard()]
Error. Early free of 0x611000804d78 type :Array{UInt8, 1}
val : Array{UInt8, (453,)}[0x08, 0x04, 0x00, 0x00, 0x00, 0x23, 0x73, 0x65, 0x6c, 0x66, 0x23, 0x00, 0x73, 0x00, 0x70, 0x00, 0x72, 0x00, 0x13, 0x88, 0xc6, 0xec, 0x09, 0x02, 0x2d, 0x24, 0x03, 0x01, 0x00, 0x09, 0x03, 0x25, 0x00, 0x0e, 0x1b, 0x1c, 0x34, 0x05, 0x01, 0x00, 0x10, 0x08, 0xd7, 0x02, 0x01, 0x63, 0x09, 0x02, 0x2d, 0x24, 0x03, 0x0d, 0x00, 0x10, 0x08, 0xd7, 0x25, 0x09, 0x04, 0x31, 0x24, 0x32, 0x38, 0x37, 0x0f, 0xda, 0x00, 0x00, 0x00, 0x09, 0x02, 0x2d, 0x24, 0x03, 0x05, 0x00, 0x09, 0x06, 0x25, 0x00, 0x09, 0x1b, 0x1c, 0x96, 0x03, 0x0d, 0x00, 0x14, 0x0d, 0x00, 0x00, 0x05, 0x02, 0x00, 0x05, 0x03, 0x00, 0x03, 0x01, 0x00, 0x09, 0x01, 0x31, 0x24, 0x33, 0x09, 0x04, 0x31, 0x24, 0x32, 0x02, 0x5b, 0x2f, 0x68, 0x6f, 0x6d, 0x65, 0x2f, 0x74, 0x62, 0x65, 0x73, 0x61, 0x72, 0x64, 0x2f, 0x4a, 0x75, 0x6c, 0x69, 0x61, 0x2f, 0x6a, 0x75, 0x6c, 0x69, 0x61, 0x2d, 0x64, 0x65, 0x76, 0x2f, 0x62, 0x75, 0x69, 0x6c, 0x64, 0x2f, 0x73, 0x61, 0x6e, 0x69, 0x74, 0x69, 0x7a, 0x65, 0x2f, 0x75, 0x73, 0x72, 0x2f, 0x73, 0x68, 0x61, 0x72, 0x65, 0x2f, 0x6a, 0x75, 0x6c, 0x69, 0x61, 0x2f, 0x73, 0x69, 0x74, 0x65, 0x2f, 0x76, 0x30, 0x2e, 0x37, 0x2f, 0x52, 0x45, 0x50, 0x4c, 0x2f, 0x73, 0x72, 0x63, 0x2f, 0x4c, 0x69, 0x6e, 0x65, 0x45, 0x64, 0x69, 0x74, 0x2e, 0x6a, 0x6c, 0x02, 0x04, 0x65, 0x76, 0x61, 0x6c, 0xb7, 0x09, 0x04, 0x31, 0x24, 0x32, 0x38, 0x02, 0x04, 0x65, 0x76, 0x61, 0x6c, 0x0f, 0x3d, 0x01, 0x00, 0x00, 0x09, 0x02, 0x2d, 0x24, 0x03, 0x0c, 0x00, 0x09, 0x07, 0x46, 0x24, 0x10, 0x08, 0xd7, 0x02, 0x13, 0x6a, 0x6c, 0x5f, 0x74, 0x6f, 0x70, 0x6c, 0x65, 0x76, 0x65, 0x6c, 0x5f, 0x65, 0x76, 0x61, 0x6c, 0x5f, 0x69, 0x6e, 0x24, 0x06, 0x02, 0x24, 0x24, 0x10, 0x08, 0xd7, 0x47, 0xb6, 0x1a, 0x02, 0x08, 0x4c, 0x69, 0x6e, 0x65, 0x45, 0x64, 0x69, 0x74, 0x03, 0x05, 0x00, 0x09, 0x02, 0x31, 0x24, 0x33, 0xb6, 0x09, 0x02, 0x2d, 0x24, 0x05, 0x04, 0x00, 0x03, 0x0c, 0x00, 0x10, 0x10, 0xd5, 0x0f, 0x16, 0x05, 0x00, 0x00, 0x00, 0x23, 0x09, 0x02, 0x2d, 0x24, 0x03, 0x03, 0x00, 0x09, 0x03, 0x25, 0xd1, 0x1a, 0x56, 0x05, 0x04, 0x00, 0x1a, 0x19, 0xc6, 0x09, 0x02, 0x27, 0x24, 0x03, 0x03, 0x00, 0xc4, 0x10, 0x10, 0xd5, 0x0f, 0x17, 0x05, 0x00, 0x00, 0x00, 0x23, 0x09, 0x01, 0x28, 0x24, 0x10, 0x10, 0x00, 0x08, 0xb8, 0x00, 0x00, 0x02, 0x10, 0x08, 0xd4, 0xc4, 0x00, 0x10, 0x10, 0xd5, 0x0f, 0x19, 0x05, 0x00, 0x00, 0x00, 0x23, 0x09, 0x01, 0x28, 0x24, 0x10, 0x08, 0xd7, 0x02, 0x02, 0x6f, 0x6b, 0x23, 0x23, 0x13, 0x88, 0xb8, 0xec, 0x0d, 0x01, 0x00, 0x0d, 0x02, 0x00, 0x0d, 0x03, 0x00, 0x24, 0x13, 0x88, 0xc4, 0xec, 0x0d, 0x00, 0x00, 0x00, 0x0e, 0x00, 0x09, 0xd1, 0x0d, 0x04, 0x00, 0x00, 0x09, 0x00, 0x02, 0x0d, 0x00, 0x00, 0x0d, 0x02, 0x00, 0x0d, 0x03, 0x00, 0x00, 0x0e, 0x24, 0x24, 0x00, 0x02, 0x0d, 0x02, 0x00, 0x0d, 0x03, 0x00, 0x23, 0x13, 0x01, 0xb8, 0xf8, 0x00, 0x00, 0x00, 0x10, 0x00]
Let's try to backtrack the missing write barrier :
Now looking for 0x611000804d80 =======
Found parent 0x7ff67e767700 0x6110007c6800 at gc.c:1557
        loc 0x6110007c6828 : field(5)
        type: Core.MethodInstance
Found lostval 0x611000804d80 at gc.c:549 oftype: Array{UInt8, 1}
Now looking for 0x6110007c6800 =======
Found parent 0x7ff67e767700 0x6110007c6580 at gc.c:1557
        loc 0x6110007c65b0 : field(6)
        type: Core.TypeMapEntry
Found lostval 0x6110007c6800 at gc.c:549 oftype: Core.MethodInstance
Missing write barrier found !
0x6110007c6580 was written a reference to 0x6110007c6800 that was not recorded

[JeffBezanson]

Perhaps related to #27952

[Keno]

The original test case is actually doing something very dangerous: Overriding the iteration protocol for BitArray. I think everybody missed it, because of the typedef on the first line. BitArray is used extensively in the system, so messing with it causes crashes (just like overriding +). What you meant to do was something like:

struct SubBoard
end
Base.IteratorSize(s::SubBoard) = Base.SizeUnknown()
function Base.iterate(b::SubBoard, state=falses(18))
    state == vcat(falses(9), trues(9)) && return nothing
    old = copy(state)
    for i = 1:9
        if !state[i] && !state[i+9]
            state[i] = true
            break
        elseif !state[i+9]
            state[i] = false
            state[i+9] = true
            break
        elseif !state[i]
            state[i+9] = false
        end
    end
    return (old, state)
end