Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash on master with CategoricalArrays #18343

Closed
nalimilan opened this issue Sep 3, 2016 · 14 comments · Fixed by #18534
Closed

Crash on master with CategoricalArrays #18343

nalimilan opened this issue Sep 3, 2016 · 14 comments · Fixed by #18534
Assignees
@vtjnash
Labels
bug Indicates an unexpected problem or unintended behavior compiler:precompilation Precompilation of modules
Milestone
0.5.x

Comments

@nalimilan
Copy link
Member

I'm getting a crash both with latest master and with 0.4.6 after loading CategoricalArrays.jl and calling gc(). You can get the code from the nl/crash branch.

   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0-dev.456 (2016-09-03 13:57 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit a9140fd* (0 days old master)
|__/                   |  x86_64-redhat-linux

julia> using CategoricalArrays
INFO: Recompiling stale cache file /home/milan/.julia/lib/v0.6/CategoricalArrays.ji for module CategoricalArrays.
julia> gc()
GC error (probable corruption) :
Allocations: 2519259 (Pool: 2518041; Big: 1218); GC: 2
<?#0x7f85f4112870::<circular reference @-1>>

signal (6): Aborted
while loading no file, in expression starting on line 0
gsignal at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)
push_root at /home/milan/Dev/julia/src/gc.c:1478
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1368
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1368
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1368
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1368
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1368
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1368
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1368
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
gc_mark_module at /home/milan/Dev/julia/src/gc.c:1218
push_root at /home/milan/Dev/julia/src/gc.c:1429
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1368
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1368
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1368
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
gc_mark_module at /home/milan/Dev/julia/src/gc.c:1218
push_root at /home/milan/Dev/julia/src/gc.c:1429
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
push_root at /home/milan/Dev/julia/src/gc.c:1466
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
gc_mark_module at /home/milan/Dev/julia/src/gc.c:1218
push_root at /home/milan/Dev/julia/src/gc.c:1429
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
gc_mark_module at /home/milan/Dev/julia/src/gc.c:1218
push_root at /home/milan/Dev/julia/src/gc.c:1429
gc_push_root at /home/milan/Dev/julia/src/gc.c:1189 [inlined]
pre_mark at /home/milan/Dev/julia/src/gc.c:1515
_jl_gc_collect at /home/milan/Dev/julia/src/gc.c:1709 [inlined]
jl_gc_collect at /home/milan/Dev/julia/src/gc.c:1865
gc at ./base.jl:145
unknown function (ip: 0x7f85e315de7f)
jl_call_method_internal at /home/milan/Dev/julia/src/julia_internal.h:189 [inlined]
jl_apply_generic at /home/milan/Dev/julia/src/gf.c:1931
do_call at /home/milan/Dev/julia/src/interpreter.c:66
eval at /home/milan/Dev/julia/src/interpreter.c:205
jl_toplevel_eval_flex at /home/milan/Dev/julia/src/toplevel.c:619 [inlined]
jl_toplevel_eval at /home/milan/Dev/julia/src/toplevel.c:641
jl_toplevel_eval_in_warn at /home/milan/Dev/julia/src/builtins.c:590
eval at ./boot.jl:234
unknown function (ip: 0x7f87f6301cdf)
jl_call_method_internal at /home/milan/Dev/julia/src/julia_internal.h:189 [inlined]
jl_apply_generic at /home/milan/Dev/julia/src/gf.c:1931
eval_user_input at ./REPL.jl:66
unknown function (ip: 0x7f85e3156a06)
jl_call_method_internal at /home/milan/Dev/julia/src/julia_internal.h:189 [inlined]
jl_apply_generic at /home/milan/Dev/julia/src/gf.c:1931
macro expansion at ./REPL.jl:97 [inlined]
#3 at ./event.jl:68
unknown function (ip: 0x7f85e314df2f)
jl_call_method_internal at /home/milan/Dev/julia/src/julia_internal.h:189 [inlined]
jl_apply_generic at /home/milan/Dev/julia/src/gf.c:1931
jl_apply at /home/milan/Dev/julia/src/julia.h:1364 [inlined]
start_task at /home/milan/Dev/julia/src/task.c:259
unknown function (ip: 0xffffffffffffffff)
Allocations: 2519259 (Pool: 2518041; Big: 1218); GC: 2
Aborted (core dumped)

Is this a duplicate of #18299?
@ViralBShah ViralBShah added the bug Indicates an unexpected problem or unintended behavior label Sep 3, 2016
@ViralBShah ViralBShah added this to the 0.5.x milestone Sep 3, 2016
@nalimilan
Copy link
Member Author

Turns out the line that triggers the crash is this one, from src/value.jl:

Base.convert{T}(::Type{Ref}, x::CategoricalValue{T}) = RefValue{T}(x)

@nalimilan
Copy link
Member Author

Oops, actually it's this one, from the same file:

Base.convert(::Type{Nullable{CategoricalValue}}, x::CategoricalValue{Nullable}) = Nullable(x)

@yuyichao yuyichao added the compiler:precompilation Precompilation of modules label Sep 3, 2016
@yuyichao
Copy link
Contributor

yuyichao commented Sep 3, 2016

Likely related to #16883, unlikely dup of #18299

Reduced code.

__precompile__(true)

module CategoricalArrays
immutable Pool{R,V}
    valindex::Vector{V}
end
immutable Value{T,R}
    pool::Pool{R,Value{T,R}}
end

Base.convert{S}(::Type{Nullable{S}}, ::Value{Nullable}) = 2
Base.convert(::Type{Nullable{Value}}, ::Value{Nullable}) = 2
Base.convert{T}(::Type{Ref}, ::Value{T}) = 3

end

@nalimilan
Copy link
Member Author

AFAICT it also happens without precompilation enabled.

@yuyichao
Copy link
Contributor

yuyichao commented Sep 3, 2016

It doesn't happen with precompilation off for me and the reduced case does have a recursive type so it's likely missing case from #16883.

Note that removing __precompile__(true) won't turn off precompilation if the packages was precompiled before. --compilecache=no is the recommended way of turning off precompilation.

@vtjnash
Copy link
Member

vtjnash commented Sep 13, 2016

this is similar to #16767, but it found a slightly different way to recurse :/

@vtjnash vtjnash self-assigned this Sep 13, 2016
@nalimilan nalimilan mentioned this issue Sep 13, 2016
Closed
@vtjnash vtjnash mentioned this issue Sep 15, 2016
Closed
vtjnash added a commit that referenced this issue Sep 15, 2016
@vtjnash
remove deserialize mode MODULE_POSTWORK
f0e6dc1 
it seems to be unnecessary

fix #18343
@vtjnash vtjnash mentioned this issue Sep 15, 2016
Merged
@quinnj
Copy link
Member

quinnj commented Sep 21, 2016

FYI, I'm still seeing crashes on current CategoricalArrays release (0.0.5) on julia 0.4.7. Both travis and locally.

@quinnj
Copy link
Member

quinnj commented Sep 21, 2016

Also see the travis log here where we see it directly with CategoricalArrays when precompile is turned on.

@yuyichao
Copy link
Contributor

I don't think the fix is backported

@nalimilan
Copy link
Member Author

nalimilan commented Sep 23, 2016 •
edited
Loading

@quinnj The crash you linked to doesn't happen on 0.5, so the fix was merged before this PR.

@yuyichao @vtjnash Any idea what commit might have fixed it, and whether it could be backported to 0.4?
EDIT: maybe #16898?

@yuyichao
Copy link
Contributor

At leas this was closed by #18534

@nalimilan
Copy link
Member Author

How backportable is that?

@yuyichao
Copy link
Contributor

..... Well, if you check that PR it already has backport 0.5 label applied so at least that's something

@nalimilan
Copy link
Member Author

Sorry, I meant backporting to 0.4. But since we don't get the crash on 0.5, that must be a different bug and fix.

vtjnash added a commit that referenced this issue Oct 11, 2016
@vtjnash
remove deserialize mode MODULE_POSTWORK
a319d6e 
it seems to be unnecessary

fix #18343

(cherry picked from commit f0e6dc1)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vtjnash vtjnash

Labels
bug Indicates an unexpected problem or unintended behavior compiler:precompilation Precompilation of modules
Projects
None yet
Milestone
0.5.x
Development

Successfully merging a pull request may close this issue.

remove deserialize mode MODULE_POSTWORK JuliaLang/julia
5 participants
@vtjnash @yuyichao @ViralBShah @nalimilan @quinnj