Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

packages with overly complicated recursively defined types aren't precompilable #16883

Closed
nalimilan opened this issue Jun 11, 2016 · 1 comment · Fixed by #16898
Closed

packages with overly complicated recursively defined types aren't precompilable #16883

nalimilan opened this issue Jun 11, 2016 · 1 comment · Fixed by #16898
Assignees
@vtjnash
Labels
compiler:precompilation Precompilation of modules

Comments

@nalimilan
Copy link
Member

nalimilan commented Jun 11, 2016 •
edited by vtjnash
Loading

edit: jwn DataFrames is unconnected to this bug. CategoricalArrays just doesn't work with precompilation (for currently unknown reasons) due to the complexity of the type definition recursion exceeding the current capabilities of the incremental deserializer

original text below:

I get this segfault reliably when I add CategoricalArrays to the REQUIRES file of DataFrames. I'm using DataFrames from latest master (JuliaData/DataFrames.jl@3400fbc) and CategoricalArrays from latest master too (JuliaData/CategoricalArrays.jl@b2476fe).

julia> using DataFrames
INFO: Precompiling module DataFrames...

signal (11): Erreur de segmentation
while loading /home/milan/.julia/DataFrames/src/DataFrames.jl, in expression starting on line 16
push_root at /home/milan/Dev/julia/src/gc.c:1245
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
push_root at /home/milan/Dev/julia/src/gc.c:1335
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
push_root at /home/milan/Dev/julia/src/gc.c:1335
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
push_root at /home/milan/Dev/julia/src/gc.c:1335
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
push_root at /home/milan/Dev/julia/src/gc.c:1335
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
push_root at /home/milan/Dev/julia/src/gc.c:1335
gc_push_root at /home/milan/Dev/julia/src/gc.c:1104
_jl_gc_collect at /home/milan/Dev/julia/src/gc.c:1607
__pool_alloc at /home/milan/Dev/julia/src/gc.c:704
newstruct at /home/milan/Dev/julia/src/julia_internal.h:67
jl_f_tuple at /home/milan/Dev/julia/src/builtins.c:653
next at ./array.jl:302
unknown function (ip: 0x7fda6d75a123)
jl_call_method_internal at /home/milan/Dev/julia/src/julia_internal.h:93
_require_from_serialized at ./loading.jl:182
unknown function (ip: 0x7fda6d86b68e)
jl_call_method_internal at /home/milan/Dev/julia/src/julia_internal.h:93
_require_from_serialized at ./loading.jl:202
require at ./loading.jl:332
unknown function (ip: 0x7fda6d86d0cb)
jl_call_method_internal at /home/milan/Dev/julia/src/julia_internal.h:93
jl_apply at /home/milan/Dev/julia/src/julia.h:1381
eval_import_path at /home/milan/Dev/julia/src/toplevel.c:411
jl_toplevel_eval_flex at /home/milan/Dev/julia/src/toplevel.c:434
jl_toplevel_eval_flex at /home/milan/Dev/julia/src/toplevel.c:446
jl_parse_eval_all at /home/milan/Dev/julia/src/ast.c:699
jl_load at /home/milan/Dev/julia/src/toplevel.c:579
include_from_node1 at ./loading.jl:426
unknown function (ip: 0x7fda6d81c80c)
jl_call_method_internal at /home/milan/Dev/julia/src/julia_internal.h:93
do_call at /home/milan/Dev/julia/src/interpreter.c:65
eval at /home/milan/Dev/julia/src/interpreter.c:173
jl_toplevel_eval_flex at /home/milan/Dev/julia/src/toplevel.c:539
jl_toplevel_eval_in_warn at /home/milan/Dev/julia/src/builtins.c:571
macro expansion; at ./none:2
unknown function (ip: 0x7fda79fb921f)
jl_call_method_internal at /home/milan/Dev/julia/src/julia_internal.h:93
jl_toplevel_eval_in_warn at /home/milan/Dev/julia/src/builtins.c:571
eval at ./boot.jl:225
unknown function (ip: 0x7fda6d7fff20)
jl_call_method_internal at /home/milan/Dev/julia/src/julia_internal.h:93
process_options at ./client.jl:243
_start at ./client.jl:322
unknown function (ip: 0x7fda6d855ce4)
jl_call_method_internal at /home/milan/Dev/julia/src/julia_internal.h:93
jl_apply at /home/milan/Dev/julia/ui/../src/julia.h:1381
main at /home/milan/Dev/julia/ui/repl.c:678
unknown function (ip: 0x7fda78f6957f)
unknown function (ip: 0x401718)
Allocations: 2043527 (Pool: 2042339; Big: 1188); GC: 2
@yuyichao yuyichao assigned vtjnash and unassigned yuyichao Jun 11, 2016
@yuyichao
Copy link
Contributor

Traced back to the deserializer which reads a Null_tag when deserializing a datatype. @vtjnash

@tkelman tkelman added the compiler:precompilation Precompilation of modules label Jun 11, 2016
@vtjnash vtjnash changed the title Segmentation fault when precompiling package packages with overly complicated recursively defined types aren't precompilable Jun 12, 2016
vtjnash added a commit that referenced this issue Jun 13, 2016
@vtjnash
reorganize some code in dump.c
214d5b4 
fix #16883

improve performance of packages with many ambiguous methods
(such as CategoricalArrays)

correctly classify primary datatypes during MODE_MODULE_POSTWORK
@vtjnash vtjnash mentioned this issue Jun 13, 2016
Merged
@yuyichao yuyichao mentioned this issue Jun 13, 2016
Closed
@nalimilan nalimilan mentioned this issue Sep 1, 2016
Closed
@yuyichao yuyichao mentioned this issue Sep 3, 2016
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vtjnash vtjnash

Labels
compiler:precompilation Precompilation of modules
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

reorganize some code in dump.c JuliaLang/julia
4 participants
@vtjnash @yuyichao @nalimilan @tkelman