Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Julia 1.0.3 crashes loading saved JuliaDB table with mempool error #26

Closed
MaximilianJHuber opened this issue Jan 9, 2019 · 5 comments
Closed

Julia 1.0.3 crashes loading saved JuliaDB table with mempool error #26

MaximilianJHuber opened this issue Jan 9, 2019 · 5 comments

Comments

@MaximilianJHuber
Copy link
Contributor

MaximilianJHuber commented Jan 9, 2019 •
edited
Loading

I am not sure whether this issue belongs here. On Julia 1.0.3, JuliaDB 0.10, and MemPool 0.1.2 (or master) I saved a table:

Table with 429003 rows, 7 columns:
Columns:
#  colname        type
──────────────────────────────────────
1  name1   String
2  name2    String
3  id1       Int64
4  id2      Int64
5  date1       Union{Missing, Date}
6  date2  Union{Missing, Date}
7  date3  Union{Missing, Date}

and when I load it back in Julia crashes with:

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7fff76cf4300 -- memmove at C:\WINDOWS\System32\msvcrt.dll (unknown line)
in expression starting at no file:0
memmove at C:\WINDOWS\System32\msvcrt.dll (unknown line)
jl_pchar_to_string at /home/Administrator/buildbot/worker/package_win64/build/src\array.c:472
unsafe_string at .\strings\string.jl:53
unknown function (ip: 000000001276EBB0)
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184
mmread at C:\Users\Max\.julia\packages\MemPool\Bw8DR\src\io.jl:135
deserialize at C:\Users\Max\.julia\packages\MemPool\Bw8DR\src\io.jl:27
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1831
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184
handle_deserialize at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Serialization\src\Serialization.jl:762
deserialize at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Serialization\src\Serialization.jl:711 [inlined]
#30 at .\none:0 [inlined]
iterate at .\generator.jl:47 [inlined]
collect at .\array.jl:619
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1831
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184
mmread at C:\Users\Max\.julia\packages\JuliaDB\R4e6y\src\serialize.jl:53
deserialize at C:\Users\Max\.julia\packages\MemPool\Bw8DR\src\io.jl:27
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1831
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184
handle_deserialize at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Serialization\src\Serialization.jl:762
mmread at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Serialization\src\Serialization.jl:711
deserialize at C:\Users\Max\.julia\packages\MemPool\Bw8DR\src\io.jl:27
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1831
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184
handle_deserialize at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Serialization\src\Serialization.jl:762
deserialize at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Serialization\src\Serialization.jl:711
handle_deserialize at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Serialization\src\Serialization.jl:801
deserialize at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Serialization\src\Serialization.jl:711
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1831
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184 [inlined]
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1537 [inlined]
jl_invoke at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:56
deserialize at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Serialization\src\Serialization.jl:708 [inlined]
#open#294 at .\iostream.jl:369
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1831
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184 [inlined]
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1537 [inlined]
jl_invoke at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:56
open at .\iostream.jl:367 [inlined]
load at C:\Users\Max\.julia\packages\JuliaDB\R4e6y\src\io.jl:181
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184
do_call at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:324
eval_value at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:430
eval_stmt_value at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:363 [inlined]
eval_body at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:678
jl_interpret_toplevel_thunk_callback at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:806
unknown function (ip: FFFFFFFFFFFFFFFE)
unknown function (ip: 0000000008BB720F)
unknown function (ip: FFFFFFFFFFFFFFFF)
jl_toplevel_eval_flex at /home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:805
jl_toplevel_eval_in at /home/Administrator/buildbot/worker/package_win64/build/src\builtins.c:622
eval at .\boot.jl:319
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184
eval_user_input at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\REPL\src\REPL.jl:85
macro expansion at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\REPL\src\REPL.jl:117 [inlined]
#28 at .\task.jl:259
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1537 [inlined]
start_task at /home/Administrator/buildbot/worker/package_win64/build/src\task.c:268
Allocations: 38776166 (Pool: 38767273; Big: 8893); GC: 84
@MaximilianJHuber
Copy link
Contributor Author

This is very likely related to this and this.

The suggested quick-and-dirty solution is to disable the garbage collector temporarily:

Base.GC.enable(false)
table = load(path)
Base.GC.enable(true)

and works for me.

@yoh-meyers yoh-meyers mentioned this issue Apr 16, 2019
Open
@joshday
Copy link
Contributor

joshday commented Apr 16, 2019

Sorry, this issue slipped past me. If this still occurs, could you post

map(typeof, columns(mytable))

I'm curious if it's WeakRefStrings-related.

@MaximilianJHuber
Copy link
Contributor Author

Another table, same issue:

Table with 16112465 rows, 6 columns:
Columns:
#  colname           type
───────────────────────────
1  id  Int32
2  start_date   Date
3  end_date     Date
4  value       String
5  type      Int64
6  flag        Bool

map(typeof, columns(mytable)) yields:

(id= Array{Int32,1}, start_date = Array{Dates.Date,1}, end_date = Array{Dates.Date,1}, value = Array{String,1}, type_id = Array{Int64,1}, flag = Array{Bool,1})

The issue persists on Julia 1.1.0, JuliaDB 0.11.2, MemPool 0.2.0.

@r3tex
Copy link

r3tex commented Jul 24, 2019

The GC walkaround works for me and allows me to read this table back.

(LocationLatitude = Dagger.DArray{Float32,1,typeof(JuliaDB.dvcat)}, 
LocationTimestamp = Dagger.DArray{Dates.DateTime,1,typeof(JuliaDB.dvcat)}, 
LocationProvider = Dagger.DArray{String,1,typeof(JuliaDB.dvcat)}, 
LocationAge = Dagger.DArray{Int64,1,typeof(JuliaDB.dvcat)}, 
RadioLteRsrp = Dagger.DArray{Int32,1,typeof(JuliaDB.dvcat)}, 
LocationLongitude = Dagger.DArray{Float32,1,typeof(JuliaDB.dvcat)}, 
gcId = Dagger.DArray{String,1,typeof(JuliaDB.dvcat)})

Still happening on Julia 1.2.0-rc2, JuliaDB 0.12.0, MemPool 0.2.0

@bwang12 bwang12 mentioned this issue Sep 12, 2019
Closed
@haavardhvarnes haavardhvarnes mentioned this issue Jun 5, 2020
Closed
@jpsamaroo
Copy link
Collaborator

Closing as this issue is rather old and JuliaDB is unmaintainted; please re-open with an MWE using MemPool's master branch if you still need help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@MaximilianJHuber @joshday @r3tex @jpsamaroo