Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Julia repeated crash when using XLSX writetable to produce a multi-tab Excel output #111

Closed
bwang12 opened this issue Sep 10, 2019 · 4 comments
Closed

Julia repeated crash when using XLSX writetable to produce a multi-tab Excel output #111

bwang12 opened this issue Sep 10, 2019 · 4 comments

Comments

@bwang12
Copy link

bwang12 commented Sep 10, 2019

My use case involves creating a multi-tab Excel with each tab containing a different dataframe. More often than not, this step would produce the error log attached below and crash Julia. The output Excel would be corrupted with the first column and column headers missing after using Excel to recover what's leftover.

My code essentially looks like the following, but I cannot actually replicate the error with this code below, it ran perfectly fine for me. On a side note, would anyone know how I can overwrite the output if the file already exists instead of just getting an error?

Thx in advance!

Code:

path = raw"XXX"

df1 = DataFrame(Dict("Test1" => ["test", "test", "test", "test", "test", "test", "test", "test", "test", "test"]))
df2 = DataFrame(Dict("Test2" => ones(20)))
df3 = DataFrame(Dict("Test3" => ones(30)))

writetable(path * raw"\testing.xlsx",
df1=( collect(eachcol(df1)), names(df1) ),
df2=( collect(eachcol(df2)), names(df2) ),
df3=( collect(eachcol(df3)), names(df3) ),
)

ERROR:

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ffd2b034300 -- memmove at C:\Windows\System32\msvcrt.dll (unknown line)
in expression starting at O:\Staff\BWang\Projects\General_Portfolio_Optimizer\Julia\julia_optimizer.jl:706
memmove at C:\Windows\System32\msvcrt.dll (unknown line)
read_buf at C:\Users\bwang.julia\packages\ZipFile\oD4uG\deps\usr\bin\libz-1.2.11.dll (unknown line)
fill_window at C:\Users\bwang.julia\packages\ZipFile\oD4uG\deps\usr\bin\libz-1.2.11.dll (unknown line)
deflate_slow at C:\Users\bwang.julia\packages\ZipFile\oD4uG\deps\usr\bin\libz-1.2.11.dll (unknown line)
deflate at C:\Users\bwang.julia\packages\ZipFile\oD4uG\deps\usr\bin\libz-1.2.11.dll (unknown line)
write at C:\Users\bwang.julia\packages\ZipFile\oD4uG\src\Zlib.jl:138
unsafe_write at C:\Users\bwang.julia\packages\ZipFile\oD4uG\src\Zlib.jl:156
unknown function (ip: 0000000066C46A29)
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2191
unsafe_write at C:\Users\bwang.julia\packages\ZipFile\oD4uG\src\ZipFile.jl:542
dump_node at .\gcutils.jl:87
print at C:\Users\bwang.julia\packages\EzXML\G0JeY\src\node.jl:310 [inlined]
print at C:\Users\bwang.julia\packages\EzXML\G0JeY\src\document.jl:55 [inlined]
#writexlsx#34 at C:\Users\bwang.julia\packages\XLSX\FJ07G\src\write.jl:67
#writexlsx at .\none:0 [inlined]
#writetable#44 at C:\Users\bwang.julia\packages\XLSX\FJ07G\src\write.jl:648
#writetable at .\none:0
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2191
#export_portfolio#14 at O:\Staff\BWang\Projects\General_Portfolio_Optimizer\Julia\julia_optimizer_misc.jl:511
#export_portfolio at .\none:0
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2191
do_call at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:323
eval_value at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:411
eval_stmt_value at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:362 [inlined]
eval_body at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:758
jl_interpret_toplevel_thunk_callback at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:884
unknown function (ip: FFFFFFFFFFFFFFFE)
unknown function (ip: 00000001277E668F)
unknown function (ip: 0000000000000006)
jl_toplevel_eval_flex at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:815jl_parse_eval_all at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\ast.c:873
include_string at .\loading.jl:1064
include_string at C:\Users\bwang.julia\packages\CodeTools\xGemk\src\eval.jl:30
unknown function (ip: 000000003BB3538A)
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2197
#118 at C:\Users\bwang.julia\packages\Atom\Wouyw\src\eval.jl:94
withpath at C:\Users\bwang.julia\packages\CodeTools\xGemk\src\utils.jl:30
withpath at C:\Users\bwang.julia\packages\Atom\Wouyw\src\eval.jl:46 [inlined]
#117 at C:\Users\bwang.julia\packages\Atom\Wouyw\src\eval.jl:93 [inlined]
with_logstate at .\logging.jl:395
with_logger at .\logging.jl:491 [inlined]
#116 at C:\Users\bwang.julia\packages\Atom\Wouyw\src\eval.jl:92 [inlined]
hideprompt at C:\Users\bwang.julia\packages\Atom\Wouyw\src\repl.jl:75
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2197
macro expansion at C:\Users\bwang.julia\packages\Atom\Wouyw\src\eval.jl:91 [inlined]
macro expansion at C:\Users\bwang.julia\packages\Media\ItEPc\src\dynamic.jl:24 [inlined]
#115 at C:\Users\bwang.julia\packages\Atom\Wouyw\src\eval.jl:86
#88 at C:\Users\bwang.julia\packages\Revise\D6h31\src\Revise.jl:1101
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2197
handlemsg at C:\Users\bwang.julia\packages\Atom\Wouyw\src\comm.jl:164
unknown function (ip: 000000005E5682D3)
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2191
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1614 [inlined]
jl_f__apply at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\builtins.c:563
#19 at .\task.jl:268
unknown function (ip: 000000005E56663A)
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2191
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1614 [inlined]
start_task at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\task.c:596
Allocations: 514056842 (Pool: 510122452; Big: 3934390); GC: 438
@felipenoris
Copy link
Owner

Hi!

I can't reproduce this bug.

The following note is unrelated to your problem, but please, be aware that DataFrames.jl exports a function named writetable. In XLSX.jl I have the policy to not export functions, so you should use

XLSX.writetable("report.xlsx", df1=( collect(eachcol(df1)), names(df1) ),
       df2=( collect(eachcol(df2)), names(df2) ),
       df3=( collect(eachcol(df3)), names(df3) ),
       )

with the XLSX prefix to use the correct method.

@felipenoris
Copy link
Owner

If the output file already exists, you can use overwrite=true, as in

XLSX.writetable("report.xlsx", df1=( collect(eachcol(df1)), names(df1) ),
       df2=( collect(eachcol(df2)), names(df2) ),
       df3=( collect(eachcol(df3)), names(df3) ),
       overwrite=true)

@bwang12
Copy link
Author

bwang12 commented Sep 12, 2019

Hi!

I can't reproduce this bug.

The following note is unrelated to your problem, but please, be aware that DataFrames.jl exports a function named writetable. In XLSX.jl I have the policy to not export functions, so you should use

XLSX.writetable("report.xlsx", df1=( collect(eachcol(df1)), names(df1) ),
       df2=( collect(eachcol(df2)), names(df2) ),
       df3=( collect(eachcol(df3)), names(df3) ),
       )

with the XLSX prefix to use the correct method.

Thanks for the prompt response!

I can confirm I am using the writetable from XLSX as I only import specific functions from DataFrames and XLSX.

Additionally, I have switched the writetable destination from a sharedrive location to my local C drive and yet still encountered the same error.

After some searching, I discovered a fix that seems to have worked nicely.
It simply involves disabling GC before the writetable function then re-enable it afterwards.

Base.GC.enable(false)
XLSX.writetable(XXXXX)
Base.GC.enable(true)

The original source is from here JuliaData/MemPool.jl#26

@bwang12 bwang12 closed this as completed Sep 12, 2019
@felipenoris
Copy link
Owner

Looks like a windows issue. At least it has a workaround. Thanks for the info!

@haavardhvarnes haavardhvarnes mentioned this issue Jun 5, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@felipenoris @bwang12