Gadfly load time regression from 0.5 to 0.6.

[glycerine]

How the heck does anyone get anything done in Julia 0.6.2 on 64-bit linux (ubuntu 16.04). It takes 22 seconds to load the plotting package Gadfly ??? It's not precompilation time, since it happens even after the package has been precompiled once. If I restart Julia, I still have to wait, sometimes even longer the second time.

# julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.2 (2017-12-13 18:08 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-pc-linux-gnu

julia> tic(); using Gadfly; toc()
elapsed time: 22.640510854 seconds
22.640510854

julia> [ctl-D to exit]
# julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.2 (2017-12-13 18:08 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-pc-linux-gnu

julia> tic(); using Gadfly; toc()
elapsed time: 28.402201721 seconds
28.402201721

julia> 

[pkofod]

How the heck does anyone get anything done

Not a very constructive tone.

Is this something you didn't experience before, or is this your first time working with Julia? Long load times of certain packages is a very well known issue.

[glycerine]

Long load times of certain packages is a very well known issue.

If this is a duplicate, where's the open ticket?

[ViralBShah]

Gadfly startup time issues should be reported in that repo and has been able discussed in gadfly.jl issues.

Please also realize that this is a community project. So it helps to be constructive and offer to pitch in.

[glycerine]

For others who will have a similar experience as my own, and search and find this issue after being astonished at the slow startup times in julia 6.2, here are links to related open issues:

GiovineItalia/Gadfly.jl#921

#15048

These appear to be long standing issues in both Gadfly and Julia 0.6.x itself.

However those currently open issues do not explain why using gets SLOWER the second time around. So we suspect that this is a novel, useful, possibly new observation.

[pkofod]

So we suspect that this is a novel, useful, possibly new observation.

And that is certainly something to investigate.

In the original comment you mention that sometimes it is slower the second time around. Is this to be interpreted as it is unpredictable,or as it is almost always slower the second time around?

Still, I think Viral Shah's comment stands, until proven to be a general JuliaLang issue, this discussion should probably be held at Gadfly.

[StefanKarpinski]

To be clear, we are very aware that compile and startup times are an issue and it is at the top of the post-1.0 priority list.

[JeffBezanson]

Does anybody happen to know if the gadfly load time is worse than in 0.6.1, or 0.6.0, or 0.5.x?

[glycerine]

Seems to be a regression. For comparison to the above, here is Julia 0.5.2 on my system, a mere 3.9 seconds versus > 20 seconds.

on Julia 0.5.2 on ubuntu 16.04 amd64

# initial build time on my system
julia> tic(); using Gadfly; toc();
INFO: Precompiling module Gadfly.
WARNING: Method definition describe(AbstractArray) in module StatsBase at /root/.julia/v0.5/StatsBase/src/scalarstats.jl:560 overwritten in module DataFrames at /root/.julia/v0.5/DataFrames/src/abstractdataframe/abstractdataframe.jl:407.
WARNING: Method definition describe(AbstractArray) in module StatsBase at /root/.julia/v0.5/StatsBase/src/scalarstats.jl:560 overwritten in module DataFrames at /root/.julia/v0.5/DataFrames/src/abstractdataframe/abstractdataframe.jl:407.
WARNING: Method definition describe(AbstractArray) in module StatsBase at /root/.julia/v0.5/StatsBase/src/scalarstats.jl:560 overwritten in module DataFrames at /root/.julia/v0.5/DataFrames/src/abstractdataframe/abstractdataframe.jl:407.
elapsed time: 87.792589576 seconds

julia> 

# `using` time:
julia> tic(); using Gadfly; toc();
WARNING: Method definition describe(AbstractArray) in module StatsBase at /root/.julia/v0.5/StatsBase/src/scalarstats.jl:560 overwritten in module DataFrames at /root/.julia/v0.5/DataFrames/src/abstractdataframe/abstractdataframe.jl:407.
elapsed time: 3.857843109 seconds

julia> tic(); using Gadfly; toc();
elapsed time: 8.88e-5 seconds


#  `using` time after pre-compilation
julia> 
root@379846b04147:~# julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.5.2 (2017-05-06 16:34 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-pc-linux-gnu

julia> tic(); using Gadfly; toc();
WARNING: Method definition describe(AbstractArray) in module StatsBase at /root/.julia/v0.5/StatsBase/src/scalarstats.jl:560 overwritten in module DataFrames at /root/.julia/v0.5/DataFrames/src/abstractdataframe/abstractdataframe.jl:407.
elapsed time: 3.460540924 seconds

julia> 

some Julia 0.6.2 timings for comparison:

$ /usr/local/julia-0.6.2/bin/julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.2 (2017-12-13 18:08 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-pc-linux-gnu

julia> tic(); using Gadfly; toc()
elapsed time: 25.549629369 seconds
25.549629369

julia> tic(); using Gadfly; toc()
elapsed time: 0.000171706 seconds
0.000171706

julia> 
$ /usr/local/julia-0.6.2/bin/julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.2 (2017-12-13 18:08 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-pc-linux-gnu

julia> tic(); using Gadfly; toc()
elapsed time: 21.39631405 seconds
21.39631405

julia> tic(); using Gadfly; toc()
elapsed time: 0.00016042 seconds
0.00016042

julia> 

[pkofod]

Is that the same version of Gadfly aswell? (look at Pkg.status("Gadfly"))

[andreasnoack]

I can reproduce the very significant slowdown between Julia 0.5.2 and 0.6.2-pre both running Gadfly 0.6.4.

[ViralBShah]

The load time regression is definitely a real issue. Reopening. @glycerine Thanks for reporting that.

[JeffBezanson]

This is a general problem mostly due to the new type system in 0.6, e.g. #24383

[glycerine]

Curious. I wouldn't think a big regression like that would be allowed to persist in the latest release. Why not back out the problem and publish an 0.6.3 with the fix immediately, if not sooner? ;)

[StefanKarpinski]

Because the cause was a bug fix and the buggy behavior was an accidental and incorrect "optimization". To fix this would be to be intentionally incorrect; otherwise, we need a correct version of such an optimization.

[JeffBezanson]

Procedural issue: this does not need to be triaged, because triage at the moment is focused on 1.0, which is a feature milestone.

[KristofferC]

Don't think a dedicated issue needs to be open about this. We all know we need to improve package load time across the board.