Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import latency issue #324

Closed
Moelf opened this issue Jan 10, 2023 · 4 comments
Closed

Import latency issue #324

Moelf opened this issue Jan 10, 2023 · 4 comments

Comments

@Moelf
Copy link
Member

Moelf commented Jan 10, 2023

julia> @time_imports using UnicodePlots
      0.2 ms  SnoopPrecompile
      3.1 ms  StaticArraysCore
    795.5 ms  StaticArrays
     69.3 ms  FixedPointNumbers
     77.7 ms  ColorTypes 9.12% compilation time
     48.2 ms  Crayons
      0.8 ms  ConstructionBase
    418.2 ms  Unitful
      1.7 ms  DataAPI
      0.7 ms  Compat
      8.0 ms  OrderedCollections
     83.6 ms  DataStructures
      0.5 ms  SortingAlgorithms
      9.8 ms  Missings
      3.8 ms  DocStringExtensions 75.18% compilation time
     89.1 ms  ChainRulesCore
     14.1 ms  ChangesOfVariables
      1.2 ms  InverseFunctions
      6.1 ms  IrrationalConstants
      0.8 ms  LogExpFunctions
      0.3 ms  StatsAPI
     24.8 ms  StatsBase
    126.2 ms  MarchingCubes
      0.2 ms  Reexport
    380.0 ms  Colors
      0.3 ms  OpenLibm_jll
     22.8 ms  Preferences
      0.4 ms  JLLWrappers
      0.7 ms  CompilerSupportLibraries_jll
      3.8 ms  OpenSpecFun_jll 86.80% compilation time
     17.3 ms  SpecialFunctions
      0.3 ms  TensorCore
    170.0 ms  ColorVectorSpace 4.10% compilation time
     16.2 ms  ColorSchemes
      0.3 ms  Requires
      0.3 ms  NaNMath
      1.5 ms  Contour
    333.9 ms  FileIO 3.41% compilation time (13% recompilation)
      0.6 ms  Bzip2_jll
      0.1 ms  Zlib_jll
      0.3 ms  FreeType2_jll
      4.8 ms  CEnum
      6.7 ms  FreeType
   1801.4 ms  UnicodePlots 2.79% compilation time

what's causing the 1.8s just by UnicodePlots?

@t-bltg
Copy link
Member

t-bltg commented Jan 10, 2023

I have default similar numbers. However, using timholy/SnoopCompile.jl#334, on 1.9.0.beta2:

$ julia
julia> using SnoopPrecompile, Preferences
julia> Preferences.set_preferences!(SnoopPrecompile, "skip_precompile" => ["UnicodePlots"])
julia> exit(0)
$ julia -e 'using UnicodePlots'  # trigger precompilation with a reduced workload
$ julia
julia> @time @time_imports using UnicodePlots
     29.5 ms  Preferences
      0.2 ms  SnoopPrecompile
      0.9 ms  Statistics
      3.3 ms  StaticArraysCore
    900.5 ms  StaticArrays
     60.6 ms  FixedPointNumbers
    126.4 ms  ColorTypes 26.56% compilation time (85% recompilation)
     35.2 ms  Crayons
      1.4 ms  ConstructionBase
   1194.2 ms  Unitful
      1.6 ms  DataAPI
      0.2 ms  Compat
     10.8 ms  OrderedCollections
    115.9 ms  DataStructures
      0.6 ms  SortingAlgorithms
     14.8 ms  Missings
      2.9 ms  DocStringExtensions
    124.3 ms  ChainRulesCore
      1.0 ms  ChangesOfVariables
      1.1 ms  InverseFunctions
      7.6 ms  IrrationalConstants
      0.8 ms  LogExpFunctions
      0.4 ms  StatsAPI
     35.5 ms  StatsBase
     86.8 ms  MarchingCubes
      0.3 ms  Reexport
    297.3 ms  Colors
      0.5 ms  OpenLibm_jll
      0.3 ms  JLLWrappers
      7.1 ms  OpenSpecFun_jll 90.91% compilation time
     71.9 ms  SpecialFunctions
      0.5 ms  TensorCore
    239.6 ms  ColorVectorSpace 5.82% compilation time (91% recompilation)
     11.8 ms  ColorSchemes
      0.2 ms  Requires
      0.3 ms  NaNMath
      1.8 ms  Contour
    206.1 ms  FileIO
      0.6 ms  Bzip2_jll
      0.2 ms  Zlib_jll
      0.4 ms  FreeType2_jll
      5.5 ms  CEnum
      8.0 ms  FreeType
    144.9 ms  UnicodePlots
  4.025836 seconds (5.26 M allocations: 346.759 MiB, 6.95% gc time, 6.02% compilation time: 17% of which was recompilation)

what's causing the 1.8s just by UnicodePlots?

I would say precompiled code loading. You can also disable this workload using UP_PRECOMPILE=false env variable Preferences (see https://discourse.julialang.org/t/psa-for-snoopprecompile-turning-off-extra-workload-for-specific-packages/92865 and https://timholy.github.io/SnoopCompile.jl/dev/snoop_pc/). But then TTFP is affected 🤷 .

@Moelf
Copy link
Member Author

Moelf commented Jan 10, 2023

lineplot(I:2),
lineplot(I:2, I:2),
lineplot(sin, -I, I),
lineplot(sin, (-I):I),
lineplot([sin, cos], -I, I),
lineplot([sin, cos], (-I):I),
lineplot(I:2, T[0:1 2:3]),
lineplot(I:2, I:2; xscale = :ln, yscale = :log10),
lineplot([Date(2020), Date(2021)], I:2),
scatterplot(I:2),
scatterplot(I:2, I:2),
scatterplot(I:2, T[0:1 2:3]),
stairs(T[1, 2, 4], T[1, 3, 4]),
barplot(["Paris", "New York"], T[2, 8]),
boxplot(T[1, 2, 3]),
boxplot(["one", "two"], [collect(I:3), collect(I:4)]),
histogram(T[1, 2, 3]),
histogram(T[1, 2, 3]; vertical = true),
polarplot(0:/ 4):π, I:5),
densityplot(T[1, 2], T[3, 4]; dscale = x -> log(1 + x)),
heatmap(repeat(collect(T, 0:4)'; outer = (5, 1))),
spy(T[1 -1 0; -1 2 1; 0 -1 1]),
contourplot(surf.(meshgrid((-I):I, (-2I):(2I))...)),
contourplot((-I):I, (-2I):(2I), surf),
surfaceplot((-I):I, (-2I):(2I), surf; lines = true),
surfaceplot((-I):I, (-2I):(2I), surf.(meshgrid((-I):I, (-2I):(2I))...)),
isosurface((-I):4, I:2, I:3, (x, y, z) -> float(x^2 + y^2 - z^2 - 1)),

this is too much I think, you probably want to condense down, it's like a 20/80 rule I guess

@ablaom
Copy link

ablaom commented Feb 15, 2023

I notice UnicodePlots has the largest load time of all the many dependencies of MLJ, which surprised me. See also JuliaAI/CategoricalDistributions.jl#49

@t-bltg
Copy link
Member

t-bltg commented Feb 22, 2023

Closing in favor of #335, since I believe that the --pkgimages=no flag or the Preferences mechanism are of help for the costly precompiled code loading (in case one is interested in a lower load time rather that an decent TTFP).

@t-bltg t-bltg closed this as completed Feb 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants