Allow precompile of methods in different modules and methods w/ keyword arguments

[quinnj]

I'm opening this w/ the intent to gather feedback/answers to some questions on precompile so we can update the docs.

• What exactly is precompile compiling? A function w/ a specific set of arguments, but what is it being compiled to? Lowered form? Post-inference? LLVM IR?
• Why doesn't precompile throw an error if it can't actually compile a function for a given set of arguments? Just returning false is fine, but just wondering why not just throw an error
• Am I correct in understanding that functions w/ keyword arguments are essentially "uncompilable"? I mean, I could say precompile(sort, (Vector{Int64},)), but it's just compiling the keyword-arg-generator function like

julia> @code_warntype sort([1,2,3])
Variables:
  #self#::typeof(sort)
  v::Array{Int64,1}

Body:
  begin
      $(Expr(:inbounds, false))
      # meta: location boot.jl Type 368
      # meta: pop location
      $(Expr(:inbounds, :pop))
      return $(Expr(:invoke, MethodInstance for #sort#8(::Array{Any,1}, ::Function, ::Array{Int64,1}), :(Base.Sort.#sort#8), :($(Expr(:foreigncall, :(:jl_alloc_array_1d), Array{Any,1}, svec(Any, Int64), :(:ccall), 2, Array{Any,1}, 0))), :(#self#), :(v)))::Array{Int64,1}
  end::Array{Int64,1

right? And since it seems the "under the hood" actual keyword methods have gensym-ed names, it wouldn't be possible to precompile those? Am I on the right track there?

• Any recommendations on identifying things that need to be precompiled? I haven't looked at the internals of SnoopCompile yet, but I guess they're doing something there (though the generated precompiles seem to assume the old call convention on 0.6/0.7). The other strategy I've taken is doing a @profile on the initial code run and seeing which lines immediately precede large inference.jl blocks. Sometimes it seems that precompiling those methods doesn't remove the need to do inference in them though; is that because there might be methods in Base/other modules that still need compiling? So can a method only be precompiled if all the methods it calls are precompiled too?

[quinnj]

For reference, the current docs for precompile are

help?> precompile
search: precompile __precompile__

  precompile(f, args::Tuple{Vararg{Any}})

  Compile the given function f for the argument tuple (of types) args, but do not execute it.

[TsurHerman]

I think a solution would be the ability to save Julia's binary state , to be resumed in a later time.
For reference #22598

[tshort]

Earlier discussion:

http://julia-programming-language.2336112.n4.nabble.com/Precompilation-and-functions-with-keyword-arguments-td32038.html

[StefanKarpinski]

This is not a 1.0 blocker as far as I can tell...

[vtjnash]

Questions have been addressed or are less relevant now than other open issues