Performance regression when using FunctionWrappers on Julia 1.4 and 1.5

[rdeits]

Copied from: https://discourse.julialang.org/t/performance-and-memory-regression-with-function-wrappers-in-julia-1-4/44589

Setup:

julia> using BenchmarkTools

julia> using FunctionWrappers: FunctionWrapper

julia> w = let closed_over=([1], [2])
         FunctionWrapper{typeof(closed_over), Tuple{}}(() -> closed_over)
       end;

Results

Julia 1.3.1:

julia> @btime $w()
  5.435 ns (0 allocations: 0 bytes)

Julia 1.4.2:

julia> @btime $w()
  9.919 ns (1 allocation: 16 bytes)

Julia 1.5.0

julia> @btime $w()
  10.112 ns (1 allocation: 32 bytes)

Julia 1.6.0-DEV.607

julia> @btime $w()
  11.846 ns (1 allocation: 32 bytes)

This turns out to be the source of a significant performance regression in Parametron.jl (which was designed to allocate zero memory).

Here's my system info:

julia> versioninfo()
Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)

@yuyichao pointed to yuyichao/FunctionWrappers.jl#14 as a related issue. I cannot reproduce the MWE from yuyichao/FunctionWrappers.jl#14 (comment) on Julia 1.5, but the slowness and extra allocation of my example here is still present in Julia 1.5.0.

[JeffBezanson]

I suspect @cfunction here is calling the argument-boxing shim instead of specialized code. @vtjnash what do you think?

[vtjnash]

Probably. As codegen has become increasingly restricted from mutating the system in arbitrary ways, we've gotten somewhat worse at this. The current design is intended to allow it to inject the new pointer to replace the shim, but we haven't implemented that update logic yet.

[wsshin]

As of Juila 1.6.1, the example doesn't seem using allocations:

julia> @btime $w()
  7.999 ns (0 allocations: 0 bytes)
([1], [2])

Here is the version info:

julia> VERSION
v"1.6.1-pre.1"

(@v1.6) pkg> status FunctionWrappers
      Status `~/.julia/environments/v1.6/Project.toml`
  [069b7b12] FunctionWrappers v1.1.2

[ViralBShah]

Seems to be 30% slower in Julia 1.8 from Julia 1.6.

[willtebbutt]

I just stumbled across this for the first time -- I've checked and this remains a problem on 1.10 (rc2).

edit: I'm assuming that the answer is no, but is there a work-around available at present?

[ancapdev]

I would love to see a language native typed function that can be bound with a type erased closure, ala what FunctionWrapper is doing. There are a few overheads that could be removed I think. Things like runtime thread state check and setup (is this the 1.8 regression?) wouldn't need to be done if the runtime knows the closure is only invoked from within the runtime itself (to leak it out you'd need a cfunction). Null check could be removed. All types could traverse the interface without being forced through the c type tunnel. Maybe there are more things? It would certainly help ergonomics as well.

[chriselrod]

@ancapdev

julia/base/opaque_closure.jl

Line 4 in c1ca0d3

@opaque ([type, ]args...) -> body

#44173

[oscardssmith]

note that opaque closures have a number of drawbacks some of which are unavoidable with the current design.

1. they are experimental (and given 2 and 3, I wouldn't be shocked if they were changed or removed in future Julia versions)
2. they are opaque. this means they can't be interpreted, ADed or run on gpus. This is a really bad state of affairs and makes them a significant pain to deal with.
3. they don't observe changes to world age. (this was intentional, but since generated functions now can deal with world age changes properly I'd be interested in @Keno's thoughts as to whether this is still a useful restriction)

[adienes]

opaque closures have a number of drawbacks

• are opaque
• don't observe changes to world age

🤔 forgive the naivety but isn't that the whole point?

[oscardssmith]

yes, but it's often not the behavior you want from a functionwrapper