precompile

[johnnychen94]

I tried to build precompile.jl with SnoopCompile for ImageQualityIndexes with the following script:

# snoopfun.jl
using ImageQualityIndexes, TestImages, ImageCore
using ImageShow

img_gray_1 = testimage("fabio_gray_512")
img_gray_2 = testimage("cameraman")
img_rgb_1 = testimage("mandril_color")
img_rgb_2 = testimage("fabio_color_512")

assess_ssim(img_gray_1, img_gray_2);
assess_ssim(img_rgb_1, img_rgb_2);
assess_psnr(img_gray_1, img_gray_2)
assess_psnr(img_rgb_1, img_rgb_2)
assess_msssim(img_gray_1, img_gray_2)
assess_msssim(img_rgb_1, img_rgb_2)

colorfulness(img_gray_1)
colorfulness(img_rgb_1)

I tried both @snoopi and @snoopc but failed to see a boost in the first invocation. What's worse, it recompiles more frequently.

SnoopCompile.@snoopc "compiles.log" include("snoopfun.jl")
data = SnoopCompile.read("compiles.log")

pc = SnoopCompile.parcel(reverse!(data[2]))
SnoopCompile.write("precompile", pc)
# copy `precompile/precompile_ImageQualityIndexes.jl to `src/`
# and add `@assert` to each of the precompile call

The following is what's generated by @snoopc:

function _precompile_()
    ccall(:jl_generating_output, Cint, ()) == 1 || return nothing
    @assert isdefined(ImageQualityIndexes, Symbol("#__ssim_map_general##kw")) && precompile(Tuple{getfield(ImageQualityIndexes, Symbol("#__ssim_map_general##kw")), NamedTuple{(:crop,), Tuple{Bool}}, typeof(ImageQualityIndexes.__ssim_map_general), Array{ColorTypes.Gray{Float64}, 2}, Array{ColorTypes.Gray{Float64}, 2}, OffsetArrays.OffsetArray{Float64, 1, Array{Float64, 1}}, Float64, Float64, Float64})
    @assert isdefined(ImageQualityIndexes, Symbol("#__ssim_map_general##kw")) && precompile(Tuple{getfield(ImageQualityIndexes, Symbol("#__ssim_map_general##kw")), NamedTuple{(:crop,), Tuple{Bool}}, typeof(ImageQualityIndexes.__ssim_map_general), Array{ColorTypes.RGB{Float64}, 2}, Array{ColorTypes.RGB{Float64}, 2}, OffsetArrays.OffsetArray{Float64, 1, Array{Float64, 1}}, Float64, Float64, Float64})
    @assert precompile(Tuple{typeof(ImageQualityIndexes._average_pooling), Array{ColorTypes.Gray{Float64}, 2}})
    @assert precompile(Tuple{typeof(ImageQualityIndexes._average_pooling), Array{ColorTypes.RGB{Float64}, 2}})
    @assert precompile(Tuple{typeof(ImageQualityIndexes.assess_msssim), Array{ColorTypes.Gray{FixedPointNumbers.Normed{UInt8, 8}}, 2}, Array{ColorTypes.Gray{FixedPointNumbers.Normed{UInt8, 8}}, 2}})
    @assert precompile(Tuple{typeof(ImageQualityIndexes.assess_msssim), Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8, 8}}, 2}, Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8, 8}}, 2}})
    @assert precompile(Tuple{typeof(ImageQualityIndexes.assess_psnr), Array{ColorTypes.Gray{FixedPointNumbers.Normed{UInt8, 8}}, 2}, Array{ColorTypes.Gray{FixedPointNumbers.Normed{UInt8, 8}}, 2}})
    @assert precompile(Tuple{typeof(ImageQualityIndexes.assess_psnr), Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8, 8}}, 2}, Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8, 8}}, 2}})
    @assert precompile(Tuple{typeof(ImageQualityIndexes.assess_ssim), Array{ColorTypes.Gray{FixedPointNumbers.Normed{UInt8, 8}}, 2}, Array{ColorTypes.Gray{FixedPointNumbers.Normed{UInt8, 8}}, 2}})
    @assert precompile(Tuple{typeof(ImageQualityIndexes.assess_ssim), Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8, 8}}, 2}, Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8, 8}}, 2}})
    @assert precompile(Tuple{typeof(ImageQualityIndexes.colorfulness), Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8, 8}}, 2}})
end

@timholy Is there anything here I did incorrectly, or simply there's no precompilation gain doing this in ImageQualityIndexes?

[timholy]

I've not had any luck either, but for your reference here's a snapshot of how I go about this. While it's not commonly done this way, I personally prefer to write out the precompile directives by hand so that I can leverage my understanding of the package internals to control stuff like coverage of array eltypes and dimensionality. (In other words, I don't use parcel or write in most cases.) Here's a session:

julia> using ImageQualityIndexes, TestImages, SnoopCompile

julia> img_gray_1, img_gray_2 = testimage("fabio_gray_512"), testimage("cameraman");

julia> tinf = @snoopi tmin=0.01 assess_ssim(img_gray_1, img_gray_2)
8-element Vector{Tuple{Float64,Core.MethodInstance}}:
 (0.014342784881591797, MethodInstance for copy(::Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple},Nothing,typeof(ImageCore.channelview),Tuple{NTuple{6,Matrix{ColorTypes.Gray{Float64}}}}}))
 (0.015332937240600586, MethodInstance for mean(::Matrix{Float64}))
 (0.02500009536743164, MethodInstance for materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(-),Tuple{SubArray{ColorTypes.Gray{Float64},2,Matrix{ColorTypes.Gray{Float64}},Tuple{Base.OneTo{Int64},Base.OneTo{Int64}},false},Matrix{ColorTypes.Gray{Float64}}}}))
 (0.029573917388916016, MethodInstance for imfilter(::Type{ColorTypes.Gray{Float64}}, ::Matrix{ColorTypes.Gray{Float32}}, ::Tuple{ImageFiltering.KernelFactors.ReshapedOneD{Float64,2,0,OffsetArrays.OffsetVector{Float64,Vector{Float64}}},ImageFiltering.KernelFactors.ReshapedOneD{Float64,2,1,OffsetArrays.OffsetVector{Float64,Vector{Float64}}}}, ::String))
 (0.042634010314941406, MethodInstance for materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(ImageQualityIndexes._mul),Tuple{SubArray{ColorTypes.Gray{Float64},2,Matrix{ColorTypes.Gray{Float64}},Tuple{Base.OneTo{Int64},Base.OneTo{Int64}},false},SubArray{ColorTypes.Gray{Float64},2,Matrix{ColorTypes.Gray{Float64}},Tuple{Base.OneTo{Int64},Base.OneTo{Int64}},false}}}))
 (0.1404099464416504, MethodInstance for materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(/),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(*),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(+),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(*),Tuple{Int64,Base.ReinterpretArray{Float64,2,ColorTypes.Gray{Float64},Matrix{ColorTypes.Gray{Float64}}}}},Float64}},Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(+),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(*),Tuple{Int64,Base.ReinterpretArray{Float64,2,ColorTypes.Gray{Float64},Matrix{ColorTypes.Gray{Float64}}}}},Float64}}}},Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(*),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(+),Tuple{Base.ReinterpretArray{Float64,2,ColorTypes.Gray{Float64},Matrix{ColorTypes.Gray{Float64}}},Base.ReinterpretArray{Float64,2,ColorTypes.Gray{Float64},Matrix{ColorTypes.Gray{Float64}}},Float64}},Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(+),Tuple{Base.ReinterpretArray{Float64,2,ColorTypes.Gray{Float64},Matrix{ColorTypes.Gray{Float64}}},Base.ReinterpretArray{Float64,2,ColorTypes.Gray{Float64},Matrix{ColorTypes.Gray{Float64}}},Float64}}}}}}))
 (0.35606908798217773, MethodInstance for imfilter(::Type{ColorTypes.Gray{Float64}}, ::MappedArrays.MappedArray{ColorTypes.Gray{Float32},2,Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}},MappedArrays.var"#2#4"{ColorTypes.Gray{Float32}},MappedArrays.var"#3#5"{ColorTypes.Gray{FixedPointNumbers.N0f8}}}, ::Tuple{ImageFiltering.KernelFactors.ReshapedOneD{Float64,2,0,OffsetArrays.OffsetVector{Float64,Vector{Float64}}},ImageFiltering.KernelFactors.ReshapedOneD{Float64,2,1,OffsetArrays.OffsetVector{Float64,Vector{Float64}}}}, ::String))
 (0.6890788078308105, MethodInstance for assess_ssim(::Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}}, ::Matrix{ColorTypes.Gray{FixedPointNumbers.N0f8}}))

I usually cherry-pick the juicy targets and put things into their corresponding packages. In this case, your most expensive MethodInstance to infer is the assess_ssim call at nearly 0.7s. Also note that the types are not too complicated, we can imagine this being sufficiently broadly useful that it's worth precompiling. So then I add this to your package definition:

function _precompile_()
    ccall(:jl_generating_output, Cint, ()) == 1 || return nothing
    eltypes = (N0f8, N0f16, Float32, Float64)        # eltypes of parametric colors
    pctypes = (Gray, RGB)                            # parametric colors
    cctypes = (Gray24, RGB24)                        # non-parametric colors
    dims  = (1, 2, 3)

    for n in dims
        for C in pctypes, T in eltypes
            precompile(assess_ssim, (Array{C{T},n}, Array{C{T},n}))
        end
        for C in cctypes
            precompile(assess_ssim, (Array{C,n}, Array{C,n}))
        end
    end
end
VERSION >= v"1.4.2" && _precompile_()

Then I quit Julia and run the above again. When I do so, sadness strikes: the assess_ssim call is still there, which means precompilation didn't "take."

So now it's time to investigate why. I already have a strong guess (I bet we'll have to ensure that imfilter precompiles), but to see whether additional efforts are likely to have any success let's first check out whether we have rampant invalidation; if we do, nothing will help and all our efforts will be a waste. So start a new session (must be Julia's master branch) and do this:

julia> using SnoopCompileCore

julia> invs = @snoopr using ImageQualityIndexes;

julia> using SnoopCompile

julia> length(uinvalidated(invs))
288

That's around where we are with ImageCore. If you do

julia> trees = invalidation_trees(invs)
14-element Vector{SnoopCompile.MethodInvalidations}:
...

you'll see that it's our deprecated convert methods in ImageCore causing the vast majority of these. So then I dev ImageCore, comment out the include("deprecations.jl"), and try again:

julia> using SnoopCompileCore

julia> invs = @snoopr using ImageQualityIndexes;
[ Info: Precompiling ImageQualityIndexes [2996bd0c-7a13-11e9-2da2-2f5ce47296a9]

julia> using SnoopCompile

julia> length(uinvalidated(invs))
102

Much better. (The trees don't show any particular problem with this package, as the triggers are from its dependencies, at least when run in a fresh session.)

Sadly, when I try again, it still doesn't "take." That could be because of one or more of these 102 remaining invalidations, but at this point I suspect another source. So then I go take a look at the implementation and see that you need imfilter for assess_ssim (which I could have suspected from the @snoopi trace, of course). But imfilter is obviously not precompiled, so it might be worth taking a step back and go visit ImageFiltering and see if I can get that working. That would definitely be worth doing, but I don't have time at the moment to tackle ImageFiltering to see if it works (sorry).

Just FYI, some other useful tricks include stuff like this:

            a = zeros(C{T}, ntuple(i->1, n))
            am = of_eltype(float(T), a)
            if isdefined(Base, :bodyfunction)
                m = which(_ssim_statistics, (GenericImage, GenericImage, Any))
                f = Base.bodyfunction(m)
                precompile(f, (Bool, typeof(_ssim_statistics), typeof(am), typeof(am)))
...

which let you circumvent some of the complexity of types and just construct instances which you then use to build good precompile directives. In general I find this approach makes my precompile files nicely track changes in implementations. I can't think of a good way to automate this, which is why I generally prefer to do stuff by hand.

[ashwanirathee]

does this get completed with #55?