Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Invalidation regressions on release-1.6 #39030

Closed
timholy opened this issue Dec 29, 2020 · 4 comments
Closed

Invalidation regressions on release-1.6 #39030

timholy opened this issue Dec 29, 2020 · 4 comments
Labels
compiler:latency Compiler latency regression Regression in behavior compared to a previous version

Comments

@timholy
Copy link
Member

timholy commented Dec 29, 2020 •
edited
Loading

This makes me sad:

julia> using SnoopCompileCore

julia> invs = @snoopr using ColorTypes;

julia> using SnoopCompile

julia> length(uinvalidated(invs))
396

That's quite a large number. But it gets worse when you see what gets invalidated:

julia> trees = invalidation_trees(invs)
2-element Vector{SnoopCompile.MethodInvalidations}:
 inserting (::Type{var"#s2"} where var"#s2"<:FixedPointNumbers.FixedPoint)(x::AbstractChar) in FixedPointNumbers at /home/tim/.julia/packages/FixedPointNumbers/HAGk2/src/FixedPointNumbers.jl:60 invalidated:
   backedges: 1: superseding (::Type{T})(x::AbstractChar) where T<:Union{AbstractChar, Number} in Base at char.jl:50 with MethodInstance for (::Type{K} where K<:Char)(::Char) (2 children)
   8 mt_cache

 inserting convert(::Type{C}, c::Colorant) where C<:Colorant in ColorTypes at /home/tim/.julia/packages/ColorTypes/RF8lb/src/conversions.jl:73 invalidated:
   backedges: 1: superseding convert(::Type{Union{}}, x) in Base at essentials.jl:203 with MethodInstance for convert(::Type{Union{}}, ::Any) (622 children)


julia> tree = trees[end]
inserting convert(::Type{C}, c::Colorant) where C<:Colorant in ColorTypes at /home/tim/.julia/packages/ColorTypes/RF8lb/src/conversions.jl:73 invalidated:
   backedges: 1: superseding convert(::Type{Union{}}, x) in Base at essentials.jl:203 with MethodInstance for convert(::Type{Union{}}, ::Any) (622 children)


julia> root = tree.backedges[1]
MethodInstance for convert(::Type{Union{}}, ::Any) at depth 0 with 622 children

julia> ascend(root)
Choose a call for analysis (q to quit):
 >   convert(::Type{Union{}}, ::Any)
       convert(::Type{Nothing}, ::Any)
         setindex!(::IdDict{Dict{String, Any}, Nothing}, ::Any, ::Any)
           push!(::Base.IdSet{Dict{String, Any}}, ::Dict{String, Any})
             parse_array_table(::Base.TOML.Parser)
               parse_table(::Base.TOML.Parser)
                 parse_toplevel(::Base.TOML.Parser)
                   tryparse(::Base.TOML.Parser)
                     parse(::Base.TOML.Parser)
...

This immediately hits parse(::Base.TOML.Parser), and thus we're back to invalidating the code that Julia uses to load other packages. Hence using is slower than it was earlier in the 1.6 dev cycle. I thought I'd felt some of that just in ordinary usage, but not until I looked were my vague suspicions confirmed.

That ColorTypes signature, convert(::Type{C}, c::Colorant) where C<:Colorant, is remiscent of the kind of work @vtjnash did on ambiguities and invalidation in #36863 and #36733, and the signatures seem ambiguous to me (Union{} is more specialized than C<:Colorant, whereas Any is less specialized than Colorant.) Does this need a fresh look?
@timholy timholy added regression Regression in behavior compared to a previous version compiler:latency Compiler latency labels Dec 29, 2020
@timholy
Copy link
Member Author

timholy commented Dec 29, 2020

StaticArrays also invalidates the corresponding Union{} constructor:

julia> using SnoopCompileCore

julia> invs = @snoopr using StaticArrays;

julia> using SnoopCompile

julia> length(uinvalidated(invs))
425

julia> trees = invalidation_trees(invs)
6-element Vector{SnoopCompile.MethodInvalidations}:
 inserting getproperty(::SOneTo{n}, s::Symbol) where n in StaticArrays at /home/tim/.julia/packages/StaticArrays/LJQEe/src/SOneTo.jl:57 invalidated:
   backedges: 1: superseding getproperty(x, f::Symbol) in Base at Base.jl:33 with MethodInstance for getproperty(::AbstractUnitRange, ::Symbol) (1 children)
   2 mt_cache

 inserting (::Type{SA})(a::AbstractArray) where SA<:StaticArray in StaticArrays at /home/tim/.julia/packages/StaticArrays/LJQEe/src/convert.jl:7 invalidated:
   backedges: 1: superseding Union{}(a...) in Core at boot.jl:247 with MethodInstance for Union{}(::Vector{var"#s120"} where var"#s120"<:Union{Float32, Float64}) (2 children)

 inserting similar(A::AbstractArray, ::Type{T}, shape::Tuple{Union{Integer, Base.OneTo, SOneTo}, Vararg{Union{Integer, Base.OneTo, SOneTo}, N} where N}) where T in StaticArrays at /home/tim/.julia/packages/StaticArrays/LJQEe/src/abstractarray.jl:152 invalidated:
   mt_backedges: 1: signature Tuple{typeof(similar), Vector{T} where T, Type, Tuple{Union{Integer, AbstractUnitRange}}} triggered MethodInstance for similar(::Vector{T} where T, ::Type{T}, ::Union{Integer, AbstractUnitRange}) where T (0 children)
                 2: signature Tuple{typeof(similar), Vector{T} where T, DataType, Tuple{Union{Integer, AbstractUnitRange}}} triggered MethodInstance for similar(::Vector{T} where T, ::DataType, ::Union{Integer, AbstractUnitRange}) (0 children)
                 3: signature Tuple{typeof(similar), Vector{T} where T, Type, Tuple{Union{Integer, AbstractUnitRange}}} triggered MethodInstance for similar(::Vector{T} where T, ::Type{T}, ::Tuple{Union{Integer, Base.OneTo}}) where T (2 children)
                 4: signature Tuple{typeof(similar), Vector{T} where T, DataType, Tuple{Union{Integer, AbstractUnitRange}}} triggered MethodInstance for similar(::Vector{T} where T, ::DataType, ::Tuple{Union{Integer, Base.OneTo}}) (2 children)

 inserting similar(::Type{A}, shape::Tuple{Union{Integer, Base.OneTo, SOneTo}, Vararg{Union{Integer, Base.OneTo, SOneTo}, N} where N}) where A<:AbstractArray in StaticArrays at /home/tim/.julia/packages/StaticArrays/LJQEe/src/abstractarray.jl:153 invalidated:
   mt_backedges: 1: signature Tuple{typeof(similar), Type{Array{Any, _A}} where _A, Tuple{Union{Integer, AbstractUnitRange}}} triggered MethodInstance for similar(::Type{Array{Any, _A}}, ::Union{Integer, AbstractUnitRange}) where _A (0 children)
                 2: signature Tuple{typeof(similar), Type{Array{_A, _B}} where _B where _A, Tuple{Union{Integer, AbstractUnitRange}}} triggered MethodInstance for similar(::Type{Array{_A, _B}}, ::Union{Integer, AbstractUnitRange}) where {_A, _B} (0 children)
                 3: signature Tuple{typeof(similar), Type{Array{String, _A}} where _A, Tuple{Union{Integer, AbstractUnitRange}}} triggered MethodInstance for similar(::Type{Array{String, _A}}, ::Union{Integer, AbstractUnitRange}) where _A (0 children)
                 4: signature Tuple{typeof(similar), Type{Array{Expr, _A}} where _A, Tuple{Union{Integer, AbstractUnitRange}}} triggered MethodInstance for similar(::Type{Array{Expr, _A}}, ::Union{Integer, AbstractUnitRange}) where _A (0 children)
                 5: signature Tuple{typeof(similar), Type{Array{Any, _A}} where _A, Tuple{Union{Integer, AbstractUnitRange}}} triggered MethodInstance for similar(::Type{Array{Any, _A}}, ::Tuple{Union{Integer, Base.OneTo}}) where _A (2 children)
                 6: signature Tuple{typeof(similar), Type{Array{_A, _B}} where _B where _A, Tuple{Union{Integer, AbstractUnitRange}}} triggered MethodInstance for similar(::Type{Array{_A, _B}}, ::Tuple{Union{Integer, Base.OneTo}}) where {_A, _B} (2 children)
                 7: signature Tuple{typeof(similar), Type{Array{String, _A}} where _A, Tuple{Union{Integer, AbstractUnitRange}}} triggered MethodInstance for similar(::Type{Array{String, _A}}, ::Tuple{Union{Integer, Base.OneTo}}) where _A (2 children)
                 8: signature Tuple{typeof(similar), Type{Array{Expr, _A}} where _A, Tuple{Union{Integer, AbstractUnitRange}}} triggered MethodInstance for similar(::Type{Array{Expr, _A}}, ::Tuple{Union{Integer, Base.OneTo}}) where _A (2 children)
   4 mt_cache

 inserting isassigned(a::StaticArray, i::Int64...) in StaticArrays at /home/tim/.julia/packages/StaticArrays/LJQEe/src/abstractarray.jl:28 invalidated:
   backedges: 1: superseding isassigned(a::AbstractArray, i::Integer...) in Base at abstractarray.jl:505 with MethodInstance for isassigned(::AbstractVecOrMat{T} where T, ::Int64, ::Int64) (20 children)

 inserting (::Type{SA})(a::StaticArray) where SA<:SizedArray in StaticArrays at /home/tim/.julia/packages/StaticArrays/LJQEe/src/convert.jl:6 invalidated:
   backedges: 1: superseding Union{}(a...) in Core at boot.jl:247 with MethodInstance for Union{}(::AbstractArray) (623 children)

@timholy
Copy link
Member Author

timholy commented Dec 30, 2020

Can anyone else replicate this? I've now tried this on other machines besides my laptop and I am not getting the same result.

@timholy
Copy link
Member Author

timholy commented Dec 30, 2020

I have no clue what happened (git assured me I had no local modifications), but a make cleanall && rm -rf usr/ && make seems to have resolved this. Sorry for the noise.

@timholy timholy closed this as completed Dec 30, 2020
@serenity4
Copy link
Contributor

serenity4 commented May 27, 2022 •
edited
Loading

I can personally reproduce this with CEnum.jl on master:

julia> using SnoopCompileCore

julia> invs = @snoopr using CEnum;

julia> using SnoopCompile

julia> length(uinvalidated(invs))
1830

julia> trees = invalidation_trees(invs)
1-element Vector{SnoopCompile.MethodInvalidations}:
 inserting convert(::Type{T1}, x::CEnum.Cenum{T2}) where {T1<:Integer, T2<:Integer} in CEnum at /home/belmant/.julia/dev/CEnum.jl/src/operators.jl:25 invalidated:
   backedges: 1: superseding convert(::Type{Union{}}, x) in Base at essentials.jl:280 with MethodInstance for convert(::Core.TypeofBottom, ::Any) (2612 children)

julia> versioninfo()
Julia Version 1.9.0-DEV.667
Commit 762561c3057 (2022-05-27 04:05 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
  Threads: 1 on 8 virtual cores

EDIT: I see this issue has already been re-reported elsewhere, sorry for the noise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
compiler:latency Compiler latency regression Regression in behavior compared to a previous version
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@timholy @serenity4