Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve inferrability of prune_graph! #3202

Merged
merged 2 commits into from
Sep 21, 2022

Conversation

ranocha
Copy link
Member

@ranocha ranocha commented Sep 16, 2022

I improved type stability to fix some invalidations. This is based on the following code:

julia> import Pkg; Pkg.activate(temp=true); Pkg.add("HDF5")

julia> using SnoopCompileCore; invalidations = @snoopr(using HDF5); using SnoopCompile

julia> length(uinvalidated(invalidations))
168

julia> trees = invalidation_trees(invalidations)
...
 inserting convert(::Type{I}, ::Type{F}) where {I<:Integer, F<:HDF5.Filters.Filter} in HDF5.Filters at ~/.julia/packages/HDF5/T4H0V/src/filters/filters.jl:428 invalidated:
...
   backedges: 1: superseding convert(::Type{Union{}}, x) in Base at essentials.jl:213 with MethodInstance for convert(::Core.TypeofBottom, ::Any) (164 children)
   # quite a lot goes back to prune_graph!

I suggest the labels latency and backport-1.8. With a recent build of Pkg based on the release-1.8 branch and this commit, I get

julia> length(uinvalidated(invalidations))
44

This fixes some invalidations when loading HDF5.jl
Comment on lines +1442 to +1445
new_vdict = Vector{Dict{VersionNumber, Int}}(undef, length(new_pvers))
for new_p0 in eachindex(new_vdict)
new_vdict[new_p0] = Dict(vn => v0 for (v0,vn) in enumerate(new_pvers[new_p0]))
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't

new_vdict = Dict{VersionNumber, Int}[Dict(vn => v0 for (v0,vn) in enumerate(new_pvers[new_p0])) for new_p0 = 1:new_np]

also work?

new_vdict[new_p0] = Dict(vn => v0 for (v0,vn) in enumerate(new_pvers[new_p0]))
end
# The code above is essentially equivalent to
# new_vdict = [Dict(vn => v0 for (v0,vn) in enumerate(new_pvers[new_p0])) for new_p0 = 1:new_np]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this whole chunk can used be written as

# explicitly writing out the loop since the generator equivalent caused type inference failure

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will be AFK in the next few days. Could you please make the changes you suggested? Otherwise, I will revisit this some time in October.

@KristofferC KristofferC added latency backport 1.8 Change should be backported to release-1.8 labels Sep 21, 2022
@KristofferC KristofferC merged commit 0d5230b into JuliaLang:master Sep 21, 2022
KristofferC pushed a commit that referenced this pull request Sep 21, 2022
* improve inferrability of `prune_graph!`

This fixes some invalidations when loading HDF5.jl

* update comment

Co-authored-by: Kristoffer Carlsson <kcarlsson89@gmail.com>
(cherry picked from commit 0d5230b)
KristofferC pushed a commit that referenced this pull request Sep 21, 2022
* improve inferrability of `prune_graph!`

This fixes some invalidations when loading HDF5.jl

* update comment

Co-authored-by: Kristoffer Carlsson <kcarlsson89@gmail.com>
(cherry picked from commit 0d5230b)
@ranocha ranocha deleted the hr/prune_graph_stability_master branch October 2, 2022 13:35
KristofferC added a commit that referenced this pull request Oct 5, 2022
* improve inferrability of `prune_graph!`

This fixes some invalidations when loading HDF5.jl

* update comment

Co-authored-by: Kristoffer Carlsson <kcarlsson89@gmail.com>
@KristofferC KristofferC removed the backport 1.8 Change should be backported to release-1.8 label Dec 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants