Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix #46778, precompile() for abstract but compileable signatures #47259

Merged
merged 1 commit into from
Nov 16, 2022

Conversation

JeffBezanson
Copy link
Member

fix #46778

@JeffBezanson JeffBezanson added compiler:precompilation Precompilation of modules bugfix This change fixes an existing bug compiler:latency Compiler latency labels Oct 20, 2022
src/gf.c Outdated Show resolved Hide resolved
@KristofferC
Copy link
Member

Bump! Good to go?

@KristofferC
Copy link
Member

KristofferC commented Nov 9, 2022

Something doesn't seem right with the caching of these signatures in a sysimage.

As an example, I add a specific precompile statement for the problematic method in #46778:

diff --git a/contrib/generate_precompile.jl b/contrib/generate_precompile.jl
index 295b24d22e..abc887d94d 100644
--- a/contrib/generate_precompile.jl
+++ b/contrib/generate_precompile.jl
@@ -186,6 +186,18 @@ if Libdl !== nothing
     """
 end
 
+Serialization = get(Base.loaded_modules,
+          Base.PkgId(Base.UUID("9e88b42a-f829-5b0c-bbe9-9e923198166b"), "Serialization"),
+          nothing)
+if Serialization !== nothing
+    @info "Precompiling deserialization..."
+    hardcoded_precompile_statements *= """
+        precompile(Tuple{typeof(Serialization.deserialize), Serialization.Serializer{Base.GenericIOBuffer{Array{UInt8, 1}}}, DataType})
+
+        """
+end
+
+
 Test = get(Base.loaded_modules,
           Base.PkgId(Base.UUID("8dfed614-e22c-5e08-85e1-65c5234f0b40"), "Test"),
           nothing)

I build a Julia with this, and then I create a serialized file:

using Serialization
struct MyStruct
    x::String
end
s = MyStruct("foo")
serialize("file.jls", s)

Now, I run Julia with --trace-compile=stderr:

julia> using Serialization

julia> struct MyStruct
           x::String
       end

julia> @time deserialize(IOBuffer(read("file.jls")));
precompile(Tuple{typeof(Serialization.deserialize), Serialization.Serializer{Base.GenericIOBuffer{Array{UInt8, 1}}}, DataType})
  0.022934 seconds (11.89 k allocations: 800.115 KiB, 98.08% compilation time)

julia> @time deserialize(IOBuffer(read("file.jls")));
  0.000084 seconds (28 allocations: 2.008 KiB)

The compile time is not removed and the signature printed is exactly the one that was supposed to be cached in the generate_precompile.jl file.

@JeffBezanson
Copy link
Member Author

Oh, it looks like generate_precompile has an explicit check for this:

            ms = length(ps) == 1 ? Base._methods_by_ftype(ps[1], 1, Base.get_world_counter()) : Base.methods(ps...)
            ms isa Vector || continue

@KristofferC
Copy link
Member

Oh...

@JeffBezanson JeffBezanson added the backport 1.9 Change should be backported to release-1.9 label Nov 16, 2022
@JeffBezanson JeffBezanson merged commit fe81138 into master Nov 16, 2022
@JeffBezanson JeffBezanson deleted the jb/fix46778 branch November 16, 2022 00:01
KristofferC pushed a commit that referenced this pull request Nov 17, 2022
KristofferC pushed a commit that referenced this pull request Nov 17, 2022
@KristofferC KristofferC mentioned this pull request Dec 14, 2022
26 tasks
@KristofferC KristofferC removed the backport 1.9 Change should be backported to release-1.9 label Dec 27, 2022
JeffBezanson added a commit that referenced this pull request Mar 30, 2023
Since #47259 most Vararg precompile statements actually work, so they should be
allowed, and for ones that don't work there's no harm in trying them.
KristofferC added a commit to JuliaLang/PackageCompiler.jl that referenced this pull request Apr 23, 2023
KristofferC added a commit to JuliaLang/PackageCompiler.jl that referenced this pull request Apr 23, 2023
KristofferC added a commit to JuliaLang/PackageCompiler.jl that referenced this pull request Apr 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix This change fixes an existing bug compiler:latency Compiler latency compiler:precompilation Precompilation of modules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

precompile fails to cache code for calls that are not fully specialized (non-concrete signature)
3 participants