Strange "missing from cache"/"does not support precompilation"/... errors

[aplavin]

Reproducing in a clean env & depot, latest Julia (1.9.3; upd same on 1.9.4 and 1.10):

pkg> add AccessorsExtra
julia> using AccessorsExtra

┌ LinearAlgebraExt [2e571e4a-e68d-5fe4-9022-ff2e7d6ac6d5]
│  ┌ Warning: Module LinearAlgebraExt with build ID ffffffff-ffff-ffff-0001-6be4cd7b7f3f is missing from the cache.
│  │ This may mean LinearAlgebraExt [2e571e4a-e68d-5fe4-9022-ff2e7d6ac6d5] does not support precompilation but is imported by a module that does.
│  └ @ Base loading.jl:1793
│  ┌ Error: Error during loading of extension LinearAlgebraExt of AccessorsExtra, use `Base.retry_load_extensions()` to retry.
│  │   exception =
│  │    1-element ExceptionStack:
│  │    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│  │    Stacktrace: <...>
└  
┌ TestExt [69cf5fdc-2bf6-5cf1-837e-a22f5027a90f]
│  ┌ Warning: Module TestExt with build ID ffffffff-ffff-ffff-0001-6be4cd7ce9e0 is missing from the cache.
│  │ This may mean TestExt [69cf5fdc-2bf6-5cf1-837e-a22f5027a90f] does not support precompilation but is imported by a module that does.
│  └ @ Base loading.jl:1793
│  ┌ Error: Error during loading of extension TestExt of AccessorsExtra, use `Base.retry_load_extensions()` to retry.
│  │   exception =
│  │    1-element ExceptionStack:
│  │    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│  │    Stacktrace: <...>
└  
[ Info: Precompiling LinearAlgebraExt [2e571e4a-e68d-5fe4-9022-ff2e7d6ac6d5]
┌ Warning: Module LinearAlgebraExt with build ID ffffffff-ffff-ffff-0001-6be52894ee31 is missing from the cache.
│ This may mean LinearAlgebraExt [2e571e4a-e68d-5fe4-9022-ff2e7d6ac6d5] does not support precompilation but is imported by a module that does.
└ @ Base loading.jl:1793
┌ Error: Error during loading of extension LinearAlgebraExt of AccessorsExtra, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│    Stacktrace: <...>

Still, it seems like code from these two extensions is actually loaded and works.
I don't have __precompile__(false) anywhere in that package, all modules should support precompilation. What these errors and warnings actually mean, and are they bugs in Julia or in package code?

[IanButterworth]

Does this reproduce on master?

[aplavin]

Not sure about master, but I see the same on 1.10rc.

[IanButterworth]

1.9.4 or master would be helpful

[aplavin]

1.9.4 same

[aplavin]

Any ideas why these messages would appear?..
And what are they – infos, warnings, or errors? :) All three labels are printed in sequence.

[aplavin]

From a slack discussion (@lassepe), these warnings/errors seem to occur when Foo is a weakdep of MyPkg, but is a strong dep of another dependency of MyPkg. If that's the case, this does seem a bug in Julia not in packages.

[aplavin]

Any updates on this?.. Everything works, but big "errors"/warnings look scary to users (:

[IanButterworth]

Do you see this on 1.10.0 & master? If so please share the output on those

[KristofferC]

(I can repro this on 1.10)

I wonder if this could be related to all the method overwrites in AccessorsExtra:

┌ AccessorsExtra → LinearAlgebraExt
│  WARNING: Method definition (::Type{Accessors.IndexLens{I} where I<:Tuple})(Tuple{Accessors.Elements}) in module Accessors at C:\Users\Kristoffer\.julia\packages\Accessors\ss6Uj\src\sugar.jl:463 overwritten in module AccessorsExtra at C:\Users\Kristoffer\.julia\packages\AccessorsExtra\h0y2R\src\AccessorsExtra.jl:87.
│  ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
└

which causes loading the AccessorsExtra package to fail at precompilation and as a fallback it loads without a precomple cache (see #52214) which then reloads the extensions etc.

AcessorsExtra has stuff like:

# some piracy:
Accessors.IndexLens(::Tuple{Elements}) = Elements()
Accessors.IndexLens(::Tuple{Properties}) = Properties()
Accessors._shortstring(prev, o::Elements) = "$prev[∗]"

...

# some piracy - should upstream:
getall(obj::AbstractDict, ::Elements) = collect(obj)
getall(obj::AbstractSet, ::Elements) = collect(obj)

Edit: I removed what I think are all method overwrites and it does not seem to fix it what I can see.

[KristofferC]

This repros with only Accessors for me (also, does not require Pkg.precompile to repro):

julia> ENV["JULIA_PKG_PRECOMPILE_AUTO"] = 0
0

julia> Base.PKG_PRECOMPILE_HOOK[] = Returns(nothing)
Returns{Nothing}(nothing)

(jl_4ZbHIu) pkg> add Accessors
...

julia> using Accessors
[ Info: Precompiling Accessors [7d9f7c33-5ae7-4f3b-8dc6-eff91059b697]
[ Info: Precompiling AccessorsDatesExt [308068fd-b978-57fd-b78f-2a52fb63dba4]
┌ Warning: Module AccessorsDatesExt with build ID ffffffff-ffff-ffff-0001-6544d38c5b81 is missing from the cache.
│ This may mean AccessorsDatesExt [308068fd-b978-57fd-b78f-2a52fb63dba4] does not support precompilation but is imported by a module that does.
└ @ Base loading.jl:1942
┌ Error: Error during loading of extension AccessorsDatesExt of Accessors, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│    Stacktrace:
│     ...
│        @ Base .\client.jl:552
└ @ Base loading.jl:1295
┌ Warning: Module AccessorsDatesExt with build ID ffffffff-ffff-ffff-0001-6544d38c5b81 is missing from the cache.
│ This may mean AccessorsDatesExt [308068fd-b978-57fd-b78f-2a52fb63dba4] does not support precompilation but is imported by a module that does.
└ @ Base loading.jl:1942
┌ Error: Error during loading of extension AccessorsDatesExt of Accessors, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│    Stacktrace:
│     ...
│        @ Base .\client.jl:552
└ @ Base loading.jl:1295
┌ Warning: Module AccessorsLinearAlgebraExt with build ID ffffffff-ffff-ffff-0001-6544f41e6c6d is missing from the cache.
│ This may mean AccessorsLinearAlgebraExt [6d540e18-2ad7-59a5-acf8-1761f4011e1b] does not support precompilation but is imported by a module that does.
└ @ Base loading.jl:1942
┌ Error: Error during loading of extension AccessorsLinearAlgebraExt of Accessors, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│    Stacktrace:
│     ...
│        @ Base .\client.jl:552
└ @ Base loading.jl:1295

[IanButterworth]

I wonder if making multiple extensions that each depend on a different packages in the sysimage is racy

[dalum]

I just updated to 1.10.0 (from 1.8.5) and have started seeing this, and it seems to be causing sysimage builds with PackageCompiler to fail in our pipeline with the error:

ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
ERROR: LoadError: Declaring __precompile__(false) is not allowed in files that are being precompiled.

Trying to see if I can make an MWE.

[aplavin]

Do you see this on 1.10.0 & master? If so please share the output on those

Yes, same output on 1.10.

I wonder if this could be related to all the method overwrites in AccessorsExtra:

Nope, not related.
Btw method overrides there are a historical remnant, these methods were just upstreamed to Accessors itself. They weren't method overrides when I opened this issue.

This repros with only Accessors for me

Recently we moved to "make everything a weakdep when possible" in Accessors, making this issue manifest for that package alone as well.

[jw3126]

This repros with only Accessors for me (also, does not require Pkg.precompile to repro):

FYI we made things stong deps again in Accessors, to circumvent this issue. People need Accessors.jl v0.1.34 to reproduce.

[KristofferC]

ERROR: Method overwriting is not permitted during Module precompilation. Use __precompile__(false) to opt-out of precompilation.

I don't think this is related to the issue here.

[KristofferC]

Just a bit of an update here. We (me and @IanButterworth) think this issue has to do with adding extensions to packages that are in the sysimage (most stdlibs on 1.10 and e.g. LinearAlgebra on 1.11).

So removing extensions for the stdlibs might enough (and worth trying). cc @jw3126

[KristofferC]

Here is a somewhat untested patch that tries to avoid the Base.loaded_modules for checking if a dependency has been loaded and instead rely on require calls (thereby trying to avoid the issue of what is and isn't in the sysimage):

diff --git a/base/Base.jl b/base/Base.jl
index 0ca13265ad..98803cf29f 100644
--- a/base/Base.jl
+++ b/base/Base.jl
@@ -627,6 +627,7 @@ function __init__()
     init_load_path()
     init_active_project()
     append!(empty!(_sysimage_modules), keys(loaded_modules))
+    empty!(required_modules)
     if haskey(ENV, "JULIA_MAX_NUM_PRECOMPILE_FILES")
         MAX_NUM_PRECOMPILE_FILES[] = parse(Int, ENV["JULIA_MAX_NUM_PRECOMPILE_FILES"])
     end
diff --git a/base/loading.jl b/base/loading.jl
index 97aa9977a7..a87df18902 100644
--- a/base/loading.jl
+++ b/base/loading.jl
@@ -1343,7 +1343,7 @@ function _insert_extension_triggers(parent::PkgId, extensions::Dict{String, Any}
             # TODO: Better error message if this lookup fails?
             uuid_trigger = UUID(weakdeps[trigger]::String)
             trigger_id = PkgId(uuid_trigger, trigger)
-            if !haskey(Base.loaded_modules, trigger_id) || haskey(package_locks, trigger_id)
+            if !(trigger_id in required_modules) || haskey(package_locks, trigger_id)
                 trigger1 = get!(Vector{ExtensionId}, EXT_DORMITORY, trigger_id)
                 push!(trigger1, gid)
             else
@@ -1965,8 +1965,11 @@ function _require_prelocked(uuidkey::PkgId, env=nothing)
     end
 end
 
+const required_modules = Set{PkgId}()
 function __require_prelocked(uuidkey::PkgId, env=nothing)
     assert_havelock(require_lock)
+    first_require = !(uuidkey in required_modules)
+    push!(required_modules, uuidkey)
     if !root_module_exists(uuidkey)
         newm = _require(uuidkey, env)
         if newm === nothing
@@ -1977,6 +1980,7 @@ function __require_prelocked(uuidkey::PkgId, env=nothing)
         # After successfully loading, notify downstream consumers
         run_package_callbacks(uuidkey)
     else
+        first_require && run_package_callbacks(uuidkey)
         warn_if_already_loaded_different(uuidkey)
         newm = root_module(uuidkey)
     end

With this I can precompile Accessors.jl with no problems, but I literally haven't tried anything else than that.

Edit: I don't think the above is completely correct.

[aplavin]

So removing extensions for the stdlibs might enough (and worth trying)

That's what we did in Accessors for now.
But note that packages do make extensions for stdlibs when it makes sense, eg https://github.com/JuliaLang/Compat.jl/blob/master/Project.toml. I guess it works fine for them?

[KristofferC]

They only have one extension for an stdlib. If they added another one, they might encounter the same problem.

The issue AFAIU is that:

• Accessors is loaded
• LinearAlgebraExt starts to precompile (because LinearAlgebra is in the sysimage)
• This loads Accessors, which loads Test (because of Accessors → InverseFunctions → Test)
• TestExt starts to precompile
• This loads Accessors which loads LinearAlgebraExt(!!!) again because LinearAlgebra is in the sysimage.
• This is a cycle.

If you only have one extension there is no way to "loop back" to the first extension as in the example above.

[atbug]

this issue has to do with adding extensions to packages that are in the sysimage

Maybe not? I see such errors with this package:

[deps]
ComputedFieldTypes = "459fdd68-db75-56b8-8c15-d717a790f88e"
Distributed = "8ba89e20-285c-5b6f-9357-94700520ee1b"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
ProgressMeter = "92933f4c-e287-5a05-a399-4b506db050ca"
SparseArrays = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
StaticArrays = "90137ffa-7385-5640-81b9-e52037218182"
UniformGrids = "3d03f9f6-d0fb-45b8-9ee8-5e9698a0a906"

[weakdeps]
Cells = "c368a612-970d-47d0-a74e-c409b3f7b97e"
SiestaInterface = "820187e7-b133-44e4-ad7a-78036505f177"

[extensions]
OrbitalLatticesSiestaExt = ["Cells", "SiestaInterface"]
OrbitalsSiestaExt = ["SiestaInterface"]

[extras]
Cells = "c368a612-970d-47d0-a74e-c409b3f7b97e"
SiestaInterface = "820187e7-b133-44e4-ad7a-78036505f177"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[targets]
test = ["Test", "Cells", "SiestaInterface"]

Many of the packages above are currently private, unfortunately. Let me know if more information is needed.

Edit: I am not sure how to reproduce the error. Will edit next time I see the error.

Edit: Stack trace:

Precompiling project...
  2 dependencies successfully precompiled in 10 seconds. 90 already precompiled.
  1 dependency had output during precompilation:
┌ TightBinding → OrbitalsSiestaExt
│  ┌ Warning: Module OrbitalsSiestaExt with build ID ffffffff-ffff-ffff-0000-3f0623d76494 is missing from the cache.
│  │ This may mean OrbitalsSiestaExt [6d573fe1-6245-5e66-b3c2-3070708ad8ae] does not support precompilation but is imported by a module that does.
│  └ @ Base loading.jl:1942
│  ┌ Error: Error during loading of extension OrbitalsSiestaExt of TightBinding, use `Base.retry_load_extensions()` to retry.
│  │   exception =
│  │    1-element ExceptionStack:
│  │    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│  │    Stacktrace:
│  │      [1] _require(pkg::Base.PkgId, env::Nothing)
│  │        @ Base ./loading.jl:1946
│  │      [2] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│  │        @ Base ./loading.jl:1806
│  │      [3] #invoke_in_world#3
│  │        @ Base ./essentials.jl:921 [inlined]
│  │      [4] invoke_in_world
│  │        @ Base ./essentials.jl:918 [inlined]
│  │      [5] _require_prelocked
│  │        @ Base ./loading.jl:1797 [inlined]
│  │      [6] _require_prelocked
│  │        @ Base ./loading.jl:1796 [inlined]
│  │      [7] run_extension_callbacks(extid::Base.ExtensionId)
│  │        @ Base ./loading.jl:1289
│  │      [8] run_extension_callbacks(pkgid::Base.PkgId)
│  │        @ Base ./loading.jl:1324
│  │      [9] run_package_callbacks(modkey::Base.PkgId)
│  │        @ Base ./loading.jl:1158
│  │     [10] __require_prelocked(uuidkey::Base.PkgId, env::String)
│  │        @ Base ./loading.jl:1813
│  │     [11] #invoke_in_world#3
│  │        @ Base ./essentials.jl:921 [inlined]
│  │     [12] invoke_in_world
│  │        @ Base ./essentials.jl:918 [inlined]
│  │     [13] _require_prelocked(uuidkey::Base.PkgId, env::String)
│  │        @ Base ./loading.jl:1797
│  │     [14] macro expansion
│  │        @ Base ./loading.jl:1784 [inlined]
│  │     [15] macro expansion
│  │        @ Base ./lock.jl:267 [inlined]
│  │     [16] __require(into::Module, mod::Symbol)
│  │        @ Base ./loading.jl:1747
│  │     [17] #invoke_in_world#3
│  │        @ Base ./essentials.jl:921 [inlined]
│  │     [18] invoke_in_world
│  │        @ Base ./essentials.jl:918 [inlined]
│  │     [19] require(into::Module, mod::Symbol)
│  │        @ Base ./loading.jl:1740
│  │     [20] include
│  │        @ Base ./Base.jl:495 [inlined]
│  │     [21] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
│  │        @ Base ./loading.jl:2216
│  │     [22] top-level scope
│  │        @ stdin:3
│  │     [23] eval
│  │        @ Core ./boot.jl:385 [inlined]
│  │     [24] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│  │        @ Base ./loading.jl:2070
│  │     [25] include_string
│  │        @ Base ./loading.jl:2080 [inlined]
│  │     [26] exec_options(opts::Base.JLOptions)
│  │        @ Base ./client.jl:316
│  │     [27] _start()
│  │        @ Base ./client.jl:552
│  └ @ Base loading.jl:1295
└  

[aplavin]

This issue is marked as closed, but still reproduces on 1.10.1, at least partially. @KristofferC

Output

   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.10.1 (2024-02-13)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.10) pkg> activate --temp
  Activating new project at `/var/folders/2j/9vtd991d201dbkh9dnx3wvy00000gq/T/jl_Jy6kID`

(jl_Jy6kID) pkg> add AccessorsExtra
   Resolving package versions...
    Updating `/private/var/folders/2j/9vtd991d201dbkh9dnx3wvy00000gq/T/jl_Jy6kID/Project.toml`
  [33016aad] + AccessorsExtra v0.1.68
    Updating `/private/var/folders/2j/9vtd991d201dbkh9dnx3wvy00000gq/T/jl_Jy6kID/Manifest.toml`
  [7d9f7c33] + Accessors v0.1.35
  [33016aad] + AccessorsExtra v0.1.68
  [a33af91c] + CompositionsBase v0.1.2
  [187b0558] + ConstructionBase v1.5.4
  [02685ad9] + DataPipes v0.3.14
  [3587e190] + InverseFunctions v0.1.12
  [1914dd2f] + MacroTools v0.5.13
  [189a3867] + Reexport v1.2.2
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [b77e0a4c] + InteractiveUtils
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [de0858da] + Printf
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [8dfed614] + Test
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v1.1.0+0
  [4536629a] + OpenBLAS_jll v0.3.23+4
  [8e850b90] + libblastrampoline_jll v5.8.0+1
Precompiling project...
  3 dependencies successfully precompiled in 4 seconds. 9 already precompiled.
  1 dependency had output during precompilation:
┌ AccessorsExtra → LinearAlgebraExt
│  ┌ Warning: Module LinearAlgebraExt with build ID ffffffff-ffff-ffff-0003-d7c6509dfd87 is missing from the cache.
│  │ This may mean LinearAlgebraExt [2e571e4a-e68d-5fe4-9022-ff2e7d6ac6d5] does not support precompilation but is imported by a module that does.
│  └ @ Base loading.jl:1948
│  ┌ Error: Error during loading of extension LinearAlgebraExt of AccessorsExtra, use `Base.retry_load_extensions()` to retry.
│  │   exception =
│  │    1-element ExceptionStack:
│  │    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│  │    Stacktrace:
│  │      [1] _require(pkg::Base.PkgId, env::Nothing)
│  │        @ Base ./loading.jl:1952
│  │      [2] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│  │        @ Base ./loading.jl:1812
│  │      [3] #invoke_in_world#3
│  │        @ ./essentials.jl:926 [inlined]
│  │      [4] invoke_in_world
│  │        @ ./essentials.jl:923 [inlined]
│  │      [5] _require_prelocked
│  │        @ ./loading.jl:1803 [inlined]
│  │      [6] _require_prelocked
│  │        @ ./loading.jl:1802 [inlined]
│  │      [7] run_extension_callbacks(extid::Base.ExtensionId)
│  │        @ Base ./loading.jl:1295
│  │      [8] run_extension_callbacks(pkgid::Base.PkgId)
│  │        @ Base ./loading.jl:1330
│  │      [9] run_package_callbacks(modkey::Base.PkgId)
│  │        @ Base ./loading.jl:1164
│  │     [10] __require_prelocked(uuidkey::Base.PkgId, env::String)
│  │        @ Base ./loading.jl:1819
│  │     [11] #invoke_in_world#3
│  │        @ ./essentials.jl:926 [inlined]
│  │     [12] invoke_in_world
│  │        @ ./essentials.jl:923 [inlined]
│  │     [13] _require_prelocked(uuidkey::Base.PkgId, env::String)
│  │        @ Base ./loading.jl:1803
│  │     [14] macro expansion
│  │        @ ./loading.jl:1790 [inlined]
│  │     [15] macro expansion
│  │        @ ./lock.jl:267 [inlined]
│  │     [16] __require(into::Module, mod::Symbol)
│  │        @ Base ./loading.jl:1753
│  │     [17] #invoke_in_world#3
│  │        @ ./essentials.jl:926 [inlined]
│  │     [18] invoke_in_world
│  │        @ ./essentials.jl:923 [inlined]
│  │     [19] require(into::Module, mod::Symbol)
│  │        @ Base ./loading.jl:1746
│  │     [20] include
│  │        @ ./Base.jl:495 [inlined]
│  │     [21] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
│  │        @ Base ./loading.jl:2222
│  │     [22] top-level scope
│  │        @ stdin:3
│  │     [23] eval
│  │        @ ./boot.jl:385 [inlined]
│  │     [24] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│  │        @ Base ./loading.jl:2076
│  │     [25] include_string
│  │        @ ./loading.jl:2086 [inlined]
│  │     [26] exec_options(opts::Base.JLOptions)
│  │        @ Base ./client.jl:316
│  │     [27] _start()
│  │        @ Base ./client.jl:552
│  └ @ Base loading.jl:1301

[vtjnash]

AFAIK, #52841 was only intended to fix some of this issue / mitigate this bug in some cases, but not actually to close it?

[vtjnash]

The relevant issue for other examples similar to this is #52511