move out Test from the sysimage

[KristofferC]

There doesn't seem to be any noticeable latency from this. Using a test file:

using Test
@testset "test" begin
    @test 1==1
end

With this PR I get:

❯ time ./julia --startup-file=no time_test.jl  
Test Summary: | Pass  Total  Time
test          |    1      1  0.0s
./julia --startup-file=no time_test.jl  0.42s user 0.77s system 447% cpu 0.266 total

and on 1.9:

❯ time julia --startup-file=no time_test.jl 
Test Summary: | Pass  Total  Time
test          |    1      1  0.0s
julia --startup-file=no time_test.jl  0.40s user 0.75s system 511% cpu 0.225 total

[KristofferC]

This prints to stderr which is not so pretty

[IanButterworth]

Can stderr be redirected or would that ruin the precompilation effectiveness?

[IanButterworth]

Also for some reason this is getting printed again again in the tests when it should only happen during pkgimage precompilation once

[KristofferC]

They all print from the same worker so maybe there is some test running a bunch of processes with --compiled-modules=no that loads Test?

[KristofferC]

It is also printed twice when compiling Julia:

    JULIA stdlib/DelimitedFiles.release.image
    JULIA stdlib/LibSSH2_jll.release.image
    JULIA stdlib/SharedArrays.release.image
    JULIA stdlib/Profile.release.image
Test Summary: | Pass  Total  Time
example       |    1      1  0.1s
    JULIA stdlib/Test.release.image
    JULIA stdlib/SparseArrays.release.image
Test Summary: | Pass  Total  Time
example       |    1      1  0.1s

Seems it should only be once (when it is compiled)?

[KristofferC]

Oh yeah, one with bounds and one without. So two total

[KristofferC]

It comes from:

julia/test/loading.jl

Lines 1089 to 1105 in bd1a664

	for (P, D, C, I, O) in Iterators.product(0:1, 0:2, 0:2, 0:1, 0:3)
	julia = joinpath(Sys.BINDIR, Base.julia_exename())
	script = """
	using Test
	let
	cf = Base.CacheFlags()
	opts = Base.JLOptions()
	@test cf.use_pkgimages == opts.use_pkgimages == $P
	@test cf.debug_level == opts.debug_level == $D
	@test cf.check_bounds == opts.check_bounds == $C
	@test cf.inline == opts.can_inline == $I
	@test cf.opt_level == opts.opt_level == $O
	end
	"""
	cmd = `$julia $(pkgimage(P)) $(opt_level(O)) $(debug_level(D)) $(check_bounds(C)) $(inline(I)) -e $script`
	@test success(pipeline(cmd; stdout, stderr))
	end

[IanButterworth]

Perhaps all these precompiles should be guarded to happen during precompilation only like SnoopCompile ones are?

[KristofferC]

I removed the usage of Test completely from this test.

[KristofferC]

I wonder how the precompile tests that fail are relying on that Test is in the sysimage.