Attempting to compile with PackageCompilerX #15

IanButterworth · 2020-01-03T22:17:57Z

I have a package that uses CImGui that I'd like to compile down to a sysimage, so I've been trying out PackageCompilerX, but it's having difficulties with CImGui, so I've given compiling CImGui alone a go.

I have the scripts setup to load the demo at examples/demo.jl, which I manually interact with once loaded, to allow PackageCompilerX to generate the precompile statements. https://github.com/ianshmean/CImGui.jl/tree/attempt_compile/dev/compilation

The process succeeds but with various precompile statement execution failures.

[ Info: PackageCompilerX: creating system image object file, this might take a while...
┌ Error: failed to execute precompile(Tuple{Type{Base.Generator{I, F} where F where I}, getfield(Main, Symbol("#17#18")), Base.UnitRange{Int64}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{getfield(Main, Symbol("#7#10")), Int64})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{getfield(Main, Symbol("#ItemGetter#23")), Ptr{Ptr{Int8}}, Int32, Ptr{Ptr{Int8}}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{getfield(Main, Symbol("#Saw#25")), Ptr{Nothing}, Int32})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{getfield(Main, Symbol("#Sin#24")), Ptr{Nothing}, Int32})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{getfield(Main, Symbol("#Square#14")), Ptr{CImGui.LibCImGui.ImGuiSizeCallbackData}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{getfield(Main, Symbol("#Step#15")), Ptr{CImGui.LibCImGui.ImGuiSizeCallbackData}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Base.collect), Base.Generator{Base.UnitRange{Int64}, getfield(Main, Symbol("#17#18"))}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowAboutWindow), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDemoWindow), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDemoWindowColumns)})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDemoWindowLayout)})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDemoWindowMisc)})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDemoWindowPopups)})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDemoWindowWidgets)})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDummyObject), String, Int64})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppAutoResize), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppConstrainedResize), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppCustomRendering), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppLayout), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppLongText), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppMainMenuBar), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppPropertyEditor), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppSimpleOverlay), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleAppWindowTitles), Base.RefValue{Bool}})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowExampleMenuFile)})
└ @ Main.anonymous none:23
┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowHelpMarker), String})
└ @ Main.anonymous none:23

The sysmimage however does begin to work, but the resulting window has graphical bugs.

i.e.

sh -c "julia-1.3 -J/home/ian/Documents/GitHub/CImGui.jl/dev/compilation/CImGuiSysImage.so -q -e '@show isdefined(Main, :CImGui); include(joinpath(dirname(dirname(pathof(CImGui))), \"examples\", \"demo.jl\"))' -i"

when it should look like

I assume this hasn't been tried before, given how new PackageCompilerX is, but I was wondering if anyone has any ideas? @Gnimuc @KristofferC ?

The text was updated successfully, but these errors were encountered:

IanButterworth · 2020-01-03T22:35:01Z

I guess my key question is whether there's anything inherently limiting about the implementation of CImGui.jl w.r.t. PackageCompilerX's approach?

KristofferC · 2020-01-03T22:41:48Z

It is not very strange that things like

┌ Error: failed to execute precompile(Tuple{typeof(Main.ShowDummyObject), String, Int64})

fail to precompile since those functions only exist in the script and not in the resulting sysimage. In fact, we should probably filter out all precompilation statements that gets defined in Main.

I don't know why the UI doesn't work properly though. Is it possible to get any debug messages out of ClmGui? Perhaps some state is not being set properly.
What happens if you run without any precompile stuff? Same thing?

Gnimuc · 2020-01-04T16:18:29Z

I'm trying to compile this simple app using PackageCompilerX, but it complains about:

julia> create_app("App", "AppCompiled")
┌ Warning: Package ModernGL has a build script, this might indicate that it is not relocatable
└ @ PackageCompilerX ~/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:365
ERROR: Cannot locate artifact 'Xorg_libXau' in '/Users/gnimuc/.julia/packages/Xorg_libXau_jll/VwzXL/Artifacts.toml'
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] #ensure_artifact_installed#41(::Pkg.BinaryPlatforms.MacOS, ::Nothing, ::Bool, ::typeof(Pkg.Artifacts.ensure_artifact_installed), ::String, ::String) at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Artifacts.jl:784
 [3] ensure_artifact_installed(::String, ::String) at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Artifacts.jl:782
 [4] bundle_artifacts(::Pkg.Types.Context, ::String) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:538
 [5] #create_app#6(::Array{String,1}, ::Array{String,1}, ::Bool, ::Bool, ::Bool, ::Bool, ::typeof(create_app), ::String, ::String) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:443
 [6] create_app(::String, ::String) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:420
 [7] top-level scope at REPL[12]:1

How could I tell it to find the artifacts that used by its dependency packages?

Gnimuc · 2020-01-04T16:24:53Z

I have the scripts setup to load the demo at examples/demo.jl, which I manually interact with once loaded, to allow PackageCompilerX to generate the precompile statements.

I don't think it's a good idea to compile examples/demo.jl as a proof-of-concept example. This script is directly translated from the C++ code and it just used hundreds of global variables for state management.

KristofferC · 2020-01-04T16:41:26Z

How could I tell it to find the artifacts that used by its dependency packages?

You shouldn't need to do anything. The error message is a bit strange since the artifact Xorg_libXau should exist in https://github.com/JuliaBinaryWrappers/Xorg_libXau_jll.jl/blob/master/Artifacts.toml...

KristofferC · 2020-01-04T16:46:41Z

Okay, that package doesn't have any artifact for macOS so we shouldn't try bundle it in that case.

KristofferC · 2020-01-04T16:50:39Z

Should be fixed by JuliaLang/PackageCompilerX.jl#69.

Gnimuc · 2020-01-05T02:43:39Z

@KristofferC Thanks for the quick fix! Now I hit some @rpath issues:

➜  App git:(compile) ✗ julia -q --project
julia> using PackageCompilerX
[ Info: Precompiling PackageCompilerX [dffaa6cc-da53-48e5-b007-4292dfcc27f1]

shell> cd ..
/Users/gnimuc/.julia/dev/Redux/examples

julia> create_app("App", "AppCompiled")
┌ Warning: Package ModernGL has a build script, this might indicate that it is not relocatable
└ @ PackageCompilerX ~/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:369
[ Info: PackageCompilerX: creating base system image (incremental=false)...
[ Info: PackageCompilerX: creating system image object file, this might take a while...
[ Info: PackageCompilerX: creating system image object file, this might take a while...
ERROR: LoadError: LoadError: error compiling igGET_FLT_MAX: could not load library "@rpath/libcimgui.dylib"
dlopen(@rpath/libcimgui.dylib, 1): image not found
Stacktrace:
 [1] top-level scope at /Users/gnimuc/.julia/packages/CImGui/M9ITG/src/CImGui.jl:22
 [2] include at ./boot.jl:328 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1105
 [4] _require(::Base.PkgId) at ./loading.jl:1053
 [5] require(::Base.PkgId) at ./loading.jl:922
 [6] require(::Module, ::Symbol) at ./loading.jl:917
 [7] include at ./boot.jl:328 [inlined]
 [8] include_relative(::Module, ::String) at ./loading.jl:1105
 [9] _require(::Base.PkgId) at ./loading.jl:1053
 [10] require(::Base.PkgId) at ./loading.jl:922
 [11] require(::Module, ::Symbol) at ./loading.jl:917
in expression starting at /Users/gnimuc/.julia/packages/CImGui/M9ITG/src/CImGui.jl:22
in expression starting at /Users/gnimuc/.julia/dev/Redux/examples/App/src/App.jl:4
caused by [exception 1]
could not load library "@rpath/libcimgui.dylib"
dlopen(@rpath/libcimgui.dylib, 1): image not found
Stacktrace:
 [1] top-level scope at /Users/gnimuc/.julia/packages/CImGui/M9ITG/src/CImGui.jl:22
 [2] include at ./boot.jl:328 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1105
 [4] _require(::Base.PkgId) at ./loading.jl:1053
 [5] require(::Base.PkgId) at ./loading.jl:922
 [6] require(::Module, ::Symbol) at ./loading.jl:917
 [7] include at ./boot.jl:328 [inlined]
 [8] include_relative(::Module, ::String) at ./loading.jl:1105
 [9] _require(::Base.PkgId) at ./loading.jl:1053
 [10] require(::Base.PkgId) at ./loading.jl:922
 [11] require(::Module, ::Symbol) at ./loading.jl:917
ERROR: failed process: Process(`/Applications/Julia-1.3.app/Contents/Resources/julia/bin/julia --color=yes --startup-file=no --compiled-modules=no '--cpu-target=generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)' --sysimage=/var/folders/ft/nd_bm3z52152069y78vb71280000gn/T/jl_E1ghWV/tmp_sys.so --project=/Users/gnimuc/.julia/dev/Redux/examples/App --output-o=/var/folders/ft/nd_bm3z52152069y78vb71280000gn/T/jl_eyh60I.o -e 'Base.reinit_stdio()
Base.init_load_path()
Base.init_depot_path()
using App
# This @eval prevents symbols from being put into Main
@eval Module() begin
    PrecompileStagingArea = Module()
    for (_pkgid, _mod) in Base.loaded_modules
        if !(_pkgid.name in ("Main", "Core", "Base"))
            eval(PrecompileStagingArea, :(const $(Symbol(_mod)) = $_mod))
        end
    end
    precompile_statements = String[]
        append!(precompile_statements, readlines("/var/folders/ft/nd_bm3z52152069y78vb71280000gn/T/jl_SeD3MB"))

    for statement in sort(precompile_statements)
        # println(statement)
        try
            Base.include_string(PrecompileStagingArea, statement)
        catch
            # See julia issue #28808
            @error "failed to execute $statement"
        end
    end
end # module
    empty!(LOAD_PATH)
    empty!(DEPOT_PATH)
'`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at ./process.jl:525 [inlined]
 [2] #run#565(::Bool, ::typeof(run), ::Cmd) at ./process.jl:440
 [3] run at ./process.jl:438 [inlined]
 [4] #create_sysimg_object_file#4(::String, ::String, ::Array{String,1}, ::Array{String,1}, ::String, ::Bool, ::typeof(PackageCompilerX.create_sysimg_object_file), ::String, ::Array{Symbol,1}) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:167
 [5] #create_sysimg_object_file at ./none:0 [inlined]
 [6] #create_sysimage#5(::String, ::String, ::Array{String,1}, ::Array{String,1}, ::Bool, ::Bool, ::Bool, ::String, ::String, ::Bool, ::typeof(create_sysimage), ::Symbol) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:275
 [7] (::PackageCompilerX.var"#kw##create_sysimage")(::NamedTuple{(:sysimage_path, :project, :incremental, :precompile_execution_file, :precompile_statements_file, :cpu_target, :base_sysimage, :compiled_modules),Tuple{String,String,Bool,Array{String,1},Array{String,1},String,String,Bool}}, ::typeof(create_sysimage), ::Symbol) at ./none:0
 [8] (::PackageCompilerX.var"#7#8"{Array{String,1},Array{String,1},Bool,Bool,String,String})() at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:463
 [9] cd(::PackageCompilerX.var"#7#8"{Array{String,1},Array{String,1},Bool,Bool,String,String}, ::String) at ./file.jl:104
 [10] #create_app#6(::Array{String,1}, ::Array{String,1}, ::Bool, ::Bool, ::Bool, ::Bool, ::typeof(create_app), ::String, ::String) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:452
 [11] create_app(::String, ::String) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:424
 [12] top-level scope at REPL[3]:1

I guess this is not supposed to happen since CImGui.jl has already switched to the new artifacts build system and hence there shouldn't be any re-locatability issues described here.

KristofferC · 2020-01-05T09:18:26Z

Fixing JuliaLang/PackageCompilerX.jl#68 might help this. I will try it out today.

KristofferC · 2020-01-05T12:26:17Z

Ok with JuliaLang/PackageCompilerX.jl#71 creating the app works well.

However, it immediately exits when started. Perhaps nothing is waiting for the UI task?

KristofferC · 2020-01-05T12:35:41Z

Actually, it is a bug in PackageCompilerX.

IanButterworth · 2020-01-05T14:07:23Z

This is all fantastic. Thanks!!
I’m looking forward to testing it out when back at a computer tomorrow. I can see if that last bug occurs for my package.

KristofferC · 2020-01-06T21:04:45Z

Got it working with incremental=true at least:

IanButterworth · 2020-01-06T22:43:58Z

I gave this a go on my system but the app window still immediately closes when runing bin/App.
I'm on the up to date kc/compilecache branch, with 1.3.1 on ubuntu 18

Gnimuc · 2020-01-07T07:30:47Z

@KristofferC I hit #15 (comment) again after you force-pushed JuliaLang/PackageCompilerX.jl#71.

KristofferC · 2020-01-07T09:04:40Z

@Gnimuc

The log showing:

Base.init_load_path()
Base.init_depot_path()
using App

maks me think that it is not running the version in the PR. It should never run using anymore:

https://github.com/KristofferC/PackageCompilerX.jl/pull/71/files#diff-c8c207e0820e5dba4d133f965fa4c6ceL114-R133

Gnimuc · 2020-01-07T09:55:27Z

indeed, I forgot to change the branch 🤦‍♂ . now I get:

julia> create_app("App", "AppCompiled", incremental=true, force=true)
┌ Warning: Package ModernGL has a build script, this might indicate that it is not relocatable
└ @ PackageCompilerX ~/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:431
[ Info: PackageCompilerX: creating system image object file, this might take a while...
fatal: error thrown and no exception handler available.
ErrorException("error compiling julia_main: error compiling init_renderer: could not load library "@rpath/libglfw.3.dylib"
dlopen(@rpath/libglfw.3.dylib, 1): image not found")
rec_backtrace at /Users/sabae/buildbot/worker/package_macos64/build/src/stackwalk.c:94
record_backtrace at /Users/sabae/buildbot/worker/package_macos64/build/src/task.c:224
jl_throw at /Users/sabae/buildbot/worker/package_macos64/build/src/task.c:461
jl_rethrow_with_add at /Users/sabae/buildbot/worker/package_macos64/build/src/codegen.cpp:791
jl_compile_linfo at /Users/sabae/buildbot/worker/package_macos64/build/src/codegen.cpp:1205
_generate_from_hint at /Users/sabae/buildbot/worker/package_macos64/build/src/gf.c:2007 [inlined]
jl_compile_now at /Users/sabae/buildbot/worker/package_macos64/build/src/gf.c:2015
jl_precompile at /Users/sabae/buildbot/worker/package_macos64/build/src/precompile.c:388
jl_write_compiler_output at /Users/sabae/buildbot/worker/package_macos64/build/src/precompile.c:34
jl_atexit_hook at /Users/sabae/buildbot/worker/package_macos64/build/src/init.c:230
main at /Applications/Julia-1.3.app/Contents/Resources/julia/bin/julia (unknown line)
ERROR: failed process: Process(`/Applications/Julia-1.3.app/Contents/Resources/julia/bin/julia --color=yes --startup-file=no '--cpu-target=generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)' --sysimage=/Applications/Julia-1.3.app/Contents/Resources/julia/lib/julia/sys.dylib --project=/Users/gnimuc/.julia/dev/Redux/examples/App --output-o=/var/folders/ft/nd_bm3z52152069y78vb71280000gn/T/jl_cGlj1j.o -e 'Base.reinit_stdio()
Base.init_load_path()
Base.init_depot_path()
@eval Module() begin
    m = Base._require_from_serialized("/Users/gnimuc/.julia/compiled/v1.3/App/6XOLX_GDaLN.ji")
    m isa Exception && throw(m)
end
# This @eval prevents symbols from being put into Main
@eval Module() begin
    PrecompileStagingArea = Module()
    for (_pkgid, _mod) in Base.loaded_modules
        if !(_pkgid.name in ("Main", "Core", "Base"))
            eval(PrecompileStagingArea, :(const $(Symbol(_mod)) = $_mod))
        end
    end
    precompile_statements = String[]
        append!(precompile_statements, readlines("/var/folders/ft/nd_bm3z52152069y78vb71280000gn/T/jl_vSd7O3"))

    for statement in sort(precompile_statements)
        # println(statement)
        try
            Base.include_string(PrecompileStagingArea, statement)
        catch
            # See julia issue #28808
            @error "failed to execute $statement"
        end
    end
end # module
Base.@ccallable function julia_main()::Cint
    try
        app = Base.loaded_modules[Base.PkgId(Base.UUID("f943f3d7-887a-4ed5-b0c0-a1d6899aa8f5"), "App")]
        app.julia_main() 
    catch
        return 1
    end
end
empty!(LOAD_PATH)
empty!(DEPOT_PATH)
'`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at ./process.jl:525 [inlined]
 [2] #run#565(::Bool, ::typeof(run), ::Cmd) at ./process.jl:440
 [3] run at ./process.jl:438 [inlined]
 [4] #create_sysimg_object_file#4(::String, ::String, ::Array{String,1}, ::Array{String,1}, ::String, ::Base.PkgId, ::typeof(PackageCompilerX.create_sysimg_object_file), ::String, ::Array{String,1}) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:200
 [5] #create_sysimg_object_file at ./none:0 [inlined]
 [6] #create_sysimage#5(::String, ::String, ::Array{String,1}, ::Array{String,1}, ::Bool, ::Bool, ::Bool, ::String, ::Nothing, ::Base.PkgId, ::typeof(create_sysimage), ::Symbol) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:334
 [7] (::PackageCompilerX.var"#kw##create_sysimage")(::NamedTuple{(:sysimage_path, :project, :incremental, :filter_stdlibs, :precompile_execution_file, :precompile_statements_file, :cpu_target, :app),Tuple{String,String,Bool,Bool,Array{String,1},Array{String,1},String,Base.PkgId}}, ::typeof(create_sysimage), ::Symbol) at ./none:0
 [8] (::PackageCompilerX.var"#9#10"{Array{String,1},Array{String,1},Bool,Bool,String,Base.PkgId,String})() at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:534
 [9] cd(::PackageCompilerX.var"#9#10"{Array{String,1},Array{String,1},Bool,Bool,String,Base.PkgId,String}, ::String) at ./file.jl:104
 [10] #create_app#8(::Array{String,1}, ::Array{String,1}, ::Bool, ::Bool, ::Bool, ::Bool, ::typeof(create_app), ::String, ::String) at /Users/gnimuc/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:515
 [11] (::PackageCompilerX.var"#kw##create_app")(::NamedTuple{(:incremental, :force),Tuple{Bool,Bool}}, ::typeof(create_app), ::String, ::String) at ./none:0
 [12] top-level scope at REPL[3]:1

KristofferC · 2020-01-07T09:59:32Z

Are you on 1.3.1? That is required for PackageCompilerX.

Gnimuc · 2020-01-07T10:03:32Z

no. 🤦‍♂ BB2 has some problems with 1.3.1 so I'm still using 1.3.0.

KristofferC · 2020-01-07T10:09:01Z

Okay, there are some bugfixes in 1.3.1 that are needed, unfortunately (in this case JuliaLang/julia#34062)

Gnimuc · 2020-01-07T12:02:32Z

With the latest JuliaLang/PackageCompilerX.jl#71, I can compile it successfully, but it immediately exits even with incremental=true:

shell> cd ~/.julia/dev/Redux/examples/App
/Users/gnimuc/.julia/dev/Redux/examples/App

julia> using PackageCompilerX
[ Info: Precompiling PackageCompilerX [dffaa6cc-da53-48e5-b007-4292dfcc27f1]

(v1.3) pkg> activate .
Activating environment at `~/.julia/dev/Redux/examples/App/Project.toml`

(App) pkg> st
Project App v0.1.0
    Status `~/.julia/dev/Redux/examples/App/Project.toml`
  [5d785b6c] CImGui v1.74.1
  [6d653af8] Redux v0.1.0 [`~/.julia/dev/Redux`]

julia> using PackageCompilerX

julia> create_app("../App", "../AppCompiled", incremental=true)
┌ Warning: Package ModernGL has a build script, this might indicate that it is not relocatable
└ @ PackageCompilerX ~/.julia/dev/PackageCompilerX/src/PackageCompilerX.jl:447
[ Info: PackageCompilerX: creating system image object file, this might take a while...

julia> run(`../AppCompiled/bin/App`)
Process(`../AppCompiled/bin/App`, ProcessExited(0))

Gnimuc · 2020-01-07T12:12:43Z

OK, it's working(both incremental=true and incremental=false) if I remove the @sync call from this line, but there're wired graphic bugs too:

KristofferC · 2020-01-07T12:41:13Z

OK, it's working if I remove the @sync call from this line

Yeah, something has to wait on the render task or everything just falls through and the main function returns.

but there're wired graphic bugs too:

I cannot repro... Strange (I'm on Linux now, my prev screenshot was mac)

KristofferC · 2020-01-07T13:15:44Z

I got the graphics bug too now...

Gnimuc · 2020-01-07T13:22:03Z

😂 Only on macOS? I moved some source code out of the App folder on Redux.jl's master. Could that be the reason for the bug?

KristofferC · 2020-01-07T13:38:12Z

No on Linux as well. Trying to figure out why... I didn't update Redux so shouldn't be the reason...

KristofferC · 2020-01-07T13:50:05Z

I only get the graphics bug when I run with a precompilation file...:

IanButterworth · 2020-01-07T13:56:36Z

Could it be linked to the speed of execution being faster and GLFW not being ready? I mean it shouldn’t, but the speed of execution is maybe the only difference?

Gnimuc · 2020-01-07T14:05:55Z

I don't know what job a precompilation file does in this case(I also didn't set up one when I was doing tests). Does that imply I did something wrong when setting up those global variables in __init__ here and here?

KristofferC · 2020-01-07T14:57:59Z

The precompile file should in theory just speed up the execution by removing some need to compile functions.
And since you get the graphics bug even without it, it feels like precompilation or not isn't the end of the story.

Maybe the DearImGui guys have some idea what would cause problems like this...

KristofferC · 2020-01-07T16:38:29Z

For posterity, #16 didn't change anything when it comes to the graphics bugs.

Gnimuc · 2020-01-07T17:10:29Z

Confirmed this is a CImGui issue. It turns out the naive OpenGL/GLFW example in this branch works well with PackageCompilerX.

Gnimuc · 2020-01-08T11:36:25Z

looks like it's working now...

versioninfo:

Julia 1.3.1
PackageCompilerX#master
Redux#master
CImGui#ogl-backend-update

IanButterworth · 2020-01-08T14:32:07Z

I'm afraid it's still immediately closing for me:

(v1.3) pkg> st
    Status `~/.julia/environments/v1.3/Project.toml`
  [5d785b6c] CImGui v1.74.1 #ogl-backend-update (https://github.com/Gnimuc/CImGui.jl.git)
  [ea656a56] CSyntax v0.3.0
  [66fc600b] ModernGL v1.1.2
  [dffaa6cc] PackageCompilerX v0.1.0 [`~/Documents/GitHub/PackageCompilerX.jl`]
  [6d653af8] Redux v0.1.0 [`~/.julia/dev/Redux`]

(v1.3) pkg> free PackageCompilerX
ERROR: cannot free a `dev`ed package that does not exist in a registry

julia> create_app("App", "AppCompiled", force=true, incremental=true)
┌ Warning: Package ModernGL has a build script, this might indicate that it is not relocatable
└ @ PackageCompilerX ~/Documents/GitHub/PackageCompilerX.jl/src/PackageCompilerX.jl:432
precompile_paths = ["/home/ian/.julia/compiled/v1.3/App/6XOLX_l6U19.ji"]
[ Info: PackageCompilerX: creating system image object file, this might take a while...

julia> run(`AppCompiled/bin/App`)
Process(`AppCompiled/bin/App`, ProcessExited(0))

KristofferC · 2020-01-08T14:35:32Z

With https://github.com/Gnimuc/Redux.jl/pull/5/files#diff-3273ac476f7170cd30f851d83be23287R28 it shouldn't. Did you update the Redux branch to that?

IanButterworth · 2020-01-08T18:29:29Z

Ah, I was on an outdated fork master. Now with that PR, the App stays open but:

IanButterworth · 2020-01-08T18:32:46Z

Also, when I manually close the window I get:

julia> run(`AppCompiled/bin/App`)
ERROR: MethodError: no method matching wait(::Nothing)
Closest candidates are:
  wait(::Base.Event) at lock.jl:286
  wait(::Task) at task.jl:248
  wait(::Base.Process) at process.jl:615
  ...
Stacktrace:
 [1] real_main() at /home/ian/Documents/GitHub/Redux.jl/examples/App/src/App.jl:28
 [2] julia_main() at /home/ian/Documents/GitHub/Redux.jl/examples/App/src/App.jl:17
 [3] julia_main() at ./none:32
ERROR: failed process: Process(`AppCompiled/bin/App`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at ./process.jl:525 [inlined]
 [2] #run#565(::Bool, ::typeof(run), ::Cmd) at ./process.jl:440
 [3] run(::Cmd) at ./process.jl:438
 [4] top-level scope at none:0

IanButterworth · 2020-01-08T18:57:09Z

t = Renderer.render(()->todo_mvc_ui(store), width=600, height=300, title="App: TodoMVC")
wait(t)

doesn't seem right, and when I revert those lines it stays open and closes correctly.. I've lost track of what change actually fixed the window not staying open..

Gnimuc · 2020-01-09T02:43:16Z

@KristofferC @ianshmean the graphic bug should be fixed now, see https://github.com/Gnimuc/Redux.jl/tree/master/examples/App.

IanButterworth · 2020-01-09T03:20:29Z

Fixed for me on Ubuntu. This is great! Thanks so much

KristofferC · 2020-01-09T10:52:18Z

This isn't related to compilation but the Redux App example looks like:

for me on Mac.

So it seems like some size is being calculated incorrectly since the buttons in the bottom are not visisble, for example.

Gnimuc · 2020-01-09T10:57:26Z

That should be fixed by this commit, please try the latest Redux.jl's master again.

BTW, I also released a precompiled-binary for macOS here.

KristofferC · 2020-01-09T11:22:53Z

Hm, it doesn't seem fixed to me. I'm using the latest master.

And boy, is Catalina annoying. Every single library needs to be explicitly approved:

Gnimuc · 2020-01-09T11:36:28Z

I can not reproduce... Do you get the same result when running https://github.com/Gnimuc/Redux.jl/blob/master/examples/todomvc/app.jl directly?

KristofferC · 2020-02-28T21:08:28Z

I tried this again now (new computer) and I am still getting the oversized window (even when using app.jl directly:

Gnimuc · 2020-02-29T04:55:08Z

@KristofferC I still cannot reproduce, could you give this Gnimuc/Redux.jl#8 a try?

KristofferC · 2020-02-29T09:41:23Z

If I change the window size for CimGui in the render loop to be half of the display sizes:

CImGui.SetNextWindowSize((div(display_w, 2), div(display_h, 2)))

in the render loop, it works perfectly:

My guess is that it has something to do with DPI and scale factors, e.g as described in https://docs.rs/winit/0.21.0/winit/dpi/index.html#how-is-the-scale-factor-calculated:

macOS: "retina displays" have a scale factor of 2.0. Otherwise, the scale factor is 1.0. Intermediate scale factors are never used. It's possible for any display to use that 2.0 scale factor, given the use of the command line.

That is why I tried to divide by two since I thought maybe there is some disagreement somewhere if things are measured in pixels with or without scale factor taken into account.

Gnimuc/CImGui.jl#15 (comment)

Gnimuc · 2020-02-29T10:33:22Z

Thanks for the info! I now can reproduce the issue on a retina display. I've pushed a new fix on Gnimuc/Redux.jl#8.

KristofferC · 2020-02-29T10:37:48Z

Yep, works perfect with the PR! Thanks :)

Gnimuc mentioned this issue Feb 4, 2020

Gadfly JuliaLang/PackageCompilerX.jl#89

Closed

Gnimuc mentioned this issue Feb 29, 2020

Adjust Window flags and fix renderer for retina displays Gnimuc/Redux.jl#8

Merged

Gnimuc added a commit to Gnimuc/Redux.jl that referenced this issue Feb 29, 2020

Fix renderer for retina dispalys

687a4a6

Gnimuc/CImGui.jl#15 (comment)

Gnimuc closed this as completed in Gnimuc/Redux.jl#8 Feb 29, 2020

Attempting to compile with PackageCompilerX #15

Attempting to compile with PackageCompilerX #15

Comments

IanButterworth commented Jan 3, 2020

IanButterworth commented Jan 3, 2020

KristofferC commented Jan 3, 2020

Gnimuc commented Jan 4, 2020

Gnimuc commented Jan 4, 2020 • edited Loading

KristofferC commented Jan 4, 2020 • edited Loading

KristofferC commented Jan 4, 2020

KristofferC commented Jan 4, 2020

Gnimuc commented Jan 5, 2020 • edited Loading

KristofferC commented Jan 5, 2020

KristofferC commented Jan 5, 2020

KristofferC commented Jan 5, 2020

IanButterworth commented Jan 5, 2020

KristofferC commented Jan 6, 2020

IanButterworth commented Jan 6, 2020

Gnimuc commented Jan 7, 2020

KristofferC commented Jan 7, 2020 • edited Loading

Gnimuc commented Jan 7, 2020

KristofferC commented Jan 7, 2020

Gnimuc commented Jan 7, 2020

KristofferC commented Jan 7, 2020 • edited Loading

Gnimuc commented Jan 7, 2020

Gnimuc commented Jan 7, 2020 • edited Loading

KristofferC commented Jan 7, 2020 • edited Loading

KristofferC commented Jan 7, 2020

Gnimuc commented Jan 7, 2020

KristofferC commented Jan 7, 2020

KristofferC commented Jan 7, 2020

IanButterworth commented Jan 7, 2020

Gnimuc commented Jan 7, 2020

KristofferC commented Jan 7, 2020 • edited Loading

KristofferC commented Jan 7, 2020

Gnimuc commented Jan 7, 2020

Gnimuc commented Jan 8, 2020

IanButterworth commented Jan 8, 2020

KristofferC commented Jan 8, 2020

IanButterworth commented Jan 8, 2020

IanButterworth commented Jan 8, 2020

IanButterworth commented Jan 8, 2020 • edited Loading

Gnimuc commented Jan 9, 2020

IanButterworth commented Jan 9, 2020

KristofferC commented Jan 9, 2020 • edited Loading

Gnimuc commented Jan 9, 2020

KristofferC commented Jan 9, 2020

Gnimuc commented Jan 9, 2020

KristofferC commented Feb 28, 2020

Gnimuc commented Feb 29, 2020

KristofferC commented Feb 29, 2020

Gnimuc commented Feb 29, 2020

KristofferC commented Feb 29, 2020

Gnimuc commented Jan 4, 2020 •

edited

Loading

KristofferC commented Jan 4, 2020 •

edited

Loading

Gnimuc commented Jan 5, 2020 •

edited

Loading

KristofferC commented Jan 7, 2020 •

edited

Loading

KristofferC commented Jan 7, 2020 •

edited

Loading

Gnimuc commented Jan 7, 2020 •

edited

Loading

KristofferC commented Jan 7, 2020 •

edited

Loading

KristofferC commented Jan 7, 2020 •

edited

Loading

IanButterworth commented Jan 8, 2020 •

edited

Loading

KristofferC commented Jan 9, 2020 •

edited

Loading