Skip to content

Make PyCall.jl AOT-compilable #651

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Mar 27, 2019
Merged

Make PyCall.jl AOT-compilable #651

merged 9 commits into from
Mar 27, 2019

Conversation

tkf
Copy link
Member

@tkf tkf commented Feb 25, 2019

This PR makes PyCall.jl compatible with PackageCompiler.jl and add some tests.

Quick usage:

$ aot/compile.jl --color=yes
...

$ aot/julia.sh --compiled-modules=no --startup-file=no --banner=no
julia> @time using PyCall
  0.001537 seconds (309 allocations: 16.688 KiB)

Slightly more detailed docs in: aot/README.md

empty!(pycall_gc)
empty!(pyexc)
empty!(pytype_queries)
empty!(permanent_strings)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this required?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes in b0e131a are the only ones required for making PyCall compilable with PackageCompiler. The ones in a9e9419 are not required but added for consistency and possible similar problems. It seems that at least adding them does not break PyCall.


write(joinpath(@__DIR__, "_julia_path"), Base.julia_cmd().exec[1])

@info "System image: $pysysout"
Copy link
Member

@stevengj stevengj Feb 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all seems like boilerplate — why does PyCall.jl need a custom compile.jl file? Can't PackageCompiler automate this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Snoop in PackageCompiler does not work due to JuliaLang/julia#31156 (a bug in Julia?). There is another "latent" issue JuliaLang/PackageCompiler.jl#192. I think PackageCompiler is still rather a moving target so using lower-level API with a bit of boilerplate makes sense. Another upside is that we can control exactly what functions are included in system image by tweaking code in aot/precompile.jl (although it is rather sparse and random ATM).

@tkf
Copy link
Member Author

tkf commented Mar 27, 2019

Rebuilt after #671: https://travis-ci.org/JuliaPy/PyCall.jl/builds/511789604 (all green)

@stevengj stevengj merged commit 2a7bb25 into JuliaPy:master Mar 27, 2019
tkf added a commit to tkf/PyCall.jl that referenced this pull request May 19, 2019
In PR JuliaPy#651, I introduced code to empty `npy_api` in `__init__` but
forgot to reset `npy_initialized`.  This introduces a bug if
`npyinitialize` is called during sysimage compilation (`--output-o`
mode) because the main body of `npyinitialize` is not run at run-time.
tkf added a commit to tkf/PyCall.jl that referenced this pull request May 19, 2019
In PR JuliaPy#651, I introduced code to empty `npy_api` in `__init__` but
forgot to reset `npy_initialized`.  This introduces a bug if
`npyinitialize` is called during sysimage compilation (`--output-o`
mode) because the main body of `npyinitialize` is not run at run-time.
tkf added a commit that referenced this pull request May 20, 2019
)

In PR #651, I introduced code to empty `npy_api` in `__init__` but
forgot to reset `npy_initialized`.  This introduces a bug if
`npyinitialize` is called during sysimage compilation (`--output-o`
mode) because the main body of `npyinitialize` is not run at run-time.
@tkf tkf deleted the aot branch April 25, 2020 03:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants