You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the macosx wheels are being built with a hard-coded CI paths (/Users/runner/hostedtoolcache/Java_Temurin-Hotspot_jdk/8.0.332-9/x64/Contents/Home/jre/lib/server). This makes them unfit for distribution.
$ otool -l jpy.cpython-38-darwin.so
...
Load command 10
cmd LC_LOAD_DYLIB
cmdsize 48
name @rpath/libjvm.dylib (offset 24)
time stamp 2 Wed Dec 31 16:00:02 1969
current version 1.0.0
compatibility version 1.0.0
...
Load command 12
cmd LC_RPATH
cmdsize 112
path /Users/runner/hostedtoolcache/Java_Temurin-Hotspot_jdk/8.0.332-9/x64/Contents/Home/jre/lib/server (offset 12)
...
Unfortunately, just removing the hard-coded LC_RPATH is not enough to produce usable wheels; @rpath/libjvm.dylib is not resolvable.
Going down a rabbit-hole, I discovered that dynamic loading / dlopen semantics are different on Linux vs OS X.
On Linux, once a library has been opened, it can be resolved via its soname:
>>> import ctypes
>>> ctypes.CDLL("libjvm.so")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.10/ctypes/__init__.py", line 374, in __init__
self._handle = _dlopen(self._name, mode)
OSError: libjvm.so: cannot open shared object file: No such file or directory
>>> ctypes.CDLL("/usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so")
<CDLL '/usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so', handle 55993695f530 at 0x7f55e89e3010>
>>> ctypes.CDLL("libjvm.so")
<CDLL 'libjvm.so', handle 55993695f530 at 0x7f55e89e32b0>
The same is not true on Mac:
>>> import ctypes
>>> ctypes.CDLL("libjvm.dylib")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/homebrew/Cellar/python@3.10/3.10.6_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ctypes/__init__.py", line 374, in __init__
self._handle = _dlopen(self._name, mode)
OSError: dlopen(libjvm.dylib, 0x0006): tried: '/opt/homebrew/lib/libjvm.dylib' (no such file), 'libjvm.dylib' (no such file), '/usr/local/lib/libjvm.dylib' (no such file), '/usr/lib/libjvm.dylib' (no such file), '/Users/deephaven/libjvm.dylib' (no such file)
>>> ctypes.CDLL("/opt/homebrew/Cellar/openjdk@11/11.0.16.1/libexec/openjdk.jdk/Contents/Home/lib/server/libjvm.dylib")
<CDLL '/opt/homebrew/Cellar/openjdk@11/11.0.16.1/libexec/openjdk.jdk/Contents/Home/lib/server/libjvm.dylib', handle 2013663c0 at 0x100ee0100>
>>> ctypes.CDLL("libjvm.dylib")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/homebrew/Cellar/python@3.10/3.10.6_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ctypes/__init__.py", line 374, in __init__
self._handle = _dlopen(self._name, mode)
OSError: dlopen(libjvm.dylib, 0x0006): tried: '/opt/homebrew/lib/libjvm.dylib' (no such file), 'libjvm.dylib' (no such file), '/usr/local/lib/libjvm.dylib' (no such file), '/usr/lib/libjvm.dylib' (no such file), '/Users/deephaven/libjvm.dylib' (no such file)
Note: if we were re-distributing a JVM we'd be able to setup a proper LC_RPATH / @rpath, but we're wanting the user to bring-their-own JVM (either as resolvable at runtime via the PATH or via JAVA_HOME).
Ultimately, I think the solution, currently in testing, is to remove both LC_RPATH and the LC_LOAD_DYLIB for libjvm.dylib; we'll have appropriate bootstrapping application logic to load libjvm.dylib, and the .so files will not try and re-find an rpath'd libjvm.dylib.
The text was updated successfully, but these errors were encountered:
This has the effect of removing the non-portable LC_LOAD_DYLIB and LC_RPATH directives for libjvm.dylib from the .so file.
Fixesjpy-consortium#79, see for more details
This has the effect of removing the non-portable LC_LOAD_DYLIB and LC_RPATH directives for libjvm.dylib from the .so file.
Fixesjpy-consortium#79, see for more details
Currently, the macosx wheels are being built with a hard-coded CI paths (
/Users/runner/hostedtoolcache/Java_Temurin-Hotspot_jdk/8.0.332-9/x64/Contents/Home/jre/lib/server
). This makes them unfit for distribution.Unfortunately, just removing the hard-coded LC_RPATH is not enough to produce usable wheels;
@rpath/libjvm.dylib
is not resolvable.Going down a rabbit-hole, I discovered that dynamic loading / dlopen semantics are different on Linux vs OS X.
On Linux, once a library has been opened, it can be resolved via its soname:
The same is not true on Mac:
https://developer.apple.com/forums/thread/132817 seems to reinforce that fact.
Note: if we were re-distributing a JVM we'd be able to setup a proper LC_RPATH / @rpath, but we're wanting the user to bring-their-own JVM (either as resolvable at runtime via the PATH or via JAVA_HOME).
Ultimately, I think the solution, currently in testing, is to remove both LC_RPATH and the LC_LOAD_DYLIB for libjvm.dylib; we'll have appropriate bootstrapping application logic to load libjvm.dylib, and the .so files will not try and re-find an rpath'd libjvm.dylib.
The text was updated successfully, but these errors were encountered: