Skip to content

Commit 4230024

Browse files
committed
RPATH Fix for portable_lib Python Extension
Problem: The _portable_lib.so Python extension built on CI couldn't find PyTorch libraries when installed locally because it had hardcoded absolute paths from the CI build environment. Error: ImportError: dlopen(.../_portable_lib.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libtorch_python.dylib Referenced from: .../executorch/extension/pybindings/_portable_lib.cpython-311-darwin.so Reason: tried: '/Users/runner/work/_temp/.../torch/lib/libtorch_python.dylib' (no such file) Root Cause: The CMake build was linking to PyTorch libraries using absolute paths from the build environment, without setting proper relative RPATHs for runtime library resolution. Solution: Added platform-specific relative RPATH settings to the portable_lib target in /Users/mnachin/executorch/CMakeLists.txt (lines 657-669): - macOS: Uses @loader_path/../../../torch/lib to find PyTorch libraries relative to the .so file location - Linux: Uses $ORIGIN/../../../torch/lib for the same purpose - Sets both BUILD_RPATH and INSTALL_RPATH to ensure consistency Impact: This allows the wheel-packaged _portable_lib.so to find PyTorch libraries regardless of the installation location, fixing the runtime linking issue when using ExecutorTorch wheels built on CI. Note: The same fix may be needed for _training_lib if it experiences similar issues.
1 parent b1b38fe commit 4230024

File tree

2 files changed

+23
-0
lines changed

2 files changed

+23
-0
lines changed

.ci/scripts/wheel/test_base.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,15 @@ class ModelTest:
4141

4242

4343
def run_tests(model_tests: List[ModelTest]) -> None:
44+
# Test that we can import the portable_lib module - verifies RPATH is correct
45+
print("Testing portable_lib import...")
46+
try:
47+
from executorch.extension.pybindings._portable_lib import _load_for_executorch
48+
print("✓ Successfully imported _load_for_executorch from portable_lib")
49+
except ImportError as e:
50+
print(f"✗ Failed to import portable_lib: {e}")
51+
raise
52+
4453
# Why are we doing this envvar shenanigans? Since we build the testers, which
4554
# uses buck, we cannot run as root. This is a sneaky of getting around that
4655
# test.

CMakeLists.txt

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -654,6 +654,20 @@ if(EXECUTORCH_BUILD_PYBIND)
654654
target_compile_options(portable_lib PUBLIC ${_pybind_compile_options})
655655
target_link_libraries(portable_lib PRIVATE ${_dep_libs})
656656

657+
# Set RPATH to find PyTorch libraries relative to the installation location
658+
# This goes from executorch/extension/pybindings up to site-packages, then to torch/lib
659+
if(APPLE)
660+
set_target_properties(portable_lib PROPERTIES
661+
BUILD_RPATH "@loader_path/../../../torch/lib"
662+
INSTALL_RPATH "@loader_path/../../../torch/lib"
663+
)
664+
else()
665+
set_target_properties(portable_lib PROPERTIES
666+
BUILD_RPATH "$ORIGIN/../../../torch/lib"
667+
INSTALL_RPATH "$ORIGIN/../../../torch/lib"
668+
)
669+
endif()
670+
657671
install(TARGETS portable_lib
658672
LIBRARY DESTINATION executorch/extension/pybindings
659673
)

0 commit comments

Comments
 (0)