Copy adolc and dependencies to install folder on linux #2930
Conversation
|
@adamkewley I started working on this a while back but got stuck trying to find a replacement for install_name_tool on unix that can be generic/available out of the box. If you can help here since you're more familiar with linux tools that would be great. Thank you |
fix filename reference on linux
|
@adamkewley We're trying to finalize 4.2, if you can take a look and fix or suggest a solution so that the linux build is usable that would be much appreciated. Thank you |
|
@mitkof6 Can you help with this PR so that linux builds are functional? Thanks |
|
Hi @aymanhab. Sorry, I just spotted this. I would be glad to help, but I am not sure that I understood the issue. Do you want to pack adolc within the OpenSim's install directory? Why not have it as an external dependency? |
|
Hi @mitkof6, the issue is not with building but with making a distribution on linux/ubuntu. We copy all the necessary files to the install directory already but the loader-path for the .so files is not modified, which causes a distribution or other forms of delivery to fail to find the dependencies. On osx, there's install_name_tool that we use as described above, I'm trying to find a similar tool readily available on linux to do the same (change rpath for so's to the install folder). Let me know if you have any questions or suggestions. Typically applications are self contained so no external dependencies, and there's no way to guarantee version/compiler compatibility. Thank you |
|
I see. I am not sure how things work on Mac, but on Linux, the philosophy is a bit different. We typically set the LD_LIBRARY_PATH to point to the new folder where the libraries are copied before running the application. This will make sure (prefer) to load the libraries located in this folder without overriding the relative paths. The drawback is that we will have to set LD_LIBRARY_PATH every time you call the application. Alternatively, there is a nice discussion that suggests the use of |
|
Also, CMake has the CMAKE_BUILD_WITH_INSTALL_RPATH. This is set in combination with |
|
Thanks @mitkof6 this could be helpful though may need a more recent version of CMake than we currently require. Will keep you posted. |
|
I am looking into this. Linux does not have the same restrictions as MAC. I will try to translate the commands. |
|
@AlbertoCasasOrtiz This PR should take care of adolc related issues on linux (both copying and rpath setting) Please review and comment as needed. In particular if you can check for both python and java/Matlab/GUI on linux that would be great. Thank you |
|
@AlbertoCasasOrtiz Can you please confirm that Ld_library_path is not needed to run the command line executables from this artifact on linux? Thank you |
|
@aymanhab I just tested this in a clean Ubuntu 18.04 environment and it is still showing the following error:
After installing it, it is still showing it. |
|
@AlbertoCasasOrtiz going back to finish this, the current status is that the libraries in question are copied but we end up with broken links for libgfortran and libquadmath and also the rpath is not updated. The latter is due to the difference in usage/syntax between otool on mac and patchelf on linux. I'll sort the latter out but would like to know the layout that works on osx for (gfortran and quadmath) and I don't have access to a mac at the moment, can you help? Thanks |
|
@AlbertoCasasOrtiz since you just done this for osx (copy and fix rpath for tropter libraries on osx), can you pickup this PR on linux to do the same? Thank you |
|
@aymanhab Sure, I'll work on this. |
In the documentation of patchelf, the option used to change the rpath is --set-rpath. --add-rpath does not exist in the documentation. More info: https://manpages.ubuntu.com/manpages/bionic/man1/patchelf.1.html
|
@aymanhab I think I just solved this issue in this branch https://github.com/opensim-org/opensim-core/tree/fix_linux_library_problems I will integrate the changes in this PR and better document my changes, and the reason of the problem as soon as I arrive today. I will test on Ubuntu 18, 20, 22 and Debian 10 and 11. |
|
@aymanhab With the latest commits, I have just tested this on a clean Ubuntu 22.04 environment and it is working. No warnings/errors related to libraries appear when starting the GUI. In order to test this, I have a fork of opensim-gui that obtains the appropriate branch for opensim-core while building. Here is the generated artifacts with this configuration: https://github.com/AlbertoCasasOrtiz/opensim-gui/actions/runs/3970774891 |
|
@aymanhab I have tested this on Ubuntu 18.04, 20.04 and 22.04. It is working now, but there is an issue with Ubuntu 18.04. Basically, looks like the issue we had on Google colab with the conda package. Builds created with Ubuntu 20.04 or above do not work on Ubuntu 18.04. The build created on Ubuntu 18.04 works perfectly. |
casadi libraries were not able to detect libcasadi.so.3.6
Update branch fix_tropter_libs_linux
|
@AlbertoCasasOrtiz Sorry to tell you this but the artifact I downloaded still shows the same problems on a fresh linux on windows. I'd leave this open. Thanks for all your efforts on fixing this rather nasty issue. |
|
@AlbertoCasasOrtiz I think this is now fixed, but will wait for a fresh ci build before declaring victory ✌️ |
|
Downloading and testng artifact on ubuntu 20.04 I can run opensim-cmd and opensense executables from arbitrary folders. Since these require loading the whole stack of libraries at runtime will declare victory and merge. Big thanks @AlbertoCasasOrtiz 🥇 |
Fixes issue #<issue_number>
opensim-org/opensim-gui#1393
Brief summary of changes
Hack made to copy moco dependencies and fix their rpath on mac is to be replicated for linux
Testing I've completed
Looking for feedback on...
CHANGELOG.md (choose one)
This change is