Installation fails for Apple M3 - ImportError: symbol not found in flat namespace

[solbes]

Hi,

I'm trying to install cvxpy with CBC solver support (which uses cylp in the background) for my new Mac M3. Installation commands go through fine, but when importing stuff I get the following error:

>>> from cylp.cy import CyClpSimplex
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/miniconda3/envs/cylp/lib/python3.12/site-packages/cylp/cy/__init__.py", line 1, in <module>
    from .CyCoinIndexedVector import CyCoinIndexedVector
ImportError: dlopen(/opt/miniconda3/envs/cylp/lib/python3.12/site-packages/cylp/cy/CyCoinIndexedVector.cpython-312-darwin.so, 0x0002): symbol not found in flat namespace '__ZN9CoinError12printErrors_E'

Any idea what's happening here? Thanks in advance.

[jyuyangma]

I met the same problem, and I have resolved it using the methods shared by neumeyerx in the following post: Closed Issue

Basically, I just did the following:

git clone https://github.com/coin-or/CyLP.git
cd CyLP
pip install -e .

and now I can import 'CyClpSimplex'.

[solbes]

I met the same problem, and I have resolved it using the methods shared by neumeyerx in the following post: Closed Issue

Basically, I just did the following:

git clone https://github.com/coin-or/CyLP.git
cd CyLP
pip install -e .

and now I can import 'CyClpSimplex'.

Thanks, this did it for me!

[tkralphs]

I looked into this a little and it seems that for some reason, the wheels produced for Mac by cibuildwheel do not include the Cbc library and other dependent libraries, which are obviously needed. On Linux, the shared libraries are packaged in the wheel and installed in a directory site-package/cylp.libs and the libraries that comprise the extension itself depend on them.

(cylp) ~/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy > ldd CyClpSimplex.cpython-39-x86_64-linux-gnu.so
        linux-vdso.so.1 (0x00007fffe9fb7000)
        libCbcSolver-5c04a2f9.so.3.10.11 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libCbcSolver-5c04a2f9.so.3.10.11 (0x00007f9294ca8000)
        libCbc-cfea62ec.so.3.10.11 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libCbc-cfea62ec.so.3.10.11 (0x00007f9294b3b000)
        libCgl-bfe47839.so.1.10.8 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libCgl-bfe47839.so.1.10.8 (0x00007f92949f7000)
        libOsiClp-77baf8fe.so.1.14.9 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libOsiClp-77baf8fe.so.1.14.9 (0x00007f9294982000)
        libClpSolver-ed09da0c.so.1.14.9 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libClpSolver-ed09da0c.so.1.14.9 (0x00007f9294908000)
        libClp-00ff5bcd.so.1.14.9 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libClp-00ff5bcd.so.1.14.9 (0x00007f92946fb000)
        libOsi-1bfe6771.so.1.13.9 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libOsi-1bfe6771.so.1.13.9 (0x00007f9294689000)
        libCoinUtils-4fbb2601.so.3.11.10 => /home/ted/Projects/miniforge3/envs/cylp/lib/python3.9/site-packages/cylp/cy/./../../cylp.libs/libCoinUtils-4fbb2601.so.3.11.10 (0x00007f9294521000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f92944eb000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f92944e1000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f92942ff000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f92941ae000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f9294193000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9294170000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9293f7e000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f9294ed6000)

But the Mac wheels don't seem to include the libraries or have them as dependencies.

Mac /opt/homebrew/lib/python3.11/site-packages/cylp/cy > otool -L CyCoinIndexedVector.cpython-311-darwin.so
CyCoinIndexedVector.cpython-311-darwin.so:
        /usr/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.8)
        /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
        /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)

On the other hand, when using Cbc installed by homebrew and building CyLP from source, this isn't the case.

Mac /opt/homebrew/lib/python3.11/site-packages/cylp/cy > otool -L CyCoinIndexedVector.cpython-311-darwin.so
CyCoinIndexedVector.cpython-311-darwin.so:
        /opt/homebrew/opt/cbc/lib/libCbcSolver.3.dylib (compatibility version 14.0.0, current version 14.11.0)
        /opt/homebrew/opt/cbc/lib/libCbc.3.dylib (compatibility version 14.0.0, current version 14.11.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)
        /opt/homebrew/opt/cgl/lib/libCgl.1.dylib (compatibility version 12.0.0, current version 12.8.0)
        /opt/homebrew/opt/clp/lib/libOsiClp.1.dylib (compatibility version 16.0.0, current version 16.9.0)
        /opt/homebrew/opt/clp/lib/libClpSolver.1.dylib (compatibility version 16.0.0, current version 16.9.0)
        /opt/homebrew/opt/clp/lib/libClp.1.dylib (compatibility version 16.0.0, current version 16.9.0)
        /opt/homebrew/opt/osi/lib/libOsi.1.dylib (compatibility version 15.0.0, current version 15.9.0)
        /opt/homebrew/opt/coinutils/lib/libCoinUtils.3.dylib (compatibility version 15.0.0, current version 15.10.0)
        /usr/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.8)
        /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
        /opt/homebrew/opt/openblas/lib/libopenblas.0.dylib (compatibility version 0.0.0, current version 0.0.0)
        /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1500.65.0)

@mkoeppe I am cc'ing you in case you know anything. I have a vague recollection that there was something that needed to be done to package the libraries in the wheel on Linux, but it's a bit strange that the libraries are not there in the wheel for Mac, since the whole package is then obviously useless.

[tkralphs]

I guess this PR is related to pypa/cibuildwheel#1691. It seems the only solution is probably to build the wheels on a machine running Apple Silicon. For now, we should probably just not publish these wheels, since they don't work.

[tkralphs]

I have deleted the wheels for arm64 in both the release assets and on Pypi, I also updated the documentation. I'm going to close this issue.

[tkralphs]

Thanks to #195, I've now uploaded wheels for MacOS on Apple Silicon and attached them to the release 0.92.2. I have tested and the wheels seem to work. Unfortunately, I have to make a new release to get these on Pypi.