-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for Apple M1 #131
Comments
I cannot reproduce. What is |
I had a similar error on my Apple M1 machine: julia> using Clp
julia> optimizer = Clp.Optimizer()
Clp.Optimizer
julia> exit()
julia(99879,0x1007ebd40) malloc: *** error for object 0xe00000000000000: pointer being freed was not allocated
julia(99879,0x1007ebd40) malloc: *** set a breakpoint in malloc_error_break to debug
signal (6): Abort trap: 6
in expression starting at REPL[4]:1
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 4250078 (Pool: 4247945; Big: 2133); GC: 3
zsh: abort /Applications/Julia-1.7.app/Contents/Resources/julia/bin/Julia julia> import Pkg; Pkg.status()
Status `~/tmp/Clp/Project.toml`
[e2554f3b] Clp v1.0.0 `https://github.com/jump-dev/Clp.jl#master` julia> versioninfo()
Julia Version 1.7.1
Commit ac5cc99908 (2021-12-22 19:35 UTC)
Platform Info:
OS: macOS (arm64-apple-darwin21.2.0)
CPU: Apple M1
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-12.0.1 (ORCJIT, cyclone) |
Apple M1 is currently tier-3 support at Julia: https://julialang.org/downloads/#supported_platforms, so it's highly likely that you'll encounter segfaults like this. Use Rosetta instead. |
Right, but I just wondered if it might work, having found this JuliaPackaging/Yggdrasil#4015. |
It compiles, but I don't know if we've ever tested all of the bugs and issues. I don't have an M1, so I'm not much help. |
We should rebuild the M1 libraries with a recent toolchain. Should I just bump Clp_jll to a recent version? M1 is almost tier 1 - not yet but very close now. |
I assume we need to rebuild the entire stack, not just Clp or Cbc. I'll take a look. |
Yeah should be the whole stack. Also MUMPS has had new releases for example. So good idea to bump the whole stack. And I would love to link to LBT as well. But one step at a time... |
To anyone stumbling across this until we get it fixed: use HiGHS.jl instead. |
The issue still exists after recompiling: (m1-support) pkg> st Clp_jll
Status `~/Code/m1-support/Project.toml`
[06985876] Clp_jll v100.1700.700+0
julia> versioninfo()
Julia Version 1.8.2
Commit 36034abf260 (2022-09-29 15:21 UTC)
Platform Info:
OS: macOS (arm64-apple-darwin21.3.0)
CPU: 8 × Apple M1
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-13.0.1 (ORCJIT, apple-m1)
Threads: 1 on 4 virtual cores
julia> run(`$(Clp_jll.clp())`);
Coin LP version 1.17.7, build Oct 25 2022
clp(4926,0x1008c8580) malloc: *** error for object 0x600000e4c2a0: pointer being freed was not allocated
clp(4926,0x1008c8580) malloc: *** set a breakpoint in malloc_error_break to debug so I guess this means it is likely an upstream problem. |
Should we file a Clp issue? |
cc @tkralphs have you ever looked at Clp on the M1? |
I have had some bug reports from people trying to use Clp/Cbc on M1, but they were related to bugs in the build system. My impression is that there are people successfully using Clp/Cbc on M1, but this is anecdotal and I have not tried myself (I also don't have an M1, but am considering getting one). I would suggest maybe just posting a Discussion asking if anyone has succeeded. |
I have access to an M1, so I'll try compiling locally, excluding all the BinaryBuilder stuff |
So I build Clp@1.17.7 using coinbrew, and it worked without any issues. oscardowson@Oscars-Mac-mini clp % ./dist/bin/clp test.mps
Coin LP version 1.17.7, build Oct 28 2022
command line - ./dist/bin/clp test.mps
At line 1 NAME
At line 2 OBJSENSE
MIN found after OBJSENSE - Coin ignores
At line 4 ROWS
At line 7 COLUMNS
At line 11 RHS
At line 13 BOUNDS
At line 15 ENDATA
Problem no_name has 1 rows, 1 columns and 1 elements
Model was imported from ./test.mps in 0.000103 seconds
Presolve 0 (-1) rows, 0 (-1) columns and 0 (-1) elements
Empty problem - 0 rows, 0 columns and 0 elements
Optimal - objective value 1
After Postsolve, objective 1, infeasibilities - dual 0 (0), primal 0 (0)
Optimal objective 1 - 0 iterations time 0.002, Presolve 0.00 So I guess this is really some problem in the BB toolchain. |
Can you give a pointer to the |
https://github.com/coin-or/coinbrew/blob/master/coinbrew but I'm pretty sure there are some differences. The non-standard-y things we do are these sorts of flags: there are also mumps and Metis: and we set an but I don't know what impact that has on M1. |
Those differences probably don't explain the segfault, but of course anything is possible. @giordano are you familiar with any toolchain issues for M1 that might be causing this failure? |
Not really. |
@tkralphs The error is non-malloced memory being freed, which suggests some memory corruption. |
I was trying to do a debug build for Jacob, but aarch64-apple-darwin fails with some weird linking errors at the end (the non-debug build works, only the debug one fails, why?!?) which honestly I don't have the time to look at for the next ~10 days. In case anyone was curious:
|
OpenBLAS version bump. Perhaps that can help. JuliaPackaging/Yggdrasil#5844 |
The missing symbol is sandbox:${WORKSPACE}/srcdir/Clp/build # c++filt _ZN17CoinIndexedVector10checkClearEv
CoinIndexedVector::checkClear() I'm not sure OpenBLAS is related (but I also don't know Clp/Coin source code) |
No - but given how old our openblas is in Coin-OR (0.3.10!), perhaps there were other issues on M1? Just being optimistic. It won't help with this issue, but we can rebuild the known working version with a new openblas. |
But when doing dynamic linking the version of OpenBLAS shouldn't matter much (if at all), the ABI has been pretty stable. And the problem is only in the debug build ( |
Ah ok, good to know. Maybe I should remove mention of OpenBLAS32 version in the coin-or builds then? Remember this is OpenBLAS32, and not OpenBLAS that is in Julia. |
Specifying the version is useful for building for compatibility purposes (very often if you build against a new version of a library then you can't use at runtime an older version on macOS), but then at runtime we can use whatever version is available. |
Thanks all for poking at this. I don't have much bandwidth for it at the moment, maybe next week. |
@quinnj: in the interim, try HiGHS instead, or manually compile Clp on M1 and use a custom binary: https://jump.dev/JuMP.jl/stable/developers/custom_solver_binaries/ |
Will give HiGHS a shot; thanks. |
A very quick search revealed why linking failed: https://github.com/coin-or/CoinUtils/blob/bbd81ae459f25ee1f665f3cc017309da5025eb81/src/CoinIndexedVector.hpp#L265-L273. CoinUtils defines |
And of course I can't use an unregistered dependency because of JuliaLang/Pkg.jl#3251 🙄 |
This is motivated by issues with CBC on Apple M1/M2 architectures: - jump-dev/Clp.jl#131 - http://github.com/daschw/SankeyPlots.jl/issues/26 Moreover there seems to be a slight performance advantage overall: HiGHS: Test Summary: | Pass Total Time LayeredLayouts.jl | 20 20 17.4s Test Summary: | Pass Total Time LayeredLayouts.jl | 20 20 19.3s CBC: Test Summary: | Pass Total Time LayeredLayouts.jl | 20 20 20.5s Test Summary: | Pass Total Time LayeredLayouts.jl | 20 20 20.6s However, the `direct/sankey_3twos.png` and `paths/sankey_3twos.png` results to be flipped with this solver:
Is there a fix for this? I'm running into the same memory error when attempting to run the optimize step using Clp on an Apple M1. |
No. Use HiGHS.jl instead |
Is there a way to build |
We're looking at rebuilding the COIN-OR stack: JuliaPackaging/Yggdrasil#8067 But really: to any readers of this issue, please use HiGHS.jl instead. |
Unfortunately, for some problems, HiGHS.jl performs very poorly when compared to Cbc. |
Do you have a reproducible example of a model? The HiGHS developers would be interested in this. |
I agree! Unfortunately, it is a reasonably complex MIP. I'll need to find a similar problem but with a reasonable size. |
Export it to an MPS file? |
Okay, I've managed to get to here, but I don't really know what I'm looking for as a next step:
|
Many thanks, Oscar, for working on this!! Since the Homebrew version of Cbc/Clp (2.10.11) works on Apple silicon, can we replicate what they did to compile the code? |
Not exactly. We compile things using Yggdrasil: https://github.com/JuliaPackaging/Yggdrasil/blob/master/C/Coin-OR/Clp/build_tarballs.jl Here's the home-brew recipe for completeness: |
So I commented out this line: So somewhere we're obvious screwing up |
It's possible that we use |
I cannot run
Clp
throughJuMP
, it crashes when as soon as I start the optimization. Here is a minimal example, where I am trying to minimizec^T x
whereA x = b
andx >=0
.The text was updated successfully, but these errors were encountered: