-
-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gitian: add ARM64 Darwin support #8312
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested successfully on:
- Intel 10.15.7
- Intel 11.6.1
- ARM 12.4
If someone wants to test the binaries on their own Mac, here are the download links:
- https://github.com/selsta/monero/suites/6386331257/artifacts/232120463 (ARM)
- https://github.com/selsta/monero/suites/6386331257/artifacts/232120464 (Intel)
Small change that is missing to fix CI:
diff --git a/.github/workflows/depends.yml b/.github/workflows/depends.yml
index 9385338de..2eb7dba74 100644
--- a/.github/workflows/depends.yml
+++ b/.github/workflows/depends.yml
@@ -46,10 +46,14 @@ jobs:
- name: "x86_64 Linux"
host: "x86_64-unknown-linux-gnu"
packages: "gperf cmake python3-zmq libdbus-1-dev libharfbuzz-dev"
- - name: "Cross-Mac"
+ - name: "Cross-Mac x86_64"
host: "x86_64-apple-darwin11"
packages: "cmake imagemagick libcap-dev librsvg2-bin libz-dev libbz2-dev libtiff-tools python-dev python3-setuptools-git"
- osx_sdk: "10.11"
+ osx_sdk: "11.1"
+ - name: "Cross-Mac aarch64"
+ host: "aarch64-apple-darwin11"
+ packages: "cmake imagemagick libcap-dev librsvg2-bin libz-dev libbz2-dev libtiff-tools python-dev python3-setuptools-git"
+ osx_sdk: "11.1"
- name: "x86_64 Freebsd"
host: "x86_64-unknown-freebsd"
packages: "clang-8 gperf cmake python3-zmq libdbus-1-dev libharfbuzz-dev"
@@ -86,11 +90,11 @@ jobs:
- name: install dependencies
run: sudo apt update; sudo apt -y install build-essential libtool cmake autotools-dev automake pkg-config bsdmainutils curl git ca-certificates ccache ${{ matrix.toolchain.packages }}
- name: prepare apple-darwin11
- if: ${{ matrix.toolchain.host == 'x86_64-apple-darwin11' }}
+ if: ${{ matrix.toolchain.host == 'x86_64-apple-darwin11' || matrix.toolchain.host == 'aarch64-apple-darwin11' }}
run: |
mkdir -p contrib/depends/SDKs contrib/depends/sdk-sources
- if [ ! -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz ]; then curl --location --fail https://bitcoincore.org/depends-sources/sdks/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz -o contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz; fi
- if [ -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz ]; then tar -C contrib/depends/SDKs -xf contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz; fi
+ if [ ! -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz ]; then curl --location --fail https://github.com/phracker/MacOSX-SDKs/releases/download/11.3/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz -o contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz; fi
+ if [ -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz ]; then tar -C contrib/depends/SDKs -xf contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz; fi
- name: prepare w64-mingw32
if: ${{ matrix.toolchain.host == 'x86_64-w64-mingw32' || matrix.toolchain.host == 'i686-w64-mingw32' }}
run: |
I did not check yet if the resulting binaries are indeed reproducible.
We used to skip the otools part of the cctools package because that build was non-deterministic, but that quirk was fixed in clang 7.0.1 and newer so we should be good to go. But yes, after I add your patch we can all build this branch and compare. |
Note to self: still need to delete depends/patches/native_cctools but will do that later |
This Maybe we should work with that going forward. |
I've merged the Mac SDK change into this PR now. So now the SDK is a regular depends package and doesn't need to be manually fetched and copied any more. Also took a stab at silencing all the libprotobuf linker warnings. Got the majority of them. |
That means you can also remove this again. diff --git a/.github/workflows/depends.yml b/.github/workflows/depends.yml
index 2eb7dba74..f20bf949f 100644
--- a/.github/workflows/depends.yml
+++ b/.github/workflows/depends.yml
@@ -49,11 +49,9 @@ jobs:
- name: "Cross-Mac x86_64"
host: "x86_64-apple-darwin11"
packages: "cmake imagemagick libcap-dev librsvg2-bin libz-dev libbz2-dev libtiff-tools python-dev python3-setuptools-git"
- osx_sdk: "11.1"
- name: "Cross-Mac aarch64"
host: "aarch64-apple-darwin11"
packages: "cmake imagemagick libcap-dev librsvg2-bin libz-dev libbz2-dev libtiff-tools python-dev python3-setuptools-git"
- osx_sdk: "11.1"
- name: "x86_64 Freebsd"
host: "x86_64-unknown-freebsd"
packages: "clang-8 gperf cmake python3-zmq libdbus-1-dev libharfbuzz-dev"
@@ -89,12 +87,6 @@ jobs:
run: ${{env.APT_SET_CONF}}
- name: install dependencies
run: sudo apt update; sudo apt -y install build-essential libtool cmake autotools-dev automake pkg-config bsdmainutils curl git ca-certificates ccache ${{ matrix.toolchain.packages }}
- - name: prepare apple-darwin11
- if: ${{ matrix.toolchain.host == 'x86_64-apple-darwin11' || matrix.toolchain.host == 'aarch64-apple-darwin11' }}
- run: |
- mkdir -p contrib/depends/SDKs contrib/depends/sdk-sources
- if [ ! -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz ]; then curl --location --fail https://github.com/phracker/MacOSX-SDKs/releases/download/11.0-11.1/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz -o contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz; fi
- if [ -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz ]; then tar -C contrib/depends/SDKs -xf contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz; fi
- name: prepare w64-mingw32
if: ${{ matrix.toolchain.host == 'x86_64-w64-mingw32' || matrix.toolchain.host == 'i686-w64-mingw32' }}
run: | |
8cbfcfc
to
2110abf
Compare
Hashes for this build
|
The libprotobuf warnings appear to be a long standing problem with Apple/ARM64 builds of libprotobuf protocolbuffers/protobuf#8958 (comment) |
Tested @selsta's binaries successfully for MacOS 10.13.6 (High Sierra). Definitely not a full on stress test or anything, but they run and seem to work just fine at first glance. |
Also got it working on Intel 10.12:
|
Great! I tried running a local depends build on my Mac (without gitian) and ran into a bunch of snags re: clang path. Also the version of QT in depends is too old for Mac ARM support. Not sure it's worth worrying about at the moment. |
4 warnings remain but it's better than before
RandomX PR was merged so all of this is ready to go now. (tevador/RandomX#241) |
230b2ca
to
76cd2fa
Compare
Also silence randomx malloc exceptions
All patching should be in _preprocess_cmd. config_cmd should just run configure and nothing else before it, otherwise $PATH is broken when it runs. Also remove redundant settings to configure, _autoconf already sets them. We leave explicit AR_FLAGS settings because the builtin rule uses ARFLAGS, and so would leave AR_FLAGS unset otherwise.
My build hashes for everything
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bb72a7b61507e1f96d05dfee4a89c626fe4647147d9a962bf77353020e7e5cb5 monero-aarch64-linux-gnu-5c27fd853.tar.bz2
41c1053da090f5ed5fcb708af88bdbf1b0b6e2bb2c52b009a20cfa2f82c9a231 monero-arm-linux-gnueabihf-5c27fd853.tar.bz2
2198d6af777ebd47199c42ca0db1e5d2b05c533eb6c32729866ece1880420f8b monero-i686-linux-gnu-5c27fd853.tar.bz2
e5e5f8340d42dcaec23e0a34ccdd61bf9387ba08674d0c4b22df7f68ed1807b9 monero-x86_64-linux-gnu-5c27fd853.tar.bz2
6837cc13ed07e21ef008ea39be9102be8bc3d5c05ac7fc71ad6e7251eed9aed3 monero-aarch64-linux-android-5c27fd853.tar.bz2
6279f3c97989077c24e7276d075d8f18244761e68bc456bba202f3c987440632 monero-arm-linux-android-5c27fd853.tar.bz2
7c6272f07282fb283b6b89593c10b3293debd135a5af1cf57a2e638ea49fe77b monero-x86_64-unknown-freebsd-5c27fd853.tar.bz2
33c0c829659657f5f423c3425098dc43e2e3e3b4e07576abbd90e3a60b9ceeb6 monero-i686-w64-mingw32-5c27fd853.zip
19f3d60c604dd3fa80f5ab0dc5beca49e4bf15ba321d16ddf7eac9417f988308 monero-x86_64-w64-mingw32-5c27fd853.zip
ca250ad0312233fbc2b091cb25574a272289f6681d526a4f8d27a98b5cc72358 monero-aarch64-apple-darwin11-5c27fd853.tar.bz2
c55bb230f9a70bccddd428691be6a751980dc709bef014aa3252e0636fc69a83 monero-x86_64-apple-darwin11-5c27fd853.tar.bz2
Hashes match.
Same here:
|
Adding ARM64 MacOSX support requires updating the MacOSX SDK to at least 11.0. Updating the SDK has a number of knock-on effects.
I'm not entirely sure we need a native_clang package, since the Ubuntu 18.04 pkg repos have clang packages up to clang-10 available. We could just use that instead. Looking for feedback here. This branch successfully builds for both x86_64-apple-darwin11 and aarch64-apple-darwin11 but I've only tested the resulting aarch64 binaries since I don't have an Intel Mac.
I haven't yet PR'd my fix for the randomx error. You can find it at hyc/RandomX@b1e0864 and it's incorporated into this branch already.