Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gitian: add ARM64 Darwin support #8312

Merged
merged 11 commits into from
May 16, 2022
Merged

Conversation

hyc
Copy link
Collaborator

@hyc hyc commented May 4, 2022

Adding ARM64 MacOSX support requires updating the MacOSX SDK to at least 11.0. Updating the SDK has a number of knock-on effects.

  • the new SDK has a newer format for library archives, which the current cctools linker doesn't recognize
  • updating cctools also needs a libtapi library specifically to parse the new library format
  • both of these packages must be built with a native compiler, so the clang stuff from cctools had to be split out into its own package
  • the compiler needs to be at least gcc-8 to build libtapi
  • lots of the depends package definitions were hardcoded to support x86_64-apple, those packages had to be fixed to use $(host) properly
  • using the newer SDK causes a link error / undefined symbol in librandomx

I'm not entirely sure we need a native_clang package, since the Ubuntu 18.04 pkg repos have clang packages up to clang-10 available. We could just use that instead. Looking for feedback here. This branch successfully builds for both x86_64-apple-darwin11 and aarch64-apple-darwin11 but I've only tested the resulting aarch64 binaries since I don't have an Intel Mac.

I haven't yet PR'd my fix for the randomx error. You can find it at hyc/RandomX@b1e0864 and it's incorporated into this branch already.

Copy link
Collaborator

@selsta selsta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested successfully on:

  • Intel 10.15.7
  • Intel 11.6.1
  • ARM 12.4

If someone wants to test the binaries on their own Mac, here are the download links:

Small change that is missing to fix CI:

diff --git a/.github/workflows/depends.yml b/.github/workflows/depends.yml
index 9385338de..2eb7dba74 100644
--- a/.github/workflows/depends.yml
+++ b/.github/workflows/depends.yml
@@ -46,10 +46,14 @@ jobs:
           - name: "x86_64 Linux"
             host: "x86_64-unknown-linux-gnu"
             packages: "gperf cmake python3-zmq libdbus-1-dev libharfbuzz-dev"
-          - name: "Cross-Mac"
+          - name: "Cross-Mac x86_64"
             host: "x86_64-apple-darwin11"
             packages: "cmake imagemagick libcap-dev librsvg2-bin libz-dev libbz2-dev libtiff-tools python-dev python3-setuptools-git"
-            osx_sdk: "10.11"
+            osx_sdk: "11.1"
+          - name: "Cross-Mac aarch64"
+            host: "aarch64-apple-darwin11"
+            packages: "cmake imagemagick libcap-dev librsvg2-bin libz-dev libbz2-dev libtiff-tools python-dev python3-setuptools-git"
+            osx_sdk: "11.1"
           - name: "x86_64 Freebsd"
             host: "x86_64-unknown-freebsd"
             packages: "clang-8 gperf cmake python3-zmq libdbus-1-dev libharfbuzz-dev"
@@ -86,11 +90,11 @@ jobs:
     - name: install dependencies
       run: sudo apt update; sudo apt -y install build-essential libtool cmake autotools-dev automake pkg-config bsdmainutils curl git ca-certificates ccache ${{ matrix.toolchain.packages }}
     - name: prepare apple-darwin11
-      if: ${{ matrix.toolchain.host == 'x86_64-apple-darwin11' }}
+      if: ${{ matrix.toolchain.host == 'x86_64-apple-darwin11' || matrix.toolchain.host == 'aarch64-apple-darwin11' }}
       run: |
         mkdir -p contrib/depends/SDKs contrib/depends/sdk-sources
-        if [ ! -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz ]; then curl --location --fail https://bitcoincore.org/depends-sources/sdks/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz -o contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz; fi
-        if [ -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz ]; then tar -C contrib/depends/SDKs -xf contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz; fi
+        if [ ! -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz ]; then curl --location --fail https://github.com/phracker/MacOSX-SDKs/releases/download/11.3/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz -o contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz; fi
+        if [ -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz ]; then tar -C contrib/depends/SDKs -xf contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz; fi
     - name: prepare w64-mingw32
       if: ${{ matrix.toolchain.host == 'x86_64-w64-mingw32' || matrix.toolchain.host == 'i686-w64-mingw32' }}
       run: |

I did not check yet if the resulting binaries are indeed reproducible.

@hyc
Copy link
Collaborator Author

hyc commented May 5, 2022

I did not check yet if the resulting binaries are indeed reproducible.

We used to skip the otools part of the cctools package because that build was non-deterministic, but that quirk was fixed in clang 7.0.1 and newer so we should be good to go. But yes, after I add your patch we can all build this branch and compare.

@hyc
Copy link
Collaborator Author

hyc commented May 5, 2022

Note to self: still need to delete depends/patches/native_cctools but will do that later

@hyc
Copy link
Collaborator Author

hyc commented May 5, 2022

This macsdk branch is the same, but makes the Mac SDK a regular depends package so it's managed automatically. https://github.com/LMDB/bitmonero/tree/macsdk

Maybe we should work with that going forward.

@hyc
Copy link
Collaborator Author

hyc commented May 6, 2022

I've merged the Mac SDK change into this PR now. So now the SDK is a regular depends package and doesn't need to be manually fetched and copied any more.

Also took a stab at silencing all the libprotobuf linker warnings. Got the majority of them.

@selsta
Copy link
Collaborator

selsta commented May 6, 2022

That means you can also remove this again.

diff --git a/.github/workflows/depends.yml b/.github/workflows/depends.yml
index 2eb7dba74..f20bf949f 100644
--- a/.github/workflows/depends.yml
+++ b/.github/workflows/depends.yml
@@ -49,11 +49,9 @@ jobs:
           - name: "Cross-Mac x86_64"
             host: "x86_64-apple-darwin11"
             packages: "cmake imagemagick libcap-dev librsvg2-bin libz-dev libbz2-dev libtiff-tools python-dev python3-setuptools-git"
-            osx_sdk: "11.1"
           - name: "Cross-Mac aarch64"
             host: "aarch64-apple-darwin11"
             packages: "cmake imagemagick libcap-dev librsvg2-bin libz-dev libbz2-dev libtiff-tools python-dev python3-setuptools-git"
-            osx_sdk: "11.1"
           - name: "x86_64 Freebsd"
             host: "x86_64-unknown-freebsd"
             packages: "clang-8 gperf cmake python3-zmq libdbus-1-dev libharfbuzz-dev"
@@ -89,12 +87,6 @@ jobs:
       run: ${{env.APT_SET_CONF}}
     - name: install dependencies
       run: sudo apt update; sudo apt -y install build-essential libtool cmake autotools-dev automake pkg-config bsdmainutils curl git ca-certificates ccache ${{ matrix.toolchain.packages }}
-    - name: prepare apple-darwin11
-      if: ${{ matrix.toolchain.host == 'x86_64-apple-darwin11' || matrix.toolchain.host == 'aarch64-apple-darwin11' }}
-      run: |
-        mkdir -p contrib/depends/SDKs contrib/depends/sdk-sources
-        if [ ! -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz ]; then curl --location --fail https://github.com/phracker/MacOSX-SDKs/releases/download/11.0-11.1/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz -o contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz; fi
-        if [ -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz ]; then tar -C contrib/depends/SDKs -xf contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz; fi
     - name: prepare w64-mingw32
       if: ${{ matrix.toolchain.host == 'x86_64-w64-mingw32' || matrix.toolchain.host == 'i686-w64-mingw32' }}
       run: |

@hyc hyc force-pushed the macarm2 branch 2 times, most recently from 8cbfcfc to 2110abf Compare May 6, 2022 15:50
@hyc
Copy link
Collaborator Author

hyc commented May 6, 2022

Hashes for this build

81d95b5ae40660f6d7bed09723dbaf09e199ffdebd2ca8c59a23b5c7236c74dc  monero-aarch64-apple-darwin11-2110abfb3.tar.bz2
b4607a69bb282d8fa99fe72659a24b3c0f0aa6f863df212d833279835bba0c85  monero-x86_64-apple-darwin11-2110abfb3.tar.bz2

@hyc
Copy link
Collaborator Author

hyc commented May 6, 2022

The libprotobuf warnings appear to be a long standing problem with Apple/ARM64 builds of libprotobuf protocolbuffers/protobuf#8958 (comment)

@jeffro256
Copy link
Contributor

Tested successfully on:

* Intel 10.15.7

* Intel 11.6.1

* ARM 12.4

If someone wants to test the binaries on their own Mac, here are the download links:

* https://github.com/selsta/monero/suites/6386331257/artifacts/232120463 (ARM)

* https://github.com/selsta/monero/suites/6386331257/artifacts/232120464 (Intel)

Small change that is missing to fix CI:

diff --git a/.github/workflows/depends.yml b/.github/workflows/depends.yml
index 9385338de..2eb7dba74 100644
--- a/.github/workflows/depends.yml
+++ b/.github/workflows/depends.yml
@@ -46,10 +46,14 @@ jobs:
           - name: "x86_64 Linux"
             host: "x86_64-unknown-linux-gnu"
             packages: "gperf cmake python3-zmq libdbus-1-dev libharfbuzz-dev"
-          - name: "Cross-Mac"
+          - name: "Cross-Mac x86_64"
             host: "x86_64-apple-darwin11"
             packages: "cmake imagemagick libcap-dev librsvg2-bin libz-dev libbz2-dev libtiff-tools python-dev python3-setuptools-git"
-            osx_sdk: "10.11"
+            osx_sdk: "11.1"
+          - name: "Cross-Mac aarch64"
+            host: "aarch64-apple-darwin11"
+            packages: "cmake imagemagick libcap-dev librsvg2-bin libz-dev libbz2-dev libtiff-tools python-dev python3-setuptools-git"
+            osx_sdk: "11.1"
           - name: "x86_64 Freebsd"
             host: "x86_64-unknown-freebsd"
             packages: "clang-8 gperf cmake python3-zmq libdbus-1-dev libharfbuzz-dev"
@@ -86,11 +90,11 @@ jobs:
     - name: install dependencies
       run: sudo apt update; sudo apt -y install build-essential libtool cmake autotools-dev automake pkg-config bsdmainutils curl git ca-certificates ccache ${{ matrix.toolchain.packages }}
     - name: prepare apple-darwin11
-      if: ${{ matrix.toolchain.host == 'x86_64-apple-darwin11' }}
+      if: ${{ matrix.toolchain.host == 'x86_64-apple-darwin11' || matrix.toolchain.host == 'aarch64-apple-darwin11' }}
       run: |
         mkdir -p contrib/depends/SDKs contrib/depends/sdk-sources
-        if [ ! -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz ]; then curl --location --fail https://bitcoincore.org/depends-sources/sdks/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz -o contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz; fi
-        if [ -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz ]; then tar -C contrib/depends/SDKs -xf contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.gz; fi
+        if [ ! -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz ]; then curl --location --fail https://github.com/phracker/MacOSX-SDKs/releases/download/11.3/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz -o contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz; fi
+        if [ -f contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz ]; then tar -C contrib/depends/SDKs -xf contrib/depends/sdk-sources/MacOSX${{ matrix.toolchain.osx_sdk }}.sdk.tar.xz; fi
     - name: prepare w64-mingw32
       if: ${{ matrix.toolchain.host == 'x86_64-w64-mingw32' || matrix.toolchain.host == 'i686-w64-mingw32' }}
       run: |

I did not check yet if the resulting binaries are indeed reproducible.

Tested @selsta's binaries successfully for MacOS 10.13.6 (High Sierra). Definitely not a full on stress test or anything, but they run and seem to work just fine at first glance.

@selsta
Copy link
Collaborator

selsta commented May 7, 2022

Also got it working on Intel 10.12:

mbp:Downloads admin$ uname -a
Darwin mbp.local 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64
mbp:Downloads admin$ sw_vers 
ProductName:	Mac OS X
ProductVersion:	10.12.6
BuildVersion:	16G29

@hyc
Copy link
Collaborator Author

hyc commented May 7, 2022

Great!

I tried running a local depends build on my Mac (without gitian) and ran into a bunch of snags re: clang path. Also the version of QT in depends is too old for Mac ARM support. Not sure it's worth worrying about at the moment.

@hyc
Copy link
Collaborator Author

hyc commented May 9, 2022

RandomX PR was merged so all of this is ready to go now. (tevador/RandomX#241)

@hyc hyc force-pushed the macarm2 branch 3 times, most recently from 230b2ca to 76cd2fa Compare May 9, 2022 17:52
hyc added 2 commits May 10, 2022 21:05
Also silence randomx malloc exceptions
All patching should be in _preprocess_cmd. config_cmd should just run
configure and nothing else before it, otherwise $PATH is broken when it runs.

Also remove redundant settings to configure, _autoconf already sets them.
We leave explicit AR_FLAGS settings because the builtin rule uses ARFLAGS,
and so would leave AR_FLAGS unset otherwise.
@hyc
Copy link
Collaborator Author

hyc commented May 10, 2022

My build hashes for everything

bb72a7b61507e1f96d05dfee4a89c626fe4647147d9a962bf77353020e7e5cb5  monero-aarch64-linux-gnu-5c27fd853.tar.bz2
41c1053da090f5ed5fcb708af88bdbf1b0b6e2bb2c52b009a20cfa2f82c9a231  monero-arm-linux-gnueabihf-5c27fd853.tar.bz2
2198d6af777ebd47199c42ca0db1e5d2b05c533eb6c32729866ece1880420f8b  monero-i686-linux-gnu-5c27fd853.tar.bz2
e5e5f8340d42dcaec23e0a34ccdd61bf9387ba08674d0c4b22df7f68ed1807b9  monero-x86_64-linux-gnu-5c27fd853.tar.bz2

6837cc13ed07e21ef008ea39be9102be8bc3d5c05ac7fc71ad6e7251eed9aed3  monero-aarch64-linux-android-5c27fd853.tar.bz2
6279f3c97989077c24e7276d075d8f18244761e68bc456bba202f3c987440632  monero-arm-linux-android-5c27fd853.tar.bz2

7c6272f07282fb283b6b89593c10b3293debd135a5af1cf57a2e638ea49fe77b  monero-x86_64-unknown-freebsd-5c27fd853.tar.bz2

33c0c829659657f5f423c3425098dc43e2e3e3b4e07576abbd90e3a60b9ceeb6  monero-i686-w64-mingw32-5c27fd853.zip
19f3d60c604dd3fa80f5ab0dc5beca49e4bf15ba321d16ddf7eac9417f988308  monero-x86_64-w64-mingw32-5c27fd853.zip

ca250ad0312233fbc2b091cb25574a272289f6681d526a4f8d27a98b5cc72358  monero-aarch64-apple-darwin11-5c27fd853.tar.bz2
c55bb230f9a70bccddd428691be6a751980dc709bef014aa3252e0636fc69a83  monero-x86_64-apple-darwin11-5c27fd853.tar.bz2

Copy link
Collaborator

@selsta selsta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bb72a7b61507e1f96d05dfee4a89c626fe4647147d9a962bf77353020e7e5cb5  monero-aarch64-linux-gnu-5c27fd853.tar.bz2
41c1053da090f5ed5fcb708af88bdbf1b0b6e2bb2c52b009a20cfa2f82c9a231  monero-arm-linux-gnueabihf-5c27fd853.tar.bz2
2198d6af777ebd47199c42ca0db1e5d2b05c533eb6c32729866ece1880420f8b  monero-i686-linux-gnu-5c27fd853.tar.bz2
e5e5f8340d42dcaec23e0a34ccdd61bf9387ba08674d0c4b22df7f68ed1807b9  monero-x86_64-linux-gnu-5c27fd853.tar.bz2

6837cc13ed07e21ef008ea39be9102be8bc3d5c05ac7fc71ad6e7251eed9aed3  monero-aarch64-linux-android-5c27fd853.tar.bz2
6279f3c97989077c24e7276d075d8f18244761e68bc456bba202f3c987440632  monero-arm-linux-android-5c27fd853.tar.bz2

7c6272f07282fb283b6b89593c10b3293debd135a5af1cf57a2e638ea49fe77b  monero-x86_64-unknown-freebsd-5c27fd853.tar.bz2

33c0c829659657f5f423c3425098dc43e2e3e3b4e07576abbd90e3a60b9ceeb6  monero-i686-w64-mingw32-5c27fd853.zip
19f3d60c604dd3fa80f5ab0dc5beca49e4bf15ba321d16ddf7eac9417f988308  monero-x86_64-w64-mingw32-5c27fd853.zip

ca250ad0312233fbc2b091cb25574a272289f6681d526a4f8d27a98b5cc72358  monero-aarch64-apple-darwin11-5c27fd853.tar.bz2
c55bb230f9a70bccddd428691be6a751980dc709bef014aa3252e0636fc69a83  monero-x86_64-apple-darwin11-5c27fd853.tar.bz2

Hashes match.

@jeffro256
Copy link
Contributor

Same here:

bb72a7b61507e1f96d05dfee4a89c626fe4647147d9a962bf77353020e7e5cb5  monero-aarch64-linux-gnu-5c27fd853.tar.bz2
41c1053da090f5ed5fcb708af88bdbf1b0b6e2bb2c52b009a20cfa2f82c9a231  monero-arm-linux-gnueabihf-5c27fd853.tar.bz2
2198d6af777ebd47199c42ca0db1e5d2b05c533eb6c32729866ece1880420f8b  monero-i686-linux-gnu-5c27fd853.tar.bz2
e5e5f8340d42dcaec23e0a34ccdd61bf9387ba08674d0c4b22df7f68ed1807b9  monero-x86_64-linux-gnu-5c27fd853.tar.bz2
f9c3f0db08c10012f91dda875452f6e8699b45895933ddde8854481c9700063c  monero-linux-0.17-res.yml

6837cc13ed07e21ef008ea39be9102be8bc3d5c05ac7fc71ad6e7251eed9aed3  monero-aarch64-linux-android-5c27fd853.tar.bz2
6279f3c97989077c24e7276d075d8f18244761e68bc456bba202f3c987440632  monero-arm-linux-android-5c27fd853.tar.bz2
b71a1403fa747633766921cd3937bc6c9c232af2017366fd1d6164ad5b53f4a8  monero-android-0.17-res.yml

7c6272f07282fb283b6b89593c10b3293debd135a5af1cf57a2e638ea49fe77b  monero-x86_64-unknown-freebsd-5c27fd853.tar.bz2
57b6def1d161115011dc3ab55b68570a9be031d7dbd71db7abd2e6b4925f2a75  monero-freebsd-0.17-res.yml

33c0c829659657f5f423c3425098dc43e2e3e3b4e07576abbd90e3a60b9ceeb6  monero-i686-w64-mingw32-5c27fd853.zip
19f3d60c604dd3fa80f5ab0dc5beca49e4bf15ba321d16ddf7eac9417f988308  monero-x86_64-w64-mingw32-5c27fd853.zip
ec49c129da5874677bcf46dac36e54578418eb925c6cb8d255ab255fce6ed5cb  monero-win-0.17-res.yml

ca250ad0312233fbc2b091cb25574a272289f6681d526a4f8d27a98b5cc72358  monero-aarch64-apple-darwin11-5c27fd853.tar.bz2
c55bb230f9a70bccddd428691be6a751980dc709bef014aa3252e0636fc69a83  monero-x86_64-apple-darwin11-5c27fd853.tar.bz2
3ac99ad1e023f0833e3baa3fd06b85a65c640a42457fa51714bccd08139c1031  monero-osx-0.17-res.yml

@luigi1111 luigi1111 merged commit 44c0470 into monero-project:master May 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants