Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aarch64 build target for release #764

Closed
neenjaw opened this issue Jul 4, 2023 · 9 comments · Fixed by #797
Closed

aarch64 build target for release #764

neenjaw opened this issue Jul 4, 2023 · 9 comments · Fixed by #797
Labels
kind: build Non-.nim changes that affect the release binary

Comments

@neenjaw
Copy link

neenjaw commented Jul 4, 2023

Hey @ee7 and @ErikSchierboom,

I switched platforms from ubuntu to macOS for my daily driver and noticed that configlet can't be fetched using the included repo script any more. Reading the documentation, I didn't see any method described to build it from source myself as a very poor nim user.

Could you advise?

Thanks

@ErikSchierboom
Copy link
Member

This has been an open issue for a while IIRC. You should be able to use the x86 version by downloading it from an individual release. That said, this is not optimal and something we should fix.

@neenjaw
Copy link
Author

neenjaw commented Jul 4, 2023

@ee7
Copy link
Member

ee7 commented Jul 14, 2023

I've been planning to add aarch64 (AKA arm64) assets for some time. But in the meantime, I thought that fetch-configlet already downloaded the x86_64 asset when running on aarch64 macOS.

It looks like that, here:

case "$(uname -m)" in
x86_64) arch='x86-64' ;;
*686*) arch='i386' ;;
*386*) arch='i386' ;;
*) arch='x86-64' ;;

@neenjaw can you confirm that https://github.com/exercism/configlet/blob/main/scripts/fetch-configlet doesn't work on your machine? And if it doesn't, what happens?

Maybe it was just an intermittent connectivity issue? Or you were using an old fetch-configlet script? There was a breaking change in the asset naming scheme last year (see 57e6c5f), and 6 months after that I stopped supporting old fetch-configlet scripts.

ee7 added a commit that referenced this issue Aug 16, 2023
Add a script to install Zig, and add a job to the build workflow that
uses `zig cc` to cross compile configlet for arm64 (AKA aarch64) Linux.

This means that the next configlet release will have two new release
assets:

    configlet_4.0.0-beta.14_linux_arm64.tar.gz
    configlet_4.0.0-beta.14_linux_arm64.tar.gz.minisig

and extracting the archive will yield the executable:

    $ file ./configlet
    ./configlet: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, stripped

Also support running `nimble build -d:zig` locally.

For now, the new executable has a large comment section:

    $ readelf -p .comment ./configlet

    String dump of section '.comment':
      [     0]  clang version 16.0.6 (https://github.com/ziglang/zig-bootstrap 1dda86241204c4649f668d46b6a37feed707c7b4)
      [...] (repeat for 39 KiB)
      [  8d82]  clang version 16.0.6 (https://github.com/ziglang/zig-bootstrap 1dda86241204c4649f668d46b6a37feed707c7b4)
      [  8deb]  Linker: LLD 16.0.6

Try to strip it later. Our `strip` doesn't support elf64-aarch64 as a
target, and `zig objcopy` doesn't have a `-R, --remove-section` option
currently. But prepare to use `llvm-strip`.

Cross-compiling for arm64 macos currently produces an error:

    CC: ../nimdir/lib/std/sysrand.nim
    /home/runner/.cache/nim/configlet_r/@m..@snimdir@slib@sstd@ssysrand.nim.c:9:10:
    fatal error: 'Security/SecRandom.h' file not found

which is due to a recent commit [1].

See some posts on cross-compiling with Zig [2][3][4] and the support table [5].

Refs: #24
Refs: #122
Refs: #764

[1] 53a75a2 ("nimble, uuid: generate UUIDs via std/sysrand, not pragmagic/uuids", 2023-08-07)
[2] https://ziglang.org/learn/overview/#cross-compiling-is-a-first-class-use-case
[3] https://andrewkelley.me/post/zig-cc-powerful-drop-in-replacement-gcc-clang.html
[4] https://www.uber.com/blog/bootstrapping-ubers-infrastructure-on-arm64-with-zig/
[5] https://ziglang.org/download/0.11.0/release-notes.html#Support-Table
@ee7 ee7 added the kind: build Non-.nim changes that affect the release binary label Aug 16, 2023
ee7 added a commit that referenced this issue Aug 17, 2023
Continue the recent `zig cc` work [1], such that the next configlet
release will have two new release assets:

    configlet_4.0.0-beta.14_macos_arm64.tar.gz
    configlet_4.0.0-beta.14_macos_arm64.tar.gz.minisig

where the archive contains the executable:

    $ file ./configlet
    configlet: Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE|HAS_TLV_DESCRIPTORS>

`configlet uuid` now uses [2] `std/sysrand`, which on macOS uses [3] the
Security Framework. So to produce the macOS arm64 configlet, it's
easiest to cross-compile from macOS x86_64, where the macOSX SDK is
already available. Make the cross-compile and install-zig scripts
support macOS, and add a build job for it.

It looks like the macos-12 GitHub runner has SDK versions 12.3 and 13.1
installed. Use the latest one.

The new executable is about 1.09 MiB, which is notably larger than the
655 KiB macOS x86_64 executable. Possible ways to reduce the size in
the future:

- Enable LTO when `zig ld` supports it [4].

- Compile it natively, when GitHub begins to provide a hosted arm64
  macOS runner [5].

- Compile it natively, using a non-GitHub arm64 macOS runner.

Refs: #24
Refs: #122
Refs: #764

[1] 0e8d665, ".github, config: use Zig to cross-compile arm64 Linux asset", 2023-08-13
[2] 53a75a2, "nimble, uuid: generate UUIDs via std/sysrand, not pragmagic/uuids", 2023-08-07
[3] https://github.com/nim-lang/Nim/blob/v2.0.0/lib/std/sysrand.nim#L235-L256
[4] https://www.github.com/ziglang/zig/issues/8680
[5] https://www.github.com/github/roadmap/issues/528
@ee7 ee7 closed this as completed in #797 Aug 18, 2023
ee7 added a commit that referenced this issue Aug 18, 2023
Continue the recent `zig cc` work [1][2][3], such that the next
configlet release will have two new release assets:

    configlet_4.0.0-beta.14_windows_arm64.zip
    configlet_4.0.0-beta.14_windows_arm64.zip.minisig

where the archive contains the executable:

    $ file ./configlet.exe
    ./configlet: PE32+ executable (console) Aarch64, for MS Windows, 6 sections

The aarch64-windows-gnu target will have Tier 1 Zig support [4].

Also make the cross-compilation jobs and scripts work on Windows, even
though we currently cross-compile the new executable from Linux.

Refs: #24
Refs: #122
Closes: #764

[1] 0e8d665, 2023-08-16, ".github, config: use Zig to cross-compile arm64 Linux asset"
[2] f280445, 2023-08-17, ".github: cross-compile arm64 macOS asset"
[3] a962b18, 2023-08-17, ".github: cross-compile riscv64 Linux asset"
[4] https://ziglang.org/download/0.11.0/release-notes.html#Support-Table
@neenjaw
Copy link
Author

neenjaw commented Aug 19, 2023

So using the script at the link you shared:

curl https://raw.githubusercontent.com/exercism/configlet/main/scripts/fetch-configlet | bash

It seems to download configlet as expected but I still get a bash: line 86: ./bin/configlet: Bad CPU type in executable error which crashes with error code 127.

I think the fetch configlet script is still downloading the inappropriate version. I added this line above line 30 in the fetch-configlet script:

+    *arm*)  arch='arm64'  ;;
     *).     arch='x86-64' ;;

Which appears to work as expected now to fetch a runnable configlet

@ee7
Copy link
Member

ee7 commented Aug 19, 2023

Hey @neenjaw, thanks for following up.

Indeed, we haven't yet updated the fetch-configlet script in either this repo or any track repo. Some rationale:

  1. My understanding was that the existing fetch-configlet script has always worked on arm64 macOS, and it downloaded the x86_64 macOS configlet.
  2. arm64 macOS can run an x86_64 macOS binary.
  3. CI in this repo currently lacks native/emulated integration testing of the non-x86_64 binaries.
  4. The x86_64 macOS binary probably doesn't perform strictly worse when running on arm64 macOS right now, because the arm64 macOS binary is built without link-time optimisation (as our linker doesn't yet support it).

I don't run macOS myself, so I don't know how automatic it is for arm64 macOS to run an x86_64 binary. But I thought it was pretty automatic.

Regarding:

curl https://raw.githubusercontent.com/exercism/configlet/main/scripts/fetch-configlet | bash

The fetch-configlet scripts weren't written with curl | bash in mind. So I want to confirm this isn't a curl | bash thing. Could you please run the below commands, and tell me what happens on your machine?

#!/usr/bin/env bash
git clone https://github.com/exercism/elixir temp-elixir-track
cd temp-elixir-track
bin/fetch-configlet # this script is identical to that in the configlet repo
file bin/configlet
bin/configlet --version
bin/configlet uuid

And could you confirm that you have Rosetta installed? If you didn't have Rosetta installed, but didn't get a good prompt to install it at the time of configlet execution, we should indeed make it a higher priority to modify the fetch-configlet script on each track to download the arm64 binary on arm64 macOS.

@neenjaw
Copy link
Author

neenjaw commented Aug 19, 2023

I have recreated the same error using elixir repository as advised. I don't get a prompt to install rosetta 2.

I manually installed rosetta with /usr/sbin/softwareupdate --install-rosetta, and it does succeed after that.

@ee7
Copy link
Member

ee7 commented Aug 19, 2023

Ok, thanks!

@ee7
Copy link
Member

ee7 commented Aug 19, 2023

But yeah, that's more friction than I assumed. Sorry for the inconvenience. I think we'll make closing #122 a higher priority, then.

In the meantime, feel free to use whichever binary you want :)

(But please let me know if you encounter anything unexpected with the arm64 binary).

@neenjaw
Copy link
Author

neenjaw commented Aug 20, 2023

the arm 64 binary seems to work as expected, no noted issues in the basic usage

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: build Non-.nim changes that affect the release binary
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants