Add aarch64-apple-darwin target #2099

arbourd · 2021-12-12T01:40:54Z

Adds a build target aarch64-apple-darwin to the release steps.

BurntSushi

I think this also needs to be added to the ci workflow, otherwise macos-arm won't be tested on PRs.

arbourd · 2021-12-15T19:25:50Z

Not 100% sure why this is failing (not a Rust dev):

     Running `/Users/runner/work/ripgrep/ripgrep/target/aarch64-apple-darwin/debug/deps/globset-5ccc9bf3313e73b3`
error: test failed, to rerun pass '-p globset --lib'

Caused by:
  could not execute process `/Users/runner/work/ripgrep/ripgrep/target/aarch64-apple-darwin/debug/deps/globset-5ccc9bf3313e73b3` (never executed)

Guessing we're building globset for aarch64 and trying to execute it? The macOS runners should be able to build arm64 binaries but don't think they'd be able to execute anything.

BurntSushi · 2021-12-15T19:35:48Z

The macOS runners should be able to build arm64 binaries but don't think they'd be able to execute anything.

How come? If that's truly the case, then I think adding macOS/arm to CI is premature.

arbourd · 2021-12-15T19:48:22Z

The Actions runners are amd64. I can easily build a Go or Swift arm64 binary on my intel amd64 machine, but if I try to execute that binary it will fail (unless it's universal, which is basically both the binaries squeezed into the same bin at double the size).

Homebrew uses its own arm64 runner VMs to build ripgrep binaries (it runs tests after that would require the runtime).

If that's truly the case, then I think adding macOS/arm to CI is premature.

True, but I'd still love to see an official binary packaged here, rather than having to rely on just Homebrew/source. I'd have to check that the bin in release.yml actually work though, haha.

BurntSushi · 2021-12-15T19:56:27Z

Basically, managing the release is already really time consuming work. Adding more release targets that are difficult to verify isn't great IMO. I'd be open to it, but I'm really not the right point-of-contact to help debug unfortunately. I'm neither a macOS nor an arm user. So if others want to chip in here and do the leg work to make sure this is all working properly, I'd be willing to bring it in. But that also means testing that the release workflow works (probably in another repo).

For other linux/arm builds, I believe we use Cross to both build binaries and run tests. Ideally we could do that here, although I doubt the infrastructure on macOS exists for such a thing.

Given the popularity of macOS/arm, like I said, I'd be willing to bring this in, but I think others will need to put in the work here.

arbourd · 2021-12-15T20:20:21Z

I'm neither a macOS nor an arm user. So if others want to chip in here and do the leg work to make sure this is all working properly, I'd be willing to bring it in. But that also means testing that the release workflow works (probably in another repo).

Totally understand. I can get the release working on my fork and verify if the binary functions. When you say "testing that the release workflow works" do you mean constantly or just the first time 😅

Ideally we could do that here, although I doubt the infrastructure on macOS exists for such a thing.

I think to get the full workflow in here we'd need actions/runner-images#2187 (comment)

Given the popularity of macOS/arm, like I said, I'd be willing to bring this in, but I think others will need to put in the work here.

Happy to help here. I don't even have an arm64 Mac yet but I will eventually and when I do I wanna be able to curl here.

Thank you for all your work on ripgrep. It's awesome and I use it everywhere ❤️

arbourd · 2021-12-15T20:27:28Z

Updated the macos runner to macos-11 (latest is still 10.15). I don't think that'll fix it but arm64 cross compilation support should be better.

arcsi42 · 2022-01-20T19:38:57Z

.github/workflows/ci.yml

+ - build: macos-arm
+ os: macos-11
+ rust: nightly
+ target: aarch64-apple-darwin


I've tried running the ci workflow without this target specified, so cross isn't used for building, and all the runs have passed so far. Not sure if that's helpful.

Suggested change

target: aarch64-apple-darwin

https://github.com/arcsi42/ripgrep/actions/runs/1725214380

Without the target is it building a universal binary?

Does cargo build universal binaries on Apple silicon? I'm not sure if it does yet, rust-lang/cargo#8875.
But we could use lipo in a macos-11 runner to merge both the x86_64 and arm64/aarch64 builds.
I have successful runs using lipo: https://github.com/arcsi42/ripgrep/actions/runs/1729830482, https://github.com/arcsi42/ripgrep/releases/tag/TEST-0.0.3

arcsi42 · 2022-01-20T20:48:47Z

I have also tried modifying the release.yml workflow and seems to be working with the macos-11 runner. Here is the passing workflow run: https://github.com/arcsi42/ripgrep/actions/runs/1725448592.

And the built artifacts could be tested by someone with an Arm Mac machine: https://github.com/arcsi42/ripgrep/releases/tag/TEST-0.0.2.

At least the file format looks okay:

$ file ./ripgrep-TEST-0.0.2-aarch64-apple-darwin/rg 
./ripgrep-TEST-0.0.2-aarch64-apple-darwin/rg: Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE|HAS_TLV_DESCRIPTORS>

ellioseven · 2022-01-21T02:32:40Z

@arcsi42 Can confirm I was able to get the version from the binary on my M1 Mac:

arcsi42 · 2022-01-21T17:13:46Z

I have also built a universal binary using lipo on a macos-11 runner: https://github.com/arcsi42/ripgrep/actions/runs/1729830482, can be downloaded from https://github.com/arcsi42/ripgrep/releases/tag/TEST-0.0.3.

Not sure what is the best way to integrate it into the release workflow or if a universal binary is needed at all, I just download previously built macos binaries and use lipo to merge them with github actions.

arbourd · 2022-01-21T18:23:32Z

@BurntSushi on the universal binary: #1737 (comment)

arbourd · 2022-10-29T22:27:05Z

I was able to build both amd64/arm64 binaries and run the tests with cargo/cross on macOS 13 with an arm64 machine @BurntSushi

We might be good to go here if the tests pass 🤞🏻

arbourd · 2022-11-07T19:28:54Z

Hey @BurntSushi, I don't mean to bother you, but could you please approve the workflow so I can continue on this?

arbourd · 2022-11-07T19:56:08Z

Thank you! So it's failing because it looks like the target aarch64-apple-darwinisn't present. I had this problem locally as well but I was able to solve it by resolving the rustup target. I'll look into this a bit more 👀

arbourd · 2022-11-07T21:39:41Z

Seems like this is still blocked by Cross because of cross-rs/cross#558. We could also use M1 runners if GitHub ever releases them.

I'll finish this PR up if either of these things ever come to fruition.

unixsurfer · 2023-07-08T13:56:42Z

Does anyone when this will be merged? I would like to use the pre-build package .

BurntSushi · 2023-07-08T14:11:14Z

@unixsurfer Use brew?

Otherwise I don't schedule my free time. It happens when it happens and if it happens. See: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#release

unixsurfer · 2023-07-08T22:20:26Z

@unixsurfer Use brew?

Otherwise I don't schedule my free time. It happens when it happens and if it happens. See: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#release

On my working laptop I can't use brew. I need to package it, using autopkg, and the easiest way is to use a pre-built dmg.

This doesn't add a universal binary, but instead keeps x86-64 and arm as distinct binaries. Fixes #1737, Closes #2099

BurntSushi · 2023-07-09T13:45:07Z

I looked into this, and the fundamental problem here is that I do not know how to run tests for the aarch64-apple-darwin target in CI. It is possible to produce a binary, but I'm not comfortable doing that without also being able to run tests. (This PR also doesn't work at all as-is because you can't use Cross to build macOS targets. You never could do that and I don't see how that would ever work. You'd need a macOS Docker image, which I'm not sure even exists. And Cross doesn't support it.)

So this looks blocked on GitHub providing macOS arm CI runners. I'm therefore going to close this out for now.

bc-geoffl · 2023-10-06T19:26:25Z

what's the latest and greatest way to install ripgrep on apple silicon without homebrew?

Is ripgrep-13.0.0-arm-unknown-linux-gnueabihf.tar.gz the correct binary from releases?

BurntSushi · 2023-10-06T19:35:42Z

@bc-geoffl No, that isn't correct. Look at its name... It has "linux" in it, so it can't be for macOS.

I'm not a macOS user, but if you want to do it without Homebrew, you can install Rust and then build ripgrep from source.

tekumara · 2023-10-06T22:25:39Z

FYI m1 arm runners are now available https://github.blog/2023-10-02-introducing-the-new-apple-silicon-powered-m1-macos-larger-runner-for-github-actions/

BurntSushi · 2023-10-06T22:35:20Z

@tekumara Wow, about time! I had not seen that. I'll try it out and if it works, I'll get binaries available for the ripgrep 14 release.

tekumara · 2023-10-06T23:49:20Z

One gotcha I've just discovered is that the m1 runners are large and not standard sized runners, and so aren't free for open-source projects, and require a billing plan:

Larger runners are only available for organizations and enterprises using the GitHub Team or GitHub Enterprise Cloud plans.
Larger runners are only billed at the per-minute rate for the amount of time workflows are executed on them. There is no cost associated with creating a larger runner that is not being used by a workflow.

BurntSushi · 2023-10-06T23:55:53Z

Sigh.

I do personally have an M2 mac mini now, so I can worst case scenario upload a binary like I do for the Debian package.

arbourd changed the title ~~Add macOS-arm64 build~~ Add aarch64-apple-darwin target Dec 12, 2021

BurntSushi requested changes Dec 13, 2021

View reviewed changes

arbourd requested a review from BurntSushi December 15, 2021 18:21

arcsi42 reviewed Jan 20, 2022

View reviewed changes

Add macOS-arm64 build

c02707a

BurntSushi pushed a commit that referenced this pull request Jul 9, 2023

ci: add macOS-arm64 build

5fdf790

This doesn't add a universal binary, but instead keeps x86-64 and arm as distinct binaries. Fixes #1737, Closes #2099

BurntSushi pushed a commit that referenced this pull request Jul 9, 2023

ci: add macOS-arm64 build

4e614d5

This doesn't add a universal binary, but instead keeps x86-64 and arm as distinct binaries. Fixes #1737, Closes #2099

BurntSushi pushed a commit that referenced this pull request Jul 9, 2023

ci: add macOS-arm64 build

9dea4eb

This doesn't add a universal binary, but instead keeps x86-64 and arm as distinct binaries. Fixes #1737, Closes #2099

BurntSushi pushed a commit that referenced this pull request Jul 9, 2023

ci: add macOS-arm64 build

7af688c

This doesn't add a universal binary, but instead keeps x86-64 and arm as distinct binaries. Fixes #1737, Closes #2099

BurntSushi pushed a commit that referenced this pull request Jul 9, 2023

ci: add macOS-arm64 build

c8a6634

This doesn't add a universal binary, but instead keeps x86-64 and arm as distinct binaries. Fixes #1737, Closes #2099

BurntSushi pushed a commit that referenced this pull request Jul 9, 2023

ci: add macOS-arm64 build

41320ef

This doesn't add a universal binary, but instead keeps x86-64 and arm as distinct binaries. Fixes #1737, Closes #2099

BurntSushi closed this Jul 9, 2023

arbourd deleted the arm64-macos branch July 11, 2023 14:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add aarch64-apple-darwin target #2099

Add aarch64-apple-darwin target #2099

arbourd commented Dec 12, 2021 •

edited

Loading

BurntSushi left a comment

arbourd commented Dec 15, 2021

BurntSushi commented Dec 15, 2021

arbourd commented Dec 15, 2021

BurntSushi commented Dec 15, 2021

arbourd commented Dec 15, 2021

arbourd commented Dec 15, 2021

arcsi42 Jan 20, 2022 •

edited

Loading

arbourd Jan 20, 2022

arcsi42 Jan 21, 2022

arcsi42 commented Jan 20, 2022

ellioseven commented Jan 21, 2022

arcsi42 commented Jan 21, 2022

arbourd commented Jan 21, 2022

arbourd commented Oct 29, 2022

arbourd commented Nov 7, 2022

arbourd commented Nov 7, 2022

arbourd commented Nov 7, 2022

unixsurfer commented Jul 8, 2023

BurntSushi commented Jul 8, 2023

unixsurfer commented Jul 8, 2023

BurntSushi commented Jul 9, 2023 •

edited

Loading

bc-geoffl commented Oct 6, 2023 •

edited

Loading

BurntSushi commented Oct 6, 2023

tekumara commented Oct 6, 2023

BurntSushi commented Oct 6, 2023

tekumara commented Oct 6, 2023

BurntSushi commented Oct 6, 2023

Add aarch64-apple-darwin target #2099

Add aarch64-apple-darwin target #2099

Conversation

arbourd commented Dec 12, 2021 • edited Loading

BurntSushi left a comment

Choose a reason for hiding this comment

arbourd commented Dec 15, 2021

BurntSushi commented Dec 15, 2021

arbourd commented Dec 15, 2021

BurntSushi commented Dec 15, 2021

arbourd commented Dec 15, 2021

arbourd commented Dec 15, 2021

arcsi42 Jan 20, 2022 • edited Loading

Choose a reason for hiding this comment

arbourd Jan 20, 2022

Choose a reason for hiding this comment

arcsi42 Jan 21, 2022

Choose a reason for hiding this comment

arcsi42 commented Jan 20, 2022

ellioseven commented Jan 21, 2022

arcsi42 commented Jan 21, 2022

arbourd commented Jan 21, 2022

arbourd commented Oct 29, 2022

arbourd commented Nov 7, 2022

arbourd commented Nov 7, 2022

arbourd commented Nov 7, 2022

unixsurfer commented Jul 8, 2023

BurntSushi commented Jul 8, 2023

unixsurfer commented Jul 8, 2023

BurntSushi commented Jul 9, 2023 • edited Loading

bc-geoffl commented Oct 6, 2023 • edited Loading

BurntSushi commented Oct 6, 2023

tekumara commented Oct 6, 2023

BurntSushi commented Oct 6, 2023

tekumara commented Oct 6, 2023

BurntSushi commented Oct 6, 2023

arbourd commented Dec 12, 2021 •

edited

Loading

arcsi42 Jan 20, 2022 •

edited

Loading

BurntSushi commented Jul 9, 2023 •

edited

Loading

bc-geoffl commented Oct 6, 2023 •

edited

Loading