Try to make a build that works for M1 Macs

[bovine3dom]

Let's see if this compiles.

Then we'll need to find a guinea pig.

Fixes #2.

[bovine3dom]

the --cpu flag doesn't seem to do anything to Nim. Maybe it needs some extra configuration: https://nim-lang.org/docs/nimc.html#crossminuscompilation

https://developer.apple.com/documentation/xcode/building_a_universal_macos_binary#3618377

[bovine3dom]

the -o: option doesn't seem to be able to override the project.

[Rummskartoffel]

--cpu: definitely does work sometimes, at least. I've successfully used it to compile x32 Windows binaries on a x64 PC.

[timotheecour]

M1 support in nim is recent, see nim-lang/Nim#16211

[bovine3dom]

Thanks for the heads up. I'd be happy to volunteer Tridactyl users as guinea pigs but:

• Tridactyl has about ~10k daily active users
• about 15% of those are OSX users
• about 5-10% of users use the native messenger (this project)

So I think it is safe to say we have approximately zero M1 users. On the other hand shipping a totally broken M1 binary would be a good way of testing that theory ; )

Thanks for your work on Nim, by the way. It's been a pleasure to write in.

[ringabout]

Just like our devel branch, v1.4.8 is built using csources_v1, which means you can use it on Apple M1 chips.

Nim 1.4.8 has already added supports for M1 Macs.

Ref https://forum.nim-lang.org/t/8031

[bovine3dom]

Thanks for the heads up!

I think we just need to check the output filenames, the install scripts and maybe bump the version of Nim used in the runners, then it should just work.

[rogchap]

Wondering if there is any progress here? You have at least 1 M1 user 😉

[bovine3dom]

I'll try to remember to do what I said I would in the comment above and then should have some binaries for you to test

If I forget please poke me :)

[mrafaeldie12]

Would be happy to be a guinea pig for this :) tridactyl/tridactyl#4298 (comment)

[winterscar]

Weekly reminder to merge this :)

[bovine3dom]

Run mv native_main native_main-x86
  mv native_main native_main-x86
  nimble build -d:danger -d:release --opt:speed -Y --cpu:arm64
  mv native_main native_main-arm64
  lipo -create -output native_main native_main-x86 native_main-arm64
  shell: /bin/bash -e {0}
  env:
    PATH: /Users/runner/.nimble/bin:/usr/local/lib/ruby/gems/3.0.0/bin:/usr/local/opt/ruby@3.0/bin:/usr/local/opt/pipx_bin:/Users/runner/.cargo/bin:/usr/local/opt/curl/bin:/usr/local/bin:/usr/local/sbin:/Users/runner/bin:/Users/runner/.yarn/bin:/Users/runner/Library/Android/sdk/tools:/Users/runner/Library/Android/sdk/platform-tools:/Library/Frameworks/Python.framework/Versions/Current/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/usr/bin:/bin:/usr/sbin:/sbin:/Users/runner/.dotnet/tools:/Users/runner/.ghcup/bin:/Users/runner/hostedtoolcache/stack/2.9.1/x64
  Verifying dependencies for tridactyl_native@
      Info: Dependency on tempfile@>= 0.1.0 already satisfied
  Verifying dependencies for tempfile@0.1.7
   Building tridactyl_native/native_main using c backend
fatal error: /Applications/Xcode_14.0.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo: native_main-x86 and native_main-arm64 have the same architectures (x86_64) and can't be in the same fat output file
Error: Process completed with exit code 1.

Need to investigate further when I get a chance, for some reason --cpu:arm64 appears to still be producing an x86_64 build

[ringabout]

nim-lang/Nim#21142 includes fixes for "ucpu fix-up for arm64 value on macos m1"

Would it help?

You can try it with https://github.com/nim-lang/nightlies/releases/tag/latest-devel

[bovine3dom]

Thanks for the suggestion. It doesn't seem to have made a difference :(

[ringabout]

Sorry, probably you need to follow https://summarity.com/nim-2-m1

task release_clang, "Build a production release (macOS)":
  --verbose
  --forceBuild:on
  --cc:clang
  --define:release
  --deepcopy:on
  --cpu:arm64
  --passC:"-flto -target arm64-apple-macos11" 
  --passL:"-flto -target arm64-apple-macos11"
  --hints:off
  --outdir:"."
  setCommand "c", "hello.nim"

note the cpu and -target parameters

[bovine3dom]

Thanks! That seems like a pain to debug without access to a MacOS machine - with each change I have to push the code and then wait ~10 minutes for the CI to pass/fail.

If someone could get it working locally first that would be a great help (or post me an M1/M2 machine ;) )

[bovine3dom]

Steps to reproduce for anyone who wants to help on an x86 macOS machine:

# clone the repo
nimble build -d:danger -d:release --opt:speed -Y # make an x86 binary
mv native_main native_main-x86
nimble build -d:danger -d:release --opt:speed -Y --cpu:arm64 # this is the line that needs fixing - it still makes an x86 binary
mv native_main native_main-arm64
lipo -create -output native_main native_main-x86 native_main-arm64 # this line errors because it realises both binaries are x86

[bovine3dom]

We're telling setup-nim-action to give the devel version of Nim, but we're only getting version 1.6. I guess we just need to wait a little while until Nim 2 is released and then maybe this will work.

[tatsushid]

Hi,

I tried to run native_main on my M1 Mac and found this PR but the compile option is still not stable, right? I also have an Intel Mac so I tested some compile options and tried to run the built object on both machines

My M1 machine OS is macOS 13.4 and Intel one is macOS 12.6.6. Nim versions on them are same, 1.6.12, installed by Homebrew. Here is the Intel Mac Nim version output

nim --version
Nim Compiler Version 1.6.12 [MacOSX: amd64]
Compiled at 2023-03-10
Copyright (c) 2006-2023 by Andreas Rumpf

active boot switches: -d:release -d:nimUseLinenoise

First, as far as I tested the command below,

nimble build -d:danger -d:release --opt:speed -Y

it built a machine's native architecture binary. It means that it made x86_64 binary on Intel Mac while it made arm64 binary on M1 Mac

Next, I tested the command

nimble build -d:danger -d:release --opt:speed -Y --cpu:arm64

but --cpu:arm64 didn't affect the built result. It simply made a machine's native architecture binary regardless of the option

As --cpu and also --os options didn't work, I tried --passC and --passL options with -target argument. After several tries by changing options, these commands seem to build a proper binary regardless of machine's architecture on which a command runs

To build x86_64 binary

nimble build --verbose -d:danger -d:release --opt:speed -Y '--passC:"-target x86_64-apple-macos10.12"' '--passL:"-target x86_64-apple-macos10.12"'

To build arm64 binary

nimble build --verbose -d:danger -d:release --opt:speed -Y '--passC:"-target arm64-apple-macos11"' '--passL:"-target arm64-apple-macos11"'

To run these commands in a terminal, --passC option and its arguments have to be surrounded by ' to pass it to background nim command properly. Without it, it causes a parse option error. It is same as --passL option

When a binary was made, its architecture could be checked like

$ file native_main.x86_64
native_main.x86_64: Mach-O 64-bit executable x86_64
$ lipo -archs native_main.x86_64
x86_64

$ file native_main.arm64
native_main.arm64: Mach-O 64-bit executable arm64
$ lipo -archs native_main.arm64
arm64

The universal binary could be made like

$ lipo -create -output native_main native_main.x86_64 native_main.arm64
$ lipo -archs native_main
x86_64 arm64
$ file native_main
native_main: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64Mach-O 64-bit executable arm64]
native_main (for architecture x86_64):	Mach-O 64-bit executable x86_64
native_main (for architecture arm64):	Mach-O 64-bit executable arm64

I tested the built universal binary on both M1 and Intel Mac and confirmed it worked properly

Hope this will help at making an auto build with GitHub

[bovine3dom]

That seems to work! @tatsushid could you check that the built binary works? https://github.com/tridactyl/native_messenger/suites/13576524430/artifacts/747719133

Thanks so much for your help :)

[tatsushid]

I tested it but it seems to only include x86_64 binary. Here is the result

% unzip -l native_main-macOS.zip
Archive:  native_main-macOS.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
   873848  06-13-2023 17:36   native_main
---------                     -------
   873848                     1 file
% ls
native_main           native_main-macOS.zip
% file native_main
native_main: Mach-O 64-bit executable x86_64
% lipo -archs native_main
x86_64

From the action log, this seems to be caused by the line 58 in .github/workflows/compile.yml. It overwrites the universal library made in the steps before I think

[bovine3dom]

Sorry, that was a silly bug, @tatsushid

I think the latest artifact - https://github.com/tridactyl/native_messenger/suites/13659414315/artifacts/754327926 - should be a real universal binary. Promisingly, it's twice the size of the other ones :)

Could you check for me? Thanks again for all your help with this.

[tatsushid]

Hi,

I downloaded the file and tested it. Yes, it worked!

Thank you for all the fixes!

Here is the my terminal output

% unzip -l native_main-macOS.zip
Archive:  native_main-macOS.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
  1816904  06-16-2023 14:21   native_main
---------                     -------
  1816904                     1 file
% unzip native_main-macOS.zip
Archive:  native_main-macOS.zip
  inflating: native_main
% chmod +x native_main
% file native_main
native_main: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64]
native_main (for architecture x86_64):	Mach-O 64-bit executable x86_64
native_main (for architecture arm64):	Mach-O 64-bit executable arm64
% lipo -archs native_main
x86_64 arm64
% ~/Devel/sandbox/native_messenger/gen_native_message.py cmd..version | ./native_main | cut -b4-
{"cmd":"version","version":"0.3.8","code":0}

[bovine3dom]

@tatsushid could you check the release build too, just in case? https://github.com/tridactyl/native_messenger/releases/download/0.4.0/native_main-macOS

The file size is much smaller. Probably due to the lack of debug symbols but it's still worth checking...

[tatsushid]

Hi,

I tested the release build on M1 Mac by running :nativeinstall command in Tridactyl then run the command in a terminal. I omitted the version string in the command because it downloads the older version

curl -fsSl https://raw.githubusercontent.com/tridactyl/native_messenger/master/installers/install.sh -o /tmp/trinativeinstall.sh && sh /tmp/trinativeinstall.sh

Then I ran :native in Tridactyl. It showed

# Native messenger is correctly installed, version 0.4.0 

so the latest binary looks working

I also checked the checksums of the automatically downloaded file and your link one. Both checksums were same so everything should be OK

[tatsushid]

Forgot to post the binary detail

% file ~/.local/share/tridactyl/native_main
/Users/demachitatsushi/.local/share/tridactyl/native_main: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64]
/Users/demachitatsushi/.local/share/tridactyl/native_main (for architecture x86_64):	Mach-O 64-bit executable x86_64
/Users/demachitatsushi/.local/share/tridactyl/native_main (for architecture arm64):	Mach-O 64-bit executable arm64

It is a proper universal binary

[bovine3dom]

Marvellous, thank you so much!