installing emsdk on Apple M1 hardware fails

[lpotter]

There is no precompiled emsdk package for Apple's new MacOSX hardware M1

lpotter@Lorns-Mac-mini emsdk % ./emsdk install latest
Error: No tool or SDK found by name 'sdk-releases-upstream-4764c5c323a474f7ba28ae991b0c9024fccca43c-64bit'.

and building emsdk fails:

./emsdk install sdk-upstream-master-64bit
...
Done installing tool 'llvm-git-master-64bit'.
Manifest error: No tool by name 'node-12.18.1-64bit' found! This may indicate an internal SDK error!
Traceback (most recent call last):
File "/Users/lpotter/emsdk/./emsdk.py", line 3082, in
sys.exit(main())
File "/Users/lpotter/emsdk/./emsdk.py", line 3062, in main
success = tool.install()
File "/Users/lpotter/emsdk/./emsdk.py", line 1840, in install
return self.install_sdk()
File "/Users/lpotter/emsdk/./emsdk.py", line 1851, in install_sdk
success = tool.install()
AttributeError: 'NoneType' object has no attribute 'install'

[sbc100]

emsdk only officially supports x86_64 today.

If you would like to try to add similar support for aarch64 macOS that be most welcome. The first level of support where we support building from source should be relatively straight forward and will probably just involve adding versions of node and python for aarch64 macOS. The second level of support where we supply pre-built binaries is likely to be a lot harder as it will involves CI builders to build and upload the packages. There is similar issue for adding support for aarch64 linux: #547.

Regardless, we should certainly be giving a better error message here.

There are two levels of support that we could add

[lpotter]

I could, but I would not know where to begin at add it. Perhaps someone could point me in the right direction, or even a similar git sha.

[sbc100]

BTW, in the mean time you should be able to build from source: https://emscripten.org/docs/building_from_source/index.html

Building from source will also allow you to find and fix any issues with llvm, binaryen and emscripten before dealing with emsdk integration issues.

[lpotter]

I thought using ./emsdk install sdk-upstream-master-64bit built from source?

This builds llvm, but seems to want to download node package.

[bobbydigitales]

Hey @lpotter, if all you want to do is run emscripten on an M1 mac then I just forced it to install the x86_64 version by changing line 112 of emsdk.py to 'machine = "x86_64'. This makes it grab the Intel build for mac which runs fine under Rosetta 2.

[lpotter]

Ok, that's one way to work around it, but it is not ideal in the long term.

[sbc100]

If you would like to work on a more long term solution (perhaps we could do it incrementally starting with pulling arm versions of python and node for example) I think we would need the help of one of you to write and test out a patch. I don't know of any other emscripten developers or users who have this hardware yet.

[bobbydigitales]

The Intel solution is working great for my needs and I don't see much need for a native ARM version. Maybe for now, M1 macs could be redirected to the Intel version with a small message. If that would be helpful, I might be able to write a patch for that so at least it works for now.

[sbc100]

That would be great. We could even link to this bug in case anyone wants to help work on it.

[lpotter]

I can certainly help, since i have hardware. Not sure I know emscripten or it's tools well enough to do the work myself, besides having my hands full with Qt WebAssembly.

[namniav]

In addition to @bobbydigitales's solution, I found that Emscripten can also be automatically installed with Homebrew running under Rosetta:arch -x86_64 brew install emscripten.
If you have no Rosetta installed yet:softwareupdate --install-rosetta.

[redblobgames]

Speaking of homebrew, I can't get openjdk working on arm, but I've tricked it into compiling emscripten for arm by symlinking to the intel/rosetta openjdk:

# install intel version of these dependencies, under rosetta
/usr/local/bin/brew install openjdk yuicompressor

# install arm version of these dependencies
/opt/homebrew/bin/brew install python@3.9 node cmake

# trickery: point to the intel versions of openjdk, yuicompressor
ln -snf /usr/local/opt/openjdk /opt/homebrew/opt/
ln -snf /usr/local/opt/yuicompressor /opt/homebrew/opt/

# install arm version of emscripten from source
/opt/homebrew/bin/brew install -s emscripten --ignore-dependencies

# trickery: point to the arm version of emscripten
ln -snf /opt/homebrew/opt/emscripten /usr/local/opt/

I can't say it's the right way to do it, but it does seem to work so far.

[sbc100]

java is no longer a hard requirement to run emscripten since the npm package of closure can also use the native and js versions.

[jerrans]

You can just patch the emsdk.py file to remove the checks for node and python tools, then install the arm64 versions them manually via brew. You can confirm the binary architecture using lipo:

jerrans-mbp-3:build-web schmidje$ lipo -info `which python3`
Non-fat file: /opt/homebrew/bin/python3 is architecture: arm64
jerrans-mbp-3:build-web schmidje$ lipo -info `which node`
Non-fat file: /opt/homebrew/bin/node is architecture: arm64

emsdk patch

[berkus]

@jerrans thanks, I will test it out.

However, a question:

The following SDKs can be compiled from source:
Manifest error: No tool by name 'node-12.9.1-64bit' found! This may indicate an internal SDK error!
Manifest error: No tool by name 'node-12.9.1-64bit' found! This may indicate an internal SDK error!
         sdk-upstream-master-64bit
         sdk-fastcomp-master-64bit

Which node-12.9.1-64bit does it mean? Is it looking for exact node version on the host machine and that's all?

because

❯ node --version
v15.5.0

❯ file (which node)
/opt/homebrew/bin/node: Mach-O 64-bit executable arm64

there's definitely aarch64 node installed.

[jerrans]

@berkus No that's just the version listed in the emsdk_manifest.json, I'm using 15.5.1 as well.

My patch is probably out of date now, if you just search the emsdk.py script for any instances of

Manifest error: No tool by name

And make sure that you just add a line to the effect of:

        if tool_name == "node-12.9.1-64bit":
          continue

That will eliminate the check for node (you may need to do the same with Python) and allow the installation to proceed.

Also make sure the node path is set to empty:

-  node_path = os.path.join(node_tool.installation_path(), 'bin')
+  node_path = ""

When emsdk uses node it just runs node_path + "node" so with node_path empty it will use the system-installed version.

I have had no problems using v15.5.1 on my M1 system.

[berkus]

Yep, I applied your patch semi-automatically and it seemed to configure. I ran into some other issues with it later though, that's missing "tools/optimize" directory and couldn't build it in the end.

[jerrans]

Ok so I've just tried pulling 1ee4a9c, applying the attached patch to emsdk.py and then running ./emsdk install sdk-upstream-main-64bit

$ node --version
v15.11.0
$ npm --version
7.6.0
$ python3 --version
Python 3.8.2

I did run into an issue at the end:

Installing tool 'emscripten-main-64bit'..
<snip>
dyld: Library not loaded: /opt/homebrew/opt/icu4c/lib/libicui18n.67.dylib
  Referenced from: /opt/homebrew/bin/node
  Reason: image not found

But actually even an npm --version failed with the same error, a brew upgrade npm fixed that.

emsdk.py.txt

[juj]

M1 support landed in #753. That should work without needing to patch, and/or needing to rely on homebrew for anything.

However, there is one gotcha to be aware: detecting M1 is done using python, and if you have installed an Intel-only python that runs under Rosetta 2 emulation, then emsdk will also behave as if it is Intel only. If you are using the version of python that is shipped with the M1 Mac, or if you have installed an ARM64 aware python, then it should detect properly.

You can check if your python interpreter is compiled to ARM64 by running

file /path/to/python3 

Use which python3 to locate the path to python, or combine the commands with e.g.

file $(which python3)

The file tool will print a line with arm64 in it, if the binary is built for native M1 ARM64 execution. (the M1 Mac bundled python will also print intel since it is a fat binary that works natively on both)

Currently node will still be run under Rosetta 2 emulation, because there does not exist an official M1 ARM61 version of Node.js yet: nodejs/build#2474 . (We could work around that by switching to building node.js ourselves, but haven't done that so far)

[akoeplinger]

@juj I'm still running into an error even after #753, I verified my python3 is native:

$ file $(which python3)
/usr/bin/python3: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
/usr/bin/python3 (for architecture x86_64):	Mach-O 64-bit executable x86_64
/usr/bin/python3 (for architecture arm64e):	Mach-O 64-bit executable arm64e

$ ./emsdk install latest
Error: No tool or SDK found by name 'sdk-releases-upstream-89202930a98fe7f9ed59b574469a9471b0bda7dd-64bit'.

[akoeplinger]

Ah looks like it works with ./emsdk install upstream-main-64bit, though that clones and builds llvm from git (via Rosetta emulated clang for some reason).

[juj]

Ah yea, good point - upstream google servers do not package up precompiled versions, but one must compile from source. That might change in the future.

[akoeplinger]

Hm it seems even the locally built llvm and binaryen are still x86_64:

$ file binaryen/main_64bit_binaryen/bin/wasm-opt
binaryen/main_64bit_binaryen/bin/wasm-opt: Mach-O 64-bit executable x86_64

$ file llvm/git/build_main_64/bin/clang
llvm/git/build_main_64/bin/clang: Mach-O 64-bit executable x86_64

$ file python/3.9.2-1_64bit/bin/python3
python/3.9.2-1_64bit/bin/python3: Mach-O 64-bit executable arm64

Only the downloaded python is arm64.

[juj]

That's odd. Any chance did you have previous build directories (main_64bit_binaryen, build_main_64) that would have existed that might have gotten CMake'd under an Intel python? Can you try deleting those build directories and reinstalling? If it still fails, can you capture the build logs to a zip here?

[sbc100]

Given how many reports we are seeing perhaps we should make the x86_64 binaries installable on M1 arm hardware so that "emsdk install latest" at least functions (even if its in emulation mode).

[bvibber]

I've submitted a quick PR as #771 which simply treats arm64 macOS as x86_64 macOS, and works fine with the prebuilt binaries on my M1 MacBook Air. This is not ideal as it fetches the x86_64 node and python.

Ideally all macOS dependencies would be built as universal binaries containing both architectures, or if that's inconvenient for the Unix-y build processes, as separate builds for both architectures made from the build bots.

Note that x86_64 macOS can build for Aarch64/arm64 macOS just fine -- and if you build as universal binaries it will run the resulting binaries for tests (by running the native side, presumably x86_64 on a VM).

[truboxl]

Here's what I did for emscripten to work on an unknown platform:
You need to have python and node executable native to the platform installed
You also need to have a C compiler installed

Then ./emsdk install llvm-git-main-64bit emscripten-main-64bit to build everything else from scratch

My point is that emscripten devs did an excellent job to allow building emscripten from scratch without prior knowledge to any platform, easy on porting jobs...

[dschuff]

Update: I've added arm64 mac builds to the emscripten-releases build system, which is what produces the precompiled binaries for emsdk. I believe with #816 emsdk should now "just work" on arm64 macs. For now it works with tot (and any emscripten-releases hash on the main branch dated after today), and the next tagged release should have it as well.

[sbc100]

Awesome news! Assuming you have done some basic testing on a real M1 mac with tot installed, I think we can close this issue.

[dschuff]

Yes, I tested tot on a real macbook pro.
I'll go ahead and close this. we can reopen or file a new bug if someone discovers any issues.

[bvibber]

Awesome news, thanks all!

[wjureczka]

Hey, I have a question, what are the steps to install emsdk/emcc on macbook m1? Still having trouble going along with instructions from docs.

Commands I follow:

git clone https://github.com/emscripten-core/emsdk.git
cd emsdk
./emsdk install tot
./emsdk activate latest
source ./emsdk_env.sh

EDIT:

I have found error, commands should be:

git clone https://github.com/emscripten-core/emsdk.git
cd emsdk
./emsdk install tot
./emsdk activate tot
source ./emsdk_env.sh

[dschuff]

Glad you figured it out. When we make the next tagged release (2.0.21, which I expect to be in the next couple of days), then it should also work with latest and 2.0.21 (and subsequent).

[penghuazhou]

source ./emsdk_env.sh

It alse cannot work in ubuntu(use paralles to install ubuntu vmware on macbook m1)
parallels@ubuntu-linux-20-04-desktop:~/Desktop/istio/emsdk$ ./emsdk install tot Cloning from https://chromium.googlesource.com/emscripten-releases... Cloning into '/home/parallels/Desktop/istio/emsdk/releases'... remote: Sending approximately 12.88 MiB ... remote: Counting objects: 30, done remote: Finding sources: 100% (30/30) remote: Total 29567 (delta 17000), reused 29565 (delta 17000) Receiving objects: 100% (29567/29567), 12.82 MiB | 3.07 MiB/s, done. Resolving deltas: 100% (17000/17000), done. Fetching latest changes to the branch/tag 'main' for '/home/parallels/Desktop/istio/emsdk/releases'... refs/heads/main Already up to date. Successfully updated and checked out branch/tag 'main' on repository '/home/parallels/Desktop/istio/emsdk/releases' Current repository version: "Mon, 13 Dec 2021 03:34:24 +0000 9059f3a910524083e5046157ce4fa5fb603fb537" error: tool or SDK not found: 'sdk-releases-upstream-9059f3a910524083e5046157ce4fa5fb603fb537-64bit'

[bvibber]

It alse cannot work in ubuntu(use paralles to install ubuntu vmware on macbook m1)

I don't think there are Linux/arm64 binaries yet, just macOS/arm64.

You'll have to build from source; use:

./emsdk install sdk-upstream-main-64bit && ./emsdk activate sdk-upstream-main-64bit

[penghuazhou]

thanks