missing pre-compiled arm binary #1405

shotor · 2020-05-04T19:23:32Z

Problem description

Can't install grpc-tools on arm architecture (raspberry pi) because of missing precompiled binaries.

node-pre-gyp ERR! stack Error: 404 status code downloading tarball https://node-precompiled-binaries.grpc.io/grpc-tools/v1.8.1/linux-arm64.tar.gz

Is it possible to build these at all?

Reproduction steps

npm init
npm install grpc-tools

Environment

Ubuntu 20.04 arm64
10-12-14
n
grpc-tools, possibly more grpc packages - haven't checked yet

Additional context

error /home/repo/grpc/node_modules/grpc-tools: Command failed.
Exit code: 1
Command: node-pre-gyp install
Arguments:
Directory: /home/repo/grpc/node_modules/grpc-tools
Output:
node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using node-pre-gyp@0.12.0
node-pre-gyp info using node@10.20.1 | linux | arm64
node-pre-gyp WARN Using request for node-pre-gyp https download
node-pre-gyp info check checked for "/home/repo/grpc/node_modules/grpc-tools/bin/grpc_tools.node" (not found)
node-pre-gyp http GET https://node-precompiled-binaries.grpc.io/grpc-tools/v1.8.1/linux-arm64.tar.gz
node-pre-gyp http 404 https://node-precompiled-binaries.grpc.io/grpc-tools/v1.8.1/linux-arm64.tar.gz
node-pre-gyp ERR! install error
node-pre-gyp ERR! stack Error: 404 status code downloading tarball https://node-precompiled-binaries.grpc.io/grpc-tools/v1.8.1/linux-arm64.tar.gz
node-pre-gyp ERR! stack     at Request.<anonymous> (/home/repo/grpc/node_modules/node-pre-gyp/lib/install.js:149:27)
node-pre-gyp ERR! stack     at Request.emit (events.js:203:15)
node-pre-gyp ERR! stack     at Request.onRequestResponse (/home/repo/grpc/node_modules/request/request.js:1059:10)
node-pre-gyp ERR! stack     at ClientRequest.emit (events.js:198:13)
node-pre-gyp ERR! stack     at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:565:21)
node-pre-gyp ERR! stack     at HTTPParser.parserOnHeadersComplete (_http_common.js:111:17)
node-pre-gyp ERR! stack     at TLSSocket.socketOnData (_http_client.js:451:20)
node-pre-gyp ERR! stack     at TLSSocket.emit (events.js:198:13)
node-pre-gyp ERR! stack     at addChunk (_stream_readable.js:288:12)
node-pre-gyp ERR! stack     at readableAddChunk (_stream_readable.js:269:11)
node-pre-gyp ERR! System Linux 5.4.0-1008-raspi
node-pre-gyp ERR! command "/usr/local/bin/node" "/home/repo/grpc/node_modules/grpc-tools/node_modules/.bin/node-pre-gyp" "install"
node-pre-gyp ERR! cwd /home/repo/grpc/node_modules/grpc-tools
node-pre-gyp ERR! node -v v10.20.1
node-pre-gyp ERR! node-pre-gyp -v v0.12.0
node-pre-gyp ERR! not ok

The text was updated successfully, but these errors were encountered:

Nilos · 2020-12-22T11:19:30Z

This is also an issue when running inside Docker on the new MacBook with the M1 Chip.

antonimmo · 2021-02-01T10:26:05Z

@Nilos Same issue brought me here 👋

Update: I managed to build and run the container by adding --platform linux/amd64 option to both build and run docker commands. You could alternatively specify platform: linux/amd64 for each docker-compose service.

remko · 2021-02-18T08:42:15Z

Given the introduction of Apple Silicon, and the fact that Docker for Mac on these machines use arm64, does it make sense to re-open this case? Building JS apps that depend on grpc (directly or indirectly) is no longer possible in Docker on Mac (or at least not easy; e.g I can't make the platform trick work with docker-compose, and there are many qemu crashes when running in emulated mode on an M1).

antonimmo · 2021-02-21T11:56:21Z

@remko is right, I'd reopen this one. Some issues I came across with just can't be sorted out that easily.

murgatroid99 · 2021-02-23T23:38:54Z

Our build process runs on GitHub Actions, so adding this is blocked on actions/runner-images#2187.

sri-vathsa · 2021-04-05T23:11:00Z

Can anyone share the instructions on how to build the binaries from the source code?

nicolasnoble · 2021-04-05T23:14:29Z

Can anyone share the instructions on how to build the binaries from the source code?

Basically https://github.com/grpc/grpc-node/blob/master/packages/grpc-tools/build_binaries.sh

This script probably requires changes, but we have no way of testing on any M1 hardware, so you'd be on your own to change it.

sri-vathsa · 2021-04-05T23:39:51Z

@nicolasnoble @murgatroid99 I am actually using trying to install the grpc-tools on AWS Graviton 2 instances. Are there any instructions on how to run the test cases? I have access to both M1 hardware and Graviton instances. I can run the test cases and share my findings/results.

shnhrrsn · 2021-04-16T22:28:56Z

For any on M1 hardware running into this, I was able to workaround it by just forcing the x64 binary and letting Rosetta deal with it:

yarn add grpc-tools --ignore-scripts
pushd node_modules/grpc-tools
node_modules/.bin/node-pre-gyp install --target_arch=x64
popd

Is there any way to include this during normal install?

murgatroid99 · 2021-04-16T23:18:51Z

I think you can pass the --target_arch=x64 argument directly to npm install. I assume it works the same way with yarn, but I don't use yarn.

antonimmo · 2021-04-17T01:32:09Z

@shnhrrsn Also, if you use nvm, you can install x86 node versions following the steps under the "Macs with M1 chip" section of https://github.com/nvm-sh/nvm#macos-troubleshooting. This one solved all my M1 related issues.

circa10a · 2021-07-07T16:14:20Z

ARM binaries should be distributed as part of releases. Especially in 2021. Many cloud providers and personal computer manufacturers are beginning to implement ARM as larger offerings. This is also troublesome for raspberry pi users who would like to run node apps. There isn't a good reason not to pre-compile these binaries, otherwise node marketshare will lessen. I'm already pretty frustrated with it and will be doing less node development as a result.

circa10a · 2021-08-02T15:55:26Z

@murgatroid99
Any chance this group can reconsider?

murgatroid99 · 2021-08-02T17:07:13Z

I said that making this change is blocked in #1405 (comment). Nothing in the linked issue appears to have changed, so it's still blocked.

artursapek · 2021-08-05T04:05:21Z

Our build process runs on GitHub Actions, so adding this is blocked on actions/virtual-environments#2187.

lol that issue is blocking SO many people

ryanberckmans · 2021-08-21T18:36:21Z

We do not intend to distribute ARM binaries for grpc-tools. I suggest generating your code on a different system and then transferring your code to your ARM system instead.

Apple Silicon is here to stay and so is grpc-node. Please reconsider :)

murgatroid99 · 2021-08-22T07:04:55Z

That comment predates the announcement of Apple Silicon. I wasn't talking about Apple Silicon. As I said in my more recent comments #1405 (comment) and #1405 (comment)

Our build process runs on GitHub Actions, so adding this is blocked on actions/runner-images#2187.

sebirdman · 2021-11-06T17:43:37Z

I'm curious why there's not a fallback setting to just compile the binaries if no prebuilt exists. Other packages do something similar.

Edit: it appears node-pre-gyp is being abused to just download files. because we're not actually using a binding to node for those tools?

murgatroid99 · 2021-11-08T18:26:52Z

I disagree with the term "abused". We are using node-pre-gyp to do what it is supposed to do: download binary files for the system the code is running on. It doesn't have a fallback to building locally because it is building protoc, which has an existing build process that uses cmake instead of node-gyp.

sebirdman · 2021-11-08T23:33:45Z

Abused might be a strong word, but it's certainly a tad misused. Their docs say it should "stand between" npm and node-gyp. If we can't fallback using the node-gyp to build, i wonder if there's a good way to wrap node-pre-gyp to just auto run ./build_binaries.sh if that fails?

Here's how i got around this without using Rosetta:

clone this repo
install your grpc-tools with yarn install --ignore-scripts
run the script ./build_binaries.sh in the grpc-tools package
copy your files over, something like this:

#!/bin/bash

cp binaries/arm64/protoc node_modules/grpc-tools/bin/protoc
cp binaries/arm64/grpc_node_plugin node_modules/grpc-tools/bin/grpc_node_plugin

nicolasnoble · 2021-11-09T00:19:32Z

Think of it more as a stop gap, really. The reality being that building protoc from pure source is impossible due to its bootstrapping nature, and so we can't write a node-gyp installer for it, which requires a very flat list of files to compile, with no intermediate compilation process.

Ideally, node-gyp should support cmake, but that's also not happening: nodejs/node-gyp#1590

Additionally, Apple has made it very difficult to build software on their new M1 hardware when using things like github actions, rendering us basically powerless to create working M1 binaries. So we're stuck between a rock and a hard place here, and there's no good solution unfortunately.

NixBiks · 2021-11-11T13:43:33Z

This is also an issue when running inside Docker on the new MacBook with the M1 Chip.

Did anyone find a fix when running in Docker?

We have a team that develops in VSCode Remote Containers (ie. docker containers) - some have M1 chips others intel. Any way to support both?

NixBiks · 2021-11-11T15:06:59Z

@shnhrrsn Also, if you use nvm, you can install x86 node versions following the steps under the "Macs with M1 chip" section of https://github.com/nvm-sh/nvm#macos-troubleshooting. This one solved all my M1 related issues.

@antonimmo That doesn't work when inside a debian-bullseye docker container (on a Mac M1 host) since arch -x86_64 zsh causes an error.

kakyoism · 2021-12-01T22:58:54Z

Looks like we must build grpc-tools ourselves for Apple Silicon?
I'm a beginner here. Could anybody kindly share a pointer to how to build it locally?

maschwenk · 2022-10-06T16:53:18Z

@murgatroid99 I updated to that new version and it worked a treat. Thank you!

hexnaught · 2022-10-07T08:03:54Z

Version 1.11.3 now has Mac M1/arm64 binaries.

Updated too and works great now without any work arounds. Thank you! 🙏

pballester · 2022-10-18T10:57:13Z

Still missing linux arm64 binary 😢

https://node-precompiled-binaries.grpc.io/grpc-tools/v1.11.3/linux-arm64.tar.gz

ghost · 2022-10-23T18:43:33Z

Version 1.11.3 now has Mac M1/arm64 binaries.

Still not available for linux (exp: if you have a docker image):

response status 404 Not Found on https://node-precompiled-binaries.grpc.io/grpc-tools/v1.11.3/linux-arm64.tar.gz

so I've added the following to my dockerfile

RUN npm_config_target_arch=x64 npm i grpc-tools

ravicious · 2022-10-24T10:30:19Z

@source-transformer Excuse me, but how does this work? Assuming you have an arm64 Linux image, how is it able to run x64 binaries? Wouldn't that result in an error anyway?

Or do you just install the package but not actually use its components which require x64 binaries?

ghost · 2022-10-24T13:08:05Z

Or do you just install the package but not actually use its components which require x64 binaries?

Correct - I should have qualified my answer with - you can't actually run grpc-tools on your container. I had tried the following:

npm_config_target_arch=x86 npm i grpc-tools

In the hopes of getting a binary that potentially could be run. But, honestly that was just a shot in the dark as to the correct label/alias for the 32 bit build/architecture. I wasn't sure how to get a listing of supported architectures.

In my case - it isn't necessary to actually run grpc-tools on my docker container - since those tools are "devDependencies" and their output is actually copied to the docker image.

huangjun0210 · 2022-12-15T09:03:40Z

Will it support linux arm64 in the future? I'm looking forward to it.

RedstoneWizard08 · 2022-12-17T02:36:14Z

A while back we made an alternative mirror for this...
https://github.com/ZCube/grpc-precompiled-binaries

huangjun0210 · 2022-12-19T04:32:07Z

A while back we made an alternative mirror for this... https://github.com/ZCube/grpc-precompiled-binaries

How to bulid linux arm64? when I follow with this document: https://github.com/grpc/grpc-node/blob/master/packages/grpc-tools/build_binaries.sh。Error occurred：

...
[ 99%] Linking CXX executable protoc
[100%] Linking CXX executable grpc_node_plugin
/usr/bin/ld: cannot find -lstdc++
collect2: error: ld returned 1 exit status
gmake[2]: *** [grpc_node_plugin] Error 1
gmake[1]: *** [CMakeFiles/grpc_node_plugin.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....
[100%] Built target protoc
gmake: *** [all] Error 2

ldconfig -p | less

        libstdc++.so.6 (libc6,AArch64) => /lib64/libstdc++.so.6
        libstdc++.so (libc6,AArch64) => /usr/local/lib/libstdc++.so
        libssl3.so (libc6,AArch64) => /lib64/libssl3.so

RedstoneWizard08 · 2022-12-30T05:11:31Z

I'll have to look again, it's been a while. I know it was pretty simple, though.

geminiyellow · 2023-01-10T06:07:47Z

2 years later this issue still open

node-pre-gyp info it worked if it ends with ok | 88s
-- | --
86 | node-pre-gyp info using node-pre-gyp@1.0.10 | 88s
87 | node-pre-gyp info using node@16.17.1 \| linux \| x64 | 88s
88 | node-pre-gyp info check checked for "/drone/src/node_modules/grpc-tools/bin/grpc_tools.node" (not found) | 88s
89 | node-pre-gyp http GET https://node-precompiled-binaries.grpc.io/grpc-tools/v1.12.3/linux-x64.tar.gz | 88s
90 | node-pre-gyp ERR! install request to https://node-precompiled-binaries.grpc.io/grpc-tools/v1.12.3/linux-x64.tar.gz failed, reason: read ECONNRESET | 88s
91 | node-pre-gyp ERR! install error

fatihky · 2023-01-17T11:06:40Z

@murgatroid99 and other contributors, are you planning to add support for the arm architecture? if you don't have enough time for that, I'd like to try to add arm support to your library.

murgatroid99 · 2023-01-17T18:38:49Z

I have no immediate plans to add this myself. If someone contributes a build change that produces ARM binaries for grpc-tools, I will accept it and publish them.

RedstoneWizard08 · 2023-01-18T05:45:13Z

As soon as I remember how I did it, I'll integrate it into the build system and make a PR. (I hope that this message showing up multiple times is just a glitch in the GitHub app on my phone.)

RedstoneWizard08 · 2023-01-22T23:11:30Z

@murgatroid99 I just finished a patch. https://github.com/RedstoneWizard08/grpc-node
Successful build: https://github.com/RedstoneWizard08/grpc-node/actions/runs/3981949215/jobs/6826020346

PR: #2333

RedstoneWizard08 · 2023-01-31T01:02:08Z

Merged!

murgatroid99 · 2023-01-31T18:26:34Z

The Linux ARM64 binary is now available in version 1.12.4.

yurks · 2023-03-01T15:08:35Z

Version 1.11.3 now has Mac M1/arm64 binaries.

actually it hasn't: #2378

MehmetSecgin · 2023-03-02T10:01:06Z

Is it planned to support M2 architecture as well?

rburdicksuki · 2024-07-17T18:07:43Z

The latest version is 1.11.3 and it is on NPM.

Which repo are we talking about here? It sounds like folks have gotten grpc-node to work on Mac M series, is there a concise set of instructions I can follow?

murgatroid99 closed this as completed May 4, 2020

murgatroid99 mentioned this issue Apr 5, 2021

Compile grpc-node binaries for ARM from source code #1737

Closed

sri-vathsa mentioned this issue Apr 15, 2021

arm arch grpc-tools installation issue agreatfool/grpc_tools_node_protoc_ts#100

Open

raksooo mentioned this issue Aug 16, 2021

[Feature request] Linux ARM support mullvad/mullvadvpn-app#952

Closed

murgatroid99 reopened this Nov 2, 2021

murgatroid99 mentioned this issue Nov 2, 2021

Can't install grpc m1 Mac #1880

Closed

adf-ncgr mentioned this issue Oct 10, 2022

some possible issues with dev config legumeinfo/gcv-docker-compose#13

Open

ravicious mentioned this issue Oct 18, 2022

Remove grpc-teleterm make target gravitational/teleport#17536

Closed

zepatrik mentioned this issue Oct 28, 2022

Upgrade deprecated grpc-tools to @grpc/grpc-js ory/keto#835

Closed

6 tasks

kameshsampath mentioned this issue Nov 13, 2022

[Question] Is there grpc-tools for linux/arm64 #2274

Closed

gregmagolan-codaio mentioned this issue Nov 30, 2022

Bump grpc-tools binary dependency to v1.11.3 which has pre-compiled M1 binaries stultuss/protoc-gen-grpc-ts#26

Merged

RedstoneWizard08 mentioned this issue Jan 22, 2023

Add support for arm64 linux. #2333

Merged

murgatroid99 closed this as completed Jan 31, 2023

jkbz64 mentioned this issue Mar 8, 2023

Installation on ARM64 - Missing Dependency (linux-arm64) Kong/insomnia#5777

Closed

1 task

stevemer mentioned this issue Feb 24, 2024

Broken at head bshepherdson/grpc-web-bazel-ts-example#23

Open

richard-to mentioned this issue Jun 6, 2024

Update grpc-tools to 1.12.4 to support Linux Arm builds google/mesop#361

Closed

missing pre-compiled arm binary #1405

missing pre-compiled arm binary #1405

Comments

shotor commented May 4, 2020 • edited Loading

Problem description

Reproduction steps

Environment

Additional context

Nilos commented Dec 22, 2020 • edited Loading

antonimmo commented Feb 1, 2021 • edited Loading

remko commented Feb 18, 2021

antonimmo commented Feb 21, 2021

murgatroid99 commented Feb 23, 2021

sri-vathsa commented Apr 5, 2021

nicolasnoble commented Apr 5, 2021

sri-vathsa commented Apr 5, 2021 • edited Loading

shnhrrsn commented Apr 16, 2021

murgatroid99 commented Apr 16, 2021

antonimmo commented Apr 17, 2021

circa10a commented Jul 7, 2021 • edited Loading

circa10a commented Aug 2, 2021

murgatroid99 commented Aug 2, 2021

artursapek commented Aug 5, 2021

ryanberckmans commented Aug 21, 2021 • edited Loading

murgatroid99 commented Aug 22, 2021

sebirdman commented Nov 6, 2021 • edited Loading

murgatroid99 commented Nov 8, 2021

sebirdman commented Nov 8, 2021

nicolasnoble commented Nov 9, 2021

NixBiks commented Nov 11, 2021

NixBiks commented Nov 11, 2021

kakyoism commented Dec 1, 2021

maschwenk commented Oct 6, 2022

hexnaught commented Oct 7, 2022

pballester commented Oct 18, 2022

ghost commented Oct 23, 2022

ravicious commented Oct 24, 2022

ghost commented Oct 24, 2022

huangjun0210 commented Dec 15, 2022

RedstoneWizard08 commented Dec 17, 2022

huangjun0210 commented Dec 19, 2022

RedstoneWizard08 commented Dec 30, 2022

geminiyellow commented Jan 10, 2023 • edited Loading

fatihky commented Jan 17, 2023

murgatroid99 commented Jan 17, 2023

RedstoneWizard08 commented Jan 18, 2023

RedstoneWizard08 commented Jan 22, 2023 • edited Loading

RedstoneWizard08 commented Jan 31, 2023

murgatroid99 commented Jan 31, 2023

yurks commented Mar 1, 2023

MehmetSecgin commented Mar 2, 2023

rburdicksuki commented Jul 17, 2024 • edited Loading

shotor commented May 4, 2020 •

edited

Loading

Nilos commented Dec 22, 2020 •

edited

Loading

antonimmo commented Feb 1, 2021 •

edited

Loading

sri-vathsa commented Apr 5, 2021 •

edited

Loading

circa10a commented Jul 7, 2021 •

edited

Loading

ryanberckmans commented Aug 21, 2021 •

edited

Loading

sebirdman commented Nov 6, 2021 •

edited

Loading

geminiyellow commented Jan 10, 2023 •

edited

Loading

RedstoneWizard08 commented Jan 22, 2023 •

edited

Loading

rburdicksuki commented Jul 17, 2024 •

edited

Loading