Publish pre-built binaries for native components

[ofrobots]

This is probably not needed for EAP, but eventually we should start using node-pre-gyp and shipping precompiled binaries to avoid users having to compile at installation time.

[aalexand]

Do we still need this? From our discussion I got impression that Node.js environments typically have the compiler installed since many packages with native components do not ship prebuilt binaries?

[ofrobots]

We do need this, but perhaps not imminently.

Most customers would have the prerequisite necessary to build the native components in the deployment environment, but many won't. Even folks who can compile will still see compile time cost incurred at deployment time, making their deployments slow.

Precompiled binaries improves the overall UX, so we should have them (eventually).

[evanlucas]

Having to compile native addons at build time slows down our builds quite a bit, so this is something that we would definitely benefit from.

[aalexand]

@evanlucas Is the profiler agent your only dependency that has to build the native part and the rest is coming precompiled, or the agent is just making things worse among others? Just curious to which extent shipping prebuilt native components is a standard among Node deps.

[evanlucas]

we currently ship 20 services that use native addons and they all are precompiled. Occasionally, one of our dependencies will not offer a prebuild (like when we bump our node version in our base docker image) and it more than triples our build times. We aren't currently shipping with the profile agent at this point, but I am about to start experimenting with it.

[aalexand]

@nolanmar511 Looking at modules called prebuild and prebuild-ci should be useful for the task.

[nolanmar511]

prebuild-ci seems to require using a personal access token (which would be associated with a github account, rather than a project). Is there a good way around that?

[nolanmar511]

https://github.com/grpc/grpc-node builds pre-compiled binaries for node. Prebuilt binaries are built with kokoro and stored in a GCS bucket
I think following their pattern would work.
That project uses scripts here to build. Scripts called by kokoro are here.

[nolanmar511]

While grpc uses kokoro for prebuilding it's binaries, I'm not certain that kokoro is right for our use case. At least, it might not be the right choice if we want to stick with our current release process.
Kokoro Release builds can be triggered with the stubby API, but there doesn't seem to be a good way to trigger it from github.

Kokoro only triggers continous builds every hour -- this seems like a long time to wait for the binaries to be built. I'm also not certain what would happen if there were another commit added after the release commit but before kokoro polls for new commits, or (assuming we use the tag to decide if we should generate binaries) if kokoro polled after the release commit but before the release was saved.

Since presubmit checks occur before the the release tag is associated with the commit, I don't think using a presubmit kokoro build to build the binaries would be optimal either.

[ofrobots]

/cc @nicolasnoble

[nolanmar511]

grpc seems to build for all combinations of these:

• Node Versions 4, 5, 6, 7, 8, and 9 (and Electron versions 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, and 1.8)
• Mac, Windows, Linux
  • For linux alpine, build all node versions for x64 (nothing else).
• Ia32, x64, arm (arm for linux only)

[aalexand]

@nolanmar511, are there separate pre-built versions per different Linux distributions?

@ofrobots One thing I am curious about is the overall importance of having the pre-built binaries. It seems that setting all this up is a non-trivial effort, both one-time and potentially ongoing maintenance. I am curious if we could defer this work. Building the extension during npm install requires having a C++ compiler but after all maybe it's not that big of a deal.

[ofrobots]

For this module we probably need:

• Node versions 6, 8, 10
• Linux (glibc), Windows, Mac
• x64 only.

It might be a fair amount of work to set this up, we consistently have heard feedback from users about the needed for per-compiled binaries, at least as far as gRPC was concerned. See up-thread aboud the request from @evanlucas for pre-compiled binaries too.

Compiling from source also impacts deployment time on AppEngine.

are there separate pre-built versions per different Linux distributions?

Alpine linux uses musl, so precompiled binaries linked against glibc do not work.

[nicolasnoble]

Sorry, just saw this notification now. Also pinging @murgatroid99 on this thread.

We're currently triggering the builds automatically per pull request to test if the PR aren't breaking the builds. We're also using the Kokoro's API to trigger the builds from any branch. We're spending a bit less than an hour to do all of these builds, but we currently only sharded in the set {MacOS, Linux, Windows}.

It is kind of possible to make glibc-linked binaries work on musl, but it's a pain, and node-pre-gyp supports adding the libc info in the URL so that's less of an issue.

Node provides alpine docker images up to 10, and we're currently building for alpine and musl using a docker script.

We're also supporting arm64 for Linux on gRPC.

Most of our build scripts can be seen here publicly:

https://github.com/grpc/grpc-node/tree/v1.11.x/packages/grpc-native-core/tools/run_tests/artifacts

There's also a few lines of Python internally at Google to hit the Kokoro API that I can't link here.

It is true that we've spent a non-trivial amount of time to write all of this and we're still spending a non-trivial amount of time to debug and nanny all of this. But we strongly believe this is best for the end user experience. I'm open to share any more of our experience if you have more questions :-)

[nolanmar511]

An example of someone wanting pre-built binaries for the sampling heap profiler https://github.com/v8/sampling-heap-profiler/issues/19, as an example when pre-built binaries could be useful.

[kilianc]

@ofrobots for the time being is it possible to have a list of dependencies this module needs when running in a barebone container? I am running alpine-node which is one of the most popular choices and I am struggling to make it pass build time.

[aalexand]

@kilianc What are the build errors?

[kilianc]

@aalexand figured it out. python2==2.7.14-r2 as virtual dep at build time and it works. I noticed that it ignores NODE_PATH when I run it and a local node_modules folder is present.

[BrianChristie]

Is this being worked on?

We would like to use precompiled binaries, specifically compatible with the node:8-alpine and node:10-alpine Docker images (musl).

[nolanmar511]

Closing this issue.

Pre-built binaries are available for Linux and Linux Alpine.
I opened google/pprof-nodejs#33 and google/pprof-nodejs#34 for getting pre-built binaries for Mac and Windows, respectively.