ARM builds

[rvagg]

I believe ARM is an important target but I'm unsure the best way to tackle CI with it. We either need to find some beefy ARM cpus somewhere that don't take 12h to compile Node or do a cross-compile then push to actual boxes to test or a distcc as @mmalecki has suggested or perhaps some crazy virtualization?

Need some input from cleverer people than me on this.

/cc @wolfeidau

[wolfeidau]

At ninjablocks we are using the a couple of odroid XU boxes (8 core Arm A15 processor) as our build agents, each has 2gig of ram and flash based storage. These systems cost about $200 each.

Build time for NodeJS is approx. 5 minutes, which is better than the 45 minutes on either beagle bone or RaspberryPi...

In our case we run Ubuntu LTS on all our ARM based systems as they are now providing security updates in line with the x86 distribution, this was added when they released 14.04.

We found native builds where way more reliable and many times faster than QEMU arm emulation offered by Amazon, last time I checked performance was worse than Raspberry Pi.

Hosted ARM is currently pretty wild west, with very few commercial services, hopefully this will change once AMDs new ARM based Opterons go on sale later this year.

Look forward to tested and supported ARM NodeJS builds!

[mmalecki]

So my idea here was to connect a few A20 boards (dual core, 1 GB RAM) over distcc and create a build cluster. So instead of having a single beefy machine as @wolfeidau does, cluster commodity hardware.

We don't have any data on how long such a cluster will take to compile node, or even how many machines this cluster will exactly have, but we're working on figuring that out.

[rvagg]

@mmalecki I don't know much about distcc but is it going to give us a standard build as if you were to compile on a single machine?

[mmalecki]

@rvagg yup! distcc was build to do exactly that, so it takes a bunch of precautions to make sure that's true, like not relaying on local header files.

Here's an excerpt from distcc overview:

distcc is a program to distribute builds of C, C++, Objective C or Objective C++ code across several machines on a network. distcc should always generate the same results as a local build, is simple to install and use, and is usually much faster than a local compile.

distcc does not require all machines to share a filesystem, have synchronized clocks, or to have the same libraries or header files installed. They can even have different processors or operating systems, if cross-compilers are installed.

[andreas-marschke]

In my mind another option to get or find hardware would be asking Linux distributions that are already deploying/building their complete stack and packages on arm.

See for example Debian who support an extensive amount of "ports" of the debian distribution on different hardware. They also use donated or hosted hardware from community members and maintainers.

As luck would have it they already support nodejs on a lot of "non-standard" arches:
https://packages.debian.org/search?keywords=nodejs
(as of this writing):

• amd64
• armel
• armhf
• i386
• kfreebsd-amd64
• kfreebsd-i386
• mipsel

So if there was a supportive developer or even the package maintainers themselves we could ask for assistance that would certainly help this task along.

Any opinions?

[andrewlow]

Debian Node builds today are useful (as they exist) but they are built in a non-standard way if we compare it to binaries created by Joyent.

They dynamically link in the openssl libraries (probably a good idea) and also libv8 (a less good idea given some of the patches Node has made to V8 in the 0.10.x stream).

So you'd really be asking them to do an entirely different build than when they do today.

There is also the gcc compile farm https://gcc.gnu.org/wiki/CompileFarm

[andreas-marschke]

@andrewlow Thanks for pointing out gcc.gnu.org. Haven't had it on my radar.

And as you were pointing out the differences between Debians build of node vs. joyent:

Wouldn't it help the nodejs project to get some queues from the package maintainers of the larger distributions? Asking them what they see is a pain, where they'd like things improved. What goes and doesn't go for their distribution as in upstream changes etc. etc. I feel like it's not for nothing that nodejs on Debian is lagging behind 4/5(!) versions behind the most recent stable releases.

The situation on CentOS however is largely different. Having Red Hat as strong contributor is a good thing. [as of this writing] Comparatively they're only a merely version behind most recent stable.

I don't know how the rest of the core team or community sees this but being an administrator for a long time now trying to support and plan a larg-ish node infrastructure seems very much of a hassle unless theres is some support from the distribution or upstream.

[dshaw]

@andreas-marschke Convincing the Debian maintainers to "do it the Node way" is challenging at best. They have a very specific way that they like to do things. If you're interested in the history, search for "debian" on the Node.js mailing list.

Many Linux admins choose to use Chris Lea's PPA which he's moved over to NodeSource. https://github.com/nodesource/distributions

[wolfeidau]

Just a suggestion but may want to ping and chat with this startup http://labs.online.net/ we have started using them for build agents, they are in beta and by all accounts the product performs well.

Saves the need for hosting your own ARM machines.

Need to verify but they may also support docker, we have it running on our much smaller ARM platform, so builds for debian will be easier and in line with your existing tooling.

[bnoordhuis]

Closing, we have ARM machines (including arm64 - woo!) hooked up.

[wolfeidau]

👍 awesome effort!