-
Notifications
You must be signed in to change notification settings - Fork 166
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cross-compile ARMv6 & ARMv7 release binaries (?) #1238
Comments
so I've tested the binaries on some different ARM devices and they work as expected:
Will test more when the next v10 test build comes out. I'll also confirm that they work on ARM64 and might fish around for more ARM devices to try on. I don't see any reason they'd be too different from the native-compiled binaries but I want to have higher confidence in this process. |
Cross-compiling sounds good to me, especially if that's how we build in CI. |
+1 |
For the machines we can containerize that sounds great, but I'd assume that some of them would be harder to get set up (I think Jenkins X requires Kubernetes). Also Jenkins X sounds cool, but it seems entirely focused on building docker apps that get deployed onto a Kubernetes cluster, which doesn't really help us. |
I'm +1, I though about cross compiling releases a number of times but was never able to move forward. This should take care of the issues we have with the Pi 1s on the release Jenkins, and actually release a binary that is more like the one we test in CI. On the down side, we lose the last barrier we have ensuring that node can be compiled natively in a Raspberry Pi. I don't think this is a big issue if we point people towards the new cross compilers (part of #1237) and tell them how to use it. If compiling natively is actually important for anyone, we could add a new subjob to the daily jobs in CI to compile and test (not to On a related thought, this is not likely to happen quickly but I think our CI should move away form the current logic of build and test on each platform. Instead, build like a release and test the resulting binaries on the platforms where they apply (similar to using the current cross compiling logic for everything). We could still have the current jobs for platforms that we don't publish binaries for, and for platforms that are too different (like newer versions) than the ones where we compile the releases. The main advantage of this would be, if we found a good way to manage temporaries, to run CI on a release before promoting. |
IIUC this has been done. |
Sadly, I think it might be time to shift to cross-compiling ARMv6 & ARMv7 release binaries. We currently build ARMv6 on Raspberry Pi 1 B+'s and ARMv7 builds on the Marvell Armada 370/XP devices that we get from Scaleway.
The ARMv7 builds haven't been speedy but are usually there on time and with little trouble. The ARMv6 builds on the other hand are a totally different matter. Depending on freshness of cache and whether the release Pi's are alive at the time, we can get delays in the order of days at the worst of times. Usually it's not this bad and we normally get nightlies out just fine every day. It's not been easy to keep the Pi's happy for this with them regularly freezing and needing a hard reboot—although this seems to have been mostly solved with a firmware upgrade and running Debian 9 (Stretch) and compiling inside Docker. Michael's backup Pi also has had mixed success, often being unresponsive to builds even though it reports to being online. I also recently had to turn down xz compression to
-6
from-9e
to save on memory and time (so much time) on the Raspberry Pis. It doesn't cost very much in .tar.xz size but it's a compromise nonetheless. So ARMv6 binaries are kind of complicated and can get a little crazy sometimes.Now we have a new dilemma. As per #1229 we are bumping binutils minimum version to >=2.26. Unfortunately, unless we jump straight to Debian 9 (Stretch) as our minimum version we have small problem since gcc 4.9.3 is bundled with 2.25. We may be able to get around this, like in #1235, but it's already meant we have to do some gymnastics with our cross compiler and I now have a custom-built one in play for the test infra (for Raspberry Pi 1,2,3 testing at least, not the plain ARMv7 tests we also run in Scaleway). See #1237.
There's also the fact that we don't even test ARMv6 binaries produced the way that we do them for release! We cross-compile for our tests, but native-compile for releases.
So, I'd like us to discuss the pros and cons of cross-compiling our release binaries. I currently have a new cross compiler set up in ci-release to test nightlies and have taken the debian8 and pi1-docker configurations out of action for Node 10+ for now. This is easily reversable of course but we'd need to solve the binutils problem on both of these builders if we go that route. I'll test over the next couple of days with nightlies and report back anything that might be cause for concern.
@nodejs/build
The text was updated successfully, but these errors were encountered: