MacStadium sponsorship

[rvagg]

@nodejs/test @nodejs/release @nodejs/tsc

I mentioned in a previous thread that we'd reached an initial agreement with MacStadium to sponsor the Node.js project with some hardware resources. This is great news of course because since our Voxer resources dropped off (because they had staff turnover and didn't realise that the couple of macMinis in their datacenter were dedicated to our use, so when they decommissioned that datacenter I guess they just got packed up and shipped off somewhere), we've been reliant on a single macMini to do everything. So that's been our primary SPOF for a little while now and has been a bit painful for all of us.

The sponsorship arrangement we have for them is initially for 6 months, after which we'll both revisit and assess the value we're providing eachother. I'm confident that we can make this a win/win like we have with other providers. For them, we'll be a source of promotion for their products, we're happy to do that and have been keen for some time now to beef up the promotional value we provide to our generous donors. They also get access to our expertise and we can assist them with testing their infrastructure and provide feedback where necessary.

The current status is that we have an account set up and running, they have provisioned a "private cloud" for us and we have begun the configuration phase. There is a bit of a learning curve in setting up the VMWare vSphere infrastructure but ultimately we get the freedom to deploy the VMs we want in the configuration that makes sense. This new capacity will allow us to greatly expand our macOS coverage, even reaching back to some older versions of the OS that Node.js claims to support.

@jbergstroem, @mhdawson and myself have been working together to understand and deploy hosts on the vSphere system. It's behind a VPN and both the setup and configuration is far from simple. So it's essential that we spread the knowledge about how this works and how we have it set up across a few people so we don't end up with yet another SPOF in the form of an individual.

This is not going to be speedy, but we've already started deploying servers and trying to automate as much of it as possible.

Personally I continue to be proud of the work we've done on the @nodejs/build team to build such an amazing and advanced test and build infrastructure on the back of donated resources. We have so many companies and individuals that have donated resources in the form of ongoing cloud-based infrastructure (compute and other, such as CDN from CloudFlare and mail from Rackspace) and one-off donations such as the early prototype ARM8 machines that ARM Holdings gave us to accelerate our ARM8 support. We've also had many individuals and small companies provide one-off donations in the form of hardware, particularly Raspberry Pis (actual and $$ to purchase them) for our cluster and supporting infrastructure (networking and power gear, etc.). I don't believe there exists a more sophisticated CI infrastructure for an open source project, and I doubt there are many commercial ones that would rival ours either. This is all because of the heavy community focus we have, where we have a focus on investing in Node.js and in the ecosystem around it—which leads us to consider how to form mutually beneficial relationships with all of our partners. I hope we can continue this into the future, although there will be challenges as the ecosystem evolves around us and the key community people (and therefore culture) changes over time. But we'll do our best, and continue to celebrate what we've all done to achieve such amazing success.

Lastly, look out for @gibfahn's talk at Node.js Interactive 2017 for a snapshot of how we're doing. I'm sure he'll talk about many of these things as well as some of the other innovative things happening around @nodejs/build.

[mhdawson]

Spent a few hours with @gdams today. We have the networking setup so that the mac machines can connect out to the internet and can be reached using ssh using the jump box server.

We have images for each of 10.8 through 10.12 and the next step is to build an initial ansible script to configure those machines.

[mhdawson]

FYI @rvagg @jbergstroem

[mhdawson]

We have a number of the machines up and connected to the CI. Now testing them with this temporary job: https://ci.nodejs.org/view/All/job/node-test-commit-osx-macstadium/

[mhdawson]

We are seeing some failures when running on the new machines. Tracking issue here: nodejs/node#17164

[mhdawson]

Also as context. Here is the PR for the ansiable scripts used to configure the machines: #971

[mhdawson]

Things to close out on this

• validate CC is working (@gdams)
• run updated ansible scripts acrosss all of the machines (@gdams)
• land the ansible PR Initial macOS ansible scripts #971
• add 10.11 and 10.12 to regular CI runs (@mhdawson)
• add 10.13 machine to setup
• complete documentation of setup
• presentation to the rest of the build WG on setup.
• check for firewall popups after a few builds (@gdams).

[refack]

@gdams can you report the status of this?
@maclover7 you had a bug to report RE the playbook?

[gdams]

@refack the only issue right now is that ccache is not on the path on jenkins. It looks like I need to add it to the java start command

[sam-github]

Closing as stale, but if anyone wants to take this up feel free to reopen.