ansible: add 1015 orka test macs #2253

AshCripps · 2020-03-26T17:15:20Z

Finally got the firewall set up so heres the draft PR whilst I finish setup.
Couple things need doing.

I Need to make a note of the NAT in place for anyone else planning to add new machines
These machines need reansibling to pick the correct hostname. (they are created off an ansibled image)
Set up second release machine
Work to set up some 10.14 machines (open ticket with macstadium about issues)

mhdawson · 2020-03-26T18:07:14Z

ansible/inventory.yml

@@ -153,6 +153,9 @@ hosts:
        macos10.11-x64-2: {ip: 207.254.58.162, port: 10004, user: administrator}
        macos10.12-x64-1: {ip: 207.254.58.162, port: 10001, user: administrator}
        macos10.12-x64-2: {ip: 207.254.58.162, port: 10002, user: administrator}
+        macos10.15-x64-1: {ip: 199.7.167.99, port: 8823, user: administrator}
+        macos10.15-x64-2: {ip: 199.7.167.100, port: 8823, user: administrator}
+        macos10.15-x64-3: {ip: 199.7.167.99, port: 8824, user: administrator}


rvagg · 2020-03-27T02:51:28Z

Maybe we should brand these in Jenkins as "Orka", so macos10.15_orka-x64-1. Gives us clarity and adds an extra bit of branding for MacStadium.

AshCripps · 2020-04-02T15:51:21Z

Ive managed to sort out the 10.14 issues and get a working VM which I will now create an image of. Ive redployed all machines to remove an 1015 machine (thought 3 here was a bit overkill) and to turn off VNC access on the VMs.

AshCripps · 2020-04-07T15:19:12Z

Ive added a release machine, reansibled to get the correct hostname and installed full xcode on it.

The remaining release setup is:

Add Release pub key (Added ive kept test on there for now while im still setting up but can someone in release please remove it when doing the other setup)
The other steps in MANUAL_STEPS.md (I dont have access to copy across .ssh and the signing secrets)

AshCripps · 2020-04-14T08:18:31Z

@nodejs/build-infra whats the process about adding IP to the jenkins firewall? is just the IP of the machine needed or the ports aswell? If its just the IPs does it make sense to add all three orka IPs to both jenkins hosts to make it easier to add new machines at our whim?

rvagg · 2020-04-14T09:21:05Z

@AshCripps just the IP addresses, and yes, we should just add all 3 I reckon. That'd be 199.7.167.99, 199.7.167.100 and 199.7.167.101?

AshCripps · 2020-04-14T09:25:27Z

@rvagg yes please those 3. Ill try to create some jenkins instances for test today.

rvagg · 2020-04-14T09:47:03Z

done, added entries:

-A jnlp -s 199.7.167.99/32 -m comment --comment macstadium-orka-cluster-1 -j ACCEPT
-A jnlp -s 199.7.167.100/32 -m comment --comment macstadium-orka-cluster-2 -j ACCEPT
-A jnlp -s 199.7.167.101/32 -m comment --comment macstadium-orka-cluster-3 -j ACCEPT

AshCripps · 2020-04-14T11:05:31Z

test ci build - https://ci.nodejs.org/job/node-test-commit-osx-AshCripps/11/

EDIT: xcode got corrupted on 10.15 I think so redownloading before I rerun the build.

AshCripps · 2020-04-14T16:06:44Z

@rvagg @nodejs/build I think we need to rethink our migration of our vsphere systems.

ATM we only need osx10.10 for the building of release sources (#2282) so I tried to pull down an image of the test machine today. Not only is there a minefield you have to go through due to the outdatedness of the vsphere when i finally started the download it was going to take over a year according to my rough maths!

So IMO that option is out and leave us with a couple of options:

We move the release source building to modern OSX (I have no idea what this entails)
or
Contact macstadium and see if they can get us a 10.10 image in our orka setup and we go from there.

LMK your thoughts

richardlau · 2020-04-14T16:14:30Z

We switched release source building from macOS to Linux for 12+ in #1777 so if we were to migrate away from macOS 10.10 it would make more sense to use the same Linux nodes as 12+ rather than attempt to go to another version of macOS.

sam-github · 2020-04-14T16:31:54Z

Sounds like getting #1777 back onto 10.x would be the best way, and also simplify (hopefully) the range of build variations we have.

rvagg · 2020-04-15T03:14:15Z

Is it worth considering hanging on to 10.10 in the old vsphere stuff until April 2021 to see out v10.x?

The other options are probably not high risk. We originally switched to linux for tarballs because bsd tar was producing tarballs that weren't as compatible as gnu tar, so we should increase compatibility by switching to linux for the sources files. I haven't heard of any issues with the switch between 10 to 12, only the opposite - complaints went away. You never know what weird edge cases people out there are relying on that we might trigger, but we could make a switch like we did for notarization and just include a big notice in the next 10.x that they've changed but we don't expect problems.

AshCripps · 2020-04-15T09:39:51Z

Is it worth considering hanging on to 10.10 in the old vsphere stuff until April 2021 to see out v10.x?

IMO I dont think its worth, seems a bit silly to hold on to such aging hardware for only 1 machine and macstadium would like to get rid of it sooner rather than later.

AshCripps · 2020-04-16T10:54:06Z

Hoping to add the 1015 and 1014 machines to the CI jobs today.
Test Runs:
Master - https://ci.nodejs.org/job/node-test-commit-osx-AshCripps/15/
12.x-staging - https://ci.nodejs.org/job/node-test-commit-osx-AshCripps/16/
10.x-staging - https://ci.nodejs.org/job/node-test-commit-osx-AshCripps/17/

Still need someone to make the release CI agent and do the final setup steps for the second 1015 release machine

EDIT: all build passed except 12.x on 1015 which failed to a known flaky test which is in process of being removed - nodejs/node#32870. Im going to add these machines to the CI jobs now

rvagg · 2020-04-21T01:50:12Z

ansible/plugins/inventory/nodejs_yaml.py

@@ -47,7 +47,7 @@
    # providers - validated for consistency
    'provider': ('azure', 'digitalocean', 'ibm', 'joyent', 'linuxonecc',
                 'macstadium', 'marist', 'mininodes', 'msft', 'osuosl',
-                 'rackspace', 'requireio', 'scaleway', 'softlayer', 'voxer',
+                 'orka', 'rackspace', 'requireio', 'scaleway', 'softlayer', 'voxer',


this is breaking master since the orka hosts were merged into secrets

Exception: Failed to parse host: Invalid provider: orka

I can fix manually to run Ansible but can we get this merged and have follow-up work later?

AshCripps mentioned this pull request Mar 26, 2020

Set up our ORKA environment #2236

Closed

10 tasks

mhdawson reviewed Mar 26, 2020

View reviewed changes

rvagg approved these changes Mar 27, 2020

View reviewed changes

mhdawson approved these changes Mar 31, 2020

View reviewed changes

AshCripps added 2 commits April 2, 2020 16:49

ansible: add 1015 orka test macs

12741e6

ansible: add 10.14 and "orka" name

1b743b5

AshCripps force-pushed the add-orka-mac branch from 51b6d73 to 1b743b5 Compare April 2, 2020 15:49

ansible: add second 10.15 release machine

c93a9c2

AshCripps force-pushed the add-orka-mac branch from 951dfdf to c93a9c2 Compare April 7, 2020 14:22

ansible: add second 10.14 machine

ecafbfc

AshCripps changed the title ~~[WIP] ansible: add 1015 orka test macs~~ ansible: add 1015 orka test macs Apr 16, 2020

rvagg reviewed Apr 21, 2020

View reviewed changes

AshCripps merged commit 5542b14 into nodejs:master Apr 21, 2020

richardlau mentioned this pull request May 19, 2020

ansible: add iinthecloud provider #2325

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ansible: add 1015 orka test macs #2253

ansible: add 1015 orka test macs #2253

AshCripps commented Mar 26, 2020 •

edited

Loading

mhdawson Mar 26, 2020

rvagg commented Mar 27, 2020

AshCripps commented Apr 2, 2020

AshCripps commented Apr 7, 2020 •

edited

Loading

AshCripps commented Apr 14, 2020

rvagg commented Apr 14, 2020

AshCripps commented Apr 14, 2020

rvagg commented Apr 14, 2020

AshCripps commented Apr 14, 2020 •

edited

Loading

AshCripps commented Apr 14, 2020

richardlau commented Apr 14, 2020

sam-github commented Apr 14, 2020

rvagg commented Apr 15, 2020

AshCripps commented Apr 15, 2020

AshCripps commented Apr 16, 2020 •

edited

Loading

rvagg Apr 21, 2020

ansible: add 1015 orka test macs #2253

ansible: add 1015 orka test macs #2253

Conversation

AshCripps commented Mar 26, 2020 • edited Loading

mhdawson Mar 26, 2020

Choose a reason for hiding this comment

rvagg commented Mar 27, 2020

AshCripps commented Apr 2, 2020

AshCripps commented Apr 7, 2020 • edited Loading

AshCripps commented Apr 14, 2020

rvagg commented Apr 14, 2020

AshCripps commented Apr 14, 2020

rvagg commented Apr 14, 2020

AshCripps commented Apr 14, 2020 • edited Loading

AshCripps commented Apr 14, 2020

richardlau commented Apr 14, 2020

sam-github commented Apr 14, 2020

rvagg commented Apr 15, 2020

AshCripps commented Apr 15, 2020

AshCripps commented Apr 16, 2020 • edited Loading

rvagg Apr 21, 2020

Choose a reason for hiding this comment

AshCripps commented Mar 26, 2020 •

edited

Loading

AshCripps commented Apr 7, 2020 •

edited

Loading

AshCripps commented Apr 14, 2020 •

edited

Loading

AshCripps commented Apr 16, 2020 •

edited

Loading