ci-public: replace multijob with freestyle for custom-suites

[refack]

I've configure a new job https://ci.nodejs.org/job/node-test-commit-custom-suites-freestyle/ (job XML attached).
This should replace https://ci.nodejs.org/job/node-test-commit-custom-suites/ in node-test-commit, as the current one is based on multijob which ignores the PLATFORM_LABEL parameter, and is unneccecerly combersome in this scenario.
Because of its limitations the current job is recursive (the flyweight job runs on a jenkins-workspace node and the actual build also runs on a jenkins-workspace node) so when the systems under load it can lead to unexpected behavior.

This is the diff from the current job https://www.diffchecker.com/MKqrFehH

[richardlau]

If this is for use in node-test-commit, does it need to be custom-suites with all the customisation, or should it be a node-test-worker (since currently the chained job is for running with --worker)?

[refack]

If this is for use in node-test-commit, does it need to be custom-suites with all the customisation, or should it be a node-test-worker (since currently the chained job is for running with --worker)?

It was designed for running customizable tests. We are already using it in three configurations:

1. to run the default suits with --worker - example
2. as part of daily-master to run the internet suite - example
3. as part of the test-v8 job to test the v8-update suite - example

So IMHO the conscumizations have proven very (re)useful.

[refack]

Due to an apparent degradation of stability, I've implemented the switch, and am monitoring the situation.

[richardlau]

@refack I agree that customisations are very useful, especially for one off runs (e.g. like the stress job). My query is whether we want to be reusing the same job in this way as part of larger jobs/flows/pipelines? For example, it's harder to see the history of --worker runs because the job history will be a mixture of the three scenarios you have outlined (plus any manual runs).

[refack]

My query is whether we want to be reusing the same job in this way as part of larger jobs/flows/pipelines?

It's a balance of useability/reusability... This case is actually a good example, as if we had 3 seperate jobs it would have been harder to identify the issue and correct the configuration.
So IMHO for the time being using it for --worker is a good solution. But we should be exploring better code reuse solutions, such as project inheritance, managed scripts and shared groovy steps.

[maclover7]

My query is whether we want to be reusing the same job in this way as part of larger jobs/flows/pipelines? For example, it's harder to see the history of --worker runs because the job history will be a mixture of the three scenarios you have outlined (plus any manual runs).

To view the history of --worker builds, you can check out the node-test-commit-custom-suites part from https://nodejs-ci-health.mmarchini.me/#/detailed-jobs

[richardlau]

@maclover7 but are all of those are --worker builds (e.g. https://ci.nodejs.org/job/node-test-commit-custom-suites/410/ isn't a --worker build)?

[refack]

FTR the issue that the jobs were running on jenkins-workspace nodes, was because I did not RTFM and missed that there's a specific way to trigger such jobs. (That is most probably why we have been seeing very frequent infra related failure these last couple of days.

[sam-github]

@richardlau is this still relevant? in-progress? already done some other way? Maybe its stale and should be closed, but its not immediately obvious, and it looks like you've refed it in an open issue.

[richardlau]

https://ci.nodejs.org/job/node-test-commit-custom-suites/ is disabled in Jenkins and has a description saying to instead use https://ci.nodejs.org/job/node-test-commit-custom-suites-freestyle/ (incidentally I've just PR'ed a change to the COLLABORATOR_GUIDE (nodejs/node#31557) in core to update the job link).

I think we can just close this out. I don't know what policy (or if we have one written down) covers what is backed up to https://github.com/nodejs/build/tree/master/jenkins/xml but it's hard to tell from a glance if the XML in this PR is current (i.e. has anyone changed the job configuration since). In any case backing up Jenkins changes from test and release CI is being handled by the proposed scripts @rvagg is working on in #2133.