ci: Add Jenkins pipeline #1899

jlebon · 2019-09-06T21:05:57Z

This is an experiment in using Jenkins pipelines for our CI. See similar
initiatives in coreos-assembler[1] and fedora-coreos-config[2].

For now, this only does the following testing:

checks commit for unintended submodule bumps
checks the minimum Rust version
builds RPMs
builds FCOS (with the new RPMs both for executing the build
itself, as well as included in the built OS)

There are dummy placeholders for where we'd actually run the vmcheck
and the compose testsuites. Let's address those trickier parts as
follow-ups.

[1] coreos/coreos-assembler#667
[2] coreos/fedora-coreos-config#131

jlebon · 2019-09-06T21:13:55Z

Split out the prep patches in #1900!

jlebon · 2019-09-06T21:25:48Z

Hmm, OK it's hitting permissions issues:

Not permitted to query list of collaborators, assuming none

Which I'm not sure why. I made it an admin and the token should have the proper scopes. Will have to debug next week.

jlebon · 2019-09-09T22:05:06Z

✔️ continuous-integration/jenkins/pr-merge — This commit looks good

Woohoo!

cgwalters · 2019-09-10T13:55:16Z

Let's use registry.svc.ci.openshift.org/coreos/cosa-buildroot:latest ?

cgwalters · 2019-09-10T13:55:42Z

(I just created the coreos project there and will soon ensure other people have access to it)

cgwalters · 2019-09-11T13:57:28Z

.cci.jenkinsfile

+  }
+},
+codestyle: {
+  coreos.pod(image: 'registry.fedoraproject.org/fedora:30', privileged: true) {


Does not need privileged

Yeah, can probably use e.g. coreos-assembler for this one. It just needs git.

cgwalters · 2019-09-11T13:57:37Z

.cci.jenkinsfile

+  }
+},
+msrv: {
+  coreos.pod(image: 'registry.fedoraproject.org/fedora:30', privileged: true) {


Does not need privileged

We do a full rpm-ostree build here too though, so we need to install packages... privileged: true here translates to runAsUser: 0.

We could use cosa-buildroot:latest, but then dependency changes would have to be done in two separate PRs, right? Which I guess isn't a big deal, though it'd be nice to keep with the atomic changes theme.

privileged: true here translates to runAsUser: 0.

Oh. So in this article - I made an argument that we shouldn't call the default docker/podman model where containers run as uid 0 "privileged".

Let's literally call the option runAsUser: 0 ?

coreos/coreos-ci-lib#3

Yup, WFM! Changed to runAsUser: 0. ⬇️

We could use cosa-buildroot:latest, but then dependency changes would have to be done in two separate PRs, right?

Another approach here is to use cosa-buildroot:latest, but still run installdeps.sh. That way, the majority of deps are no-ops, but it still closes the gap to keep changes atomic.

Done! ⬇️

cgwalters · 2019-09-11T13:58:02Z

.cci.jenkinsfile

+
+stage("Test") {
+parallel vmcheck: {
+  coreos.pod(image: 'quay.io/coreos-assembler/coreos-assembler:latest', privileged: true, kvm: true) {


Do we really need privileged instead of just kvm?

Implementation TBD, though I think we could get away with just kvm. I commented out the whole section for now.

cgwalters · 2019-09-11T13:58:49Z

.cci.jenkinsfile

+  }
+}
+
+stage("Test") {


This is a no-op right now right? Maybe best to comment out the whole section?

cgwalters · 2019-09-11T14:00:47Z

.cci.jenkinsfile

+        cd packaging
+        make -f Makefile.dist-packaging rpm
+      """
+      stash includes: 'packaging/**/*.rpm', name: 'rpms'


This is an interesting approach. Worth contrasting with e.g. #1893 where what I was trying to do was get closer to a model where we build a derived container. You could imagine e.g. that we generate a FROM coreos-assembler container and then test that potentially multiple ways.

That said, we need RPMs for composes, so I'm OK with this approach too.

Hmm, I think if we want to drop privs even more, we'll need a mix of the two. E.g. an initial derived container build which builds the RPMs and installs them on top of cosa-buildroot:latest, and then using that as part of Jenkins pipeline. I'm not sure if there's an easy way to do this though. We could use the OpenShift Jenkins pipeline plugin and use the openshiftBuild() step, though that'd require creating a buildconfig first, which really wants e.g. a throwaway namespace like Prow gives us.

Or we could mix Prow and Jenkins. E.g. have the pipeline wait until the Prow build is done? Prow can natively trigger Jenkins jobs, though I'm not sure if we'd still be able to leverage the native integration into GitHub like we do now.

Edit: this isn't a direct reply to your specific comment here really, just some brainstorming.

jlebon · 2019-09-12T21:14:45Z

Updated! ⬆️

cgwalters · 2019-09-13T14:18:40Z

bot, retest this pelase

jlebon · 2019-09-13T16:22:58Z

bot, retest this please

jlebon · 2019-09-13T16:33:45Z

bot, retest this please

This is an experiment in using Jenkins pipelines for our CI. See similar initiatives in coreos-assembler[1] and fedora-coreos-config[2]. For now, this only does the following testing: - checks commit for unintended submodule bumps - checks the minimum Rust version - builds RPMs - builds FCOS (with the new RPMs both for executing the build itself, as well as included in the built OS) There are dummy placeholders for where we'd actually run the vmcheck and the compose testsuites. Let's address those trickier parts as follow-ups. [1] coreos/coreos-assembler#667 [2] coreos/fedora-coreos-config#131

cgwalters · 2019-09-13T21:02:25Z

@rh-atomic-bot r+ 9b9b2ed

rh-atomic-bot · 2019-09-13T21:02:31Z

⌛ Testing commit 9b9b2ed with merge d24bdb6...

This is an experiment in using Jenkins pipelines for our CI. See similar initiatives in coreos-assembler[1] and fedora-coreos-config[2]. For now, this only does the following testing: - checks commit for unintended submodule bumps - checks the minimum Rust version - builds RPMs - builds FCOS (with the new RPMs both for executing the build itself, as well as included in the built OS) There are dummy placeholders for where we'd actually run the vmcheck and the compose testsuites. Let's address those trickier parts as follow-ups. [1] coreos/coreos-assembler#667 [2] coreos/fedora-coreos-config#131 Closes: #1899 Approved by: cgwalters

rh-atomic-bot · 2019-09-13T21:56:04Z

💔 Test failed - status-papr

jlebon · 2019-09-13T22:28:26Z

@rh-atomic-bot retry

rh-atomic-bot · 2019-09-13T22:28:31Z

⌛ Testing commit 9b9b2ed with merge 74e93af...

This is an experiment in using Jenkins pipelines for our CI. See similar initiatives in coreos-assembler[1] and fedora-coreos-config[2]. For now, this only does the following testing: - checks commit for unintended submodule bumps - checks the minimum Rust version - builds RPMs - builds FCOS (with the new RPMs both for executing the build itself, as well as included in the built OS) There are dummy placeholders for where we'd actually run the vmcheck and the compose testsuites. Let's address those trickier parts as follow-ups. [1] coreos/coreos-assembler#667 [2] coreos/fedora-coreos-config#131 Closes: #1899 Approved by: cgwalters

rh-atomic-bot · 2019-09-13T23:38:47Z

💔 Test failed - status-papr

cgwalters · 2019-09-15T15:09:00Z

@rh-atomic-bot retry

rh-atomic-bot · 2019-09-15T15:09:05Z

⌛ Testing commit 9b9b2ed with merge 830e974...

This is an experiment in using Jenkins pipelines for our CI. See similar initiatives in coreos-assembler[1] and fedora-coreos-config[2]. For now, this only does the following testing: - checks commit for unintended submodule bumps - checks the minimum Rust version - builds RPMs - builds FCOS (with the new RPMs both for executing the build itself, as well as included in the built OS) There are dummy placeholders for where we'd actually run the vmcheck and the compose testsuites. Let's address those trickier parts as follow-ups. [1] coreos/coreos-assembler#667 [2] coreos/fedora-coreos-config#131 Closes: #1899 Approved by: cgwalters

rh-atomic-bot · 2019-09-15T16:06:45Z

☀️ Test successful - status-papr
Approved by: cgwalters
Pushing 830e974 to master...

cgwalters · 2019-09-16T13:18:08Z

Pushing 830e974 to master...

Hmm, looks like rh-atomic-bot lost permissions with the move? Looking

cgwalters · 2019-09-16T13:18:44Z

Or alternatively...let's stop using Homu now?

jlebon · 2019-09-16T14:42:30Z

I think it was just missing from the branch protection whitelist. I added it now. Let's confirm that:
@rh-atomic-bot retry

Or alternatively...let's stop using Homu now?

Yup, WFM too. (Will be sad to see it go, though I suppose we must move with the times.)

rh-atomic-bot · 2019-09-16T14:42:35Z

⌛ Testing commit 9b9b2ed with merge 357ccfc...

This is an experiment in using Jenkins pipelines for our CI. See similar initiatives in coreos-assembler[1] and fedora-coreos-config[2]. For now, this only does the following testing: - checks commit for unintended submodule bumps - checks the minimum Rust version - builds RPMs - builds FCOS (with the new RPMs both for executing the build itself, as well as included in the built OS) There are dummy placeholders for where we'd actually run the vmcheck and the compose testsuites. Let's address those trickier parts as follow-ups. [1] coreos/coreos-assembler#667 [2] coreos/fedora-coreos-config#131 Closes: #1899 Approved by: cgwalters

jlebon · 2019-09-16T16:31:00Z

@rh-atomic-bot retry

rh-atomic-bot · 2019-09-16T16:31:05Z

⌛ Testing commit 9b9b2ed with merge c6bcbf9...

This is an experiment in using Jenkins pipelines for our CI. See similar initiatives in coreos-assembler[1] and fedora-coreos-config[2]. For now, this only does the following testing: - checks commit for unintended submodule bumps - checks the minimum Rust version - builds RPMs - builds FCOS (with the new RPMs both for executing the build itself, as well as included in the built OS) There are dummy placeholders for where we'd actually run the vmcheck and the compose testsuites. Let's address those trickier parts as follow-ups. [1] coreos/coreos-assembler#667 [2] coreos/fedora-coreos-config#131 Closes: #1899 Approved by: cgwalters

jlebon · 2019-09-16T17:58:14Z

@rh-atomic-bot retry

rh-atomic-bot · 2019-09-16T17:58:19Z

⌛ Testing commit 9b9b2ed with merge 727d4b6...

This is an experiment in using Jenkins pipelines for our CI. See similar initiatives in coreos-assembler[1] and fedora-coreos-config[2]. For now, this only does the following testing: - checks commit for unintended submodule bumps - checks the minimum Rust version - builds RPMs - builds FCOS (with the new RPMs both for executing the build itself, as well as included in the built OS) There are dummy placeholders for where we'd actually run the vmcheck and the compose testsuites. Let's address those trickier parts as follow-ups. [1] coreos/coreos-assembler#667 [2] coreos/fedora-coreos-config#131 Closes: #1899 Approved by: cgwalters

rh-atomic-bot · 2019-09-17T04:37:57Z

💥 Test timed out

jlebon · 2019-09-17T12:57:11Z

@rh-atomic-bot retry

rh-atomic-bot · 2019-09-17T12:57:17Z

⌛ Testing commit 9b9b2ed with merge ce396fd...

This is an experiment in using Jenkins pipelines for our CI. See similar initiatives in coreos-assembler[1] and fedora-coreos-config[2]. For now, this only does the following testing: - checks commit for unintended submodule bumps - checks the minimum Rust version - builds RPMs - builds FCOS (with the new RPMs both for executing the build itself, as well as included in the built OS) There are dummy placeholders for where we'd actually run the vmcheck and the compose testsuites. Let's address those trickier parts as follow-ups. [1] coreos/coreos-assembler#667 [2] coreos/fedora-coreos-config#131 Closes: #1899 Approved by: cgwalters

rh-atomic-bot · 2019-09-17T14:01:41Z

💔 Test failed - status-papr

jlebon · 2019-09-17T14:03:01Z

@rh-atomic-bot retry

rh-atomic-bot · 2019-09-17T14:03:07Z

⌛ Testing commit 9b9b2ed with merge 7480da2...

This is an experiment in using Jenkins pipelines for our CI. See similar initiatives in coreos-assembler[1] and fedora-coreos-config[2]. For now, this only does the following testing: - checks commit for unintended submodule bumps - checks the minimum Rust version - builds RPMs - builds FCOS (with the new RPMs both for executing the build itself, as well as included in the built OS) There are dummy placeholders for where we'd actually run the vmcheck and the compose testsuites. Let's address those trickier parts as follow-ups. [1] coreos/coreos-assembler#667 [2] coreos/fedora-coreos-config#131 Closes: #1899 Approved by: cgwalters

rh-atomic-bot · 2019-09-18T00:38:04Z

💥 Test timed out

jlebon · 2019-09-18T15:15:26Z

@rh-atomic-bot retry

rh-atomic-bot · 2019-09-18T15:15:31Z

⌛ Testing commit 9b9b2ed with merge 46ab7d1...

rh-atomic-bot · 2019-09-18T16:09:29Z

☀️ Test successful - status-papr
Approved by: cgwalters
Pushing 46ab7d1 to master...

jlebon force-pushed the jenkins branch from be5bd99 to dc85c8a Compare September 10, 2019 13:14

jlebon marked this pull request as ready for review September 10, 2019 13:14

cgwalters reviewed Sep 11, 2019

View reviewed changes

jlebon force-pushed the jenkins branch from dc85c8a to 649063d Compare September 12, 2019 21:14

jlebon force-pushed the jenkins branch 2 times, most recently from 9c1709c to e80e058 Compare September 13, 2019 13:15

jlebon force-pushed the jenkins branch from e80e058 to 9b9b2ed Compare September 13, 2019 20:59

jlebon added the reviewed/approved label Sep 16, 2019

rh-atomic-bot closed this in 46ab7d1 Sep 18, 2019

jlebon deleted the jenkins branch April 23, 2023 23:31

ci: Add Jenkins pipeline #1899

ci: Add Jenkins pipeline #1899

Conversation

jlebon commented Sep 6, 2019

jlebon commented Sep 6, 2019

jlebon commented Sep 6, 2019

jlebon commented Sep 9, 2019

cgwalters commented Sep 10, 2019 • edited Loading

cgwalters commented Sep 10, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jlebon Sep 12, 2019 • edited Loading

Choose a reason for hiding this comment

jlebon commented Sep 12, 2019

cgwalters commented Sep 13, 2019

jlebon commented Sep 13, 2019

jlebon commented Sep 13, 2019

cgwalters commented Sep 13, 2019

rh-atomic-bot commented Sep 13, 2019

rh-atomic-bot commented Sep 13, 2019

jlebon commented Sep 13, 2019

rh-atomic-bot commented Sep 13, 2019

rh-atomic-bot commented Sep 13, 2019

cgwalters commented Sep 15, 2019

rh-atomic-bot commented Sep 15, 2019

rh-atomic-bot commented Sep 15, 2019

cgwalters commented Sep 16, 2019

cgwalters commented Sep 16, 2019

jlebon commented Sep 16, 2019

rh-atomic-bot commented Sep 16, 2019

jlebon commented Sep 16, 2019

rh-atomic-bot commented Sep 16, 2019

jlebon commented Sep 16, 2019

rh-atomic-bot commented Sep 16, 2019

rh-atomic-bot commented Sep 17, 2019

jlebon commented Sep 17, 2019

rh-atomic-bot commented Sep 17, 2019

rh-atomic-bot commented Sep 17, 2019

jlebon commented Sep 17, 2019

rh-atomic-bot commented Sep 17, 2019

rh-atomic-bot commented Sep 18, 2019

jlebon commented Sep 18, 2019

rh-atomic-bot commented Sep 18, 2019

rh-atomic-bot commented Sep 18, 2019

cgwalters commented Sep 10, 2019 •

edited

Loading

jlebon Sep 12, 2019 •

edited

Loading