Skip to content

Commit

Permalink
jobs/build: don't try to complete ongoing builds
Browse files Browse the repository at this point in the history
Since 973bcf9 ("jobs/build: rerun `build-arch` if previous build is
incomplete"), there is a race possible where the `build` job rerun logic
could kick in before the `release` jobs initially triggered for that
build has finished. We don't want to queue builds in that case.

Gate the rerun logic on whether the multi-arch locks and release lock
are taken.

It's theoretically possible but highly unlikely that we probe the lock
status before the previous `release` job takes it. Ideally, we would
have a way to directly take the lock and "transfer" its ownership to the
job we trigger. Anyway, if that somehow happens, it would result in the
job being run twice, which is safe.
  • Loading branch information
jlebon authored and dustymabe committed Feb 27, 2023
1 parent 3aeb368 commit fcea566
Showing 1 changed file with 15 additions and 3 deletions.
18 changes: 15 additions & 3 deletions jobs/build.Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -265,9 +265,10 @@ lock(resource: "build-${params.STREAM}") {
// Nothing changed since the latest build. Check if it's missing
// some arches and retrigger `build-arch` only for the missing
// arches, and the follow-up `release` job. Match the exact src
// config commit that was used. Skip if not uploading since it's
// required for multi-arch.
if (uploading) {
// config commit that was used. But only do this if there isn't
// already outstanding work in progress for that build ID. Skip if
// not uploading since it's required for multi-arch.
if (uploading && !buildid_has_work_pending(buildID, additional_arches)) {
def builds = readJSON file: "builds/builds.json"
assert buildID == builds.builds[0].id
def missing_arches = additional_arches - builds.builds[0].arches
Expand Down Expand Up @@ -542,3 +543,14 @@ def run_release_job(buildID) {
]
}
}

def buildid_has_work_pending(buildID, arches) {
def locked = true
// these locks match the ones in the release job
def locks = arches.collect{[resource: "release-${buildID}-${it}"]}
lock(resource: "release-${params.STREAM}", extra: locks, skipIfLocked: true) {
// NB: `return` here wouldn't actually return from the function
locked = false
}
return locked
}

0 comments on commit fcea566

Please sign in to comment.