Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add script to mark build agents offline for shutdown. #1006

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Add script to mark build agents offline for shutdown. #1006

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0cfa64a
Add script to mark build agents offline for shutdown.
nuclearsandwich Jun 23, 2023
71e611c
Merge branch 'master' into import-drain
cottsay Mar 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions doc/ongoing_operations.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,31 @@ No arguments means import from the default location(s).
We do this for things like new releases of the core ROS python tools.


Stop package building jobs without blocking package imports
-----------------------------------------------------------

Package (source and binary) build jobs for deb rely on a subordinate import package job to run on the repo host before the job can complete.
If the build farm needs to be brought into shutdown the standard shutdown procedure won't work because import package jobs will not be allowed to start.
In order to do a clean shutdown the build agents must be brought offline while allowing the repo host to continue running. For build farms with fixed agent pools this can be accomplished using the following groovy script

.. code-block:: groovy

import hudson.model.labels.LabelAtom
import hudson.slaves.OfflineCause.UserCause

cause = new UserCause(Jenkins.get().getUser("nuclearsandwich"), "Allow package imports to catch up")
Comment on lines +89 to +92
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use User.current() here instead?

Suggested change
import hudson.model.labels.LabelAtom
import hudson.slaves.OfflineCause.UserCause
cause = new UserCause(Jenkins.get().getUser("nuclearsandwich"), "Allow package imports to catch up")
import hudson.model.labels.LabelAtom
import hudson.model.User
import hudson.slaves.OfflineCause.UserCause
cause = new UserCause(User.current(), "Allow package imports to catch up")


label1 = new LabelAtom("buildagent")
label2 = new LabelAtom("buildagent_arm64")

for (comp in Jenkins.get().getComputers()) {
if (comp.getAssignedLabels().contains(label1) || comp.getAssignedLabels().contains(label2)) {
comp.setTemporarilyOffline(true, cause)
}
}

Once all source and binary packaging jobs have completed the farm can be put into the standard shutdown mode to complete the process.

Perform action on a set of jobs
-------------------------------

Expand Down
Loading