-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: Add cleanup script to manage build agent disk space #381
Conversation
Codecov Report
@@ Coverage Diff @@
## master #381 +/- ##
==========================================
- Coverage 91.06% 90.78% -0.29%
==========================================
Files 36 36
Lines 3493 3493
==========================================
- Hits 3181 3171 -10
- Misses 312 322 +10
Continue to review full report at Codecov.
|
# Regular maintenance performed on a buildkite agent to control disk usage | ||
# | ||
|
||
echo --- Delete all exited containers first |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
prefer printf BUILTIN to echo
( | ||
set -x | ||
exited=$(docker ps -aq --no-trunc --filter "status=exited") | ||
if [[ -n "$exited" ]]; then |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
double quotes unnecessary in bash's builtin [[
set -x | ||
dangling=$(docker images --filter 'dangling=true' -q --no-trunc | sort | uniq) | ||
if [[ -n "$dangling" ]]; then | ||
docker rmi "$dangling" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks to me like dangling is multiple strings... when you pass "$dangling" to docker rmi, you pass a single argument
consider
dangling=( $(docker ...) )
docker rmi "${dangliing[@]}"
echo "--- Delete /tmp files older than 1 day owned by $(whoami)" | ||
( | ||
set -x | ||
find /tmp -maxdepth 1 -user "$(whoami)" -mtime +1 -print0 | xargs -0 rm -rf |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
whoami isn't always around, as such introduces a dependency, consider ${USER} or
…torage (anza-xyz#381) #### Problem The current implementation of TieredStorage::file_size() requires a sys-call to provide the file size. #### Summary of Changes Add len() API to TieredStorageReader, and have HotStorageReader() implement the API using Mmap::len(). #### Test Plan Update existing unit-test to also verify HotStorageReader::len().
Docker containers/images in particular need to be actively pruned to avoid running out of disk