drone-queue-cloudwatch is a Lambda function that publishes queued Drone builds to Cloudwatch Metrics. Metric dimensions are built from the node labels specified by a given build.
This tool allows you to leverage AWS autoscaling across multiple Drone worker groups. Some of our CI workloads need to run on GPU instances, others don't.
In order for the code to actually publish metrics, you must use node routing.
When drone-queue-cloudwatch inspects builds, it uses the node labels to build the Cloudwatch metrics dimensions.
This is meant to be run as a cron job.
See the terraform/ directory for reference.
The zip artifact is accessible in the aai-oss S3 bucket in us-west-2
- To use a specific version of the code, use the
drone-queue-cloudwatch/<commit sha>.zipobject key - To use the latest version, use the
drone-queue-cloudwatch/latest.zipobject key
Note: You want to use the "Sum" statistic when configuring autoscaling. You should also treat missing data as "Not Breaching"
This application assumes each Drone worker group passes the same DRONE_RUNNER_LABELS to all workers.
For example, one group would have these labels:
class=standardos=linux
And another these:
class=gpuos=linux
Your Drone file would specify either
node:
os: linux
class: gpuor
node:
os: linux
class: standardYour autoscaling trigger would launch more instances / containers for a given worker group based on how many queued builds there are.
To prevent scale in events from disrupting running builds, use a metrics expression to ensure you don't scale down too far if there are running builds.