v6.7.0

v6.7.0 (2023-09-14)

Full Changelog

Security

⚠️ This release partially fixes a medium-severity security vulnerability. We recommend upgrading to v6.7.1 or v5.22.5.

• Affected versions: All prior versions of Elastic CI Stack
• Impact: Privilege escalation to root on Linux agent instances
• Required privileges: Users that can run user-controlled commands on agents (e.g. by pushing a branch to a repo that triggers a build with those changes)
• Attack vector: A specially crafted build can abuse the fix-buildkite-agent-builds-permissions script to run commands as root on subsequent builds
• Fix: Improved input validation in fix-buildkite-agent-builds-permissions #1212 (@DrJosh9000)
• Alternative workarounds: Deploy a pre-bootstrap hook to prevent execution of fix-buildkite-agent-builds-permissions during a build

Thanks to Nick Nam of Atredis Partners for reporting the vulnerability.

Changed

• Update to scaler v1.6.0 #1213 (@DrJosh9000)
• Bump buildkite-agent to v3.55.0 #1214 (@DrJosh9000)

Internal

• Fix ami_source_filter #1211 (@DrJosh9000)

Upgrading

Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v6.7.0/aws-stack.yml

If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):

Documentation

See the README for this release.

v5.22.4

v5.22.4 (2023-09-14)

Full Changelog

Security

⚠️ This release partially fixes a medium-severity security vulnerability. We recommend upgrading to v6.7.1 or v5.22.5.

• Affected versions: All prior versions of Elastic CI Stack
• Impact: Privilege escalation to root on Linux agent instances
• Required privileges: Users that can run user-controlled commands on agents (e.g. by pushing a branch to a repo that triggers a build with those changes)
• Attack vector: A specially crafted build can abuse the fix-buildkite-agent-builds-permissions script to run commands as root on subsequent builds
• Fix: Improved input validation in fix-buildkite-agent-builds-permissions #1215 (@DrJosh9000)
• Alternative workarounds: Deploy a pre-bootstrap hook to prevent execution of fix-buildkite-agent-builds-permissions during a build

Thanks to Nick Nam of Atredis Partners for reporting the vulnerability.

Upgrading

Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v5.22.4/aws-stack.yml

If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):

Documentation

See the README for this release.

v6.6.0

v6.6.0 (2023-09-07)

Full Changelog

Fixed

• Fix instance storage mount script fails when instance storage not available #1206 (@triarius)

Changed

• Bump buildkite-agent to v3.54.0 #1207 (@DrJosh9000)

Upgrading

Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v6.6.0/aws-stack.yml

If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):

Documentation

See the README for this release.

v6.5.0

v6.5.0 (2023-08-31)

Full Changelog

Changed

• Bump buildkite-agent to v3.53.0 #1204 (@DrJosh9000)

Upgrading

Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v6.5.0/aws-stack.yml

If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):

Documentation

See the README for this release.

v6.4.0

v6.4.0 (2023-08-24)

Full Changelog

Changed

• Bump docker-compose to v2.20.3 #1201 (@triarius)
• Bump buildkite-agent to v3.52.1 #1200 (@triarius)
• Change the Community Slack links in documentation to Forum ones #1199 (@mcncl)

Internal

• Prevent tag builds from publishing a latest template when they are not "on the main branch" #1197 (@triarius)

Upgrading

Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v6.4.0/aws-stack.yml

If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):

Documentation

See the README for this release.

v6.3.0

Known Issues

⚠️ Buildkite Agent v3.51.0 has a known issue with the buildkite-agent step export command. This is fixed in v3.52.0.

v6.3.0 (2023-08-16)

Full Changelog

Changed

• Bump buildkite-agent to v3.51.0 #1193 (@triarius)
• Bump git-lfs to v3.4.0 #1191 (@triarius)

Fix

• Fix mdadm is not installed, leading to broken instance storage when there is more than one volumes #1190 (@triarius)

Internal

• Incorporated CHANGELOG for v5.22.3 #1189 (@triarius)

Upgrading

Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v6.3.0/aws-stack.yml

If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):

Documentation

See the README for this release.

v6.2.0

v6.2.0 (2023-08-09)

Full Changelog

Changed

• Change base image to Windows Server 2019 w/o containers and install Docker CE (v24.0.5) #1180 (@triarius)
• Add cost allocation tags to EBS volumes #1171 (@keatmin)

Fixed

• Add missing authorized keys systemd units #1184 (@sj26)
• Fix instance storage docker dir not created #1181 (@triarius)
• Fix set -e fails from environment hooks #1179 (@triarius)

Upgrading

Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v6.2.0/aws-stack.yml

If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):

Documentation

See the README for this release.

v5.22.3

v5.22.3 (2023-08-10)

Full Changelog

Changed

• Bump buildkite-agent to v3.50.4 #1186 (@triarius)
• Use Windows Server 2019 base image and Docker CE #1187 (@triarius)

Upgrading

Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v5.22.3/aws-stack.yml

If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):

Documentation

See the README for this release.

v6.1.0

v6.1.0 (2023-08-01)

Full Changelog

Changed

• Bump buildkite-agent to v3.50.4 #1177 (@DrJosh9000)
• Disable client side pager for aws-cli v2 for the buildkite-agent user #1174 (@triarius)
• Add ScalerMinPollInterval param #1173 (@amartani)

Upgrading

Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v6.1.0/aws-stack.yml

If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):

Documentation

See the README for this release.

v6.0.0

v6.0.0 (2023-07-26)

Full Changelog

Changed

• Upgrade base image to Amazon Linux 2023 #1122 (@triarius)
  • Many packages have been added, upgraded, or removed since Amazon Linux 2. We've explicitly called out what's been intentionally left out by us below. Refer to docs.aws.amazon.com/linux/al2023/ug/compare-with-al2.html for the changes Amazon have made.
• Publish template to both main and master #1129 (@triarius)
• Increase job cancel grace period to 60s #1144 (@triarius)
• Allow the MaxSize to be 0 #1140 (@triarius)
• Default EC2 instance names to stack name #1137 (@triarius)
• Rename the parameter InstanceType to InstanceTypes #1138 (@triarius)
• Rename the parameter ManagedPolicyARN to ManagedPolicyARNs #1138 (@triarius)
• Rename the parameter SecurityGroupId to SecurityGroupIds #1128 (@triarius)
• Rename the parameter EnableAgentGitMirrorsExperiment to BuildkiteAgentEnableGitMirrors #1123 (@triarius)
• Enable the ansi-timestamps setting if and only if BuildkiteAgentTimestampLines parameter is "false" #1132 (@triarius)
• Bump buildkite-agent-scaler to v1.5.0 #1169 (@tomellis91)
• Bump docker compose to v2.20.2 #1150 (@triarius)
• Bump buildx to v0.11.2 #1150 (@triarius)

Added

• Support running and building multi-platform docker images #1139 #1122 #1149 (@triarius)
• Support i4g instance types #1138 (@triarius)
• Added the parameter SpotAllocationStrategy #1130 (@triarius)
• Added the parameter ScalerEventScheduleRate to control the rate at which buildkite-agent-scaler is invoked #1169 (@tomellis91)

Fixed

• Guard against BUILDKITE_AGENT_ENABLE_GIT_MIRRORS not being set in startup script #1135 (@triarius)

Removed

• Remove deprecated SpotPrice parameter #1130 (@triarius)
• Removed packages. These packages are either not available on Amazon Linux 2023, or not installed by default on the base image we use. We have decided to not install them, as suitable replacements may be found.
  • Python 2
  • OpenSSL v1.0
  • AWS CLI v1
  • Docker-Compose v1
    • The docker-compose executable will prepend the --compatibility flag to docker-compose v2 #1148 (@triarius)
  • Cronie

Known Issues

• If you invoke docker compose with the docker-compose command, the --compatibilty flag will be prepended for you. This will be the case with the docker-compose-buildkite-plugin unless you specify cli-version: 2 in your plugin config.
• Docker build errors from docker compose will now exit with status code 17 or 18 when previously they would have exited with code 1. Please adjust your retry rules accordingly.
• The docker group is now a system group. Previously, its group ID was hard-coded to be 1001, but now it could take some value from 999 descending. If you relied on the GID being hard-coded to 1001 in your builds, they may have permission errors. We recommend you use something like the bash expression getent group docker | awk -F: '{print $3}' to detect what the ID of the docker group is instead.

Upgrading

Perform a CloudFormation stack update with the following URL:
https://s3.amazonaws.com/buildkite-aws-stack/v6.0.0/aws-stack.yml

If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):

Documentation

See the README for this release.