Upgrade Base Image to Amazon Linux 2023 #1122
Conversation
Python2 is long deprecates and is not in the amazon linux 2023 repos
triarius
force-pushed
the
pdp-695-amazon-linux-2023-for-elastic-ci-stack
branch
from
1fc70e1
to
1e431db
Compare
aws-cli v2 automatically uses signature v4: https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingAWSSDK.html#UsingAWSSDK-move-to-Sig4
Some tabs have to be kept because <<-HEREDOCs ignore tabs but not spaces. Also standardise some spacing.
The http calls are nicer because we don't have to format the output with `cut`
triarius
force-pushed
the
pdp-695-amazon-linux-2023-for-elastic-ci-stack
branch
from
1e431db
to
8752ac0
Compare
| }, | ||
| { | ||
| "type": "shell", | ||
| "script": "scripts/upgrade-kernel.sh" |
| @@ -1,2 +1,2 @@ | |||
| buildkite-agent:1001:1 | |||
| buildkite-agent:993:1 | |||
There was a problem hiding this comment.
The docker rpm installed from the repos automatically creates a docker group with this gid.
There was a problem hiding this comment.
My spidey senses tingle a bit with this change.
I vaguely remember some fragile things hanging off these subgid mappings.
But, it's probably fine.
As long as we test it with a rootless container? Maybe via Buildkite's docker plugin or something?
| @@ -24,4 +24,3 @@ KillMode=process | |||
|
|
|||
| [Install] | |||
| WantedBy=multi-user.target | |||
| DefaultInstance=1 | |||
There was a problem hiding this comment.
This is not needed as this unit is not used as a template. Leaving it in spits out a warning in the journal
| cat <<< "$(jq '."data-root"="/mnt/ephemeral/docker"' /etc/docker/daemon.json)" > /etc/docker/daemon.json | ||
| fi | ||
|
|
||
| # Customise address pools | ||
| cat <<<"$(jq '."default-address-pools"=[{"base":"172.17.0.0/12","size":20},{"base":"192.168.0.0/16","size":24}]' /etc/docker/daemon.json)" >/etc/docker/daemon.json | ||
|
|
||
| systemctl restart docker |
There was a problem hiding this comment.
Need to restart docker to get the new settings.
It's meant to be installed locally, not on EC2 instances
There was a problem hiding this comment.
These are now systemd timers
There was a problem hiding this comment.
Moved to
elastic-ci-stack-for-aws/packer/linux/scripts/install-utils.sh
Lines 30 to 35 in 00e7a8b
There was a problem hiding this comment.
Moved to
There was a problem hiding this comment.
Moved to
There was a problem hiding this comment.
This isn't necessary, the session manager plugin is used to allow logging into managed nodes with the aws-cli. A buildkite-agent typically (arguably ever) does not need to do that. It DOES need to be logged into by ssm, but the ssm-agent is there for that. We install the ssm agent on
and start it here:
| if [ "${MACHINE}" == "x86_64" ]; then | ||
| echo "Downloading docker-compose..." | ||
| sudo curl -Lsf -o /usr/bin/docker-compose https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-Linux-x86_64 | ||
| sudo chmod +x /usr/bin/docker-compose | ||
| docker-compose --version | ||
| elif [[ "${MACHINE}" == "aarch64" ]]; then | ||
| sudo yum install -y gcc-c++ libffi-devel openssl11 openssl11-devel python3-devel | ||
|
|
||
| # docker-compose depends on the cryptography package, v3.4 of which | ||
| # introduces a build dependency on rust; let's avoid that for now. | ||
| # https://github.com/pyca/cryptography/blob/master/CHANGELOG.rst#34---2021-02-07 | ||
| # This should be unpinned ASAP; hopefully docker-compose will offer binary | ||
| # download for arm64 at some point: | ||
| # https://github.com/docker/compose/issues/7472 | ||
| CONSTRAINT_FILE="/tmp/docker-compose-pip-constraint" | ||
| echo 'cryptography<3.4' >"$CONSTRAINT_FILE" | ||
| echo 'urllib3<2' >"$CONSTRAINT_FILE" | ||
| sudo pip3 install --constraint "$CONSTRAINT_FILE" "docker-compose==${DOCKER_COMPOSE_VERSION}" | ||
|
|
||
| docker-compose version | ||
| else | ||
| echo "No docker compose option configured for arch ${MACHINE}" | ||
| exit 1 | ||
| fi | ||
|
|
| @@ -1,2 +1,2 @@ | |||
| buildkite-agent:1001:1 | |||
| buildkite-agent:993:1 | |||
There was a problem hiding this comment.
My spidey senses tingle a bit with this change.
I vaguely remember some fragile things hanging off these subgid mappings.
But, it's probably fine.
As long as we test it with a rootless container? Maybe via Buildkite's docker plugin or something?
| }, | ||
| { | ||
| "type": "shell", | ||
| "script": "scripts/install-nvme-cli.sh" |
There was a problem hiding this comment.
Most of these have been folded into other scripts.
| if [[ "$(uname -m)" == "aarch64" ]]; then | ||
| AGENT_ARCH="arm64" | ||
| else | ||
| AGENT_ARCH="amd64" | ||
| fi |
There was a problem hiding this comment.
Elsewhere you've got a nice case statement, how about doing that consistently here?
| if [[ "$(uname -m)" == "aarch64" ]]; then | |
| AGENT_ARCH="arm64" | |
| else | |
| AGENT_ARCH="amd64" | |
| fi | |
| case $(uname -m) in | |
| x86_64) ARCH=amd64;; | |
| aarch64) ARCH=arm64;; | |
| *) ARCH=unknown;; | |
| esac |
(although this is very nitpicky, feel free to ignore me 😅)
There was a problem hiding this comment.
There are many more examples of sprinkled throughout the code base. I can't accept this as is thought, the variable was $AGENT_ARCH, but the suggestion has it as $ARCH. It's probably nicer to standardise on $ARCH if it does not exist in this script, though.
There was a problem hiding this comment.
Ah whoops, i missed that $AGENT_ARCH! Glad someone is paying attention 😅
There was a problem hiding this comment.
Moved to
…t instead of "success" Co-authored-by: paul david <423357+toothbrush@users.noreply.github.com>
|
Ah interesting, and yes i imagine that would be clearest - allowing a wholesale copy-pastable command to be easy to see. Thanks for indulging my nits, i am learning things about goss too 👍 On 7 Jun 2023, at 10:03, Narthana Epa ***@***.***> wrote:
@triarius commented on this pull request.
In goss.yaml:
+ # Check docker userns is enabled
+ docker info --format='{{ `{{range .SecurityOptions}}{{if eq . "name=userns"}}{{"success"}}{{end}}{{end}}` }}':
Yeah, and mine has the opposite issue.
I could put the whole line in a nested template, that might be the best of both words.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>
|
They need to have some level of nested templates because otherwise, goss will attempt to evaluate the templates that are intended to be evaluated by docker. Also, the elements of `stdout` are regexes, so we adapt it to test for each expected element of the list. Goss will report which regex did not match the output, so we can use this determine which plugin are missing. It won't be able to tell if there are other plugins, but that is very much a feature.
…all-elastic-stack.sh
triarius
force-pushed
the
pdp-695-amazon-linux-2023-for-elastic-ci-stack
branch
from
db766d3
to
3c7c3a5
Compare
This was removed in the migration to Amazon Linux 3 with the belief that there was no use case for the Session Manager Plugin being present on the agent[1]. It has since been realised that it is quite useful. For example, the agent may be used to start ECS Tasks which perform work in other environments/network configurations (such as Database migrations during deployments). 1. #1122 (comment)
This was removed in the migration to Amazon Linux 3 with the belief that there was no use case for the Session Manager Plugin being present on the agent[1]. It has since been realised that it is quite useful. For example, the agent may be used to start ECS Tasks which perform work in other environments/network configurations (such as Database migrations during deployments). 1. #1122 (comment)
This was removed in the migration to Amazon Linux 3 with the belief that there was no use case for the Session Manager Plugin being present on the agent[1]. It has since been realised that it is quite useful. For example, the agent may be used to start ECS Tasks which perform work in other environments/network configurations (such as Database migrations during deployments). 1. #1122 (comment)
This was removed in the migration to Amazon Linux 3 with the belief that there was no use case for the Session Manager Plugin being present on the agent[1]. It has since been realised that it is quite useful. For example, the agent may be used to start ECS Tasks which perform work in other environments/network configurations (such as Database migrations during deployments). 1. #1122 (comment)
Thanks for bearing with us everyone, we have finally gotten around to baking Amazon Linux 2023 base images and getting them to run builds!
Although @toothbrush made a valiant effort in #1103, their, and our previous attempts were DoA as the user data startup scripts were failing for various reasons. Thus, the instance would be marked as unhealthy soon after they booted and the ASG would continuously be booting new instances to replace them.
While fixing this, I took the opportunity to upgrade a few packages, and I've preferred to install these through the AL2023 repos as much as possible. Some older packages have been replaced where their functionality was provided by other packages that are available by default.
Consider adding these to the changelog at the time of release.
Added
Changed
Upgraded
Removed
CI