Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

apiserver: use direct paths for ephemeral storage #133

Merged
merged 1 commit into from
Sep 9, 2024

Conversation

yeazelm
Copy link
Contributor

@yeazelm yeazelm commented Sep 8, 2024

Issue number:

Closes # bottlerocket-os/bottlerocket#4181

Description of changes:
The path for ephemeral storage devices is created by udev on early boot after ephemeral-storage init is called but if you call ephemeral-storage bind immediately after this, the path may not exist, causing the resulting commands to fail. This commit switches to a path that is known to exist to avoid this issue.

Testing done:
Tested on no, single, and multi-device Instance store EC2 instance types to confirm that when used with bootstrap-commands, they no longer exhibit the error.
From a booted instance:

# journalctl -D /.bottlerocket/rootfs/var/log/journal/ -u bootstrap-commands
localhost systemd[1]: Starting Bootstrap Commands...
localhost bootstrap-commands[6281]: 22:42:47 [INFO] Processing bootstrap command '000-test-bootstrap-commands' ...
internal bootstrap-commands[6281]: 22:42:48 [INFO] Turning off bootstrap command '000-test-bootstrap-commands'
internal bootstrap-commands[6281]: 22:42:48 [INFO] Successfully ran bootstrap command '000-test-bootstrap-commands'
internal systemd[1]: Finished Bootstrap Commands.

root@admin]# sheltie
bash-5.1# mount | grep md
ramfs on /run/credentials/systemd-sysusers.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-tmpfiles-setup-dev.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-sysctl.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-tmpfiles-setup.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
/dev/md127 on /mnt/.ephemeral type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=256k,sunit=512,swidth=1024,noquota)
/dev/md127 on /var/lib/containerd type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=256k,sunit=512,swidth=1024,noquota)
/dev/md127 on /var/lib/kubelet type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=256k,sunit=512,swidth=1024,noquota)
/dev/md127 on /var/log/pods type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=256k,sunit=512,swidth=1024,noquota)

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

@yeazelm
Copy link
Contributor Author

yeazelm commented Sep 9, 2024

I found an issue with reboots, this method won't find the device after the reboot so we need a way to find this after the fact.

@yeazelm yeazelm marked this pull request as draft September 9, 2024 15:40
@yeazelm
Copy link
Contributor Author

yeazelm commented Sep 9, 2024

^ Added --homehost=any to the create commands to force the behavior needed for the array to come back after reboot.

The path for ephemeral storage devices is created by udev on early boot
after ephemeral-storage init is called but if you call ephemeral-storage
bind immediately after this, the path may not exist, causing the
resulting commands to fail. This commit switches to a path that is known
to exist to avoid this issue.

Signed-off-by: Matthew Yeazel <yeazelm@amazon.com>
@yeazelm yeazelm marked this pull request as ready for review September 9, 2024 16:57
@yeazelm
Copy link
Contributor Author

yeazelm commented Sep 9, 2024

^ Updated the labeling to be ephemeral for the filesystem.

@yeazelm yeazelm merged commit d5f4fc3 into bottlerocket-os:develop Sep 9, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants