Bare Metal Raid 5

Hetzner - boot from raid-5 over 3x1.92TB NVME.

Boot into rescue system and type installimage. Select Ubuntu.

• change HOSTNAME
• remove line PART swap ... (See swap-on-raid is bad)
• change line PART / to PART / ext4 8G

Thereafter the partitions look like this (make sure there is no swap:

/dev/nvme0n1p1      4096     528383     524288  256M EFI System
/dev/nvme0n1p2    528384    2625535    2097152    1G Linux RAID
/dev/nvme0n1p3   2625536   11014143    8388608    4G Linux RAID

/dev/nvme1n1p1      4096     528383     524288  256M EFI System
/dev/nvme1n1p2    528384    2625535    2097152    1G Linux RAID
/dev/nvme1n1p3   2625536   11014143    8388608    4G Linux RAID

/dev/nvme2n1p1      4096     528383     524288  256M EFI System
/dev/nvme2n1p2    528384    2625535    2097152    1G Linux RAID
/dev/nvme2n1p3   2625536   11014143    8388608    4G Linux RAID

/dev/nvme2n1p1         261868         0     261868   0% /boot/efi     # p1, could be any of nvme[012]
/dev/md0              1012428    262020     698084  28% /boot         # p2
/dev/md1              8144384   4702052    3007032  61% /             # p3

md0 : active raid1 nvme0n1p2[0] nvme2n1p2[1] nvme1n1p2[2]
      1046528 blocks super 1.2 [3/3] [UUU]

md1 : active raid5 nvme0n1p3[0] nvme2n1p3[1] nvme1n1p3[3]
      8378368 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

REBOOT.

Continue reading Preparing the SSC.

fdisk /dev/nvme0n1
# n -> +64G
# t -> swap
# n -> +512M  - new partition (for /sf/config)
# t -> 29     - type to 'Linux RAID'
# n
# t -> 29

repeat for /dev/nvme1n1 and /dev/nvme2n1. Verify with fdisk -l that all look the same.

addswap /dev/nvme0n1p4
addswap /dev/nvme1n1p4
addswap /dev/nvme2n1p4
swapon -a
swapon -s

mdadm --create /dev/md5 --level=5 --raid-devices=3 /dev/nvme0n1p5 /dev/nvme1n1p5 /dev/nvme2n1p5
mdadm --detail --scan  --verbose /dev/md5 | head -n1 | tee -a /etc/mdadm/mdadm.conf

mdadm --create /dev/md6 --level=5 --raid-devices=3 /dev/nvme0n1p6 /dev/nvme1n1p6 /dev/nvme2n1p6
mdadm --detail --scan  --verbose /dev/md6 | head -n1 | tee -a /etc/mdadm/mdadm.conf

update-initramfs -u
# Wait until the rebuild has finished.
cat /proc/mdstat

addsf /dev/md6

echo LUKSPASSWORD=$(head -c 1024 /dev/urandom | tr -dc '[:alpha:]' | head -c 22)
cryptsetup luksFormat /dev/md5
cryptsetup luksOpen /dev/md5 sfluks
mkfs -t xfs /dev/mapper/sfluks
mkdir /sec
mount -o nofail,noatime /dev/mapper/sfluks /sec

mv /sf/config /sec || mkdir /sf
ln -sf /sec/config /sf/config

After each reboot you need to:

cryptsetup luksOpen /dev/md5 sfluks
mount -o nofail,noatime /dev/mapper/sfluks /sec

Continue reading Installing the SSC.