[Microshift] Disk resize not works with microshift preset

[praveenkumar]

As per docs https://access.redhat.com/documentation/en-us/red_hat_build_of_microshift/4.12/html/installing/microshift-install-rpm#microshift-install-rpm-preparing_microshift-install-rpm we need to configure instance to have a logical volume manager (LVM) volume group (VG) with sufficient capacity for the persistent volumes (PVs) of your workload. As of now we are using https://github.com/crc-org/snc/blob/master/microshift.sh#L101 to set the lvm root size to 15G and start the VM with 31G of disk size https://github.com/crc-org/snc/blob/master/microshift.sh#L137 .

So the disk layout looks like

$ lsblk 
NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda           252:0    0   31G  0 disk 
|-vda1        252:1    0  200M  0 part /boot/efi
|-vda2        252:2    0  800M  0 part /boot
`-vda3        252:3    0   30G  0 part 
  `-rhel-root 253:0    0   15G  0 lvm  /sysroot

$ sudo fdisk -l
Disk /dev/vda: 31 GiB, 33285996544 bytes, 65011712 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: AEF4BC58-766E-4B6B-8892-DB02BC73A2BA

Device       Start      End  Sectors  Size Type
/dev/vda1     2048   411647   409600  200M EFI System
/dev/vda2   411648  2050047  1638400  800M Linux filesystem
/dev/vda3  2050048 65009663 62959616   30G Linux LVM


Disk /dev/mapper/rhel-root: 15 GiB, 16106127360 bytes, 31457280 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

$ sudo pvdisplay 
  --- Physical volume ---
  PV Name               /dev/vda3
  VG Name               rhel
  PV Size               30.02 GiB / not usable 2.00 MiB
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              7685
  Free PE               3845
  Allocated PE          3840
  PV UUID               rtrpuW-0gc6-OOi8-EoXf-cUyH-i973-nD2riy

$ sudo lvdisplay 
  --- Logical volume ---
  LV Path                /dev/rhel/root
  LV Name                root
  VG Name                rhel
  LV UUID                95GLN0-IMZJ-MKxD-ILsN-dfrV-l1bX-7kQ56R
  LV Write Access        read/write
  LV Creation host, time localhost.localdomain, 2023-04-25 02:31:42 -0400
  LV Status              available
  # open                 1
  LV Size                15.00 GiB
  Current LE             3840
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:0

$ sudo vgdisplay 
  --- Volume group ---
  VG Name               rhel
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  4
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               <30.02 GiB
  PE Size               4.00 MiB
  Total PE              7685
  Alloc PE / Size       3840 / 15.00 GiB
  Free  PE / Size       3845 / <15.02 GiB
  VG UUID               BOSiVx-VB0a-RyHO-55AW-AVd2-2Vn9-Qd9wY0

In our current resize disk codebase we don't handle to expand this disk partition structure so we need to have a different code path to extend the disk size.

$ growpart /dev/vda 3
$ lsblk 
$ pvs
$ pvresize /dev/vda3

[cfergeau]

Iirc, rhcos has a systemd unit automatically growing the partitions on boot if there's space available? Maybe we could reuse a similar approach here?

[praveenkumar]

Iirc, rhcos has a systemd unit automatically growing the partitions on boot if there's space available? Maybe we could reuse a similar approach here?

@cfergeau which systemd unit file is that, I can try to use it?

[praveenkumar]

@cfergeau you meant https://www.freedesktop.org/software/systemd/man/systemd-repart.html or https://www.freedesktop.org/software/systemd/man/systemd-growfs.html one because both are not available in RHCOS by default

[praveenkumar]

By default the current disk structure in microshift is #3624 (comment) , here /sysroot is mounted as lv which is 15G and then 15G is free for the openshift storage workload ( for Persistent volumes)

In today meeting we discussed some of the scenario what should a user expect when he wants to extend the disk size using --disk-size option.

1. Whatever user provide as --disk-size we just add that size to the root LV which is mounted the /sysroot so if user put --disk-size 40 then he should see 25G for sysroot and we don't change the free space which is by default 15G.
2. We don't extend the /sysroot LV and just add the extra provided space to free space so now user see the 15G for sysroot which is default and 25G as free space for openshift workload
3. We create a separate config option which allow users to specify what they want, do they want to extend the /sysroot because they need to pull more images for there workload or they want to extend free space because they have application which require more PV's.
4. We extend it half for the /sysroot and half for the free space.

any other scenario if I miss something ..

[cfergeau]

1. sounds fine to me, but this will need some documentation. We can adjust the behaviour later with additional configuration option, ... when we get questions about k8s persistent storage.

[praveenkumar]

It is now merged with suggestion #3624 (comment)

[gbraad]

It is necessary to understand what /sysroot and 'freespace' are used for.

• Where do the containers live?
• What about persistent storage?