[RFE] qemu IDs in disks and butane

[till]

Current situation

Related: #1057

I am currently pre-processing my butane configs, so that's why this is a go template:

storage:
  filesystems:
    - path: /my-disk
      device: /dev/disk/by-id/{{.QemuDisk}}
      format: ext4
      label: CUSTOMER
      wipe_filesystem: true
      with_mount_unit: true

When flatcar is booted on OpenStack the disk ID is the only predictable identifier to refer to the disk on first boot (to format and to label it). This ID is not the full disk ID, but truncated on that platform which is (I think) currently not documented or mentioned in the docs and instead non predictable device names (such as /dev/sdX) are often used in examples.

Impact

Names such as /dev/sdX are not predictable and mounting might fail when we use those in butane. It's a 50/50 chance that the node provisions when they are used.

The IDs from qemu are obscure and require truncating so they match created IDs from OpenStack which can be mapped to the system.

Here's a snippet from my preprocessor for the butane config:

	tpl, err := ignition.Render(c.fs, "resources/"+tmplFile, ignition.TemplateVars{
		Hostname:  name,
		PublicKey: c.publicKey,
		// openstack
		QemuDisk: fmt.Sprintf("scsi-0QEMU_QEMU_HARDDISK_%s", c.customerVolume[:20]),
	})

The customerVolume is the OpenStack ID of the volume I created — and then the first 20 characters of it.

Ideal future situation

This is a two in one:

1. when you mount/format a filesystem via ignition (on OpenStack), you need to refer to the disk — somehow — ideally without parsing OpenStack IDs (but also not sure if this is achievable or within the scope of Flatcar Linux)
2. the resulting mount unit could use /dev/disk/by-label when a label was supplied (with_mount_unit: true)

Implementation options

[ Optional: please provide one or more options for implementing the feature requested ]

Additional information

Looked around, but I can't find much, but here is a /dev/disk/by-* output, I can share more if that is needed/helpful:

core@node-001 ~ $ ls -lah /dev/disk/by-uuid/    
total 0
drwxr-xr-x.  2 root root 180 Oct  6 13:14 .
drwxr-xr-x. 10 root root 200 Oct  6 13:13 ..
lrwxrwxrwx.  1 root root  10 Oct  6 13:14 0338e1b3-ae30-4c18-aa0c-7d4ef32f152c -> ../../sda3
lrwxrwxrwx.  1 root root  10 Oct  6 13:14 0471460d-648f-404c-98b7-07b7675daac1 -> ../../sda4
lrwxrwxrwx.  1 root root   9 Oct  6 13:14 2023-10-06-14-14-47-00 -> ../../sr0
lrwxrwxrwx.  1 root root  10 Oct  6 13:14 2E51-C0EE -> ../../sda1
lrwxrwxrwx.  1 root root  10 Oct  6 13:14 631d441d-2f25-4aee-81d0-253005fd81fa -> ../../sda6
lrwxrwxrwx.  1 root root   9 Oct  6 13:14 7640e4ff-1acd-4e95-be26-d55b50cea208 -> ../../sdb
lrwxrwxrwx.  1 root root  10 Oct  6 13:14 b4569ff1-e591-4df3-8f90-b5798f8a38b7 -> ../../sda9
core@node-001 ~ $ ls -lah /dev/disk/by-id/  
total 0
drwxr-xr-x.  2 root root 280 Oct  6 13:14 .
drwxr-xr-x. 10 root root 200 Oct  6 13:13 ..
lrwxrwxrwx.  1 root root   9 Oct  6 13:14 ata-QEMU_DVD-ROM_QM00001 -> ../../sr0
lrwxrwxrwx.  1 root root  10 Oct  6 13:14 dm-name-usr -> ../../dm-0
lrwxrwxrwx.  1 root root  10 Oct  6 13:14 dm-uuid-CRYPT-VERITY-6bb063668e6a40b788474138cf9eb83a-usr -> ../../dm-0
lrwxrwxrwx.  1 root root   9 Oct  6 13:14 scsi-0QEMU_QEMU_HARDDISK_32b0089e-e51e-48e4-a -> ../../sda
lrwxrwxrwx.  1 root root  10 Oct  6 13:14 scsi-0QEMU_QEMU_HARDDISK_32b0089e-e51e-48e4-a-part1 -> ../../sda1
lrwxrwxrwx.  1 root root  10 Oct  6 13:14 scsi-0QEMU_QEMU_HARDDISK_32b0089e-e51e-48e4-a-part2 -> ../../sda2
lrwxrwxrwx.  1 root root  10 Oct  6 13:14 scsi-0QEMU_QEMU_HARDDISK_32b0089e-e51e-48e4-a-part3 -> ../../sda3
lrwxrwxrwx.  1 root root  10 Oct  6 13:14 scsi-0QEMU_QEMU_HARDDISK_32b0089e-e51e-48e4-a-part4 -> ../../sda4
lrwxrwxrwx.  1 root root  10 Oct  6 13:14 scsi-0QEMU_QEMU_HARDDISK_32b0089e-e51e-48e4-a-part6 -> ../../sda6
lrwxrwxrwx.  1 root root  10 Oct  6 13:14 scsi-0QEMU_QEMU_HARDDISK_32b0089e-e51e-48e4-a-part7 -> ../../sda7
lrwxrwxrwx.  1 root root  10 Oct  6 13:14 scsi-0QEMU_QEMU_HARDDISK_32b0089e-e51e-48e4-a-part9 -> ../../sda9
lrwxrwxrwx.  1 root root   9 Oct  6 13:14 scsi-0QEMU_QEMU_HARDDISK_401211b8-720d-4d1f-8 -> ../../sdb

[jepio]

The entries under /dev/disk/by-path should be stable assuming you use a stable device model in openstack/qemu

[till]

The entries under /dev/disk/by-path should be stable assuming you use a stable device model in openstack/qemu

What is a stable device model?

[jepio]

qemu defines versioned "machine" models - pc or q35, and they define how things are plugged together internally. The "/dev/disk/by-path" link is derived from that.

I don't know anything about openstack so i cant tell you how they expose this or what actions at the openstack level would lead to changes. But that path is probably the most identifier you can get.

[till]

Ah, I can investigate. Thank you! We use an OpenStack distro from Virtuozzo, so I don't know either.

I just wanted to open this, as @pothos mentioned that on the discussion.

Maybe an example in the docs would be good as well? I can PR that if that's acceptable.

[pothos]

For the topic of specifying the disk I wondered sometimes thought about a way of referring to to the boot disk or a free disk through something like a variable. With new systemd versions there is also /dev/disk/by-diskseq/ but I think this could just serve as a workaround and is not yet the solution (I hope that systemd generates some nicer references instead of having this as Ignition-only feature).

For the mount unit, yes, Ignition should rather prefer the label it can find from the filesystem. That said, you could also first create a partition with a partlabel, then use that for the filesystem entry and you'll also have this partlabel name be used for the mount unit. → Maybe we should document that more explicitly.