Debian Init: Fall back to zfs.cache if /dev/disk/by-id is missing

[siboulet]

On my system (Debian Wheezy Xen virtual machine) /dev/disk/by-id is not populated. The zfs-mount init script by default looks for disks in /dev/disk/by-id.

USE_DISK_BY_ID needs to be set explicitly to 'no' in /etc/default/zfs for pools to be automatically imported on boot.

The init script should look if /dev/disk/by-id exists and fallback to zfs.cache.

[FransUrbo]

The init script should look if /dev/disk/by-id exists and fallback to zfs.cache.

Sounds reasonable, and I should have done that if it weren't for the fact
that the cache file is going away 'in a not to distant future'....

The USE_DISK_BY_ID is there for a reason - for YOU to choose. The
default is to use disk/by-id because it protects from reordering, which
is quite a problem...

So I'd probably (most likely) not going to address this....

[aarcane]

if the zpool.cache file is missing, does device reordering even matter anymore across reboots?

[FransUrbo]

if the zpool.cache file is missing, does device reordering even matter anymore across reboots?

What do you mean? Disk reordering ALWAYS matter, that's the whole point of disk/by-id (etc)...

[aarcane]

The cache and the by-id paths are commonly used together because the cached
device names cause a zpool import failure when devices suddenly change name
on reboot. Without the cache file, the pool would be imported by name
only, meaning that the available devices in any given folder will be
scanned, and the correct devices will be selected at import time regardless
of their past names.

[FransUrbo]

The cache and the by-id paths are commonly used together

You CAN'T use them together! You'd get a

-c is incompatible with -d

# zpool import -c /etc/zfs/zpool.cache -d /dev/disk/by-id
-c is incompatible with -d
usage:
        import [-d dir] [-D]
        import [-d dir | -c cachefile] [-F [-n]] <pool | id>
        import [-o mntopts] [-o property=value] ... 
            [-d dir | -c cachefile] [-D] [-f] [-m] [-N] [-R root] [-F [-n]] -a
        import [-o mntopts] [-o property=value] ... 
            [-d dir | -c cachefile] [-D] [-f] [-m] [-N] [-R root] [-F [-n]]
            <pool | id> [newpool]

[aarcane]

The cache file causes the cached names to be used, which is how the cache
file and by-id ate used together. The -d option, unless told not to,
generates a cache file* that includes among other things, the path to the
device nodes. Those paths can be changeable sdxn names, which causes
subsequent import failure, or they can be by-id paths, which is the common
use because they don't change on reboot. If the cache file is removed
completely, the names are not cached in any meaningful way, and can change
on reboot without ill effect.

*the cache file is generated as a normal part of the zpool import process.
The -d option just doesn't inhibit that.

[FransUrbo]

I'm completely at a loss now... What do you want!?

At the top of the issue you said that your machine does not populate /dev/disk/by-id. You yourself indicated that setting the USE_DISK_ID to 'no' for it to work on your system

I told you that that was exactly it's purpose - for YOU to choose which.

Also, I looked through the script file again (for another issue), and the import actually happens (is tried) using the cache file FIRST and only if that import fails, does it use /dev/disk/by-id. This will most likely be changed 'eventually', but for now that's how it works...

Also, again, if you rely on the cache file for everything to work, you will (eventually) be very disappointed - because it WILL be removed 'soon'.

[aarcane]

The op and I are two separate people. I apologize for confusing you.

[siboulet]

I didn't want to start a debate on the cache file.

I'm new to ZFS. My first experiment was to try ZFS in our test environment inside a Debian Wheezy VM running on Xen. Took me a few hours to figure why my ZFS pool where not becoming available after reboot. Turns out my VM doesn't have the /dev/disk/by-id populated.

Ideally I'd want the init script to figure my system doesn't have /dev/disk/by-id and fallback to trying with the cache file.

Or, at least the init script should warn me, and point me to the USE_DISK_BY_ID option in /etc/default/zfs.

Also, I looked through the script file again (for another issue), and the import actually happens (is tried) using the cache file FIRST and only if that import fails, does it use /dev/disk/by-id. This will most likely be changed 'eventually', but for now that's how it works...

Not on my system (debian-zfs).

I'm not exactly sure which init script the latest debian-zfs package use, I don't seems to be able to find it in the repository. However, the logic of the version I have is similar to the one I can find in Git here:

https://github.com/zfsonlinux/zfs/blob/master/etc/init.d/zfs.lsb.in#L68

If USE_DISK_BY_ID is set, it'll use /dev/disk/by-id independently if /dev/disk/by-id actually exists or not.

[FransUrbo]

Ideally I'd want the init script to figure my system doesn't have /dev/disk/by-id and fallback to trying with the cache file.

NOW I'm starting to understand what the whole issue is about! Sorry, but I understood 'initrd' (as in the initial boot disk/image), not 'init.d' (as in the startup scripts)!!

That part I can easily fix... Give me a couple of hours and I'll have a new version (in the snapshots package repository) made.

[FransUrbo]

Ok, I've created 0.6.3-2~wheezy packages and pushed them to the snapshot repository. Please test and comment...

[siboulet]

I'm having some problems building from snapshot/debian/wheezy/0.6.3-2_wheezy (and 0.6.3-3). I followed the instructions here: https://github.com/zfsonlinux/pkg-zfs.

dpkg-source: info: building zfs-linux using existing ./zfs-linux_0.6.3.orig.tar.gz
patching file config/dkms.m4
patching file config/user.m4
Hunk #1 FAILED at 2.
1 out of 1 hunk FAILED
patching file config/zfs-build.m4
Hunk #1 succeeded at 87 (offset 1 line).
dpkg-source: info: fuzz is not allowed when applying patches
dpkg-source: info: if patch '0002-Prevent-manual-builds-in-the-DKMS-source.patch' is correctly applied by quilt, use 'quilt refresh' to update it
dpkg-source: error: LC_ALL=C patch -t -F 0 -N -p1 -u -V never -g0 -E -b -B .pc/0002-Prevent-manual-builds-in-the-DKMS-source.patch/ --reject-file=- < pkg-zfs.orig.cOjkqy/debian/patches/0002-Prevent-manual-builds-in-the-DKMS-source.patch gave error exit status 1

I reviewed the patch and have commented on zfsonlinux/pkg-zfs@37f49f3

[FransUrbo]

This is not a support forum. If you have another problem with something else, do NOT use an issue which is about something completely different.

Please either start another issue, or (in this case) take it to the list.

[siboulet]

@FransUrbo Don't be rude. I'm just trying to help and compile the fix you pushed to the snapshot repository. Anyway, thanks for your patience. Closing this. Will follow changes in zfsonlinux/pkg-zfs@37f49f3

[FransUrbo]

It wasn't intended to be rude. It's hard enough to keep track of real issues (i.e. bugs) without the clutter of questions of all kinds...

Not being able to build packages IS (or might be - in this case it isn't) an issue, but it is NOT about "Debian Init: Fall back to zfs.cache if /dev/disk/by-id is missing". Two completely different things.. Next time, try to keep focus and on target...

Btw, I created 0.6.3-4~wheezy with your findings. Please try...

[siboulet]

I managed to test the new package, unfortunately it doesn't work. I suggest the following fix: siboulet/pkg-zfs@6c02edc. I can't create a PR since the snapshot tag you're working on isn't referenced by any branch in the Github repo. Thanks