Script zfs-functions missing from generated .deb packages, causes kernel panic #7904

ReimuHakurei · 2018-09-14T03:18:14Z

System information

Type	Version/Name
Distribution Name	Ubuntu Server
Distribution Version	18.04.1
Linux Kernel	4.15.0-34-generic
Architecture	amd64
ZFS Version	0.8.0-rc1

Describe the problem you're observing

When building DEB packages of ZFSonLinux with 'make deb', the script 'zfs-functions' does not end up in any of the generated DEB files.

At the very least, this results in a kernel panic when attempting to boot, if you are using a ZFS root filesystem. I would imagine it probably breaks other stuff too, but have not tested this.

The file does get generated in the source tree at ./etc/init.d/zfs-functions. Manually copying this to /etc/zfs/ (which is where Ubuntu expects it) allows the system to boot and function normally.

Modifying the DEB file to manually place this file in /etc/zfs/ also results in a package that installs and works correctly.

Describe how to reproduce the problem

Compile ZFS, then run 'make deb'. Attempt to use the generated packages.

Include any warning/errors/backtraces from the system logs

N/A

ghfields · 2019-01-30T19:53:43Z

@ColinIanKing This seems like something you fix distro side. Is there something that can be done here in the upstream that can help everyone?

mtippmann · 2019-02-04T22:15:34Z

Could this bug be added back to the 0.8.0 milestone? This breaks updating to 0.8.0 via the -rc tarballs here if you run Debian/Ubuntu and use ZFS as root filesystem.

This comes totally as a surprive as the zfs-initramfs package is installed and update-initramfs works fine.

Most users of the packages are likely building dkms packages and some are also using initramfs - even if it's not officially recommend/supported - including this script avoiding breakage would be nice :)

ReimuHakurei · 2019-02-04T22:23:57Z

For what it's worth, I'm currently building some functional packages for my personal use on all of my systems, so I'll post a link here when I'm done with that.

EDIT: https://reimuhakurei.net/zfs/

mtippmann · 2019-02-04T23:33:06Z

If anyone has this problem and access to serial console or something similiar: Add break=premountto linux cmdline in bootloader and you can then add the file i.e. pasting with cat <<'EOF'>>/etc/zfs/zfs-functions however over ipmi SoL pasting the whole file did break, but it worked in smaller pieces.

Adding the file to /etc/zfs/zfs-functions and running update-initramfs -k all -u works you can check with lsinitramfs initrd.img | grep zfs

cwedgwood · 2019-05-21T17:43:16Z

How does this work on rpm based systems?

A quick build and inspection of the generated .rpms I don't see the /etc/zfs/zfs-functions there either.

mtippmann · 2019-05-21T20:50:53Z

@cwedgwood rpm-based systems use dracut - that is probably completly different? In this case the file is missing from the zfs-initramfs package and is not included in the initramfs - but there is a dedicated dracut package that might work different - so maybe the problem is debian only? Also the contents of the rpm do not matter, it's only the initrd contents that need to work correctly.

cwedgwood · 2019-05-23T06:04:43Z

@mtippmann yeah, that makes sense ... i did a very quick hack to show some thoughts on what might make sense so people can comment, it almost certainly needs some tweaks/fixes before it's ready

mtippmann · 2019-05-23T09:12:58Z

@cwedgwood thanks for taking care of this!

dreamcat4 · 2019-07-25T12:39:00Z

Yep... This made my zfs boot install unbootable today. While trying to upgrade from ubuntu's inbuilt 0.7.12 to 0.8.1 Kernel panics now. Without having that file there.

Tried booting back to the previous kernel but same issue.

ColinIanKing · 2019-07-25T12:45:15Z

OK, to be clear, zfs-functions should be copied to /etc/zfs is the fix? I can add that to the ubuntu zfs packaging now.

Deltik · 2019-07-25T12:47:21Z

Yes, @ColinIanKing, having zfs-functions copied to /etc/zfs on the target system fixes this issue.

ColinIanKing · 2019-07-25T13:16:11Z

FYI: The Ubuntu packaging for zfsutls-linux 0.8.1 already copies this file to /etc/zfs, the debian/zfsutils-linux.install performs this at the installation stage.

Deltik · 2019-07-25T13:19:35Z

@ColinIanKing: Ubuntu's packaging never had this issue, as far as I know. The issue is with the .deb files built by make deb in this project.

ColinIanKing · 2019-07-25T13:22:53Z

Ah, apologies for my misunderstanding. Sorry for the noise.

c0d3z3r0 · 2019-08-01T17:07:16Z

This should be fixed by #9089. @ReimuHakurei could you test that, please?

ReimuHakurei · 2019-08-01T19:05:47Z

That looks like it would probably fix it, but it's been a while since I built packages so I'll have to double-check. I'll give it a whirl tomorrow.

ReimuHakurei · 2019-08-03T09:14:31Z

That PR actually throws a compilation error when I run 'make deb' on my system, saying zfs-functions and etc/default/zfs can't be found.

c0d3z3r0 · 2019-08-03T09:48:55Z

@ReimuHakurei Unfortunately I lost my crystal ball... scnr Can you share your build log, please? What debian release?

ReimuHakurei · 2019-08-05T07:58:49Z

The full log is quite large, but here's the relevant part:

[ -x ./scripts/zfs-tests.sh ] && \
    ./scripts/zfs-tests.sh -c
make[5]: Entering directory '/tmp/zfs-build-acingram-dl7bLSTg/BUILD/zfs-0.8.0'
make[5]: Nothing to be done for 'install-exec-am'.
make  install-data-hook
make[6]: Entering directory '/tmp/zfs-build-acingram-dl7bLSTg/BUILD/zfs-0.8.0'
make[6]: Nothing to be done for 'install-data-hook'.
make[6]: Leaving directory '/tmp/zfs-build-acingram-dl7bLSTg/BUILD/zfs-0.8.0'
make[5]: Leaving directory '/tmp/zfs-build-acingram-dl7bLSTg/BUILD/zfs-0.8.0'
make[4]: Leaving directory '/tmp/zfs-build-acingram-dl7bLSTg/BUILD/zfs-0.8.0'
make[3]: Leaving directory '/tmp/zfs-build-acingram-dl7bLSTg/BUILD/zfs-0.8.0'
make[2]: Leaving directory '/tmp/zfs-build-acingram-dl7bLSTg/BUILD/zfs-0.8.0'
+ find /tmp/zfs-build-acingram-dl7bLSTg/BUILDROOT/zfs-0.8.0-1.x86_64/usr/local/lib -name *.la -exec rm -f {} ;
+ /usr/lib/rpm/brp-compress
+ /usr/lib/rpm/brp-strip /usr/bin/strip
+ /usr/lib/rpm/brp-strip-static-archive /usr/bin/strip
+ /usr/lib/rpm/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump
Processing files: zfs-0.8.0-1.x86_64
error: File not found: /tmp/zfs-build-acingram-dl7bLSTg/BUILDROOT/zfs-0.8.0-1.x86_64/usr/local/etc/zfs/zfs-functions
error: File not found: /tmp/zfs-build-acingram-dl7bLSTg/BUILDROOT/zfs-0.8.0-1.x86_64/etc/default/zfs


RPM build errors:
    File not found: /tmp/zfs-build-acingram-dl7bLSTg/BUILDROOT/zfs-0.8.0-1.x86_64/usr/local/etc/zfs/zfs-functions
    File not found: /tmp/zfs-build-acingram-dl7bLSTg/BUILDROOT/zfs-0.8.0-1.x86_64/etc/default/zfs
Makefile:1335: recipe for target 'rpm-common' failed
make[1]: *** [rpm-common] Error 1
make[1]: Leaving directory '/home/acingram/zfs-test/zfs'
Makefile:1294: recipe for target 'rpm-utils' failed
make: *** [rpm-utils] Error 2

ReimuHakurei · 2019-08-05T14:37:36Z

Also likely related: the zfs-function script doesn't seem to actually get built. It's compiled from a file named zfs-functions.in, which has some paths that get filled in to it. I just compiled 0.8.1 stock from source to update my .debs I install on my systems, and had to do that manually because that file wasn't ever built during compilation.

c0d3z3r0 · 2019-08-05T15:58:36Z

The full log is quite large, but here's the relevant part:

I really need the full log to check this. You can upload text files to Github, so please do ;)

Also likely related: the zfs-function script doesn't seem to actually get built. It's compiled from a file named zfs-functions.in, which has some paths that get filled in to it. I just compiled 0.8.1 stock from source to update my .debs I install on my systems, and had to do that manually because that file wasn't ever built during compilation.

My PR fixes that.

ReimuHakurei · 2019-08-05T16:41:42Z

out.log

Try that.

c0d3z3r0 · 2019-08-05T18:16:10Z

@ReimuHakurei thanks, I'll have a look at this

c0d3z3r0 · 2019-08-05T18:54:14Z

@ReimuHakurei I can't reproduce this. Even the tests are all fine, see #9089.

this line confuses me....

Installing zfs-0.8.0-1.src.rpm

There is no commit hash... just to be sure... you tried to build this one #9089, right?

Which are your configure flags?

ReimuHakurei · 2019-08-06T01:20:36Z

Yeah, I did it from that PR. No configure flags.

I'll nuke the directory tree and try it on a clean tree again.

ReimuHakurei · 2019-08-06T02:35:11Z

Alright, I nuked the entire tree and restarted from scratch. This time it built DEBs but zfs-functions is not in the zfs DEB it builds.

I used the commands from this build guide, just like I always do when building ZFS:

https://github.com/zfsonlinux/zfs/wiki/Building-ZFS

Exact sequence of build commands that I ran:

git clone https://github.com/c0d3z3r0/zfs.git
cd zfs
git checkout for-upstream/initramfs_fixes
sh autogen.sh
./configure
make -s -j$(nproc)
make deb

So either I'm an idiot (which is quite possible), or there's still something broken there. Also, the files it generates are labeled 0.8.0-1 still, not 0.8.1-1.

c0d3z3r0 · 2019-08-06T05:40:19Z

Hmm... I will check this again later on a fresh Ubuntu Server instance. Maybe that makes a difference

c0d3z3r0 · 2019-08-06T12:47:17Z

@ReimuHakurei I just reworked the rpm spec again. Could you do another test, please?

ReimuHakurei · 2019-08-11T02:00:11Z

I just built DEBs and did a test install with ZFS root in a virtual machine, and it did boot and appear to function. The generated packages still have the dependencies being broken, but I'm not sure anything can be done about that as long as they're being translated from RPMs with alien.

c0d3z3r0 · 2019-08-11T07:56:31Z

Great, thanks for testing!

* contrib/initramfs: include /etc/default/zfs and /etc/zfs/zfs-functions At least debian needs /etc/default/zfs and /etc/zfs/zfs-functions for its initramfs. Include both in build when initramfs is configured. * contrib/initramfs: include 60-zvol.rules and zvol_id Include 60-zvol.rules and zvol_id and set udev as predependency instead of debians zdev. This makes debians additional zdev hook unneeded. * Correct initconfdir substitution for some distros Not every Linux distro is using @sysconfdir@/default but @initconfdir@ which is already determined by configure. Let's use it. * systemd: prevent possible conflict between systemd and sysvinit Systemd will not load a sysvinit service if a unit exists with the same name. This prevents conflicts between sysvinit and systemd. In ZFS there is one sysvinit service that does not have a systemd service but a target counterpart, zfs-import.target. Usually it does not make any sense to install both but it is possisble. Let's prevent any conflict by masking zfs-import.service by default. This does not harm even if init.d/zfs-import does not exist. Reviewed-by: Chris Wedgwood <cw@f00f.org> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Tested-by: Alex Ingram <reimu@reimuhakurei.net> Tested-by: Dreamcat4 <dreamcat4@gmail.com> Signed-off-by: Michael Niewöhner <foss@mniewoehner.de> Closes #7904 Closes #9089

* contrib/initramfs: include /etc/default/zfs and /etc/zfs/zfs-functions At least debian needs /etc/default/zfs and /etc/zfs/zfs-functions for its initramfs. Include both in build when initramfs is configured. * contrib/initramfs: include 60-zvol.rules and zvol_id Include 60-zvol.rules and zvol_id and set udev as predependency instead of debians zdev. This makes debians additional zdev hook unneeded. * Correct initconfdir substitution for some distros Not every Linux distro is using @sysconfdir@/default but @initconfdir@ which is already determined by configure. Let's use it. * systemd: prevent possible conflict between systemd and sysvinit Systemd will not load a sysvinit service if a unit exists with the same name. This prevents conflicts between sysvinit and systemd. In ZFS there is one sysvinit service that does not have a systemd service but a target counterpart, zfs-import.target. Usually it does not make any sense to install both but it is possisble. Let's prevent any conflict by masking zfs-import.service by default. This does not harm even if init.d/zfs-import does not exist. Reviewed-by: Chris Wedgwood <cw@f00f.org> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Tested-by: Alex Ingram <reimu@reimuhakurei.net> Tested-by: Dreamcat4 <dreamcat4@gmail.com> Signed-off-by: Michael Niewöhner <foss@mniewoehner.de> Closes openzfs#7904 Closes openzfs#9089

* contrib/initramfs: include /etc/default/zfs and /etc/zfs/zfs-functions At least debian needs /etc/default/zfs and /etc/zfs/zfs-functions for its initramfs. Include both in build when initramfs is configured. * contrib/initramfs: include 60-zvol.rules and zvol_id Include 60-zvol.rules and zvol_id and set udev as predependency instead of debians zdev. This makes debians additional zdev hook unneeded. * Correct initconfdir substitution for some distros Not every Linux distro is using @sysconfdir@/default but @initconfdir@ which is already determined by configure. Let's use it. * systemd: prevent possible conflict between systemd and sysvinit Systemd will not load a sysvinit service if a unit exists with the same name. This prevents conflicts between sysvinit and systemd. In ZFS there is one sysvinit service that does not have a systemd service but a target counterpart, zfs-import.target. Usually it does not make any sense to install both but it is possisble. Let's prevent any conflict by masking zfs-import.service by default. This does not harm even if init.d/zfs-import does not exist. Reviewed-by: Chris Wedgwood <cw@f00f.org> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Tested-by: Alex Ingram <reimu@reimuhakurei.net> Tested-by: Dreamcat4 <dreamcat4@gmail.com> Signed-off-by: Michael Niewöhner <foss@mniewoehner.de> Closes #7904 Closes #9089

behlendorf added the Type: Building Indicates an issue related to building binaries label Sep 17, 2018

behlendorf added this to the 0.8.0 milestone Sep 17, 2018

behlendorf removed this from the 0.8.0 milestone Oct 12, 2018

cwedgwood mentioned this issue May 23, 2019

[RFC] 'Fixes' for initramfs creation #8796

Closed

12 tasks

ghfields mentioned this issue Jul 31, 2019

initramfs fixes #9089

Merged

12 tasks

behlendorf closed this as completed in #9089 Aug 16, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Script zfs-functions missing from generated .deb packages, causes kernel panic #7904

Script zfs-functions missing from generated .deb packages, causes kernel panic #7904

ReimuHakurei commented Sep 14, 2018

ghfields commented Jan 30, 2019

mtippmann commented Feb 4, 2019 •

edited

Loading

ReimuHakurei commented Feb 4, 2019 •

edited

Loading

mtippmann commented Feb 4, 2019 •

edited

Loading

cwedgwood commented May 21, 2019

mtippmann commented May 21, 2019

cwedgwood commented May 23, 2019

mtippmann commented May 23, 2019

dreamcat4 commented Jul 25, 2019

ColinIanKing commented Jul 25, 2019

Deltik commented Jul 25, 2019

ColinIanKing commented Jul 25, 2019

Deltik commented Jul 25, 2019

ColinIanKing commented Jul 25, 2019

c0d3z3r0 commented Aug 1, 2019

ReimuHakurei commented Aug 1, 2019

ReimuHakurei commented Aug 3, 2019

c0d3z3r0 commented Aug 3, 2019

ReimuHakurei commented Aug 5, 2019

ReimuHakurei commented Aug 5, 2019

c0d3z3r0 commented Aug 5, 2019

ReimuHakurei commented Aug 5, 2019

c0d3z3r0 commented Aug 5, 2019

c0d3z3r0 commented Aug 5, 2019

ReimuHakurei commented Aug 6, 2019

ReimuHakurei commented Aug 6, 2019

c0d3z3r0 commented Aug 6, 2019 •

edited

Loading

c0d3z3r0 commented Aug 6, 2019

ReimuHakurei commented Aug 11, 2019

c0d3z3r0 commented Aug 11, 2019

Script zfs-functions missing from generated .deb packages, causes kernel panic #7904

Script zfs-functions missing from generated .deb packages, causes kernel panic #7904

Comments

ReimuHakurei commented Sep 14, 2018

System information

Describe the problem you're observing

Describe how to reproduce the problem

Include any warning/errors/backtraces from the system logs

ghfields commented Jan 30, 2019

mtippmann commented Feb 4, 2019 • edited Loading

ReimuHakurei commented Feb 4, 2019 • edited Loading

mtippmann commented Feb 4, 2019 • edited Loading

cwedgwood commented May 21, 2019

mtippmann commented May 21, 2019

cwedgwood commented May 23, 2019

mtippmann commented May 23, 2019

dreamcat4 commented Jul 25, 2019

ColinIanKing commented Jul 25, 2019

Deltik commented Jul 25, 2019

ColinIanKing commented Jul 25, 2019

Deltik commented Jul 25, 2019

ColinIanKing commented Jul 25, 2019

c0d3z3r0 commented Aug 1, 2019

ReimuHakurei commented Aug 1, 2019

ReimuHakurei commented Aug 3, 2019

c0d3z3r0 commented Aug 3, 2019

ReimuHakurei commented Aug 5, 2019

ReimuHakurei commented Aug 5, 2019

c0d3z3r0 commented Aug 5, 2019

ReimuHakurei commented Aug 5, 2019

c0d3z3r0 commented Aug 5, 2019

c0d3z3r0 commented Aug 5, 2019

ReimuHakurei commented Aug 6, 2019

ReimuHakurei commented Aug 6, 2019

c0d3z3r0 commented Aug 6, 2019 • edited Loading

c0d3z3r0 commented Aug 6, 2019

ReimuHakurei commented Aug 11, 2019

c0d3z3r0 commented Aug 11, 2019

mtippmann commented Feb 4, 2019 •

edited

Loading

ReimuHakurei commented Feb 4, 2019 •

edited

Loading

mtippmann commented Feb 4, 2019 •

edited

Loading

c0d3z3r0 commented Aug 6, 2019 •

edited

Loading