-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix dkms installation of deb packages created with Alien. #15415
Conversation
I now did setup a testing environment with virtualbox. On Fedora I discovered an error in my code regarding the %preun scriptlet logic, which didn't turn up on Ubuntu or Mint. On Fedora I tested the RPM installation with zfs git master (2.2.99). Testing procedure was: All tests go well in regard to the changes I made with this pull request. Two things I did notice on Fedora:
Installing only zfs-dkms afterwards, doesn't install zfs though. The other thing is, that dkms seems to have problems cleaning the build area (or at least thinks it has). Because everything seems just fine after this:
Maybe it's a result of me testing a dozens of things, maybe not. But I don't think it's related to my changes. So to sum up: |
That patch badly breaks things on my Debian bookworm amd64 (vanilla, up to date at the time of writing). After having applied the patch, I can't produce the .deb packages any more:
I don't know what this is all about, and right now, I don't have the time to dig in. We'd be very happy if you could come to rescue and fix your patch, because 2.1.13 won't compile either, so currently we can't use recent ZFS versions in debian. Thank you very much, and best regards, Binarus |
The last force-push was just adding this: The %pre scriptlet requires dkms. This makes sure it's there at that point. |
@Binarus About your issue with 2.1.13, I'd need more info. |
@AllKind Thank you very much for your time! I just have tried compiling 2.2.0 for 8 hours now, to no avail. But back to our problem with 2.1.13: I have posted it also on the mailing list 9 days ago, but got no reaction. Well, the mailing list is the wrong place :-) The following is a literal copy of my post there: Hi all, I am currently trying to compile OpenZFS 2.1.13 on a Debian bullseye stock system, following the process that is outline here:
I don't checkout the master branch, though; instead, I unpack the respective asset (
I don't care about that one, though, because I always could live well with the non-native (rpm-converted) packages. On the other hand, that command is shown in the instructions mentioned above, so I think the problem is worth being reported.
There are a few lines more; I'll put the whole excerpt at the end of this message. Obviously, the linker can't find zfs_proctitle anywhere, either because it isn't there or it can't find the library it is in. What would be the easiest way to mitigate this issue? Best regards, and thank you very much in advance, Binarus
|
@Binarus zfs 2.1.3 does not support the native-* targets yet. But in general I think you are using the wrong guide. So if you want to build a 2.1.13 package, or a 2.2.0 package (the old way), avoiding the dkms install problem. I suggest using my patch ;-) Something like that should do the trick:
for building 2.2.0:
for building 2.1.13:
For the last step, I always copy the packages I want (i.e. without debug) installed to another folder and install from there. |
@Binarus |
@AllKind Thank you very much for your effort and for your time! In the meantime, I have managed to build a working ZFS 2.2.0 following the old way (that always had worked reliably before). I was kind of surprised by the contents of the page you linked, and of course I'll follow the procedures outlined there next time. It's probably better to build native debian packages than "aliens". The only thing in the new guide that I don't understand is why we should do Although it will be off-topic again, I would like to describe why it took me more than 8 hours to finally build 2.2.0. I am convinced that there are quite a few folks out there who encounter the same problems. The first problem were error messages that occurred when trying to install It took me a lot of time until I figured out that theses error messages were coming from a After having manually removed that package, installation of the debs that had been generated during the ZFS 2.2.0 build went without issues. But as soon as I used Then I remembered that several months ago I had issued But yesterday, I didn't issue Undoubtedly, I didn't have my most brilliant time yesterday, so this was hard to find out. The solution was simple, though: So I have to apologize for the noise. It was my own mistake. The build process of ZFS 2.2.0 works correctly, and so do the artifacts and debian packages that are generated. As noted above, I have only tested the classical method with the alienized packages that is described in the first link I gave. I don't have the time for further investigations because these problems plus a few new problems with ZFSBootMenu have now cost me two full days. In general, I personally find it a bit questionable that |
I found a problem with |
Turns out Here is an example output (the
As you can see in the last line, I already caught the action in my last change. As another consequence So I came up with this in
It's safe to assume the dkms sources are in /usr/src/. Other things added: Tests were run like this on Fedora38 and Ubuntu 22.04: 1 - Fresh install of zfs 2.1.12. Result: All checks passed! Edit: btw, could be this also fixes #15336. |
That's in the part for the pre-compiled
Most likely you hit: #15336. Just as a note. Next time put reports like yours into the Issue tracker, not here. |
Of course, the correct variable is |
@AllKind Thank you very much again! I didn't foresee that it would become an extended discussion. I thought that this would be the right place because the subject of this PR is the fix for a wrong behavior for the DEB packages and I found it appropriate to comment that this fix in its original form didn't work for me. Of course, in the future I'll follow your advice / suggestion and open a new issue instead. It's absolutely great that you fixed it now. Since I have ZFS 2.2.0 running on two servers since yesterday, I can't try your fixes (the servers are production). But I'll upgrade ZFS to 2.2.1 as soon as it is released and open a new issue if I find any problems with it (supposing that your patches are merged until then). Please allow a final remark for people who come across this page: The broken DEB installation had another side effect. Even after having successfully installed ZFS 2.2.0, the DKMS build did not run when upgrading or downgrading the kernel. I could solve that problem by installing the new kernel first and then manually running the DKMS build for that kernel. The procedure roughly did go like that:
When building, I was on kernel Then Best regards, Binarus |
Short summary of my last push:1 - Changed the location to retrieve the old zfs versions to Testing:Again I did extensive testing. Long version:1 - Changed the location to retrieve the old zfs versions to If someone has a better suggestion for a glob pattern, than 2 - Re-introduced the dkms add/remove parameter
3 - Added checking and handling of the broken You can see how successful it is in the gist I created. c: It does not do any harm. If the 4 - Added
EndnoteAfter now 4 complete days of reading and debugging, I'm pretty sure I covered it all. (But LOL - you never know what you don't know) Have a nice day! |
Just wanted to say a big thanks again for all your efforts and your time. I'll test it when OpenZFS 2.2.1 is available and report back. I'm hoping that the reason for your absence is a pleasant one! Best regards, Binarus |
Just added some comments and rebased. |
Thanks for this patch, got 2.2.0 running on my Debian 10 Server. |
Fwiw the amount of workarounds required makes me feel uneasy. Although that's a me issue 😅 I would kindly ask is that we get to the bottom what caused this in the first place. There are seemingly 2-3 different PRs and another 2-3 issues that discuss that topic. And so far I'm failing to see one that summarises the root/cause of the problem. If that's a dkms bug, please open an issue/PR and cc me. Thanks in advance. |
@evelikov For (I guess) a long time this was the rpm .spec: https://github.com/openzfs/zfs/blob/zfs-2.1.12/rpm/generic/zfs-dkms.spec.in As the author of this PR #13182 pointed out:
But the introduction of the Edit: the dangling symlink problem definitely came from the broken scriptlet, which never removed it from dkms tree, while the package itself (meaning the sources in /usr/src) was removed. We don't have enough information (and I don't think it's worth the time) to untangle each possible combination of the above mentioned problems, which lead to the 3 different issues that were reported. Important is, that my PR fixes it all ;-) |
Alien does not honour the %posttrans hook. So move the dkms uninstall/install scripts to the %pre/%post hooks in case of package install/upgrade. In case of package removal, handle that in %preun. Add removal of all old dkms modules. Add checking for broken 'dkms status'. Handle that as good as possible and warn the user about it. Also add more verbose messages about what we are doing. Signed-off-by: Mart Frauenlob <AllKind@fastest.cc>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for working through this and carefully testing it. Based on the positive feedback I'm going to pull this in.
Alien does not honour the %posttrans hook. So move the dkms uninstall/install scripts to the %pre/%post hooks in case of package install/upgrade. In case of package removal, handle that in %preun. Add removal of all old dkms modules. Add checking for broken 'dkms status'. Handle that as good as possible and warn the user about it. Also add more verbose messages about what we are doing. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Mart Frauenlob <AllKind@fastest.cc> Closes openzfs#15415
Alien does not honour the %posttrans hook. So move the dkms uninstall/install scripts to the %pre/%post hooks in case of package install/upgrade. In case of package removal, handle that in %preun. Add removal of all old dkms modules. Add checking for broken 'dkms status'. Handle that as good as possible and warn the user about it. Also add more verbose messages about what we are doing. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Mart Frauenlob <AllKind@fastest.cc> Closes openzfs#15415
Alien does not honour the %posttrans hook. So move the dkms uninstall/install scripts to the %pre/%post hooks in case of package install/upgrade. In case of package removal, handle that in %preun. Add removal of all old dkms modules. Add checking for broken 'dkms status'. Handle that as good as possible and warn the user about it. Also add more verbose messages about what we are doing. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Mart Frauenlob <AllKind@fastest.cc> Closes openzfs#15415 (cherry picked from commit 9ce567c)
Alien does not honour the %posttrans hook. So move the dkms uninstall/install scripts to the %pre/%post hooks in case of package install/upgrade. In case of package removal, handle that in %preun. Add removal of all old dkms modules. Add checking for broken 'dkms status'. Handle that as good as possible and warn the user about it. Also add more verbose messages about what we are doing. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Mart Frauenlob <AllKind@fastest.cc> Closes #15415 (cherry picked from commit 9ce567c)
FTR: this commit applies perfectly (via patch -p1) on top of zfs-2.1.14, but to no avail: DKMS remains broken in the packages resulting from EDIT: The above was with intermediate commit #0e92318; with the last one listed above (#9ce567c) it seems to have worked. Will know after my next reboot :-) apologies for the initially false report. |
Alien does not honour the %posttrans hook. So move the dkms uninstall/install scripts to the %pre/%post hooks in case of package install/upgrade. In case of package removal, handle that in %preun. Add removal of all old dkms modules. Add checking for broken 'dkms status'. Handle that as good as possible and warn the user about it. Also add more verbose messages about what we are doing. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Mart Frauenlob <AllKind@fastest.cc> Closes openzfs#15415
I'm still finding that 2.1.15 has regressed compared to previous 2.1.x versions with regard to the dkms install. In a chroot environment the module does not build:
In 2.1.14 with c5bbd80 reverted the install works and the zfs-dkms.postinst script looks like:
In 2.1.15:
I can get back to a working situation for my case by reverting: |
First off, dkms does not seem to be chroot save at all: dell/dkms#145
So dkms cannot find the kernel headers... Wouldn't the easiest solution be, to provide it with an environment where they can be found? I think usually /lib/modules/KERNEL_VERSION/build being a symbolic link to them. |
I took a quick look at the postinst from dkms and at first sight the only difference seems to be, that it passes the kernel version to the dkms install command.
|
The chroot environment is an offline build of a virtual machine disk image and uses a different kernel from the host so installing those headers is not the solution. Other software using dkms does not have this problem and until recent 2.1.x versions zfs did not have an issue. We've been building this way since 0.6.x releases. The common.postinst has handling exactly for the situation that the environment is a chroot:
|
Bit of a bummer I wasn't aware of that problem. My patch was around for months and was applied to 3 ZFS versions before. Another option I thought of is to get kernel version and kernel source from configure. |
I didn't have time to set up a testing VM yet and don't know enough about your (or in general) chroot setup, so I ask you to test before I create a PR.
|
Hi - just acknowledge I've seen the proposed change but I need to find some time for testing. |
I just noticed that instead of:
probably would be better |
Hi, With the patch (and the one line change above) the postinst was generated as:
This resulted in the module being built as expected in the chroot vm build. |
That's good news! That would be of tremendous help! Thank you! |
Correct, I did not set KVERS or KSRC. I'll have to examine the build pipeline to see where the variables can be set. I'll update once I've worked it out... |
Alien does not honour the %posttrans hook.
So move the dkms uninstall/install scripts to the %pre/%post hooks in case of package install/upgrade.
In case of package removal, handle that in %preun.
Add removal of all old dkms modules.
Add checking for broken 'dkms status'. Handle that as good as possible and warn the user about it.
Also add more verbose messages about what we are doing.
Motivation and Context
This is my first pull request ever. It's been a while since I used git and never before in a collaborating matter.
So please bear with me and I hope not to make too many mistakes.
I don't have any experience with building rpms and transforming them to deb packages.
But I dug myself through https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/ and https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html to get an idea of the things involved here.
Edit: Fixes: #15328, #15336 and #15253.
Description
In commit #13182 the rpm scriptlet hook
%posttrans
was introduced.But as it looks to me, Alien does not honour it, or there is no equivalent script hook for debian packages.
So to make sure the uninstall / install of the dkms modules happens in case of install/upgrade, I moved them to the %pre / %post hooks.
In case of package removal, handle that in %preun.
Also I added a check before running each command, to avoid a non zero exit status for the scripts.
Additionally I added verbose messages about what we are doing.
This change seems to ensure, that for each rpm hook a debian script is created by Alien.
Also I think it will always uninstall the dkms modules before installation of the new package and always will install the dkms modules after the new package has been installed.
How Has This Been Tested?
On my PC with Linux Mint 21.2, which is based on Ubuntu 22.04, I installed the zfs-dkms .deb file.
The dkms version used is 2.8.7.
In case of a fresh install, or an upgrade / re-install, everything went as expected.
Also the remove/purge actions yielded the expected result.
Edit: More tests were done. Descriptions in the replies below.
Types of changes
Checklist:
Signed-off-by
.