rpm repository manual minor version uptick

[azzid]

System information

Type	zfs-kmod
Distribution Name	CentOS
Distribution Version	7
Linux Kernel	3.10.0
Architecture	x86_64
ZFS Version	0.8.3-1
SPL Version	0.8.3-1

Describe the problem you're observing

Upgrading from CentOS 7.7 to 7.8 is my first minor release upgrade since starting to use zfsonlinux. It seem to me that the process is somewhat cumbersome. Maybe I've missed something obvious - but this is the issue I see:

http://download.zfsonlinux.org/epel/7.7/kmod/$basearch/ and http://download.zfsonlinux.org/epel/7.8/kmod/$basearch/ are tightly knit to their respective minor release, and their paths are "hard coded" into /etc/yum.repos.d/zfs.repo. This means I have to do manual intervention whenever centos change their repos from 7.7 to 7.8 - a change which is not necessarily obvious to me as their repo-files never change. Their paths simply have a 7 in the path which is a symlink to whatever is current.

Thinking that my zfs repo-file was unneccessarily specific by pointing to 7.7 I tried changing it to just 7 to mimic how centos works. This did not give me a current repo, but a really old one:

[root@host ~]# yum check-update
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * epel: ftp.cica.es
zfs-kmod                                                                                     | 2.9 kB  00:00:00     
Not using downloaded zfs-kmod/repomd.xml because it is older than what we have:
  Current   : Fri May  8 03:14:37 2020
  Downloaded: Fri May  5 23:34:53 2017

I was thinking I could get better control over the situation if I could mirror the entire repo and monitor the mirror for what is currently $latest, but the only supported way to download rpm's from download.zfsonlinux.org is using reposync - which only downloads what's necessary for the particular machine it is run on.

Should it not be possible to configure a centos machine using zfs in an way that would only require that you run yum upgrade to go from one minor release to the next?

Describe how to reproduce the problem

yum upgrade after new centos minor release

Include any warning/errors/backtraces from the system logs

n/a

[shodanshok]

@azzid this is a possible duplicate of #9421
I suggest you closing this issue and continuing on the other one.

[azzid]

@azzid this is a possible duplicate of #9421

I do not believe these two to be duplicates. Your suggestion

As a side note, how do you feel about having a zfs-release.el7_latest tracking the latest RHEL 7.x release, with both 0.7.x and 0.8.x, maybe with kmod enabled by default, for anyone closely following the latest RHEL/CentOS point release?

is in the same vein as this issue in that it's trying to be more compatible with the CentOS release schedule, but the issues themselves are separate.

[tonyhutter]

I think what's being proposed here is technically possible. Let's go down the rabbit hole...

Let's say we did update zfs.repo to use $releasever instead of hardcoding the major.minor:

  [zfs-kmod]
  name=ZFS on Linux for EL7 - kmod
- baseurl=http://download.zfsonlinux.org/epel/7.8/kmod/$basearch/
+ baseurl=http://download.zfsonlinux.org/epel/$releasever/kmod/$basearch/

That would allow you to use a single zfs-release.rpm for all version of Centos 7 ($releasever == 7). So problem solved! Not quite...

The centos versions have different kernels. So lets say we built zfs-0.8.3 for both Centos 7.7 (3.10.0-1062 kernel) and Centos 7.8 (3.10.0-1127). (in reality, we only built it for 7.7, but go with me here). We would need two kmod packages, one for each kernel. So which kmod-zfs-0.8.3-1.el7.x86_64.rpm would you put in the epel/7/kmod/x86_64/ repo dir? That's the issue.

One solution that may work, is to have multiple kmod-zfs package for a release, one for each supported kernel. Like:

kmod-zfs-0.8.3-1.3.10.0-1062.el7.x86_64.rpm (Centos 7.7)
kmod-zfs-0.8.3-1.3.10.0-1127.el7.x86_64.rpm (Centos 7.8)

Which would each put their modules respectively in the right /lib/modules directories, like:

/lib/modules/3.10.0-1062.9.1.el7.x86_64/extra/zfs/zfs.ko

.. and ...

/lib/modules/3.10.0-1127.9.1.el7.x86_64/extra/zfs/zfs.ko

One issue that may arise is that we don't currently "Requires: <specific_kernel_version>" in the kmod .spec file. Instead we just check that the functions we're using in the kernel are the same ones we compiled against:

rpm -qpR kmod-zfs-0.8.3-1.el7.x86_64.rpm  | grep kernel
...
kernel(add_disk) = 0xbc28fd2e
kernel(add_timer) = 0xbe2c0274
kernel(add_wait_queue_exclusive) = 0xee46885d
kernel(alloc_disk) = 0x7959fc3f
kernel(autoremove_wake_function) = 0xc8b57c27
...

It's very possible that these functions would be exactly the same between Centos 7 minor versions. So I don't know what would happen if we did have:

kmod-zfs-0.8.3-1.3.10.0-1062.el7.x86_64.rpm
kmod-zfs-0.8.3-1.3.10.0-1127.el7.x86_64.rpm 

... in the same epel/7/kmod/x86_64 repo. They would both have the same kernel functions as requirements. Which one would yum match? You could probably get around the ambiguity by having a hard "Requires: <specific_kernel_versio >" for each one. That also makes it nice if Centos upgrades to a newer kernel that we don't have kmod-zfs for yet - you'd probably get a warning saying: "kmod-zfs" requires < older kernel >, can't upgrade the kernel yet", which would be what we want.

So that takes care of kmod-zfs.

What about the userspace RPMs though? Would we ever want a different zfs-0.8.3-1.el7.x86_64.rpm for say Centos 7.7 vs Centos 7.8? We probably don't care. Most likey they would work on all the minor versions. People are most likely going to be running the latest Centos 7 anyway.

Just thinking out loud...

[shodanshok]

@azzid Sorry, I linked the wrong issue. See here for the correct one: #5015 (comment)

@tonyhutter yours are valid concerns. I am not a fan of specifying the exact kernel version - what will happen when upgrading to a minor kernel bugfix release, for example from kernel-3.10.0-957.10.1.el7.x86_64 to kernel-3.10.0-957.12.1.el7.x86_64? Do you need to list each kernel bugfix release? If so, it can quickly become unpractical.

I think most of the benefit can be reaped by adding a, say, zfs-release.el7_latest repo which simply tracks the kernel available in the latest CentOS 7.x point release (and zfs-release.el8_latest for the latest CentOS 8.x).

Or, even simpler, let http://download.zfsonlinux.org/epel/7/kmod/$basearch/ track the latest CentOS 7 release, rather than a very old 7.0 release. This is, by the way, the same method used by RH/CentOS: for example, http://centos.mirror.garr.it/centos/7/ points to the latest point release, while http://centos.mirror.garr.it/centos/7.0/ is used for the actual 7.0 release.

Am I missing something?

[tonyhutter]

@shodanshok I like your idea of having:

epel/7/kmod/x86_64/
epel/7.7/kmod/x86_64/
epel/7.6/kmod/x86_64/
...

...with epel/7/kmod/x86_64 always being the latest. Especially since you could easily specify the custom version you want with yum --releasever=7.7 install zfs if you needed to. I'll give it a try on our test server when I get some free cycles and see what issues arise.

[stale]

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.

[dghodgson]

Commenting as this is still an issue with CentOS 8.x

@tonyhutter did you happen to run into any issues in testing?

[tonyhutter]

@dghodgson thanks for the heads-up, this totally fell off my radar (I'll blame the pandemic...) The overall idea is still totally sound. Realistically though, it's going to be at least a month or longer before I can generate and test a new zfs-release rpm, due to some prior commitments. Please ping me if I haven't posted an update here in 6 weeks or so.

Assigning myself this PR.

[dghodgson]

Appreciate the update! Let me know if there's anything I can do to help test