Add rhel 8+ dnf support

[Stromweld]

Signed-off-by: Corey Hemminger hemminger@hotmail.com

[Stromweld]

this should fix #47

[Stromweld]

I am not sure why the DCO is not working.

[rndmh3ro]

So I tested it on Rocky Linux 8 and it works great. However on Rocky Linux 9 it fails:

[root@99149d9e5ef0 /]# /usr/libexec/platform-python -c 'import dnf; base = dnf.Base(); base.read_all_repos(); base.fill_sack(); q = base.sack.query(); list = list(q.upgrades()); res = ["{\"name\":\""+x.name+"\",\version\":\""+x.version+"-"+x.release+"\",\"arch\":\""+x.arch+"\",\"repository\":\""+x.reponame+"\"}" for x in list]; print("{\"available\":["+",".join(res)+"]}")'
Errors during downloading metadata for repository 'baseos':
  - Status code: 404 for http://dl.rockylinux.org/$contentdir/9/BaseOS/x86_64/os/repodata/repomd.xml (IP: 10.121.39.112)
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/dnf/repo.py", line 573, in load
    ret = self._repo.load()
  File "/usr/lib64/python3.9/site-packages/libdnf/repo.py", line 331, in load
    return _repo.Repo_load(self)
libdnf._error.Error: Failed to download metadata for repo 'baseos': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.9/site-packages/dnf/base.py", line 399, in fill_sack
    self._add_repo_to_sack(r)
  File "/usr/lib/python3.9/site-packages/dnf/base.py", line 139, in _add_repo_to_sack
    repo.load()
  File "/usr/lib/python3.9/site-packages/dnf/repo.py", line 580, in load
    raise dnf.exceptions.RepoError(str(e))
dnf.exceptions.RepoError: Failed to download metadata for repo 'baseos': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

Can you please check this?

[Stromweld]

That looks to be an error either in the inter tubes or in the repo config file. Code is working but if dns can’t get package list from repo it’s going to fail. You should get the same error running ‘dnf upgrade’ manually.

[rndmh3ro]

That's what I thought, too. However with dnf upgrade it works.

[Stromweld]

Ok so that is odd and I'm not sure what the difference is but I found the same issue as you on rockylinux-9 but on almalinux-9 which is where I tested and got the code working originally it works just fine.

[root@default-almalinux-9 ~]# /usr/libexec/platform-python -c 'import dnf; base = dnf.Base(); base.read_all_repos(); base.fill_sack(); q = base.sack.query(); list = list(q.upgrades()); res = ["{\"name\":\""+x.name+"\",\"version\":\""+x.version+"-"+x.release+"\",\"arch\":\""+x.arch+"\",\"repository\":\""+x.reponame+"\"}" for x in list]; print("{\"available\":["+",".join(res)+"]}")'
{"available":[{"name":"selinux-policy-devel","version":"34.1.29-1.el9_0.2","arch":"noarch","repository":"appstream"},{"name":"NetworkManager","version":"1.36.0-5.el9_0","arch":"x86_64","repository":"baseos"},{"name":"NetworkManager-libnm","version":"1.36.0-5.el9_0","arch":"x86_64","repository":"baseos"},{"name":"NetworkManager-team","version":"1.36.0-5.el9_0","arch":"x86_64","repository":"baseos"},{"name":"NetworkManager-tui","version":"1.36.0-5.el9_0","arch":"x86_64","repository":"baseos"},{"name":"almalinux-gpg-keys","version":"9.0-4.el9","arch":"x86_64","repository":"baseos"},{"name":"almalinux-release","version":"9.0-4.el9","arch":"x86_64","repository":"baseos"},{"name":"almalinux-repos","version":"9.0-4.el9","arch":"x86_64","repository":"baseos"},{"name":"ca-certificates","version":"2022.2.54-90.2.el9_0","arch":"noarch","repository":"baseos"},{"name":"cryptsetup-libs","version":"2.4.3-4.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"curl","version":"7.76.1-14.el9_0.4","arch":"x86_64","repository":"baseos"},{"name":"curl","version":"7.76.1-14.el9_0.5","arch":"x86_64","repository":"baseos"},{"name":"dbus-broker","version":"28-5.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"expat","version":"2.2.10-12.el9_0.2","arch":"x86_64","repository":"baseos"},{"name":"glibc","version":"2.34-28.el9_0.2","arch":"x86_64","repository":"baseos"},{"name":"glibc-common","version":"2.34-28.el9_0.2","arch":"x86_64","repository":"baseos"},{"name":"glibc-gconv-extra","version":"2.34-28.el9_0.2","arch":"x86_64","repository":"baseos"},{"name":"glibc-langpack-en","version":"2.34-28.el9_0.2","arch":"x86_64","repository":"baseos"},{"name":"gnupg2","version":"2.3.3-2.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel","version":"5.14.0-70.17.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel","version":"5.14.0-70.22.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel","version":"5.14.0-70.26.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel-core","version":"5.14.0-70.17.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel-core","version":"5.14.0-70.22.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel-core","version":"5.14.0-70.26.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel-modules","version":"5.14.0-70.17.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel-modules","version":"5.14.0-70.22.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel-modules","version":"5.14.0-70.26.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel-tools","version":"5.14.0-70.17.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel-tools","version":"5.14.0-70.22.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel-tools","version":"5.14.0-70.26.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel-tools-libs","version":"5.14.0-70.17.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel-tools-libs","version":"5.14.0-70.22.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"kernel-tools-libs","version":"5.14.0-70.26.1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"libarchive","version":"3.5.3-2.el9_0","arch":"x86_64","repository":"baseos"},{"name":"libcurl","version":"7.76.1-14.el9_0.4","arch":"x86_64","repository":"baseos"},{"name":"libcurl","version":"7.76.1-14.el9_0.5","arch":"x86_64","repository":"baseos"},{"name":"libdnf","version":"0.65.0-5.1.el9_0.alma","arch":"x86_64","repository":"baseos"},{"name":"libgcrypt","version":"1.10.0-4.el9_0","arch":"x86_64","repository":"baseos"},{"name":"libgcrypt","version":"1.10.0-5.el9_0","arch":"x86_64","repository":"baseos"},{"name":"libsss_certmap","version":"2.6.2-4.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"libsss_idmap","version":"2.6.2-4.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"libsss_nss_idmap","version":"2.6.2-4.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"libsss_sudo","version":"2.6.2-4.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"libxml2","version":"2.9.13-1.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"openldap","version":"2.6.2-1.el9_0","arch":"x86_64","repository":"baseos"},{"name":"openssh","version":"8.7p1-10.el9_0","arch":"x86_64","repository":"baseos"},{"name":"openssh-clients","version":"8.7p1-10.el9_0","arch":"x86_64","repository":"baseos"},{"name":"openssh-server","version":"8.7p1-10.el9_0","arch":"x86_64","repository":"baseos"},{"name":"openssl","version":"3.0.1-41.el9_0","arch":"x86_64","repository":"baseos"},{"name":"openssl-libs","version":"3.0.1-41.el9_0","arch":"x86_64","repository":"baseos"},{"name":"pam","version":"1.5.1-9.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"pcre2","version":"10.37-5.el9_0","arch":"x86_64","repository":"baseos"},{"name":"pcre2-syntax","version":"10.37-5.el9_0","arch":"noarch","repository":"baseos"},{"name":"procps-ng","version":"3.3.17-5.el9_0","arch":"x86_64","repository":"baseos"},{"name":"python3-hawkey","version":"0.65.0-5.1.el9_0.alma","arch":"x86_64","repository":"baseos"},{"name":"python3-libdnf","version":"0.65.0-5.1.el9_0.alma","arch":"x86_64","repository":"baseos"},{"name":"rsync","version":"3.2.3-9.el9_0.2","arch":"x86_64","repository":"baseos"},{"name":"selinux-policy","version":"34.1.29-1.el9_0.2","arch":"noarch","repository":"baseos"},{"name":"selinux-policy-targeted","version":"34.1.29-1.el9_0.2","arch":"noarch","repository":"baseos"},{"name":"sssd-client","version":"2.6.2-4.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"sssd-common","version":"2.6.2-4.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"sssd-kcm","version":"2.6.2-4.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"sssd-nfs-idmap","version":"2.6.2-4.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"systemd","version":"250-6.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"systemd-libs","version":"250-6.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"systemd-pam","version":"250-6.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"systemd-rpm-macros","version":"250-6.el9_0.1","arch":"noarch","repository":"baseos"},{"name":"systemd-udev","version":"250-6.el9_0.1","arch":"x86_64","repository":"baseos"},{"name":"tzdata","version":"2022c-1.el9_0","arch":"noarch","repository":"baseos"},{"name":"vim-minimal","version":"8.2.2637-16.el9_0.2","arch":"x86_64","repository":"baseos"},{"name":"vim-minimal","version":"8.2.2637-16.el9_0.3","arch":"x86_64","repository":"baseos"}]}

[Stromweld]

looking at the mirrorlist= line in both repos and the url output from the error I'm seeing that rockylinux adds this $rltype to the end of it's url where almalinux doesn't have that. The rest of variables in the url are substituted properly except that one in rocky-9.

[rndmh3ro]

This command seems to work:

/usr/libexec/platform-python -c 'import dnf; base = dnf.Base(); conf = base.conf; conf.substitutions["rltype"] = ""; base.read_all_repos(); base.fill_sack(); q = base.sack.query(); list = list(q.upgrades()); res = ["{\"name\":\""+x.name+"\",\"version\":\""+x.version+"-"+x.release+"\",\"arch\":\""+x.arch+"\",\"repository\":\""+x.reponame+"\"}" for x in list]; print("{\"available\":["+",".join(res)+"]}")'

The added part is conf = base.conf; conf.substitutions["rltype"] = ""; - this worked for me in almalinux 8 and 9 as well as rockylinux 8 and 9.

[Stromweld]

That'll work for the mirror list $rltype. Another issue is if you switch from mirrorlist to baseurl then the $contentdir variable will fail. I'm trying to figure out a way to have it load the variables in /etc/dnf/vars directory where those live so that it'll future proof it and not be broken if there is a change to any of those variables.

[Stromweld]

I found the definitions in the substitutions class for loading them from the vars files at /usr/lib/python3.9/site-packages/dnf/conf/substitutions.py but don't know python enough to figure out how to use it in command we have.

[Stromweld]

ok I figured it out:

/usr/libexec/platform-python -c 'import dnf; base = dnf.Base(); conf = base.conf; conf.substitutions.update_from_etc(conf.installroot); conf.substitutions._update_from_env(); base.read_all_repos(); base.fill_sack(); q = base.sack.query(); list = list(q.upgrades()); res = ["{\"name\":\""+x.name+"\",\"version\":\""+x.version+"-"+x.release+"\",\"arch\":\""+x.arch+"\",\"repository\":\""+x.reponame+"\"}" for x in list]; print("{\"available\":["+",".join(res)+"]}")'

added conf = base.conf; conf.substitutions.update_from_etc(conf.installroot); conf.substitutions._update_from_env(); this will load substitutions from the vars files and from environment variables if they exist.

[rndmh3ro]

Looks good and works now. Can you please sign-off your commits (https://github.com/dev-sec/linux-patch-baseline/pull/57/checks?check_run_id=8582075495)? Then I'll merge this.

[Stromweld]

Done DCO has passed now.

[Stromweld]

Can this also get an official tag release?