centos_epel build fails with 404 when also passing mirror option

[bigpick]

Info

When passing in the centos_epel distribution in combination with a mirror it fails; there is no trailing /os in the mirrors. Building -r 8. In every mirror I checked, there is no os after the x86_64.

...
‣ Mounting API VFS...
‣ Mounting API VFS complete.
CentOS-8 - AppStream                                                                                                       8.3 MB/s | 6.2 MB     00:00
CentOS-8 - Base                                                                                                            3.1 MB/s | 2.3 MB     00:00
CentOS-8 - Extras                                                                                                           19 kB/s | 8.1 kB     00:00
CentOS-8 - Plus                                                                                                            949 kB/s | 593 kB     00:00
name=Extra Packages for Enterprise Linux 8 - x86_64                                                                        718  B/s | 249  B     00:00
Errors during downloading metadata for repository 'epel':
  - Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/os/repodata/repomd.xml (IP: 8.43.85.72)
Error: Failed to download metadata for repo 'epel': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
‣ Unmounting API VFS...
‣ Unmounting API VFS complete.
‣ Unmounting Package Cache...
‣ Unmounting Package Cache complete.
‣ Unmounting image...
‣ Unmounting image complete.
‣ Detaching image file...
‣ Detaching image file complete.
Traceback (most recent call last):
  File "/usr/lib64/python3.8/runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.8/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/root/.local/lib/python3.8/site-packages/mkosi/__main__.py", line 28, in <module>
    main()
  File "/root/.local/lib/python3.8/site-packages/mkosi/__main__.py", line 22, in main
    run_verb(a)
  File "/root/.local/lib/python3.8/site-packages/mkosi/__init__.py", line 5497, in run_verb
    build_stuff(args)
  File "/root/.local/lib/python3.8/site-packages/mkosi/__init__.py", line 5310, in build_stuff
    raw, tar, root_hash = build_image(args, root, do_run_build_script=False, cleanup=True)
  File "/root/.local/lib/python3.8/site-packages/mkosi/__init__.py", line 5128, in build_image
    install_distribution(args, root, do_run_build_script=do_run_build_script, cached=cached_tree)
  File "/root/.local/lib/python3.8/site-packages/mkosi/__init__.py", line 2681, in install_distribution
    install[args.distribution](args, root, do_run_build_script)
  File "/usr/lib64/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/root/.local/lib/python3.8/site-packages/mkosi/__init__.py", line 2122, in install_centos
    invoke_dnf_or_yum(args, root, repos, packages)
  File "/root/.local/lib/python3.8/site-packages/mkosi/__init__.py", line 2010, in invoke_dnf_or_yum
    invoke_dnf(args, root, repositories, packages)
  File "/root/.local/lib/python3.8/site-packages/mkosi/__init__.py", line 1698, in invoke_dnf
    run(cmdline)
  File "/root/.local/lib/python3.8/site-packages/mkosi/__init__.py", line 194, in run
    return subprocess.run(cmdline, check=check, **kwargs)
  File "/usr/lib64/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['dnf', '-y', '--config=/var/tmp/mkosi-a1tay8dv/dnf.conf', '--best', '--allowerasing', '--releasever=8', '--installroot=/var/tmp/mkosi-a1tay8dv/root', '--disablerepo=*', '--enablerepo=AppStream', '--enablerepo=BaseOS', '--enablerepo=extras', '--enablerepo=centosplus', '--enablerepo=epel', '--setopt=keepcache=1', '--setopt=install_weak_deps=0', '--nodocs', 'install', 'centos-release', 'systemd', 'epel-release']' returned non-zero exit status 1.

Simply removing the /os from the epel_url seems to fix it:

diff --git a/mkosi/__init__.py b/mkosi/__init__.py
index f9234eb..2f98143 100644
--- a/mkosi/__init__.py
+++ b/mkosi/__init__.py
@@ -2055,7 +2055,7 @@ def install_centos_new(args: CommandLineArguments, root: str, epel_release: int)
         baseos_url = f"baseurl={args.mirror}/centos/{args.release}/BaseOS/x86_64/os"
         extras_url = f"baseurl={args.mirror}/centos/{args.release}/extras/x86_64/os"
         centosplus_url = f"baseurl={args.mirror}/centos/{args.release}/centosplus/x86_64/os"
-        epel_url = f"baseurl={args.mirror}/epel/{epel_release}/Everything/x86_64/os"
+        epel_url = f"baseurl={args.mirror}/epel/{epel_release}/Everything/x86_64"
     else:
         appstream_url = f"mirrorlist=http://mirrorlist.centos.org/?release={args.release}&arch=x86_64&repo=AppStream"
         baseos_url = f"mirrorlist=http://mirrorlist.centos.org/?release={args.release}&arch=x86_64&repo=BaseOS"

Examples

All using mkosi v8.

Test command:

sudo mkosi -d centos_epel -r 8 --mirror $MIRROR

Various mirrors tried:

• CentOS: http://mirror.centos.org
• Redhat: https://download-cc-rdu01.fedoraproject.org/pub
• Redhat: https://download-ib01.fedoraproject.org/pub
• University of Utah CHPC: https://mirror.chpc.utah.edu
• Coastal Carolina University: http://mirror.coastal.edu
• U. of Maryland: https://mirror.umd.edu/fedora
• Siena College: http://mirror.siena.edu/

The issue is not present when not specifying a mirror, presumably since that does not append the extra /os.