-
Notifications
You must be signed in to change notification settings - Fork 145
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[master] feat: build Fedora 41 RPM packages (carry #1055) #1058
[master] feat: build Fedora 41 RPM packages (carry #1055) #1058
Conversation
Let's see if CI is happy, but it' possible we may need something similar as; |
Interesting; compiling works, but looks like the verify script needs an update;
|
Ugh... looks like Fedora 41 ships with dnf5 (https://github.com/rpm-software-management/dnf5), which is a complete rewrite of the command? On Fedora 40, docker run -it --rm fedora:41
dnf config-manager --help
No such command: config-manager. Please use /usr/bin/dnf --help
It could be a DNF plugin command, try: "dnf install 'dnf-command(config-manager)'"
dnf install 'dnf-command(config-manager)'
dnf config-manager --help
usage: dnf config-manager [-c [config file]] [-q] [-v] [--version] [--installroot [path]] [--nodocs] [--noplugins] [--enableplugin [plugin]] [--disableplugin [plugin]] [--releasever RELEASEVER] [--setopt SETOPTS] [--skip-broken] [-h] [--allowerasing] [-b | --nobest] [-C]
[-R [minutes]] [-d [debug level]] [--debugsolver] [--showduplicates] [-e ERRORLEVEL] [--obsoletes] [--rpmverbosity [debug level name]] [-y] [--assumeno] [--enablerepo [repo]] [--disablerepo [repo] | --repo [repo]] [--enable | --disable] [-x [package]]
[--disableexcludes [repo]] [--repofrompath [repo,path]] [--noautoremove] [--nogpgcheck] [--color COLOR] [--refresh] [-4] [-6] [--destdir DESTDIR] [--downloadonly] [--comment COMMENT] [--bugfix] [--enhancement] [--newpackage] [--security]
[--advisory ADVISORY] [--bz BUGZILLA] [--cve CVES] [--sec-severity {Critical,Important,Moderate,Low}] [--forcearch ARCH] [--save] [--add-repo URL] [--dump] [--dump-variables] [--set-enabled | --set-disabled]
[section ...]
manage dnf configuration options and repositories
General DNF options:
-c [config file], --config [config file]
config file location
-q, --quiet quiet operation
-v, --verbose verbose operation
--version show DNF version and exit
--installroot [path] set install root
--nodocs do not install documentations
--noplugins disable all plugins
--enableplugin [plugin]
enable plugins by name
--disableplugin [plugin]
disable plugins by name
--releasever RELEASEVER
override the value of $releasever in config and repo files
--setopt SETOPTS set arbitrary config and repo options
--skip-broken resolve depsolve problems by skipping packages
-h, --help, --help-cmd
show command help
--allowerasing allow erasing of installed packages to resolve dependencies
-b, --best try the best available package versions in transactions.
--nobest do not limit the transaction to the best candidate
-C, --cacheonly run entirely from system cache, don't update cache
-R [minutes], --randomwait [minutes]
maximum command wait time
-d [debug level], --debuglevel [debug level]
debugging output level
--debugsolver dumps detailed solving results into files
--showduplicates show duplicates, in repos, in list/search commands
-e ERRORLEVEL, --errorlevel ERRORLEVEL
error output level
--obsoletes enables dnf's obsoletes processing logic for upgrade or display capabilities that the package obsoletes for info, list and repoquery
--rpmverbosity [debug level name]
debugging output level for rpm
-y, --assumeyes automatically answer yes for all questions
--assumeno automatically answer no for all questions
--enablerepo [repo] Temporarily enable repositories for the purpose of the current dnf command. Accepts an id, a comma-separated list of ids, or a glob of ids. This option can be specified multiple times.
--disablerepo [repo] Temporarily disable active repositories for the purpose of the current dnf command. Accepts an id, a comma-separated list of ids, or a glob of ids. This option can be specified multiple times, but is mutually exclusive with `--repo`.
--repo [repo], --repoid [repo]
enable just specific repositories by an id or a glob, can be specified multiple times
--enable enable repos with config-manager command (automatically saves)
--disable disable repos with config-manager command (automatically saves)
-x [package], --exclude [package], --excludepkgs [package]
exclude packages by name or glob
--disableexcludes [repo], --disableexcludepkgs [repo]
disable excludepkgs
--repofrompath [repo,path]
label and path to an additional repository to use (same path as in a baseurl), can be specified multiple times.
--noautoremove disable removal of dependencies that are no longer used
--nogpgcheck disable gpg signature checking (if RPM policy allows)
--color COLOR control whether color is used
--refresh set metadata as expired before running the command
-4 resolve to IPv4 addresses only
-6 resolve to IPv6 addresses only
--destdir DESTDIR, --downloaddir DESTDIR
set directory to copy packages to
--downloadonly only download packages
--comment COMMENT add a comment to transaction
--bugfix Include bugfix relevant packages, in updates
--enhancement Include enhancement relevant packages, in updates
--newpackage Include newpackage relevant packages, in updates
--security Include security relevant packages, in updates
--advisory ADVISORY, --advisories ADVISORY
Include packages needed to fix the given advisory, in updates
--bz BUGZILLA, --bzs BUGZILLA
Include packages needed to fix the given BZ, in updates
--cve CVES, --cves CVES
Include packages needed to fix the given CVE, in updates
--sec-severity {Critical,Important,Moderate,Low}, --secseverity {Critical,Important,Moderate,Low}
Include security relevant packages matching the severity, in updates
--forcearch ARCH Force the use of an architecture
Config-manager command-specific options:
--save save the current options (useful with --setopt)
--add-repo URL add (and enable) the repo from the specified file or url
--dump print current configuration values to stdout
--dump-variables print variable values to stdout
--set-enabled enable repos (automatically saves)
--set-disabled disable repos (automatically saves)
section configuration sections to manage, "main" or repoid Fedora 41: docker run -it --rm fedora:41 dnf config-manager --help
Usage:
dnf5 [GLOBAL OPTIONS] config-manager <COMMAND> ...
Description:
Manage main and repositories configuration, variables and add new repositories.
Commands:
addrepo Add repositories from the specified configuration file or define a new repository using user options
setopt Set configuration and repositories options
unsetopt Unset/remove configuration and repositories options
setvar Set variables
unsetvar Unset/remove variables |
so I started updating the verify script to work with docker-ce-packaging/install-containerd-helpers Lines 29 to 30 in d757a76
It looks like --enable-repo=REPO_ID,... Enable additional repositories. List option. Supports globs, can be specified multiple times.
--disable-repo=REPO_ID,... Disable repositories. List option. Supports globs, can be specified multiple times. But if those are the equivalent to the --enablerepo [repo] Temporarily enable repositories for the purpose of the current dnf command. Accepts an id, a comma-separated list of ids, or a glob of ids. This option can be specified multiple
times.
--disablerepo [repo] Temporarily disable active repositories for the purpose of the current dnf command. Accepts an id, a comma-separated list of ids, or a glob of ids. This option can be specified
multiple times, but is mutually exclusive with `--repo`. So the alternative is to use However, that doesn't support glob / wildcards, so we may need to do a dnf repo list --all docker-ce-*
repo id repo name status
docker-ce-stable Docker CE Stable - aarch64 enabled
docker-ce-stable-debuginfo Docker CE Stable - Debuginfo aarch64 disabled Perhaps using dnf repo list --json --enabled docker-ce-*
[
{
"id":"docker-ce-stable",
"name":"Docker CE Stable - aarch64",
"is_enabled":true
}
] However, I ran into an issue; it looks like dnf config-manager addrepo --from-repofile="https://download.docker.com/linux/fedora/docker-ce.repo"
Error in added repository configuration file. Cannot set repository option "#1=
": Option "#1" not found After some debugging, it looks like the problem is in empty lines in the file; individual entries work (so it's not an issue in any of those), but the empty lines between them; i.e., this fails; [docker-ce-stable]
name=Docker CE Stable - $basearch
baseurl=https://download.docker.com/linux/fedora/$releasever/$basearch/stable
enabled=1
gpgcheck=1
gpgkey=https://download.docker.com/linux/fedora/gpg
[docker-ce-stable-debuginfo]
name=Docker CE Stable - Debuginfo $basearch
baseurl=https://download.docker.com/linux/fedora/$releasever/debug-$basearch/stable
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/fedora/gpg dnf config-manager addrepo --from-repofile=./test.repo --overwrite
Error in added repository configuration file. Cannot set repository option "#1=
": Option "#1" not found And this works; [docker-ce-stable]
name=Docker CE Stable - $basearch
baseurl=https://download.docker.com/linux/fedora/$releasever/$basearch/stable
enabled=1
gpgcheck=1
gpgkey=https://download.docker.com/linux/fedora/gpg
[docker-ce-stable-debuginfo]
name=Docker CE Stable - Debuginfo $basearch
baseurl=https://download.docker.com/linux/fedora/$releasever/debug-$basearch/stable
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/fedora/gpg |
And it looks like it's a confirmed bug, and .... 😂 😂 the reproducer is.. our
So; confirmed at least that the issue is because of the empty lines, but not sure yet when the fix will ship, so possibly we have to fix it up;
|
On a fun sidetone; I discovered that |
Yes, it is supported in
No. If I know correctly, in Note: It can be used in |
Globs are supported in the repository id. And it documentation for See the examples at the end of the documentation. I purposely wrote the examples with globs in the id repository, hoping it will help someone. |
Ah, thanks! I'm not a user of a rpm-based distro myself, so sometimes need to do a bit more digging for the right place to search
Yes, that's usually the fun challenge; we're packaging for various distros and distro-versions, so where possible, we try to use options that are common between versions. Scripts we use in packaging don't always have to be "polished", we can do some detection there, but for our documentation (and installation script), it m more important, because, well, users don't read documentation (so may be copying the wrong commands), and we also know many users try to install packages on various derivative distros (most of which we don't actively "support", and at least don't test)
Oh, perfect! Thank you for that; I didn't spot that part, that's useful. Interesting; looks like |
install-containerd-helpers
Outdated
@@ -27,7 +27,12 @@ function install_rpm_containerd() { | |||
if command -v dnf5; then | |||
dnf --version | |||
|
|||
dnf config-manager addrepo --save-filename=docker-ce.repo --from-repofile="${REPO_URL}" | |||
# FIXME(thaJeztah); strip empty lines as workaround for https://github.com/rpm-software-management/dnf5/issues/1603 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The dnf5 config-manager
bug affects empty lines and comment lines (starting with the #
character).
install-containerd-helpers
Outdated
dnf config-manager addrepo --save-filename=docker-ce.repo --from-repofile="${REPO_URL}" | ||
# FIXME(thaJeztah); strip empty lines as workaround for https://github.com/rpm-software-management/dnf5/issues/1603 | ||
TMP_REPO_FILE="$(mktemp --dry-run)" | ||
curl -fsSL "${REPO_URL}" | tr -s '\n' > "${TMP_REPO_FILE}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tr -s '\n'
only removes empty lines. Comment lines must also be removed. May be replace tr -s '\n'
with grep '^[^#]'
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Yes, I was considering comment-lines as well, but our .repo
files don't have them, so I ultimately only accounted for the empty lines.
It should (hopefully 🤞) just be temporary, and we can revert the commit once fixed.
FWIW: I lifted those commits to a separate PR (mostly for visibility); and we just merged them; #1059
Signed-off-by: Robert Sturla <robertsturla@outlook.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is the equivalent of docker/containerd-packaging#390 for containerd packages, but unlike for containerd packages, we currently do not run into this issue when building docker-ce packages. We're installing this as a precaution, but perhaps it's not needed. go1.21 and up have a patch that enforces the use of ld.gold to work around a bug in GNU binutils. See; - golang/go#22040. - golang/go@cd77738 Fedora 41 and up has a fixed version of binutils, and no longer requires that patch, but may fail without ld.gold installed; /usr/bin/gcc -Wl,-z,now -Wl,-z,nocopyreloc -fuse-ld=gold -o $WORK/b001/exe/a.out -rdynamic /tmp/go-link-1738353519/go.o /tmp/go-link-1738353519/000000.o /tmp/go-link-1738353519/000001.o /tmp/go-link-1738353519/000002.o /tmp/go-link-1738353519/000003.o /tmp/go-link-1738353519/000004.o /tmp/go-link-1738353519/000005.o /tmp/go-link-1738353519/000006.o /tmp/go-link-1738353519/000007.o /tmp/go-link-1738353519/000008.o /tmp/go-link-1738353519/000009.o /tmp/go-link-1738353519/000010.o /tmp/go-link-1738353519/000011.o /tmp/go-link-1738353519/000012.o /tmp/go-link-1738353519/000013.o /tmp/go-link-1738353519/000014.o /tmp/go-link-1738353519/000015.o /tmp/go-link-1738353519/000016.o /tmp/go-link-1738353519/000017.o /tmp/go-link-1738353519/000018.o /tmp/go-link-1738353519/000019.o /tmp/go-link-1738353519/000020.o /tmp/go-link-1738353519/000021.o /tmp/go-link-1738353519/000022.o /tmp/go-link-1738353519/000023.o /tmp/go-link-1738353519/000024.o -O2 -g -lresolv -O2 -g -lpthread -O2 -g -ldl -O2 -g collect2: fatal error: cannot find 'ld' Fedora's build of Go carries a patch for that, but it's not (yet) in upstream; - https://src.fedoraproject.org/rpms/golang/blob/a867bd88a656c1d6e91e7b18bab696dc3fcf1e77/f/0006-Default-to-ld.bfd-on-ARM64.patch - https://src.fedoraproject.org/rpms/golang/c/a867bd88a656c1d6e91e7b18bab696dc3fcf1e77?branch=rawhide Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
3206300
to
563068b
Compare
# FIXME(thaJeztah): workaround for building on Fedora 41 on arm64 | ||
# | ||
# This is the equivalent of https://github.com/docker/containerd-packaging/pull/390 | ||
# for containerd packages, but unlike for containerd packages, we currently do | ||
# not run into this issue when building docker-ce packages. We're installing | ||
# this as a precaution, but perhaps it's not needed. | ||
# | ||
# go1.21 and up have a patch that enforces the use of ld.gold to work around | ||
# a bug in GNU binutils. See; | ||
# - https://github.com/golang/go/issues/22040. | ||
# - https://github.com/golang/go/commit/cd77738198ffe0c4a1db58352c89f9b2d2a4e85e | ||
# | ||
# Fedora 41 and up has a fixed version of binutils, and no longer requires that | ||
# patch, but may fail without ld.gold installed; | ||
# | ||
# /usr/bin/gcc -Wl,-z,now -Wl,-z,nocopyreloc -fuse-ld=gold -o $WORK/b001/exe/a.out -rdynamic /tmp/go-link-1738353519/go.o /tmp/go-link-1738353519/000000.o /tmp/go-link-1738353519/000001.o /tmp/go-link-1738353519/000002.o /tmp/go-link-1738353519/000003.o /tmp/go-link-1738353519/000004.o /tmp/go-link-1738353519/000005.o /tmp/go-link-1738353519/000006.o /tmp/go-link-1738353519/000007.o /tmp/go-link-1738353519/000008.o /tmp/go-link-1738353519/000009.o /tmp/go-link-1738353519/000010.o /tmp/go-link-1738353519/000011.o /tmp/go-link-1738353519/000012.o /tmp/go-link-1738353519/000013.o /tmp/go-link-1738353519/000014.o /tmp/go-link-1738353519/000015.o /tmp/go-link-1738353519/000016.o /tmp/go-link-1738353519/000017.o /tmp/go-link-1738353519/000018.o /tmp/go-link-1738353519/000019.o /tmp/go-link-1738353519/000020.o /tmp/go-link-1738353519/000021.o /tmp/go-link-1738353519/000022.o /tmp/go-link-1738353519/000023.o /tmp/go-link-1738353519/000024.o -O2 -g -lresolv -O2 -g -lpthread -O2 -g -ldl -O2 -g | ||
# collect2: fatal error: cannot find 'ld' | ||
# | ||
# Fedora's build of Go carries a patch for that, but it's not (yet) in upstream; | ||
# - https://src.fedoraproject.org/rpms/golang/blob/a867bd88a656c1d6e91e7b18bab696dc3fcf1e77/f/0006-Default-to-ld.bfd-on-ARM64.patch | ||
# - https://src.fedoraproject.org/rpms/golang/c/a867bd88a656c1d6e91e7b18bab696dc3fcf1e77?branch=rawhide | ||
# | ||
# As a workaround; install binutils-gold | ||
RUN if [ "$(arch)" = 'aarch64' ]; then dnf -y install binutils-gold; fi |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@vvoland this one should be ready for review; I pushed a commit to install the binutils-gold
, but mostly as a precaution; happy to drop it as well (at least made it a separate commit).
Apologies, if I saw this thread I'd have helped out. |
You're welcome! I had to carry the PR because Jenkins is configured to not allow external contributors to make changes in the Jenkinsfile itself, so it wouldn't test it in CI. The |
Is there something I should have ran from my local machine before submitting the PR in the first place? Some way that I could have identified the dnf5 issue before CI? The build was fine, but I wasn't aware of any validation scripts Edit: |
- What I did
It's nearing that time again. A new Fedora release.
This is part 2 of 2 for packaging Docker CE for Fedora 41. Part 1 is awaiting review in the docker/containerd-packaging repository.
Fedora have not (yet) set a concrete date for the EOL of Fedora 41, however it will be in or around November 2025.
- How I did it
The same as the previous two releases - added a line for Fedora 41 in the Jenkinsfile, updated the list of OSes in the Makefile and created a new Dockerfile using the new Fedora 41 image.
- How to verify it
I have ran
make fedora
from the./rpms
directory, and a RPM file was successfully created.- Description for the changelog
Enable Docker CE builds for Fedora 41
- A picture of a cute animal (not mandatory but encouraged)