Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debian/Ubuntu package versions contain codename which "break" dist upgrades #1315

Open
2 of 3 tasks
twouters opened this issue Oct 14, 2021 · 9 comments
Open
2 of 3 tasks

Comments

@twouters
Copy link

  • This is a bug report
  • This is a feature request
  • I searched existing issues before opening this one

Expected behavior

A dist-upgrade/full-upgrade (e.g. buster->bullseye) updates the currently installed docker packages to those built for the target OS version.

Actual behavior

A dist-upgrade does not update docker packages. I don't think this currently breaks anything because the builds seem to be compatible, but the package versioning scheme could be reconsidered.

The reason for this is that the package version contains the release codename which are not alphabetically ordered: 5:20.10.9~3-0~debian-buster is considered newer than 5:20.10.9~3-0~debian-bullseye

A solution could be to use the release version instead of codename, but I think it requires an epoch update.

Steps to reproduce the behavior

Replace buster repo's with bullseye repo's and perform a dist-upgrade:

➜  debian10 vagrant ssh
Linux buster 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
vagrant@buster:~$ sudo apt install curl
Reading package lists... Done
Building dependency tree       
Reading state information... Done
curl is already the newest version (7.64.0-4+deb10u2).
0 upgraded, 0 newly installed, 0 to remove and 18 not upgraded.
vagrant@buster:~$ curl -fsSL https://get.docker.com -o get-docker.sh
vagrant@buster:~$ sudo sh get-docker.sh 
# Executing docker install script, commit: 93d2499759296ac1f9c510605fef85052a2c32be
+ sh -c apt-get update -qq >/dev/null
+ sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq apt-transport-https ca-certificates curl gnupg >/dev/null
+ sh -c curl -fsSL "https://download.docker.com/linux/debian/gpg" | gpg --dearmor --yes -o /usr/share/keyrings/docker-archive-keyring.gpg
+ sh -c echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian buster stable" > /etc/apt/sources.list.d/docker.list
+ sh -c apt-get update -qq >/dev/null
+ sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends  docker-ce-cli docker-scan-plugin docker-ce >/dev/null
+ version_gte 20.10
+ [ -z  ]
+ return 0
+ sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq docker-ce-rootless-extras >/dev/null
+ sh -c docker version
Client: Docker Engine - Community
 Version:           20.10.9
 API version:       1.41
 Go version:        go1.16.8
 Git commit:        c2ea9bc
 Built:             Mon Oct  4 16:08:42 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.9
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.8
  Git commit:       79ea9d3
  Built:            Mon Oct  4 16:06:50 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.11
  GitCommit:        5b46e404f6b9f661a205e28d59c982d3634148f8
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

================================================================================

To run Docker as a non-privileged user, consider setting up the
Docker daemon in rootless mode for your user:

    dockerd-rootless-setuptool.sh install

Visit https://docs.docker.com/go/rootless/ to learn about rootless mode.


To run the Docker daemon as a fully privileged service, but granting non-root
users access, refer to https://docs.docker.com/go/daemon-access/

WARNING: Access to the remote API on a privileged Docker daemon is equivalent
         to root access on the host. Refer to the 'Docker daemon attack surface'
         documentation for details: https://docs.docker.com/go/attack-surface/

================================================================================

vagrant@buster:~$ sudo sed -i s/buster/bullseye/g /etc/apt/sources.list /etc/apt/sources.list.d/docker.list
vagrant@buster:~$ sudo sed -i s/bullseye\\/updates/bullseye-security/g /etc/apt/sources.list
vagrant@buster:~$ sudo apt update
Hit:1 http://deb.debian.org/debian bullseye InRelease
Get:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Hit:3 https://download.docker.com/linux/debian bullseye InRelease               
Hit:4 http://deb.debian.org/debian bullseye-updates InRelease                   
Hit:5 http://deb.debian.org/debian bullseye-backports InRelease
Get:6 http://security.debian.org/debian-security bullseye-security/main Sources [38.6 kB]
Get:7 http://security.debian.org/debian-security bullseye-security/main amd64 Packages [47.4 kB]
Get:8 http://security.debian.org/debian-security bullseye-security/main Translation-en [31.7 kB]
Fetched 162 kB in 1s (314 kB/s)                                 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
308 packages can be upgraded. Run 'apt list --upgradable' to see them.
vagrant@buster:~$ sudo apt list --upgradable | grep docker

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

vagrant@buster:~$ apt policy docker-ce
docker-ce:
  Installed: 5:20.10.9~3-0~debian-buster
  Candidate: 5:20.10.9~3-0~debian-buster
  Version table:
 *** 5:20.10.9~3-0~debian-buster 100
        100 /var/lib/dpkg/status
     5:20.10.9~3-0~debian-bullseye 500
        500 https://download.docker.com/linux/debian bullseye/stable amd64 Packages
     5:20.10.8~3-0~debian-bullseye 500
        500 https://download.docker.com/linux/debian bullseye/stable amd64 Packages
     5:20.10.7~3-0~debian-bullseye 500
        500 https://download.docker.com/linux/debian bullseye/stable amd64 Packages
     5:20.10.6~3-0~debian-bullseye 500
        500 https://download.docker.com/linux/debian bullseye/stable amd64 Packages
vagrant@buster:~$ sudo apt install docker-ce=5:20.10.9~3-0~debian-bullseye
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Suggested packages:
  aufs-tools cgroupfs-mount | cgroup-lite
Recommended packages:
  git libltdl7 pigz
The following packages will be DOWNGRADED:
  docker-ce
0 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 308 not upgraded.
Need to get 21.2 MB of archives.
After this operation, 21.5 kB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.

Output of docker version:

Client: Docker Engine - Community
 Version:           20.10.9
 API version:       1.41
 Go version:        go1.16.8
 Git commit:        c2ea9bc
 Built:             Mon Oct  4 16:08:55 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.9
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.8
  Git commit:       79ea9d3
  Built:            Mon Oct  4 16:07:01 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.11
  GitCommit:        5b46e404f6b9f661a205e28d59c982d3634148f8
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.6.3-docker)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 20.10.9
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 5b46e404f6b9f661a205e28d59c982d3634148f8
 runc version: v1.0.2-0-g52b36a2
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.10.0-8-amd64
 Operating System: Debian GNU/Linux 11 (bullseye)
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 473.1MiB
 Name: bullseye
 ID: UTIF:CRIH:J6N4:MIXA:ONNW:RYBQ:ITLH:IC5G:H6HL:FB6V:GUWJ:CJ6F
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.)

@twouters twouters changed the title Debian/Ubuntu package versions contain codename which break dist upgrades Debian/Ubuntu package versions contain codename which "break" dist upgrades Oct 14, 2021
@thaJeztah
Copy link
Member

Hmm.. interesting yes, so, IIUC, the situation in your case was;

  1. you're running debian buster, with docker 20.10.9 installed
  2. upgraded to bullseye
  3. updated the apt config to point to the bullseye packages
  4. install docker again in an attempt to switch it to use the bullseye version of docker 20.10.9 instead of the buster package

And 4. failed, because it compares installed version with available version (which hits the problem you described where bullseye < buster (alphabetically).

Interesting; hadn't received reports about this case before (a more common case is where users forget step 3., and are still refering to package from the old version).

I'm wondering how this is usually dealt with, and how the same would work for other packages. For example (taking ubuntu 20.04, 21.04, and 21.10 as an example);

docker run -it --rm ubuntu:20.04 sh -c 'apt-get update -qq && apt-cache madison curl'
      curl | 7.68.0-1ubuntu2.7 | http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
      curl | 7.68.0-1ubuntu2.7 | http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
      curl | 7.68.0-1ubuntu2 | http://archive.ubuntu.com/ubuntu focal/main amd64 Packages

docker run -it --rm ubuntu:21.04 sh -c 'apt-get update -qq && apt-cache madison curl'
      curl | 7.74.0-1ubuntu2.3 | http://archive.ubuntu.com/ubuntu hirsute-updates/main amd64 Packages
      curl | 7.74.0-1ubuntu2.3 | http://security.ubuntu.com/ubuntu hirsute-security/main amd64 Packages
      curl | 7.74.0-1ubuntu2 | http://archive.ubuntu.com/ubuntu hirsute/main amd64 Packages


docker run -it --rm ubuntu:21.10 sh -c 'apt-get update -qq && apt-cache madison curl'
      curl | 7.74.0-1.3ubuntu2 | http://archive.ubuntu.com/ubuntu impish/main amd64 Packages

Installing/updating curl after upgrading from 20.04 to 21.04 would work, because 21.04 has a newer version of curl, but when upgrading from 21.04 to 21.10 (if I understand correctly), the same problem would occur, as both have 7.74.0-1ubuntu2 ?

Incrementing the epoch would be an option of course, but feels like something to avoid, as we'd have to increment it for each distro (and version) separately 🤔

@thaJeztah
Copy link
Member

oh! I see I misread the ubuntu versions; looks like there, they used .3ubuntu2 as suffix? Let me check some other packages

@thaJeztah
Copy link
Member

Looks like it's all a bit random:

docker run -it --rm ubuntu:21.04 sh -c 'apt-get update -qq && apt-cache madison jq nano dpkg-sig golang'
       jq | 1.6-2.1ubuntu1 | http://archive.ubuntu.com/ubuntu hirsute/main amd64 Packages
      nano | 5.4-2build1 | http://archive.ubuntu.com/ubuntu hirsute/main amd64 Packages
  dpkg-sig | 0.13.1+nmu4 | http://archive.ubuntu.com/ubuntu hirsute/universe amd64 Packages
    golang | 2:1.16~0ubuntu1 | http://archive.ubuntu.com/ubuntu hirsute/main amd64 Packages


docker run -it --rm ubuntu:21.10 sh -c 'apt-get update -qq && apt-cache madison jq nano dpkg-sig golang'
        jq | 1.6-2.1ubuntu2 | http://archive.ubuntu.com/ubuntu impish/main amd64 Packages
      nano | 5.6.1-1build1 | http://archive.ubuntu.com/ubuntu impish/main amd64 Packages
  dpkg-sig | 0.13.1+nmu4 | http://archive.ubuntu.com/ubuntu impish/universe amd64 Packages
    golang | 2:1.17~0ubuntu1 | http://archive.ubuntu.com/ubuntu impish/main amd64 Packages

@twouters
Copy link
Author

twouters commented Oct 27, 2021

I just realized that the epoch doesn't need to be increased since the codename is the very last portion of the version string and any version digit before that could be increased.

5:20.10.9~3-0~debian-bullseye << 5:20.10.9~3-1~debian-11

Any update to the package version would trigger an update even if the codename remains in use, so the issue is very minor.

@cpuguy83
Copy link
Collaborator

cpuguy83 commented Nov 1, 2021

/cc @tianon

I think this should be debian11 rather than debian-11 just based on convention?

@tianon
Copy link

tianon commented Nov 1, 2021

Doh, yep -- I'd probably do something like deb11u0 to mimic Debian's own stable update version scheme. The u0 allows for doing a u1, u2, etc if it's necessary to make a change before the next upstream release (and that's specific to a single Debian release, so bumping the -1~ portion doesn't make sense).

That ~3 is also really curious -- I guess it's been a bit since I looked at how that version string is generated, but that feels like it's probably not exactly right either?

@tianon
Copy link

tianon commented Nov 1, 2021

Just to illustrate, my own packaging repository for moby-engine has version 20.10.7-3, but when it gets built I repack it to include a per-suite suffix like ~deb11u0 (making the final result something like 20.10.7-3~deb11u0 -- upstream version 20.10.7, packaging revision 3, built for Debian 11 with no suite-specific revisions, although the way I maintain it makes a ~deb11u1 very unlikely because I'd do a -4 instead and just rebuild everywhere to keep the sources the same).

@thaJeztah
Copy link
Member

That ~3 is also really curious -- I guess it's been a bit since I looked at how that version string is generated, but that feels like it's probably not exactly right either?

ISTR that was added at some point when the switch was made to CalVer, but I'd have to do some digging in git history as I wasn't directly involved in those changes in packaging at the time.

Looking if there's some logical number / value we could use for this; would VERSION_ID from /etc/os-release work?

In debian, that'd be

VERSION_ID="11"

On Ubuntu, e.g.;

VERSION_ID="18.04"

(basically, looking if there's something we could add, without having to add a manual number, for each distro and/or distro version 🤔)

@tianon
Copy link

tianon commented Nov 1, 2021

Yeah, VERSION_ID would probably work reasonably well -- you could also combine it with ID to get it "unique", and for Ubuntu's sake probably remove any dots?

source /etc/os-release
echo "$ID${VERSION_ID//./}"

That gives values like debian11 and ubuntu2004 (although leaving the period would also be pretty harmless).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants