apt update failing on 1.25.1

[totallyben]

Running apt update on 1.25.1/latest gives the following:

W: GPG error: http://deb.debian.org/debian bookworm InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 648ACFD622F3D138 NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY F8D2585B8783D481
E: The repository 'http://deb.debian.org/debian bookworm InRelease' is not signed.
W: GPG error: http://deb.debian.org/debian bookworm-updates InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E[131](https://gitlab.totallydev.com/gritzb/citest/-/jobs/217530#L131)
E: The repository 'http://deb.debian.org/debian bookworm-updates InRelease' is not signed.
W: GPG error: http://deb.debian.org/debian-security bookworm-security InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 54404762BBB6E853 NO_PUBKEY BDE6D2B9216EC7A8
E: The repository 'http://deb.debian.org/debian-security bookworm-security InRelease' is not signed.
E: Problem executing scripts APT::Update::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
E: Sub-process returned an error code

[totallyben]

Actually, this can probably be ignored as it is only happening on our self-hosted GitLab pipelines. apt update seems to run fine when running on a copy of the image locally

[imroj801]

I am also running into some issues with the apt update process after this upgrade. Not sure exactly why this is occurring by mine is showing up in ado pipelines.

##[error]#0 5.197 The following packages have unmet dependencies:
##[error]#0 5.347 nginx-extras : Depends: nginx (= 1.22.1-9) but 1.25.1-1bookworm is to be installed
##[error]#0 5.347 Depends: libnginx-mod-http-geoip (= 1.22.1-9) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-http-image-filter (= 1.22.1-9) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-http-perl (= 1.22.1-9) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-http-xslt-filter (= 1.22.1-9) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-mail (= 1.22.1-9) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-stream (= 1.22.1-9) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-stream-geoip (= 1.22.1-9) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-http-cache-purge (>= 1:2.3-4) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-http-dav-ext (>= 1:3.0.0-3~) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-http-echo (>= 1:0.63-4~) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-http-fancyindex (>= 1:0.5.2-3~) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-http-geoip2 (>= 1:3.4-3~) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-http-headers-more-filter (>= 1:0.34-3~) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-http-lua (>= 1:0.10.23-1~) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-http-subs-filter (>= 1:0.6.4-4~) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-http-uploadprogress (>= 1:0.9.2-3~) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-nchan (>= 1:1.3.6+dfsg-2~) but it is not going to be installed
##[error]#0 5.347 Depends: libnginx-mod-stream-geoip2 (>= 1:3.4-3~) but it is not going to be installed
##[error]#0 5.351 E: Unable to correct problems, you have held broken packages.

[totallyben]

It's an odd one, we reverted to 1.24.0 and everything worked fine again.

We found that running curl during our pipeline returned error code 6 - could not resolve host so we are guessing the apt issue is related to that for us. We did try on varying hosts just to ensure it wasn't the specific domain we were calling.

It looks like there was a new debian docker release a few days ago also so if there is an underlying issue it might be linked to that.

[imroj801]

I reverted to 1.25.0 and that did the trick for us.

[thresheek]

Hello @totallyben,

What docker versions and seccomp libraries do you run on the hosts where it works and where it doesnt?

@imroj801 I don't think issue is the same as seen by the original poster.

[totallyben]

Hi @thresheek,

I haven't checked those versions, however, we have resolved our specific issue by upgrading the Docker in Docker (dind) image that our pipelines were using for building images. We were using the image docker:18-dind, which is a few years old now, so updating that to the most recent version fixed the problem.

[thorsteinssonh]

There are still package conflicts if installing nginx-extras on the latest nginx image.
In a really old issue it was suggested to uninstall nginx and install nginx-extras, which is odd, but will see if that keeps things stable.

[thresheek]

@thorsteinssonh nginx-extras is not compatible with the nginx packages shipped in this image. There is no support for that combination.