Latest version may fail to be detected if manifest list of latest version is missing some architectures

[aliculPix4D]

Is there an existing issue for this?

• I have searched the existing issues

Package ecosystem

Docker

Package manager version

v0.212.0

Updated dependency

PR example: https://github.com/Pix4D/dependabot-bug-report/pull/1

#<Dependabot::Dependency:0x00007fbc552953b0 @name="python", @version="3.11.0rc2", @requirements=[{:requirement=>nil, :groups=>[], :file=>"/Dockerfile", :source=>{:registry=>nil, :tag=>"3.11.0rc2"}}], @previous_version="3.10.5", @previous_requirements=[{:requirement=>nil, :groups=>[], :file=>"/Dockerfile", :source=>{:registry=>nil, :tag=>"3.10.5"}}], @package_manager="docker", @removed=false>

What you expected to see, versus what you actually saw

Short summary of the problem:

Due to a current issue with release process for Python Docker images (see: python/cpython#98397 and docker-library/python#763) Dependabot Docker wrongly proposes the updates of all Python images with initial version >3.10 to a 3.11.0rc2 release candidate.

Example PR from a Github Dependabot: https://github.com/Pix4D/dependabot-bug-report/pull/1

Explanation why and how it occurs:

We can first list sha reported by Docker cli for Python images:

docker images --digests | grep python
python       3         sha256:07954b3474d37287cfea283d501b68346766ab1adcd0f3e9be06cd9323f490d6
python       3         sha256:3e15ea73ac5c1f9408d8dc9eeb8297ece8e727f3004129a829519230c7de720c
python       3.10.8    sha256:07954b3474d37287cfea283d501b68346766ab1adcd0f3e9be06cd9323f490d6
python       3.10.8    sha256:3e15ea73ac5c1f9408d8dc9eeb8297ece8e727f3004129a829519230c7de720c
python       latest    sha256:07954b3474d37287cfea283d501b68346766ab1adcd0f3e9be06cd9323f490d6
python       latest    sha256:3e15ea73ac5c1f9408d8dc9eeb8297ece8e727f3004129a829519230c7de720c

Sha reported by Dependabot for each of the tags by using digest_of(tag):

latest: sha256:3e15ea73ac5c1f9408d8dc9eeb8297ece8e727f3004129a829519230c7de720c,
3.10:   sha256:3e15ea73ac5c1f9408d8dc9eeb8297ece8e727f3004129a829519230c7de720c,
3.10.8: sha256:07954b3474d37287cfea283d501b68346766ab1adcd0f3e9be06cd9323f490d6

We can see that different sha is reported for 3.10.8 and this is the main reason why all tags >3.10.0 are marked as prerelease by Dependabot and are allowed to update to release candidates versions.
Additionally, 3.10.0 version will be marked as latest version to which all Python versions lower than 3.10.0 can upgrade. i.e if you try to update from 3.8.

SHA reported by using DockerRegistry2 directly:

irb(main):002:0> client=DockerRegistry2::Registry.new("https://registry.hub.docker.com")
irb(main):003:0> client.digest("library/python","latest")
=> "sha256:e9ebb760d74df0c28447c2f67050800c3392a3987ed77ce493545c7d3ad24c9d"
irb(main):004:0> client.digest("library/python","3.10")
=> "sha256:e9ebb760d74df0c28447c2f67050800c3392a3987ed77ce493545c7d3ad24c9d"
irb(main):005:0> client.digest("library/python","3.10.8")
=> "sha256:e9ebb760d74df0c28447c2f67050800c3392a3987ed77ce493545c7d3ad24c9d"

Note that the same tag (e9ebb760d) is reported by Dockerhub.

This discrepancy occurs because in Dependabot you override the headers method of DockerRegistry2::Registry class.

dependabot-core/docker/lib/dependabot/docker/update_checker.rb

Line 25 in eaf59fb

def headers(payload: nil, bearer_token: nil)

which results in wrong updates for any Python image with initial version >3.10 (i.e from 3.10.5 to 3.11.0rc2):

#<Dependabot::Dependency:0x00007fbc552953b0 @name="python", @version="3.11.0rc2", @requirements=[{:requirement=>nil, :groups=>[], :file=>"/Dockerfile", :source=>{:registry=>nil, :tag=>"3.11.0rc2"}}], @previous_version="3.10.5", @previous_requirements=[{:requirement=>nil, :groups=>[], :file=>"/Dockerfile", :source=>{:registry=>nil, :tag=>"3.10.5"}}], @package_manager="docker", @removed=false>

The main reason for this is that the Dockerhub images of 3.10.8 are missing the linux/mips64le architecture compared to latest and 3.10 tags (currently).
python/cpython#98397

Similar issue: docker-library/python#763, but linux/mips64le build is still missing and failing.

Even though the main cause for this is the release process for Python Docker images, I am left wondering if this kind of issue could be prevented in the future by improving the Dependabot code.

I leave this for you to decide :)

[deivid-rodriguez]

Wow, this is such a nicely written curated report, thanks so much! 💜

I investigated a bit the history of this. We started monkey patching https://github.com/deitch/docker_registry2 at 58b2624 in order to fix incorrect digest updates reported at #2962.

However, it sounds like the monkey patch may not work well for regular tag updates.

So, my proposal is:

• Instead of money patching the headers docker_registry2 use, add a new API method to it, say reg.manifest_list_digest(repo, tag).
• Then use digest when updating regular tags, and manifest_list_digest when updating tags pinned to specific sha's.

[aliculPix4D]

python 3.11 is out so as expected this issue "fixed" itself (for now) and dependabot now proposes the correct update: https://github.com/Pix4D/dependabot-bug-report/pull/2

[deivid-rodriguez]

Great, I think this is worth keeping open still, right?

[aliculPix4D]

Great, I think this is worth keeping open still, right?

Yes, my opinion is that this is still worth fixing in the Dependabot code, to prevent this issue occurring again in the future.