hash mismatch on web vs cli

[sneak]

Apologies in advance if this is the wrong repo. Happy to take this elsewhere if this is the wrong forum.

Aren't those supposed to be the same?

[cpuguy83]

@tonistiigi @dmcgowan

Any ideas here?

When I pull ruby:latest I indeed get the same digest that @sneak has posted.
When I pull by digest using the sha from docker hub there's nothing to download and docker images shows it as the same image.

ruby                                               <none>                                     sha256:7c77d7351acbf335aeda71bce3ef60403ce703de87064b885f340592e97cc11f   556e761e192c        9 days ago          842MB
ruby                                               <none>                                     sha256:b516edb5cfe1091b6dabdfe97180f3bcc6c01c3439500d5ab720b46a3e75d5da   556e761e192c        9 days ago          842MB

[cpuguy83]

Actually, the b51 sha is the sha for the manifest list rather than the manifest of the linux/amd64 image.

You can see this with docker manifest inspect on the b51 sha, the linux/amd64 sha is the one from the website... and indeed if you do a pull by digest it will match up.

$ docker manifest inspect ruby@sha256:b516edb5cfe1091b6dabdfe97180f3bcc6c01c3439500d5ab720b46a3e75d5da
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2002,
         "digest": "sha256:7c77d7351acbf335aeda71bce3ef60403ce703de87064b885f340592e97cc11f",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2002,
         "digest": "sha256:e4156f720f07e5368afe8128c6e467c212270d70a80482ba62a060c85ab6cc85",
         "platform": {
            "architecture": "arm",
            "os": "linux",
            "variant": "v5"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2002,
         "digest": "sha256:ecdee1666cfb640f69db2264b5763e1fc02a0b4fbd2bf34f52e9113962e02b9a",
         "platform": {
            "architecture": "arm",
            "os": "linux",
            "variant": "v7"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2002,
         "digest": "sha256:7fbf8b1731b367f8808fe56b08a3c5caeac55958920e1a083518c8e933945356",
         "platform": {
            "architecture": "arm64",
            "os": "linux",
            "variant": "v8"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2003,
         "digest": "sha256:425b7ae0bf5d67b6bc8ab5e46667550edde478dbfd67691ac35cd6461644de2e",
         "platform": {
            "architecture": "386",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2003,
         "digest": "sha256:a8ee02c02456bf545832f7eea5b6d7db37a85a3fd0312bfa2c19185cdf6e97bf",
         "platform": {
            "architecture": "ppc64le",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2002,
         "digest": "sha256:2e09b75eb8fd6734f2492134d7507a7a207f6a3f12a720baf22486ad06aacd7e",
         "platform": {
            "architecture": "s390x",
            "os": "linux"
         }
      }
   ]
}

[sneak]

Where is the b51 hash displayed on the website?

[sneak]

Oh, also, copying the hashes from the website is maddening multiple times per day. On Chrome on macOS it selects the DIGEST: part too no matter how you drag, so then you end up pasting and having to delete the prefix.

Putting the manifest digest there, as well as putting it in its own element that selects just from sha256: onward (or, better yet, forces the selection to imagename@sha256:<hash> so that the whole specifier can be copy/pasted would be ideal.

[cpuguy83]

The website does not appear to show the digest of the manifest list. Likewise Docker should probably display the the resolved manifest digest instead of the manifest list digest.

[sneak]

I think using/displaying the higher (manifest) hash makes a lot more sense; then your Dockerfiles that use it in FROM can be architecture-independent.

[tonistiigi]

cc @manishtomar

[thaJeztah]

Thanks @tonistiigi - was already in conversation with the hub team; I'll open internal tickets to improve this

[sneak]

I think this explanation defangs this very worrisome output too:

When I saw "newer image"... 😰

Perhaps this output could be improved a bit.

[thaJeztah]

When I saw "newer image"... 😰
Perhaps this output could be improved a bit.

That message has been in place since docker 1.3.0 (#8213), and while it's not "perfect", changing such messages should not be taken lightly; even though they're not a "formal" contract, many scripts in existence may be parsing the output / checking for these messages, so changing them could break people.

[sneak]

That sounds suspiciously like "This factually incorrect output is how we've always done it!" 😉

Perhaps regularly breaking the non-formalized contracts is a good way to ensure agility in the future.

[thaJeztah]

The message was intended to indicate "the local image was updated with a newer (different?) image you just pulled".

"regularly breaking" is not always an option with literally millions of active installs.

As to the digest being shown, perhaps both the "manifest list" digest, and the digest for the selected should be shown. Currently it's confusing, because the digest is shown underneath the image name, but is actually referring to the digest for the selected architecture.

Something like this;

w.r.t. "selecting" the digest for copy/pasting; I think there's an issue on the backlog for adding a "copy to clipboard" button, which would make it easier to consume.

I tried selecting by mouse, both on Chrome and Safari on macOS, but didn't run into the issue you described initially, but was able to reproduce when moving the cursor "up" while dragging. Not sure what the best solution is for that (perhaps a wider padding would help)

[sneak]

Figured out steps to reproduce selection issue. Double click inside the hash value itself, then hold the second click and drag left. The moment you select the 6 of sha256, it selects all the way left, including DIGEST:

Selecting right from the s works okay, but I am habituated over years (apparently below the level of conscious realization, even!) to immediately start with the largest click target and work outward from there.

I think "copy to clipboard" is overkill. Just double-clicking (or even maybe single clicking?) anywhere inside the element should set the selection to exactly x/y@algo:fullhash. I am fairly certain anyone using cryptographic hashes is familiar with smashing cmd/ctrl-c.

[Snuupy]

Hi, you can verify by following https://gist.github.com/killbus/875b180741951980798c8fdeff04c46d

[thaJeztah]

Looks like an update was deployed to Docker Hub to fix the presentation; docker/hub-feedback#1925 (comment)

Closing this as I've just added the digest on hub 🎉 The index digest will be displayed when present now.