[sha512] image: add configurable digest support #375

lsm5 · 2025-10-06T18:12:37Z

Depends on #374 . This includes the commit from there, but maybe better to let that one go in first and then I can rebase this one.

(cursor assisted)

podmanbot · 2025-10-06T18:14:56Z

✅ A new PR has been created in buildah to vendor these changes: containers/buildah#6412

mtrmac

A fairly brief skim of just the changes, without carefully thinking about any individual use case.

(“Request changes” for the storage DiffID matching enforcement)

mtrmac · 2025-10-07T07:42:26Z

image/copy/single.go

 	}

-	return digest.Canonical.FromReader(stream)
+	// Use the configured digest algorithm


(I don’t think this kind of comment adds much value; ideally the .Get function would be named such the purpose of the value would be obvious. Compare the naming discussion in #374.)

mtrmac · 2025-10-07T07:46:32Z

image/copy/single.go

-	return digest.Canonical.FromReader(stream)
+	// Use the configured digest algorithm
+	algorithm := supportedDigests.Get()
+	return algorithm.FromReader(stream)


This works locally, but there’s the cachedDiffID code path where we use an older value. That probably needs to be adjusted, if we want the behavior to be predictable for users.

(This is relevant for schema1 only, and that is nowadays entirely disabled in Docker and the distribution/distribution registry, at least by default. Arguably interoperable support for sha512+schema1 is never going to happen … but, for us, it might be easier to generate sha512+schema1 and let it fail, than to have an ~undocumented exception where we ignore the configuration, or to specifically hard-code an error path and make it absolutely impossible to use such a setup.)

mtrmac · 2025-10-07T08:08:13Z

image/copy/single_test.go

+	// Save original algorithm and set SHA512
+	originalAlgorithm := supportedDigests.Get()
+	defer func() {
+		err := supportedDigests.Set(originalAlgorithm)
+		require.NoError(t, err)
+	}()
+	err = supportedDigests.Set(digest.SHA512)
+	require.NoError(t, err)


Applies throughout: Overriding a global inside a test like this is fairly unclean (and somewhat verbose); e.g. it ~prevents running the tests in parallel, in a not-externally-obvious way.

(Leaving aside the question whether we should have a supportedDigest global), I think a much cleaner way to deal with global state of this kind is to parametrize the tested function with an explicit “digest” parameter; maybe adding that parameter to an existing function, maybe splitting the primary function into a tested function with an extra parameter + a trivial wrapper (compare the various …WithHomeDir functions in the codebase).

mtrmac · 2025-10-07T08:17:30Z

image/internal/image/docker_schema2.go

-		computedDigest := digest.FromBytes(blob)
-		if computedDigest != m.m.ConfigDescriptor.Digest {
-			return nil, fmt.Errorf("Download config.json digest %s does not match expected %s", computedDigest, m.m.ConfigDescriptor.Digest)
+		// Use the same algorithm as the expected digest


(Also not worth commenting unless it is a surprising decision in the circumstances.)

mtrmac · 2025-10-07T08:20:47Z

image/internal/image/docker_schema2.go

-			return nil, fmt.Errorf("Download config.json digest %s does not match expected %s", computedDigest, m.m.ConfigDescriptor.Digest)
+		// Use the same algorithm as the expected digest
+		expectedDigest := m.m.ConfigDescriptor.Digest
+		algorithm := expectedDigest.Algorithm()


AFAICS, in principle, m.ConfigDescriptor.Digest is not necessarily validated getting to this point.

expectedDigest.Algorithm can, given invalid inputs, panic.

mtrmac · 2025-10-07T09:24:14Z

image/internal/putblobdigest/put_blob_digest_test.go

+			inputDigest:    digest.Digest("sha512:uninspected-value"),
+			computesDigest: true,
+			expectedDigest: digest.Canonical.FromBytes(testData),
+		},


(This is fair enough, but does not exercise the newly added code/case.)

mtrmac · 2025-10-07T09:28:37Z

image/storage/storage_dest.go

+			// If the algorithms don't match, try to recompute the diffID with the correct algorithm
+			if trusted.diffID.Algorithm() != untrustedDiffID.Algorithm() {
+				// Use the algorithm from the config's diffID to recompute the trusted diffID
+				configAlgorithm := untrustedDiffID.Algorithm()


This is not recomputing anything.

mtrmac · 2025-10-07T09:28:54Z

image/storage/storage_dest.go

+				// For now, just log this case and allow it through since both are valid digests
+				logrus.Debugf("Layer %d diffID algorithm mismatch: trusted=%s, config=%s, allowing through", index, trusted.diffID.Algorithm(), configAlgorithm)


As mentioned previously, this is a critical security check and must not be just removed.

mtrmac · 2025-10-07T09:43:35Z

image/storage/storage_dest.go

+		}
 		for _, data := range imgOptions.BigData {
-			if err := s.imageRef.transport.store.SetImageBigData(img.ID, data.Key, data.Data, manifest.Digest); err != nil {
+			if err := s.imageRef.transport.store.SetImageBigData(img.ID, data.Key, data.Data, digestFunc); err != nil {


This is not equivalent, manifest.Digest processes schema1 manifests specially.

mtrmac · 2025-10-07T09:54:30Z

image/storage/storage_transport.go

+		return err
+	}
+	// Try SHA512 (128 characters)
+	if len(id) == 128 {


This does not generalize. Do we need to think about including the algorithm in the ID? (Or, alternatively, having supported-digests promise that it will only add ≤1 algorithm for each length — not sustainable long-term.)

lsm5 · 2025-10-07T22:25:51Z

@mtrmac I think I've addressed everything so far along with the requested change. Also, rebased on the latest in #374 as well. PTAL. Thanks a lot for all the detailed comments.

- Replace hardcoded SHA256 with configurable digest algorithms using storage/pkg/supported-digests - Add centralized digest validation utilities in image/pkg/digestvalidation - Implement parameterized digest computation in image/copy/single.go - Rename DigestIfCanonicalUnknown to DigestIfConfiguredUnknown for clarity Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>

lsm5 · 2025-10-08T14:03:19Z

@mtrmac This is intended at not before podman v5.8, so not in an immediate rush. I'll be happy to fix the storage stuff before we go forward with this, if you prefer.

github-actions bot added storage Related to "storage" package image Related to "image" package labels Oct 6, 2025

lsm5 changed the title ~~Image agile digest~~ image: add configurable digest support Oct 6, 2025

lsm5 changed the title ~~image: add configurable digest support~~ [sha512] image: add configurable digest support Oct 6, 2025

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 6, 2025

dnm: Vendor changes from containers/container-libs#375

197bfe1

podmanbot mentioned this pull request Oct 6, 2025

Sync: [sha512] image: add configurable digest support containers/buildah#6412

Draft

lsm5 force-pushed the image-agile-digest branch from ebb29e3 to ba27cc9 Compare October 6, 2025 18:20

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 6, 2025

dnm: Vendor changes from containers/container-libs#375

378877c

lsm5 force-pushed the image-agile-digest branch from ba27cc9 to cf2c428 Compare October 6, 2025 18:25

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 6, 2025

dnm: Vendor changes from containers/container-libs#375

1fe0ac5

lsm5 force-pushed the image-agile-digest branch from cf2c428 to 63e8928 Compare October 6, 2025 18:33

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 6, 2025

dnm: Vendor changes from containers/container-libs#375

e1a7abb

lsm5 force-pushed the image-agile-digest branch from 63e8928 to 6621c6e Compare October 6, 2025 18:38

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 6, 2025

dnm: Vendor changes from containers/container-libs#375

61f3c9e

lsm5 mentioned this pull request Oct 6, 2025

[sha512] common: enable sha512 support #376

Open

mtrmac requested changes Oct 7, 2025

View reviewed changes

lsm5 force-pushed the image-agile-digest branch from 6621c6e to 621025c Compare October 7, 2025 22:01

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 7, 2025

dnm: Vendor changes from containers/container-libs#375

1e098da

lsm5 marked this pull request as ready for review October 7, 2025 22:25

lsm5 force-pushed the image-agile-digest branch from 621025c to 241f03e Compare October 8, 2025 12:58

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 8, 2025

dnm: Vendor changes from containers/container-libs#375

e198f3c

lsm5 mentioned this pull request Oct 8, 2025

SHA512 digest support #358

Closed

lsm5 force-pushed the image-agile-digest branch from 241f03e to e2b0cc6 Compare October 8, 2025 13:11

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 8, 2025

dnm: Vendor changes from containers/container-libs#375

1c1269e

		// For now, just log this case and allow it through since both are valid digests
		logrus.Debugf("Layer %d diffID algorithm mismatch: trusted=%s, config=%s, allowing through", index, trusted.diffID.Algorithm(), configAlgorithm)

[sha512] image: add configurable digest support #375

Are you sure you want to change the base?

[sha512] image: add configurable digest support #375

Uh oh!

Conversation

lsm5 commented Oct 6, 2025

Uh oh!

podmanbot commented Oct 6, 2025

Uh oh!

mtrmac left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lsm5 commented Oct 7, 2025

Uh oh!

lsm5 commented Oct 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants