Next steps for schema 1 deprecation?

[jonjohnsonjr]

Tell us about your request
I'm curious about the timeline and plans around deprecating support for schema 1 images. I see that they have been deprecated in the docker client with Target For Removal In Release: v20.10. Any estimates on the date for that release?

It would be great if we had a client release that we can guarantee will not pull or push a schema 1 image to avoid problems like docker-library/golang#269. (Also, I'm curious if anyone from Docker can identify how widespread this issue is among official images. I would run into quota issues if I tried to determine this via the registry API.)

Do you plan to deprecate schema 1 images in Docker Hub? It would seem reasonable to me to continue serving schema 1 images that were uploaded as schema 1 indefinitely, but would Docker Hub ever start rejecting new uploads?

Will Docker Hub ever stop performing the schema 1 down-conversion for backward compatibility? What about the manifest list -> schema 2 down-conversion? This came up in opencontainers/distribution-spec#212.

To be clear, I'm asking because I want schema 1 down-conversion to go away, as it would resolve some ambiguity around registry behavior and unblock some other conversations. I imagine once Docker Hub announces schema 1 deprecation, most other registries will follow suit. I understand that deprecating a feature of a production service is delicate, so I don't expect anything to happen immediately.

Which service(s) is this request for?
Both the docker client and Docker Hub.

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Some of the official images are accidentally referencing schema 1 manifests (docker-library/golang#269 (comment)), and some clients do not handle that well. I could update these clients, but it's not worth the effort if this problem will just go away.

Are you currently working around the issue?
Yes.

[thaJeztah]

/cc @nebuk89 @justincormack (for tomorrow's review session)

[justincormack]

I did some preliminary analysis for the distribution-spec issue and it looks like there are still a fair number of old clients pulling from Hub, before the conversion date; it is a small percentage but we would be breaking some people. I will try to get some better data to understand the scope better.

[tianon]

Hopefully the review session went well? 🙏

We've had another instance in docker-library/php#1086 of some kind of hiccup causing docker push to push a schema1 instead of schema2 -- IMO schema1 is still reasonable to serve if it's been pushed that way and is historical, but I think blocking pushes of new schema1 images should be a pretty safe step? 😇 ❤️

(Especially given containerd can't even parse them 😅)

[jonjohnsonjr]

I see that schema 1 pushing was disabled in moby/moby#41295 (outside of an environment variable override), which is now released 🎉

I imagine once v20.10.x reaches broader adoption, it would be more reasonable to start thinking about disabling schema 1 uploads from the registry side?

[justincormack]

We are going to be working on a full schema 1 phase out plan, involving removing all support over time.

Initial plan is to remove all support from the next Docker engine release and block uploads to Docker Hub (we will look at how many there are). We need to work out a strategy to deal with existing images. I want to remove serving v2 images as v1, but that may take a little longer.

[thaJeztah]

PR for removal in Moby / Docker Engine; moby/moby#42300