Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker Hub's new policy limiting image retention #995

Closed
tomkerkhove opened this issue Aug 14, 2020 · 21 comments
Closed

Docker Hub's new policy limiting image retention #995

tomkerkhove opened this issue Aug 14, 2020 · 21 comments
Labels
feature-request All issues for new features that have not been committed to governance needs-discussion stale All issues that are marked as stale due to inactivity

Comments

@tomkerkhove
Copy link
Member

Docker Hub's new policy limiting image retention is going to throttle pulls for free accounts to 100 pulls / 6h for anonymous users and 200 for authenticated users:

image

Less impact is that they will remove images that are inactive for 6m.

Checking with CNCF what their guidance is but we might need to consider moving our images: cncf/foundation#106

Related: https://www.docker.com/pricing/retentionfaq

@tomkerkhove tomkerkhove added needs-discussion feature-request All issues for new features that have not been committed to governance labels Aug 14, 2020
@zroubalik
Copy link
Member

Is this something that CNCF could cover?

@tomkerkhove
Copy link
Member Author

They are looking at it in cncf/foundation#106

@BretFisher
Copy link

more info on technicals of pull limits, with more blog/faq coming next week https://docs.docker.com/docker-hub/download-rate-limit/

@zroubalik
Copy link
Member

https://github.blog/2020-09-01-introducing-github-container-registry/

@turbaszek
Copy link
Contributor

turbaszek commented Sep 2, 2020

https://github.blog/2020-09-01-introducing-github-container-registry/

Exactly my thought @zroubalik

@tomkerkhove
Copy link
Member Author

We can move but turns out we won't be impacted by Docker Hub changes as the pull limitations are on the puller side.

However, we can move everything here if we want to. If we do so, we should have a transition period to push to both places for now.

The real question is, other than the new DH policy, what would be the benefit of migrating?

@turbaszek
Copy link
Contributor

The real question is, other than the new DH policy, what would be the benefit of migrating?

I would say that the only benefit is that "everything is in one place" and maybe pushing images on CI may work faster... but I do not think that this is worth a migration.

@tomkerkhove
Copy link
Member Author

For CI I fully agree, official ones I'm not sure yet since DH is still a bit the place to be.

But if they give better analysis than DH I'm all in. I'll ask around.

@zroubalik
Copy link
Member

We can move but turns out we won't be impacted by Docker Hub changes as the pull limitations are on the puller side.

I haven't followed the changes on Docker Hub deeply, so pardon my ignorance, but could you please elaborate on this?

@tomkerkhove
Copy link
Member Author

There are two major changes:

  • Image pull retention
  • Image pull throttling

If an image tag is not pulled for x days on free account, it will be removed. I'd have to check what the exact amount is but I think it's 60 or 90 days but you can easily automate it to keep them alive.

Pull throttling will avoid people from pulling too fast. Initially the thought was that it would be based on total pull count on an image tag, but instead it's on the client side. So if I'm logged in to DH they will throttle based on that. If it's anonymous pulls they will throttle based on IP. This means that if one user pulls a lot, they will be throttled but not our other users which was my main concern.

@zroubalik
Copy link
Member

zroubalik commented Sep 3, 2020

Thanks for the explanation!

Pull throttling will avoid people from pulling too fast. Initially the thought was that it would be based on total pull count on an image tag...

That was my (false) understanding, got it now.

@zroubalik
Copy link
Member

So agree, that we don't have to change the things that we have. We can migrate later if it is worth or needed.

@BretFisher
Copy link

Quick numbers on changes starting November 1st:

Retention Limits: If an image repository on a free account hasn't had a single pull of any tag in 6 months, the repository is deleted.

Pull Limits: Tracked by IP, Hub will limit pulls to 100 in the last 6 hours. 200 if the pull is authenticated.

If either of these are using a paid account (starts at $5/mo), limits are removed (unlimited retention and pulls).

@jaythamke
Copy link

jaythamke commented Oct 22, 2020

Quick numbers on changes starting November 1st:

Retention Limits: If an image repository on a free account hasn't had a single pull of any tag in 6 months, the repository is deleted.

Pull Limits: Tracked by IP, Hub will limit pulls to 100 in the last 6 hours. 200 if the pull is authenticated.

If either of these are using a paid account (starts at $5/mo), limits are removed (unlimited retention and pulls).

Hi @BretFisher ,
Any idea - how to measure the number images pulled by a non paid docker account? Thanks
see - SO post

@BretFisher
Copy link

I don't believe Docker Hub provides pull stats to users beyond the "total pulls" estimate shown in the Web UI.

@jaythamke
Copy link

I don't believe Docker Hub provides pull stats to users beyond the "total pulls" estimate shown in the Web UI.

Yes, I have also checked the docker hub api spec but did not find any related to my specific need. I thought I might have missed something, hence asked. Thanks for reply.

@tomkerkhove
Copy link
Member Author

Nope, they don't provide it indeed, however since the throttling is on the client side based on IP we are good to go for now.

@tomkerkhove
Copy link
Member Author

One of the maintainers will have to fill in https://www.docker.com/community/open-source/application; happy to pick this up if need be @zroubalik @jeffhollan

@zroubalik
Copy link
Member

@tomkerkhove go ahead if you can do it, please.

@tomkerkhove
Copy link
Member Author

@stale
Copy link

stale bot commented Oct 14, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale All issues that are marked as stale due to inactivity label Oct 14, 2021
SpiritZhou pushed a commit to SpiritZhou/keda that referenced this issue Jul 18, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
…kedacore#995)

* Update documents to reference float64 for GCP pub sub and stackdriver

Signed-off-by: Eric Takemoto <24865872+octothorped@users.noreply.github.com>

* Update yaml example to be float for GCP pub sub

Signed-off-by: Eric Takemoto <24865872+octothorped@users.noreply.github.com>

Signed-off-by: Eric Takemoto <24865872+octothorped@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request All issues for new features that have not been committed to governance needs-discussion stale All issues that are marked as stale due to inactivity
Projects
None yet
Development

No branches or pull requests

5 participants