Umbrella issue: k8s.gcr.io => registry.k8s.io solution #1834

BobyMCbobs · 2021-03-25T22:30:09Z

Umbrella issue: k8s.gcr.io => registry.k8s.io solution #1834

This markdown is synced from https://hackmd.io/gN-1GeSpSgyNSvmjKSULbg?edit to #1834 (comment) manually by @BobyMCbobs

Scope: https://github.com/kubernetes/k8s.io/wiki/New-Registry-url-for-Kubernetes-(registry.k8s.io)

Design Doc: https://docs.google.com/document/d/1yNQ7DaDE5LbDJf9ku82YtlKZK0tcg5Wpk9L72-x2S2k/edit (shared w/ dev@kubernetes.io and SIG mailing list)

Board: https://github.com/orgs/kubernetes/projects/77

DRAFT AIs that need filled turned into tickets:
https://github.com/orgs/kubernetes/projects/77/views/2?filterQuery=is%3Adraft

What exactly are you doing? (and how?)

stp-ip · 2021-03-26T12:38:29Z

Correct link as Github parsed wrong I guess: https://hackmd.io/@TKToYPauRJ-u_mNRBOh4HQ/HJBH3QF4_

thockin · 2021-03-26T23:34:51Z

This finally forced me to disassemble the registry protocol a bit. Interesting. I picked a simple image I know:

$ curl -i https://k8s.gcr.io/v2/git-sync/git-sync/manifests/v3.2.2
HTTP/2 200 
docker-distribution-api-version: registry/2.0
content-type: application/vnd.docker.distribution.manifest.list.v2+json
content-length: 1670
docker-content-digest: sha256:6a543fb2d1e92008aad697da2672478dcfac715e3dddd33801d772da6e70cf24
date: Fri, 26 Mar 2021 22:20:30 GMT
server: Docker Registry
x-xss-protection: 0
x-frame-options: SAMEORIGIN
alt-svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1572,
         "digest": "sha256:85d203d29623d5e7489751812d628e29d0e22075c94a2e99681ecf70be3977ad",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1572,
         "digest": "sha256:31ba6a8e4f1aad8a9c42d97cac8752aaa0e4a92a5b2a3457e597020645fc6a0c",
         "platform": {
            "architecture": "arm",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1572,
         "digest": "sha256:690188a4785caa356d2d98a806524f6f9aa4663a8c43be11fbd9dd5379a01fc9",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1572,
         "digest": "sha256:21487b58352611e67ca033a96f59f1ba47f3e377f5f2e365961c35829bc68ff7",
         "platform": {
            "architecture": "ppc64le",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1572,
         "digest": "sha256:41f3ac440284018ce19b78a8e39a3e99c701a6d7c90fdf7204e180a9715ca7e3",
         "platform": {
            "architecture": "s390x",
            "os": "linux"
         }
      }
   ]
}

I picked the last blob:

$ curl -i https://k8s.gcr.io/v2/git-sync/git-sync/blobs/sha256:41f3ac440284018ce19b78a8e39a3e99c701a6d7c90fdf7204e180a9715ca7e3
HTTP/2 302 
docker-distribution-api-version: registry/2.0
location: https://storage.googleapis.com/us.artifacts.k8s-artifacts-prod.appspot.com/containers/images/sha256:41f3ac440284018ce19b78a8e39a3e99c701a6d7c90fdf7204e180a9715ca7e3
content-type: application/json
date: Fri, 26 Mar 2021 22:21:42 GMT
server: Docker Registry
cache-control: private
x-xss-protection: 0
x-frame-options: SAMEORIGIN
alt-svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
accept-ranges: none
vary: Accept-Encoding

{"errors":[]}

So maybe this is not so hard as I feared?

If we catch URLs of the form https://reg.k8s.io/v2/<name>/manifests/<tag> we can redirect those to k8s.gcr.io (which has global replicas) or some other "canonical" source for metadata. I don't know if docker clients would trip over a literal redirect or what, so worst case we'd have to proxy that data (yuck).

Then we catch URLs of the form https://reg.k8s.io/v2/<name>/blobs/<digest> and redirect to one of the backends. As you point out, we have to do the split-horizon (geo IP) ourselves (yuck).

What I don't know is what tools or public IP databases or other resources are available for the 2nd part. The more we can outsource, the better. But a proof-of-concept would be cool!

I spent a bit of time trying to coax the Google cloud LB to distinguish /v2/<name>/manifests/<ref> from /v2/<name>/blobs/<digest> so the 1st part could simply be a cloud LB rule. Alas it only matches on prefixes. It might be possible to use Content-Type or Accept headers to tell the difference (suggested: match Accept header with blob mime type). If we could do that, then the only thing we'd have to own would be the 2nd part.

I suspect that a model which requires providers to answer our DNS will be more difficult overall.

BobyMCbobs · 2021-03-28T19:25:31Z

Correct link as Github parsed wrong I guess: https://hackmd.io/@TKToYPauRJ-u_mNRBOh4HQ/HJBH3QF4_

@stp-ip, thank you. I've updated the description

justaugustus · 2021-03-29T18:24:12Z

Great to see this discussion happening!

A few things I'd like to see:

this reconciled with the existing artifact management KEP for @brendandburns + @justinsb: https://github.com/kubernetes/enhancements/tree/master/keps/sig-release/1732-artifact-management
this reconciled with the longer-term artifact management epic: [Epic] Artifact Management sig-release#1372 (currently assigned to me)
discussion about existing CNCF projects (like Harbor) as a potential solution: Harbor as a cross-cloud global registry for the Kubernetes Project goharbor/harbor#14411
an idea of intended assignees from the WG K8s Infra side (I'm on point for SIG Release)
feedback from @kubernetes/sig-release-leads @kubernetes/release-engineering

These discussions/decisions impact release delivery, so I'd really love to see them happening in venues where @kubernetes/release-managers are hanging out.

BobyMCbobs · 2021-03-29T19:39:38Z

@thockin, thank you for your input!

If we catch URLs of the form https://reg.k8s.io/v2/<name>/manifests/<tag> we can redirect those to k8s.gcr.io (which has global replicas) or some other "canonical" source for metadata. I don't know if docker clients would trip over a literal redirect or what, so worst case we'd have to proxy that data (yuck).

I suspect that clients may be fine with redirects

Then we catch URLs of the form https://reg.k8s.io/v2/<name>/blobs/<digest> and redirect to one of the backends. As you point out, we have to do the split-horizon (geo IP) ourselves (yuck).

I'm unsure the capability of Google CloudDNS or the use of LoadBalancers to achieve this, community-hosting it might be the option (I'm still investigating otherwise).

I spent a bit of time trying to coax the Google cloud LB to distinguish /v2/<name>/manifests/<ref> from /v2/<name>/blobs/<digest> so the 1st part could simply be a cloud LB rule. Alas it only matches on prefixes. It might be possible to use Content-Type or Accept headers to tell the difference (suggested: match Accept header with blob mime type). If we could do that, then the only thing we'd have to own would be the 2nd part.

This would mean declaring a rule to rewrite the URL that redirects to a DNS host that uses split-horizon DNS which will then go to a blobs server at the nearest cloud provider?

BobyMCbobs · 2021-03-29T19:48:23Z

@justaugustus, appreciate your comments!

Great to see this discussion happening!

A few things I'd like to see:

this reconciled with the existing artifact management KEP for @brendandburns + @justinsb: https://github.com/kubernetes/enhancements/tree/master/keps/sig-release/1732-artifact-management

Thank you, I'll take a read of the KEP.

this reconciled with the longer-term artifact management epic: kubernetes/sig-release#1372 (currently assigned to me)

Totally [epic], I'll check it out as well

discussion about existing CNCF projects (like Harbor) as a potential solution: goharbor/harbor#14411

Absolutely! I've got the two proposals for either Distribution or Harbor. Both are wonderful pieces of software.

an idea of intended assignees from the WG K8s Infra side (I'm on point for SIG Release)

feedback from @kubernetes/sig-release-leads @kubernetes/release-engineering

That would be lovely!

These discussions/decisions impact release delivery, so I'd really love to see them happening in venues where @kubernetes/release-managers are hanging out.

I'll get in contact regarding this issue with folks. I look forward to coordinating a solution with ya'll 😃

thockin · 2021-03-29T20:03:02Z

> I spent a bit of time trying to coax the Google cloud LB to distinguish

/v2/<name>/manifests/<ref> from /v2/<name>/blobs/<digest> so the 1st part could simply be a cloud LB rule. Alas it only matches on prefixes. It might be possible to use Content-Type or Accept headers to tell the difference (suggested: match Accept header with blob mime type). If we could do that, then the only thing we'd have to own would be the 2nd part.

This would mean declaring a rule to rewrite the URL that redirects to a

DNS host that uses split-horizon DNS which will then go to a blobs server at the nearest cloud provider? Either 302 redirect to `blob.k8s.io` which uses DNS split horizon (which requires the backends to host certs for that SAN) or 302 to `blob.k8s.io` which is code we host that does the GeoIP lookup, picks a best backend, and then 302s again to that backend. The advantage of the latter is that the backends don't need special certs. If we can't coax the cloud LB to do this for us, it starts to look more like: 1) User pulls foo:bar 2) Client hits `reg.k8s.io/v2/foo/manifests/bar` 3) Receive that at a program we run (nginx or bespoke or ...) 4) Redirect to `k8s.gcr.io/v2/foo/manifests/bar` 5) Metadata fetched 6) Client hits `/v2/foo/blobs/<digest>` 7) Received at same program as step 3 8) GeoIP lookup, backend select 9) Redirect to `<backend>/v2/foo/blobs/<digest>` 10) Repeat steps 6-10 for each blob 11) Image is pulled

…

On Mon, Mar 29, 2021 at 12:39 PM Caleb Woodbine ***@***.***> wrote: @thockin <https://github.com/thockin>, thank you for your input! If we catch URLs of the form https://reg.k8s.io/v2/<name>/manifests/<tag> we can redirect those to k8s.gcr.io (which has global replicas) or some other "canonical" source for metadata. I don't know if docker clients would trip over a literal redirect or what, so worst case we'd have to proxy that data (yuck). I suspect that clients may be fine with redirects Then we catch URLs of the form https://reg.k8s.io/v2/<name>/blobs/<digest> and redirect to one of the backends. As you point out, we have to do the split-horizon (geo IP) ourselves (yuck). I'm unsure the capability of Google CloudDNS or the use of LoadBalancers to achieve this, community-hosting it might be the option (I'm still investigating otherwise). I spent a bit of time trying to coax the Google cloud LB to distinguish /v2/<name>/manifests/<ref> from /v2/<name>/blobs/<digest> so the 1st part could simply be a cloud LB rule. Alas it only matches on prefixes. It might be possible to use Content-Type or Accept headers to tell the difference (suggested: match Accept header with blob mime type). If we could do that, then the only thing we'd have to own would be the 2nd part. This would mean declaring a rule to rewrite the URL that redirects to a DNS host that uses split-horizon DNS which will then go to a blobs server at the nearest cloud provider? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1834 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKWAVBO573V2ZMDD27MCCLTGDJQTANCNFSM4Z2H3FUQ> .

BobyMCbobs · 2021-03-29T22:23:48Z

Either 302 redirect to blob.k8s.io which uses DNS split horizon (which requires the backends to host certs for that SAN) or 302 to blob.k8s.io which is code we host that does the GeoIP lookup, picks a best backend, and then 302s again to that backend. The advantage of the latter is that the backends don't need special certs.

Would you say that a small webserver to do 302 redirects may be easier or more maintainable than split-horizon?

If we can't coax the cloud LB to do this for us, it starts to look more like:

User pulls foo:bar

Client hits reg.k8s.io/v2/foo/manifests/bar

Receive that at a program we run (nginx or bespoke or ...)

Redirect to k8s.gcr.io/v2/foo/manifests/bar

Metadata fetched

Client hits /v2/foo/blobs/<digest>

Received at same program as step 3

GeoIP lookup, backend select

Redirect to <backend>/v2/foo/blobs/<digest>

Repeat steps 6-10 for each blob

Image is pulled

This is a really clear flow!

@thockin, thank you!

rikatz · 2021-03-31T20:46:34Z

Can I do an attempt into Cloud Run and check if instead we run a machine, running a function that does the redirect wouldn't be cheaper (probably not!) and better? :D

Edit: @justinsb was fair enough saying that we probably can run this inside the AAA and have not so much problems as well, so yeah, let's see how we can use a redirector inside Kubernetes

hh · 2021-04-08T04:12:34Z

I've been searching out a few ASNs for larger cloud providers that likely hit our existing infra.
Once we use these to understand which providers are costing the CNCF the most, we can approach to redirect to a local solution. If anyone from these providers wants to help narrow down which ASNs are part of their cloud offerings, that would help.

Microsoft
Amazon
Google
- https://www.peeringdb.com/org/574
- https://bgp.he.net/AS139190#_prefixes Google Cloud Indonesia
- https://bgp.he.net/AS139070#_prefixes Google Cloud Korea
- https://bgp.he.net/AS45566#_prefixes Google Corporate Network in APAC
- https://bgp.he.net/AS15169#_prefixes Google LLC
- https://bgp.he.net/AS19527#_prefixes Google LLC AS19527
- https://bgp.he.net/AS36040#_prefixes Google LLC AS36040
- https://bgp.he.net/AS43515#_prefixes Google LLC AS43515
- https://bgp.he.net/AS16550#_prefixes Google Private Cloud
Alibaba Group
Tencent Cloud
- https://bgp.he.net/AS132591#_prefixes Tencent Cloud
- https://bgp.he.net/AS132203#_prefixes Tencent Global
- https://bgp.he.net/AS45090#_prefixes Tencent-CN
Equinox
- https://www.peeringdb.com/org/2
- Very long list of ASNs...

stp-ip · 2021-04-08T09:15:30Z

Digital Ocean
- https://www.peeringdb.com/net/6494
- https://bgp.he.net/AS14061#_prefixes
Huawei
- https://www.peeringdb.com/net/18382
- https://bgp.he.net/AS136907#_prefixes
Baidu
- https://www.peeringdb.com/net/6517
- https://bgp.he.net/AS55967#_prefixes

There are a few other providers that could result in traffic, but above felt like a good additional selection of the bigger ones.
Full reference of a list of providers: https://docs.google.com/spreadsheets/d/1LxSqBzjOxfGx3cmtZ4EbB_BGCxT_wlxW_xgHVVa23es/edit#gid=0

So depending on how much traffic is done by those above we could always dig deeper. Let's see what the stats say for the listed providers and then happy to dig into the smaller providers.

BobyMCbobs · 2021-04-09T00:07:08Z

I believe this is the list of ASNs for Equinix Metal:

- 8545 - 9989 - 12085 - 12188 - 14609 - 15734 - 15830 - 15830 - 15830 - 15830 - 15830 - 15830 - 15830 - 16243 - 16397 - 16553 - 17819 - 17941 - 19930 - 21371 - 23637 - 23686 - 24115 - 24121 - 24989 - 24990 - 26592 - 27224 - 27272 - 27330 - 27566 - 29154 - 29884 - 32323 - 32550 - 34209 - 35054 - 43147 - 47886 - 47886 - 54588 - 54825 - 62421 - 64275 - 137840 - 139281 - 264220 - 265376 - 266849 - 270119 - 394749

BobyMCbobs · 2021-04-13T20:55:51Z

ASNs in k8s.io repo: #1914

thockin · 2021-04-16T23:21:45Z

Would you say that a small webserver to do 302 redirects may be easier or more maintainable than split-horizon?

Yes. My thinking is mostly around TLS - if we do split horizon, the real backends have to offer certs for our names. If we 302, they do not. There are a number of GeoIP libs for Go that could be viable. Other than that, the logic seems simple enough to prototype. We could throw it into the aaa cluster as a quick test.

BobyMCbobs · 2021-04-20T04:26:32Z

Would you say that a small webserver to do 302 redirects may be easier or more maintainable than split-horizon?

Yes. My thinking is mostly around TLS - if we do split horizon, the real backends have to offer certs for our names. If we 302, they do not. There are a number of GeoIP libs for Go that could be viable. Other than that, the logic seems simple enough to prototype. We could throw it into the aaa cluster as a quick test.

Thank you @thockin for your comments.

Regarding using a service to perform a redirect, the behavour of something like docker pull registry.k8s.io/{{.Image}}:

it will pull tagging the image as registry.k8s.io/{{.Image}}, not what it is at the container registry
due to Envoy being exposed with an Ingress host that has TLS, it appears to not matter about the TLS at the actual container registry

ref: https://ii.coop/blog/rerouting-container-registries-with-envoy/#the-implementation

justinsb · 2021-04-20T13:29:45Z

We could throw it into the aaa cluster as a quick test.

Do you mean deploying https://github.com/kubernetes/k8s.io/tree/main/artifactserver as a test?

BobyMCbobs · 2021-04-20T21:00:27Z

related: #1758

BobyMCbobs · 2021-05-19T05:00:59Z

I deployed Envoy as well as Distribution on a cluster in the k8s-infra-ii-sandbox project from this Org file
https://github.com/cncf-infra/prow-config/blob/master/infra/gcp/README.org#envoy

justinsb · 2021-05-19T14:12:05Z

@BobyMCbobs can we try deploying artifactserver as well?

BobyMCbobs · 2021-05-20T01:16:15Z

@BobyMCbobs can we try deploying artifactserver as well?

Yes! I've deployed it to https://artifacts.ii-sandbox.bobymcbobs-oitq.pair.sharing.io at the moment
https://github.com/cncf-infra/prow-config/blob/dc681e5d79d85af47df5f01ebcf281bf193de666/infra/gcp/README.org#artifactserver

I am currently trying to adapt the source to provide the same 302 functionality as what Envoy is providing.

k8s-triage-robot · 2021-08-18T01:43:35Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

puerco · 2021-08-18T01:45:22Z

/remove-lifecycle stale

BenTheElder · 2022-01-27T07:36:33Z

see: https://docs.google.com/document/d/1yNQ7DaDE5LbDJf9ku82YtlKZK0tcg5Wpk9L72-x2S2k/ (shared with dev@kubernetes.io mailinglist and the SIG mailing list) for some recent discussion on this topic.

Related: - kubernetes/k8s.io#1834 Test different AWS AMI with registry-sandbox.k8s.io Signed-off-by: Arnaud Meukam <ameukam@gmail.com>

BobyMCbobs · 2022-09-15T22:17:35Z

Update 📰 🎉

The redirecting from registry.k8s.io to k8s.gcr.io and prod-registry-k8s-io-$REGION.s3.dualstack.us-east-2.amazonaws.com is instantiated and there is automated replication between the buckets.
There is a registry-sandbox.k8s.io, for staging, including an auto-deploy from main. The staging environment is also used in CI jobs.
The repo for the redirector is available at https://github.com/kubernetes/registry.k8s.io.
It has been a huge effort with collaborations between many folks in sig-k8s-infra and sig-release.

cc @kubernetes/sig-k8s-infra

BenTheElder · 2023-03-15T04:26:19Z

I think we can close this.

This is at https://registry.k8s.io now and is generally implemented.

What remains is phasing over users, which we're tracking elsewhere.

BenTheElder · 2023-03-16T21:16:38Z

/close

k8s-ci-robot · 2023-03-16T21:16:48Z

@BenTheElder: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ameukam added area/artifacts Issues or PRs related to the hosting of release artifacts for subprojects priority/backlog Higher priority than priority/awaiting-more-evidence. labels Mar 25, 2021

BobyMCbobs changed the title ~~k8s.gcr.io => registry.k8s.io solution~~ Umbrella issue: k8s.gcr.io => registry.k8s.io solution Apr 8, 2021

xaleeks mentioned this issue Apr 13, 2021

decision: Evaluate options for primary container image storage projectcontour/contour#3366

Closed

ameukam added sig/release Categorizes an issue or PR as relevant to SIG Release. wg/k8s-infra labels Apr 16, 2021

BobyMCbobs mentioned this issue May 20, 2021

[WIP] Update artifactserver with conditions and config file #2068

Closed

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 18, 2021

ameukam added a commit to ameukam/test-infra that referenced this issue Feb 7, 2022

kops: test alternative registry for periodics-distros

593e796

Related: - kubernetes/k8s.io#1834 Test different AWS AMI with registry-sandbox.k8s.io Signed-off-by: Arnaud Meukam <ameukam@gmail.com>

ameukam mentioned this issue Feb 7, 2022

kops: test alternative registry for periodics-distros kubernetes/test-infra#25155

Merged

puerco mentioned this issue Feb 8, 2022

The Big Image Promoter Refactor™ kubernetes-sigs/promo-tools#494

Merged

ameukam added a commit to ameukam/test-infra that referenced this issue Feb 9, 2022

kops: test alternative registry for periodics-distros

8abf16c

Related: - kubernetes/k8s.io#1834 Test different AWS AMI with registry-sandbox.k8s.io Signed-off-by: Arnaud Meukam <ameukam@gmail.com>

ameukam mentioned this issue Feb 16, 2022

[Umbrella issue] Migrate from k8s.gcr.io to registry.k8s.io #3411

Closed

9 tasks

This was referenced Mar 24, 2022

switch the default kubeadm image registry to registry.k8s.io kubernetes/kubeadm#2671

Closed

switch the default image registry in image builder to registry.k8s.io kubernetes-sigs/image-builder#845

Closed

hh mentioned this issue Apr 1, 2022

oci-proxy should redirect per-AWS region kubernetes/registry.k8s.io#39

Closed

riaankleinhans moved this to Backlog in registry.k8s.io (SIG K8S Infra) Apr 4, 2022

riaankleinhans added this to registry.k8s.io (SIG K8S Infra) Apr 4, 2022

neolit123 mentioned this issue Jun 7, 2022

update kubeadm pages to use registry.k8s.io kubernetes/website#34163

Merged

z0rc mentioned this issue Jun 7, 2022

Switch from k8s.gcr.io to registry.k8s.io vmware-tanzu/sonobuoy#1734

Closed

ameukam moved this from Backlog to Breakdown TODO in registry.k8s.io (SIG K8S Infra) Jun 22, 2022

BobyMCbobs mentioned this issue Jul 20, 2022

registry.k8s.io S3 bucket auditing #3992

Closed

divya-mohan0209 mentioned this issue Oct 14, 2022

Update basic-stateful-set.md kubernetes/website#37178

Closed

twz123 mentioned this issue Nov 2, 2022

Switch from k8s.gcr.io to registry.k8s.io k0sproject/k0s#2346

Merged

16 tasks

riaankleinhans moved this from Breakdown TODO to Backlog in registry.k8s.io (SIG K8S Infra) Jan 4, 2023

riaankleinhans moved this from Backlog to Breakdown TODO in registry.k8s.io (SIG K8S Infra) Jan 4, 2023

afbjorklund mentioned this issue Mar 10, 2023

Make old k8s releases also use the new registry kubernetes/minikube#16017

Closed

k8s-ci-robot closed this as completed Mar 16, 2023

github-project-automation bot moved this from Breakdown TODO to Reporting, Audit & Logging in registry.k8s.io (SIG K8S Infra) Mar 16, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Umbrella issue: k8s.gcr.io => registry.k8s.io solution #1834

Umbrella issue: k8s.gcr.io => registry.k8s.io solution #1834

BobyMCbobs commented Mar 25, 2021 •

edited

Loading

stp-ip commented Mar 26, 2021

thockin commented Mar 26, 2021

BobyMCbobs commented Mar 28, 2021

justaugustus commented Mar 29, 2021

BobyMCbobs commented Mar 29, 2021

BobyMCbobs commented Mar 29, 2021

thockin commented Mar 29, 2021 via email

BobyMCbobs commented Mar 29, 2021

rikatz commented Mar 31, 2021 •

edited

Loading

hh commented Apr 8, 2021

stp-ip commented Apr 8, 2021

BobyMCbobs commented Apr 9, 2021

BobyMCbobs commented Apr 13, 2021

thockin commented Apr 16, 2021

BobyMCbobs commented Apr 20, 2021

justinsb commented Apr 20, 2021

BobyMCbobs commented Apr 20, 2021 •

edited

Loading

BobyMCbobs commented May 19, 2021

justinsb commented May 19, 2021

BobyMCbobs commented May 20, 2021

k8s-triage-robot commented Aug 18, 2021

puerco commented Aug 18, 2021

BenTheElder commented Jan 27, 2022

BobyMCbobs commented Sep 15, 2022

BenTheElder commented Mar 15, 2023

BenTheElder commented Mar 16, 2023

k8s-ci-robot commented Mar 16, 2023

Umbrella issue: k8s.gcr.io => registry.k8s.io solution #1834

Umbrella issue: k8s.gcr.io => registry.k8s.io solution #1834

Comments

BobyMCbobs commented Mar 25, 2021 • edited Loading

Umbrella issue: k8s.gcr.io => registry.k8s.io solution #1834

What exactly are you doing? (and how?)

stp-ip commented Mar 26, 2021

thockin commented Mar 26, 2021

BobyMCbobs commented Mar 28, 2021

justaugustus commented Mar 29, 2021

BobyMCbobs commented Mar 29, 2021

BobyMCbobs commented Mar 29, 2021

thockin commented Mar 29, 2021 via email

BobyMCbobs commented Mar 29, 2021

rikatz commented Mar 31, 2021 • edited Loading

hh commented Apr 8, 2021

stp-ip commented Apr 8, 2021

BobyMCbobs commented Apr 9, 2021

BobyMCbobs commented Apr 13, 2021

thockin commented Apr 16, 2021

BobyMCbobs commented Apr 20, 2021

justinsb commented Apr 20, 2021

BobyMCbobs commented Apr 20, 2021 • edited Loading

BobyMCbobs commented May 19, 2021

justinsb commented May 19, 2021

BobyMCbobs commented May 20, 2021

k8s-triage-robot commented Aug 18, 2021

puerco commented Aug 18, 2021

BenTheElder commented Jan 27, 2022

BobyMCbobs commented Sep 15, 2022

BenTheElder commented Mar 15, 2023

BenTheElder commented Mar 16, 2023

k8s-ci-robot commented Mar 16, 2023

BobyMCbobs commented Mar 25, 2021 •

edited

Loading

rikatz commented Mar 31, 2021 •

edited

Loading

BobyMCbobs commented Apr 20, 2021 •

edited

Loading