Migrate image registry to `registry.k8s.io` #4720

ArkaSaha30 · 2023-03-17T04:51:36Z

This PR will migrate image registry from k8s.gcr.io to registry.k8s.io.
Fixes: kubernetes/k8s.io#4780, #4704

Signed-off-by: ArkaSaha30 arkasaha30@gmail.com

XinShuYang · 2023-03-20T03:03:43Z

Hi, please also add #4704 to commit message, then it can be closed after your PR is merged. @ArkaSaha30

XinShuYang · 2023-03-20T03:07:06Z

/test-windows-all
/test-all

ArkaSaha30 · 2023-03-20T15:44:33Z

Done @XinShuYang, please review.

antoninbas

LGTM, but the change to pkg/apiserver/openapi/zz_generated.openapi.go needs to be reverted.

antoninbas · 2023-03-20T18:08:22Z

pkg/apiserver/openapi/zz_generated.openapi.go

@@ -4855,7 +4855,7 @@ func schema_k8sio_api_core_v1_ContainerImage(ref common.ReferenceCallback) commo
 				Properties: map[string]spec.Schema{
 					"names": {
 						SchemaProps: spec.SchemaProps{
-							Description: "Names by which this image is known. e.g. [\"k8s.gcr.io/hyperkube:v1.0.7\", \"dockerhub.io/google_containers/hyperkube:v1.0.7\"]",
+							Description: "Names by which this image is known. e.g. [\"registry.k8s.io/hyperkube:v1.0.7\", \"dockerhub.io/google_containers/hyperkube:v1.0.7\"]",


The Github workflow that checks that all auto-generated files are up-to-date is failing for this PR. You need to revert the change to this file.

This file is auto-generated so making a manual change to it (I assume you ran a sed command when working on this PR) is typically incorrect. When we update our K8s dependencies in the future, the reference to k8s.gcr.io will go away, but in the mean time, having it around is not an issue.

Signed-off-by: ArkaSaha30 <arkasaha30@gmail.com>

XinShuYang · 2023-03-21T08:58:26Z

/test-windows-all
/test-all

antoninbas

LGTM, thanks for your contribution @ArkaSaha30 !

XinShuYang · 2023-03-22T02:52:44Z

ci/jenkins/test.sh

@@ -395,7 +395,7 @@ function deliver_antrea_windows {
        # Some tests need us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.13 image but it is not for windows/amd64 10.0.17763
        # Use e2eteam/agnhost:2.13 instead
        harbor_images=("sigwindowstools-kube-proxy:v1.18.0" "agnhost:2.13" "agnhost:2.13" "agnhost:2.13" "agnhost:2.29" "e2eteam-jessie-dnsutils:1.0" "e2eteam-jessie-dnsutils:1.0" "e2eteam-pause:3.2" "e2eteam-pause:3.2" "e2eteam-busybox:1.29-windows-amd64-1809")
-        antrea_images=("sigwindowstools/kube-proxy:v1.18.0" "e2eteam/agnhost:2.13" "us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.13" "k8sprow.azurecr.io/kubernetes-e2e-test-images/agnhost:2.13" "k8s.gcr.io/e2e-test-images/agnhost:2.29" "e2eteam/jessie-dnsutils:1.0" "gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0" "e2eteam/pause:3.2" "k8s.gcr.io/pause:3.2" "docker.io/library/busybox:1.29")
+        antrea_images=("sigwindowstools/kube-proxy:v1.18.0" "e2eteam/agnhost:2.13" "us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.13" "k8sprow.azurecr.io/kubernetes-e2e-test-images/agnhost:2.13" "registry.k8s.io/e2e-test-images/agnhost:2.29" "e2eteam/jessie-dnsutils:1.0" "gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0" "e2eteam/pause:3.2" "registry.k8s.io/pause:3.2" "docker.io/library/busybox:1.29")


The antrea_images is used to generate a new tag for the Harbor image to avoid pull rate limit problems. As we still have a Windows Docker testbed with an old version of Kubernetes, the images used in the conformance tests on these testbeds are still in the k8s.gcr.io repository. Perhaps, for deliver_antrea_windows, we can add a new registry.k8s.io tag for e2e tests, but still keep the existing k8s.gcr.io tag for old version conformance tests. What's your opinion? @antoninbas

Sorry @XinShuYang, I didn't see your comment before merging this.
I guess we may need to tag the image twice (once with k8s.gcr.io and once with registry.k8s.io) to keep the tests running?

It feels like there should be a better way to handle this. The current solution seems a bit brittle, as both the Harbor registry and this script need to be updated when a new image is required by the conformance tests.

Also, one thing that comes to mind is that rate limiting doesn't apply to the K8s registries, so why don't we pull them directly?

is that why jenkins-windows-conformance failed?

@antoninbas Yes, our windows docker testbed is running with old version kubernetes, which means its conformance tests still use images from k8s.gcr.io repo. In CI pipeline we pull image from harbor first(which doesn't have pull rate limit problem) and add the k8s.gcr.io tag to these images, then set the image pull policy of conformance/networkpolicy tests to IfNotPresent to avoid pulling images from these unavailable repo during running tests.

One of my points is that there should be no rate limiting for k8s.gcr.io (unless I am mistaken), so why do we pull these from Harbor?

It's possible that images with same version differ on harbor and dockerhub. When I ran "docker pull e2eteam-pause:3.2" on windows node, I received the error message "no matching manifest for windows/amd64 10.0.17763 in the manifest list entries". However, pulling image from "projects.registry.vmware.com/antrea/e2eteam-pause:3.2" worked fine.

Perhaps, for deliver_antrea_windows, we can add a new registry.k8s.io tag for e2e tests, but still keep the existing k8s.gcr.io tag for old version conformance tests.

@antoninbas @XinShuYang do you recommend to open a new PR for this once we have resolved the best way to tag/pull the image?

I will let @XinShuYang decide what is the best course of action here.
Ideally we would not have to list all images like that (or maybe our testbed could be upgraded?).

antoninbas · 2023-03-22T18:44:04Z

@tnqn could you exclude this commit from the v1.11 release, while we figure out potential issues with Windows testing?

tnqn · 2023-03-23T01:36:57Z

@tnqn could you exclude this commit from the v1.11 release, while we figure out potential issues with Windows testing?

Sure, this will not be included in release-1.11.

Add necessary image tags which was removed by (antrea-io#4720). For antrea-io#4704 Signed-off-by: Shuyang Xin <gavinx@vmware.com>

Add necessary image tags which was removed by (#4720). For #4704 Signed-off-by: Shuyang Xin <gavinx@vmware.com>

Signed-off-by: ArkaSaha30 <arkasaha30@gmail.com>

Add necessary image tags which was removed by (antrea-io#4720). For antrea-io#4704 Signed-off-by: Shuyang Xin <gavinx@vmware.com>

ArkaSaha30 mentioned this pull request Mar 17, 2023

[Umbrella Issue] Migrate CNCF Ecosystem projects from k8s.gcr.io to registry.k8s.io kubernetes/k8s.io#4780

Open

70 tasks

antoninbas reviewed Mar 20, 2023

View reviewed changes

Migrate image registry to registry.k8s.io

d0b13f0

Signed-off-by: ArkaSaha30 <arkasaha30@gmail.com>

ArkaSaha30 force-pushed the registry-migration branch from e064028 to d0b13f0 Compare March 21, 2023 05:11

antoninbas approved these changes Mar 21, 2023

View reviewed changes

XinShuYang reviewed Mar 22, 2023

View reviewed changes

antoninbas merged commit 219adc4 into antrea-io:main Mar 22, 2023

XinShuYang added a commit to XinShuYang/antrea that referenced this pull request Apr 3, 2023

Fix Windows CI failures by updating image tags

5f3f3b9

Add necessary image tags which was removed by (antrea-io#4720). For antrea-io#4704 Signed-off-by: Shuyang Xin <gavinx@vmware.com>

XinShuYang mentioned this pull request Apr 3, 2023

Fix Windows CI failures by updating image tags #4801

Merged

XinShuYang added a commit to XinShuYang/antrea that referenced this pull request Apr 5, 2023

Fix Windows CI failures by updating image tags

cda56a9

Add necessary image tags which was removed by (antrea-io#4720). For antrea-io#4704 Signed-off-by: Shuyang Xin <gavinx@vmware.com>

XinShuYang added a commit to XinShuYang/antrea that referenced this pull request Apr 5, 2023

Fix Windows CI failures by updating image tags

bbc93a5

Add necessary image tags which was removed by (antrea-io#4720). For antrea-io#4704 Signed-off-by: Shuyang Xin <gavinx@vmware.com>

tnqn pushed a commit that referenced this pull request Apr 6, 2023

Fix Windows CI failures by updating image tags (#4801)

4ef8eec

Add necessary image tags which was removed by (#4720). For #4704 Signed-off-by: Shuyang Xin <gavinx@vmware.com>

jainpulkit22 pushed a commit to urharshitha/antrea that referenced this pull request Apr 28, 2023

Migrate image registry to registry.k8s.io (antrea-io#4720)

0a712ed

Signed-off-by: ArkaSaha30 <arkasaha30@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate image registry to `registry.k8s.io` #4720

Migrate image registry to `registry.k8s.io` #4720

ArkaSaha30 commented Mar 17, 2023 •

edited

Loading

XinShuYang commented Mar 20, 2023

XinShuYang commented Mar 20, 2023

ArkaSaha30 commented Mar 20, 2023

antoninbas left a comment

antoninbas Mar 20, 2023

XinShuYang commented Mar 21, 2023

antoninbas left a comment

XinShuYang Mar 22, 2023

antoninbas Mar 22, 2023

antoninbas Mar 22, 2023

XinShuYang Mar 23, 2023 •

edited

Loading

antoninbas Mar 23, 2023

XinShuYang Mar 23, 2023

ArkaSaha30 Mar 23, 2023

antoninbas Mar 24, 2023

antoninbas commented Mar 22, 2023

tnqn commented Mar 23, 2023

Migrate image registry to registry.k8s.io #4720

Migrate image registry to registry.k8s.io #4720

Conversation

ArkaSaha30 commented Mar 17, 2023 • edited Loading

XinShuYang commented Mar 20, 2023

XinShuYang commented Mar 20, 2023

ArkaSaha30 commented Mar 20, 2023

antoninbas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

XinShuYang commented Mar 21, 2023

antoninbas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

XinShuYang Mar 23, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antoninbas commented Mar 22, 2023

tnqn commented Mar 23, 2023

Migrate image registry to `registry.k8s.io` #4720

Migrate image registry to `registry.k8s.io` #4720

ArkaSaha30 commented Mar 17, 2023 •

edited

Loading

XinShuYang Mar 23, 2023 •

edited

Loading