Upgrade Go from 1.19.10 and 1.20.5 to 1.19.11 and 1.20.6 #988

flatcar-infra · 2023-07-17T07:26:45Z

Subject says it all.

/update-sdk

CI: http://jenkins.infra.kinvolk.io:8080/job/container/job/sdk/941/cldsv/

dongsupark · 2023-07-18T07:12:50Z

200+ tests failed with the following messages:

cluster.go:117: http: invalid Host header

Added on-hold.

github-actions · 2023-07-18T07:13:03Z

Test report for 3665.0.0+nightly-20230714-2100-INTERMEDIATE / amd64 arm64

Platforms tested : qemu_uefi-amd64 qemu_uefi-arm64

bpf.execsnoop 🟢 Succeeded: qemu_uefi-amd64 (1)

bpf.local-gadget 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.basic 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.cgroupv1 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.cloudinit.basic 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.cloudinit.multipart-mime 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.cloudinit.script 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.disk.raid0.data 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.disk.raid0.root 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.disk.raid1.data 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.disk.raid1.root 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.etcd-member.discovery 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.etcd-member.etcdctlv3 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.etcd-member.v2-backup-restore 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.filesystem 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.flannel.udp 🟢 Succeeded: qemu_uefi-amd64 (1)

cl.flannel.vxlan 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.instantiated.enable-unit 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.kargs 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.luks 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.oem.indirect 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.oem.indirect.new 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.oem.regular 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.oem.regular.new 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.oem.reuse 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.oem.wipe 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.symlink 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.translation 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.v1.btrfsroot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.v1.ext4root 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.v1.groups 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.v1.once 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.v1.sethostname 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.v1.users 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.v1.xfsroot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.v2.btrfsroot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.v2.ext4root 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.v2.users 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.v2.xfsroot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.v2_1.ext4checkexisting 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.v2_1.swap 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.ignition.v2_1.vfat 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.install.cloudinit 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.internet 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.locksmith.cluster 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.misc.falco 🟢 Succeeded: qemu_uefi-amd64 (1)

cl.network.initramfs.second-boot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.network.listeners 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.network.wireguard 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.omaha.ping 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.osreset.ignition-rerun 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.overlay.cleanup 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.swap_activation 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.sysext.boot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.sysext.fallbackdownload # SKIP 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.toolbox.dnf-install 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.update.badverity 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.update.grubnop 🟢 Succeeded: qemu_uefi-amd64 (1)

cl.update.reboot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.users.shells 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

cl.verity 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.auth.verify 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.ignition.groups 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.ignition.once 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.ignition.resource.local 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.ignition.resource.remote 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.ignition.resource.s3.versioned 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.ignition.security.tls 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.ignition.sethostname 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.ignition.systemd.enable-service 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.locksmith.reboot 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.locksmith.tls 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.selinux.boolean 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.selinux.enforce 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.tls.fetch-urls 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

coreos.update.badusr 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

devcontainer.docker 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

devcontainer.systemd-nspawn 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

docker.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

docker.btrfs-storage 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

docker.containerd-restart 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

docker.lib-coreos-dockerd-compat 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

docker.network 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

docker.selinux 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

docker.torcx-manifest-pkgs 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

docker.userns 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

kubeadm.v1.24.14.calico.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

kubeadm.v1.24.14.calico.cgroupv1.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (4) ❌ Failed: qemu_uefi-arm64 (1, 2, 3)

Diagnostic output for qemu_uefi-arm64, run 3

    L1: " Error: _cluster.go:117: I0720 12:31:17.416418    1614 version.go:256] remote version is much newer: v1.27.4; falling back to: stable-1.24"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.16"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.16"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.16"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.16"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L9: "cluster.go:117: I0720 12:31:31.805636    1782 version.go:256] remote version is much newer: v1.27.4; falling back to: stable-1.24"
    L10: "cluster.go:117: [init] Using Kubernetes version: v1.24.16"
    L11: "cluster.go:117: [preflight] Running pre-flight checks"
    L12: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.3?]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 7.505011 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: 5qcqbm.t03tnpui6uu8865a"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.3:6443 --token 5qcqbm.t03tnpui6uu8865a _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:ed2a7575b122662a369bbc3f314f0f13bdf45bd4777d1614864fdf6d9d4b8a8e "
    L78: "cluster.go:117: namespace/tigera-operator created"
    L79: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L80: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L81: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L82: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L83: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L84: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L85: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L86: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L87: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L88: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L89: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L90: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L91: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L92: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L93: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L94: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L95: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L96: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L97: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L98: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L99: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L100: "cluster.go:117: serviceaccount/tigera-operator created"
    L101: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L102: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:117: deployment.apps/tigera-operator created"
    L104: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L105: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L106: "cluster.go:117: installation.operator.tigera.io/default created"
    L107: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.24.14.calico.cgroupv1.base/nginx_deployment (93.77s)"
    L110: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L111: " "

Diagnostic output for qemu_uefi-arm64, run 2

    L1: " Error: _cluster.go:117: I0720 12:27:15.607085    1611 version.go:256] remote version is much newer: v1.27.4; falling back to: stable-1.24"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.16"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.16"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.16"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.16"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L9: "cluster.go:117: I0720 12:27:28.623700    1778 version.go:256] remote version is much newer: v1.27.4; falling back to: stable-1.24"
    L10: "cluster.go:117: [init] Using Kubernetes version: v1.24.16"
    L11: "cluster.go:117: [preflight] Running pre-flight checks"
    L12: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.5?]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 9.505940 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: kyqc6r.8tyt01a04p1py3j4"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.5:6443 --token kyqc6r.8tyt01a04p1py3j4 _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:3800085e1f0ad3d1ad3a44c984362ca3632bba12ae473733fc2c7ef4558fa1f5 "
    L78: "cluster.go:117: namespace/tigera-operator created"
    L79: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L80: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L81: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L82: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L83: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L84: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L85: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L86: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L87: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L88: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L89: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L90: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L91: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L92: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L93: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L94: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L95: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L96: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L97: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L98: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L99: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L100: "cluster.go:117: serviceaccount/tigera-operator created"
    L101: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L102: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:117: deployment.apps/tigera-operator created"
    L104: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L105: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L106: "cluster.go:117: installation.operator.tigera.io/default created"
    L107: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.24.14.calico.cgroupv1.base/nginx_deployment (93.79s)"
    L110: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L111: " "

Diagnostic output for qemu_uefi-arm64, run 1

    L1: "  "
    L2: " Error: _cluster.go:117: I0720 12:09:11.666400    1601 version.go:256] remote version is much newer: v1.27.4; falling back to: stable-1.24"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.16"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.16"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.16"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.16"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L9: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L10: "cluster.go:117: I0720 12:09:26.938993    1766 version.go:256] remote version is much newer: v1.27.4; falling back to: stable-1.24"
    L11: "cluster.go:117: [init] Using Kubernetes version: v1.24.16"
    L12: "cluster.go:117: [preflight] Running pre-flight checks"
    L13: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L14: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L15: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L16: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L17: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L18: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L19: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.5?5]"
    L20: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L22: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L27: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L28: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L29: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L30: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L35: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L36: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L37: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L40: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L41: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L42: "cluster.go:117: [apiclient] All control plane components are healthy after 7.003651 seconds"
    L43: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L44: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L45: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L47: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L48: "cluster.go:117: [bootstrap-token] Using token: hq5hhm.obvgn4smy5ekbbag"
    L49: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L53: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L54: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L55: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L56: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L57: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L58: "cluster.go:117: "
    L59: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L60: "cluster.go:117: "
    L61: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L62: "cluster.go:117: "
    L63: "cluster.go:117:   mkdir -p $HOME/.kube"
    L64: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L65: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L66: "cluster.go:117: "
    L67: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L68: "cluster.go:117: "
    L69: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L70: "cluster.go:117: "
    L71: "cluster.go:117: You should now deploy a pod network to the cluster."
    L72: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L73: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L74: "cluster.go:117: "
    L75: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L76: "cluster.go:117: "
    L77: "cluster.go:117: kubeadm join 10.0.0.55:6443 --token hq5hhm.obvgn4smy5ekbbag _"
    L78: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:95a9a8c462ee98cae6cdcd8bfb6f1ce4269775dc3502771a4829c6f8ae71a0e1 "
    L79: "cluster.go:117: namespace/tigera-operator created"
    L80: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created"
    L81: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created"
    L82: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created"
    L83: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created"
    L84: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created"
    L85: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created"
    L86: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created"
    L87: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created"
    L88: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created"
    L89: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created"
    L90: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created"
    L91: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created"
    L92: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created"
    L93: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created"
    L94: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created"
    L95: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created"
    L96: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created"
    L97: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created"
    L98: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created"
    L99: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created"
    L100: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created"
    L101: "cluster.go:117: serviceaccount/tigera-operator created"
    L102: "cluster.go:117: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
    L103: "cluster.go:117: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
    L104: "cluster.go:117: deployment.apps/tigera-operator created"
    L105: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
    L106: "cluster.go:117: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
    L107: "cluster.go:117: installation.operator.tigera.io/default created"
    L108: "cluster.go:117: apiserver.operator.tigera.io/default created"
    L109: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L110: "--- FAIL: kubeadm.v1.24.14.calico.cgroupv1.base/nginx_deployment (93.74s)"
    L111: "kubeadm.go:313: nginx is not deployed: ready replicas should be equal to 1: null_"
    L112: " "

kubeadm.v1.24.14.cilium.base 🟢 Succeeded: qemu_uefi-amd64 (2); qemu_uefi-arm64 (1) ❌ Failed: qemu_uefi-amd64 (1)

Diagnostic output for qemu_uefi-amd64, run 1

    L1: " Error: _cluster.go:117: I0720 12:16:47.851999    1494 version.go:256] remote version is much newer: v1.27.4; falling back to: stable-1.24"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.16"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.16"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.16"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.16"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L9: "cluster.go:117: I0720 12:16:59.150813    1660 version.go:256] remote version is much newer: v1.27.4; falling back to: stable-1.24"
    L10: "cluster.go:117: [init] Using Kubernetes version: v1.24.16"
    L11: "cluster.go:117: [preflight] Running pre-flight checks"
    L12: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.1?43]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 5.002102 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: q9b8fj.1v672nctr6q8t1kt"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.143:6443 --token q9b8fj.1v672nctr6q8t1kt _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:116fb7bb80af383b43fabeffab93c79b9bce01111e6c81e04637fcce325654a4 "
    L78: "cluster.go:117: i  Using Cilium version 1.12.1"
    L79: "cluster.go:117: ? Auto-detected cluster name: kubernetes"
    L80: "cluster.go:117: ? Auto-detected datapath mode: tunnel"
    L81: "cluster.go:117: ? Auto-detected kube-proxy has been installed"
    L82: "cluster.go:117: i  helm template --namespace kube-system cilium cilium/cilium --version 1.12.1 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,extraConfig.cluster-pool-ipv4-?cidr=192.168.0.0/17,extraConfig.enable-endpoint-routes=true,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vx?lan"
    L83: "cluster.go:117: i  Storing helm values file in kube-system/cilium-cli-helm-values Secret"
    L84: "cluster.go:117: ? Created CA in secret cilium-ca"
    L85: "cluster.go:117: ? Generating certificates for Hubble..."
    L86: "cluster.go:117: ? Creating Service accounts..."
    L87: "cluster.go:117: ? Creating Cluster roles..."
    L88: "cluster.go:117: ? Creating ConfigMap for Cilium version 1.12.1..."
    L89: "cluster.go:117: i Manual overwrite in ConfigMap: enable-endpoint-routes=true"
    L90: "cluster.go:117: i Manual overwrite in ConfigMap: cluster-pool-ipv4-cidr=192.168.0.0/17"
    L91: "cluster.go:117: ? Creating Agent DaemonSet..."
    L92: "cluster.go:117: ? Creating Operator Deployment..."
    L93: "cluster.go:117: ? Waiting for Cilium to be installed and ready..."
    L94: "cluster.go:117: ? Cilium was successfully installed! Run _cilium status_ to view installation health"
    L95: "cluster.go:117: daemonset.apps/cilium patched"
    L96: "cluster.go:117: ?[33m    /??_"
    L97: "cluster.go:117: ?[36m /???[33m___/?[32m??_?[0m    Cilium:         ?[32mOK?[0m"
    L98: "cluster.go:117: ?[36m ___?[31m/??_?[32m__/?[0m    Operator:       ?[32mOK?[0m"
    L99: "cluster.go:117: ?[32m /???[31m___/?[35m??_?[0m    Hubble:         ?[36mdisabled?[0m"
    L100: "cluster.go:117: ?[32m ___?[34m/??_?[35m__/?[0m    ClusterMesh:    ?[36mdisabled?[0m"
    L101: "cluster.go:117: ?[34m    ___/"
    L102: "cluster.go:117: ?[0m"
    L103: "cluster.go:117: Deployment       cilium-operator    "
    L104: "cluster.go:117: DaemonSet        cilium             "
    L105: "cluster.go:117: Containers:      cilium             "
    L106: "cluster.go:117:                  cilium-operator    "
    L107: "cluster.go:117: Cluster Pods:    0/0 managed by Cilium"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.24.14.cilium.base/node_readiness (92.08s)"
    L110: "kubeadm.go:295: nodes are not ready: ready nodes should be equal to 2: 1"
    L111: "--- FAIL: kubeadm.v1.24.14.cilium.base/IPSec_encryption (65.64s)"
    L112: "cluster.go:117: 2023/07/20 12:19:33 Unable to start gops: write /home/core/.config/gops/3220: no space left on device"
    L113: "cluster.go:117: "
    L114: "cluster.go:117: Error: Unable to determine status:  timeout while waiting for status to become successful: context deadline exceeded"
    L115: "cluster.go:130: __/opt/bin/cilium status --wait --wait-duration 1m__ failed: output ?[33m    /????_"
    L116: "?[36m /?????[33m___/?[32m????_?[0m    Cilium:         ?[31m1 errors?[0m, ?[33m1 warnings?[0m"
    L117: "?[36m ___?[31m/????_?[32m__/?[0m    Operator:       ?[32mOK?[0m"
    L118: "?[32m /?????[31m___/?[35m????_?[0m    Hubble:         ?[36mdisabled?[0m"
    L119: "?[32m ___?[34m/????_?[35m__/?[0m    ClusterMesh:    ?[36mdisabled?[0m"
    L120: "?[34m    ___/"
    L121: "?[0m"
    L122: "Deployment        cilium-operator    Desired: 1, Ready: ?[32m1/1?[0m, Available: ?[32m1/1?[0m"
    L123: "DaemonSet         cilium             Desired: 2, Ready: ?[33m1/2?[0m, Available: ?[33m1/2?[0m, Unavailable: ?[31m1/2?[0m"
    L124: "Containers:       cilium             Running: ?[32m1?[0m, Pending: ?[32m1?[0m"
    L125: "cilium-operator    Running: ?[32m1?[0m"
    L126: "Cluster Pods:     3/3 managed by Cilium"
    L127: "Image versions    cilium-operator    quay.io/cilium/operator-generic:v1.12.1@sha256:93d5aaeda37d59e6c4325ff05030d7b48fabde6576478e3fdbfb9bb4a68ec4a1: 1"
    L128: "cilium             quay.io/cilium/cilium:v1.12.1@sha256:ea2db1ee21b88127b5c18a96ad155c25485d0815a667ef77c2b7c7f31cab601b: 2"
    L129: "Errors:           cilium             cilium          1 pods of DaemonSet cilium are not ready"
    L130: "Warnings:         cilium             cilium-g57mn    pod is pending, status Process exited with status 1_"
    L131: " "
    L132: "  "

kubeadm.v1.24.14.cilium.cgroupv1.base 🟢 Succeeded: qemu_uefi-amd64 (2); qemu_uefi-arm64 (1) ❌ Failed: qemu_uefi-amd64 (1)

Diagnostic output for qemu_uefi-amd64, run 1

    L1: " Error: _cluster.go:117: I0720 12:12:33.483237    1606 version.go:256] remote version is much newer: v1.27.4; falling back to: stable-1.24"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.24.16"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.24.16"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.24.16"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.24.16"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.7"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.6-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.8.6"
    L9: "cluster.go:117: I0720 12:12:44.504166    1775 version.go:256] remote version is much newer: v1.27.4; falling back to: stable-1.24"
    L10: "cluster.go:117: [init] Using Kubernetes version: v1.24.16"
    L11: "cluster.go:117: [preflight] Running pre-flight checks"
    L12: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L13: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L14: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.9?5]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 5.502848 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: yg1qwh.kcc26uanuggyvv1f"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.95:6443 --token yg1qwh.kcc26uanuggyvv1f _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:b3d63573175263d6b7f29b4b74a5aaed06e0b62313e393f9ad58c4eb63392e2c "
    L78: "cluster.go:117: i  Using Cilium version 1.12.1"
    L79: "cluster.go:117: ? Auto-detected cluster name: kubernetes"
    L80: "cluster.go:117: ? Auto-detected datapath mode: tunnel"
    L81: "cluster.go:117: ? Auto-detected kube-proxy has been installed"
    L82: "cluster.go:117: i  helm template --namespace kube-system cilium cilium/cilium --version 1.12.1 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,extraConfig.cluster-pool-ipv4-?cidr=192.168.0.0/17,extraConfig.enable-endpoint-routes=true,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vx?lan"
    L83: "cluster.go:117: i  Storing helm values file in kube-system/cilium-cli-helm-values Secret"
    L84: "cluster.go:117: ? Created CA in secret cilium-ca"
    L85: "cluster.go:117: ? Generating certificates for Hubble..."
    L86: "cluster.go:117: ? Creating Service accounts..."
    L87: "cluster.go:117: ? Creating Cluster roles..."
    L88: "cluster.go:117: ? Creating ConfigMap for Cilium version 1.12.1..."
    L89: "cluster.go:117: i Manual overwrite in ConfigMap: enable-endpoint-routes=true"
    L90: "cluster.go:117: i Manual overwrite in ConfigMap: cluster-pool-ipv4-cidr=192.168.0.0/17"
    L91: "cluster.go:117: ? Creating Agent DaemonSet..."
    L92: "cluster.go:117: ? Creating Operator Deployment..."
    L93: "cluster.go:117: ? Waiting for Cilium to be installed and ready..."
    L94: "cluster.go:117: ? Cilium was successfully installed! Run _cilium status_ to view installation health"
    L95: "cluster.go:117: daemonset.apps/cilium patched"
    L96: "cluster.go:117: ?[33m    /??_"
    L97: "cluster.go:117: ?[36m /???[33m___/?[32m??_?[0m    Cilium:         ?[32mOK?[0m"
    L98: "cluster.go:117: ?[36m ___?[31m/??_?[32m__/?[0m    Operator:       ?[32mOK?[0m"
    L99: "cluster.go:117: ?[32m /???[31m___/?[35m??_?[0m    Hubble:         ?[36mdisabled?[0m"
    L100: "cluster.go:117: ?[32m ___?[34m/??_?[35m__/?[0m    ClusterMesh:    ?[36mdisabled?[0m"
    L101: "cluster.go:117: ?[34m    ___/"
    L102: "cluster.go:117: ?[0m"
    L103: "cluster.go:117: Deployment       cilium-operator    "
    L104: "cluster.go:117: DaemonSet        cilium             "
    L105: "cluster.go:117: Containers:      cilium             "
    L106: "cluster.go:117:                  cilium-operator    "
    L107: "cluster.go:117: Cluster Pods:    0/0 managed by Cilium"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.24.14.cilium.cgroupv1.base/node_readiness (91.72s)"
    L110: "kubeadm.go:295: nodes are not ready: ready nodes should be equal to 2: 1_"
    L111: " "
    L112: "  "

kubeadm.v1.24.14.flannel.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

kubeadm.v1.24.14.flannel.cgroupv1.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

kubeadm.v1.25.10.calico.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

kubeadm.v1.25.10.cilium.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

kubeadm.v1.25.10.flannel.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

kubeadm.v1.26.5.calico.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

kubeadm.v1.26.5.cilium.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

kubeadm.v1.26.5.flannel.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

kubeadm.v1.27.2.calico.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

kubeadm.v1.27.2.cilium.base 🟢 Succeeded: qemu_uefi-amd64 (2); qemu_uefi-arm64 (1) ❌ Failed: qemu_uefi-amd64 (1)

Diagnostic output for qemu_uefi-amd64, run 1

    L1: " Error: _cluster.go:117: W0720 12:08:24.697194    1493 images.go:80] could not find officially supported version of etcd for Kubernetes v1.27.4, falling back to the nearest etcd version (3.5.7-0)"
    L2: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.27.4"
    L3: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.27.4"
    L4: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.27.4"
    L5: "cluster.go:117: [config/images] Pulled registry.k8s.io/kube-proxy:v1.27.4"
    L6: "cluster.go:117: [config/images] Pulled registry.k8s.io/pause:3.9"
    L7: "cluster.go:117: [config/images] Pulled registry.k8s.io/etcd:3.5.7-0"
    L8: "cluster.go:117: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.10.1"
    L9: "cluster.go:117: [init] Using Kubernetes version: v1.27.4"
    L10: "cluster.go:117: [preflight] Running pre-flight checks"
    L11: "cluster.go:117: [preflight] Pulling images required for setting up a Kubernetes cluster"
    L12: "cluster.go:117: [preflight] This might take a minute or two, depending on the speed of your internet connection"
    L13: "cluster.go:117: [preflight] You can also perform this action in beforehand using _kubeadm config images pull_"
    L14: "cluster.go:117: W0720 12:08:35.022913    1647 checks.go:835] detected that the sandbox image __registry.k8s.io/pause:3.8__ of the container runtime is inconsistent with that used by kubeadm. It is rec?ommended that using __registry.k8s.io/pause:3.9__ as the CRI sandbox image."
    L15: "cluster.go:117: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
    L16: "cluster.go:117: [certs] Generating __ca__ certificate and key"
    L17: "cluster.go:117: [certs] Generating __apiserver__ certificate and key"
    L18: "cluster.go:117: [certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 10.0.0.4?8]"
    L19: "cluster.go:117: [certs] Generating __apiserver-kubelet-client__ certificate and key"
    L20: "cluster.go:117: [certs] Generating __front-proxy-ca__ certificate and key"
    L21: "cluster.go:117: [certs] Generating __front-proxy-client__ certificate and key"
    L22: "cluster.go:117: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
    L23: "cluster.go:117: [certs] External etcd mode: Skipping etcd/server certificate generation"
    L24: "cluster.go:117: [certs] External etcd mode: Skipping etcd/peer certificate generation"
    L25: "cluster.go:117: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
    L26: "cluster.go:117: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
    L27: "cluster.go:117: [certs] Generating __sa__ key and public key"
    L28: "cluster.go:117: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
    L29: "cluster.go:117: [kubeconfig] Writing __admin.conf__ kubeconfig file"
    L30: "cluster.go:117: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
    L31: "cluster.go:117: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
    L32: "cluster.go:117: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
    L33: "cluster.go:117: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
    L34: "cluster.go:117: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
    L35: "cluster.go:117: [kubelet-start] Starting the kubelet"
    L36: "cluster.go:117: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
    L37: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-apiserver__"
    L38: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
    L39: "cluster.go:117: [control-plane] Creating static Pod manifest for __kube-scheduler__"
    L40: "cluster.go:117: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__. This can take up to 30m0s"
    L41: "cluster.go:117: [apiclient] All control plane components are healthy after 4.502022 seconds"
    L42: "cluster.go:117: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
    L43: "cluster.go:117: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
    L44: "cluster.go:117: [upload-certs] Skipping phase. Please see --upload-certs"
    L45: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]"
    L46: "cluster.go:117: [mark-control-plane] Marking the node localhost as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]"
    L47: "cluster.go:117: [bootstrap-token] Using token: oh6n45.zpdblo50yrjpebz0"
    L48: "cluster.go:117: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
    L49: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
    L50: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
    L51: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
    L52: "cluster.go:117: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
    L53: "cluster.go:117: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
    L54: "cluster.go:117: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
    L55: "cluster.go:117: [addons] Applied essential addon: CoreDNS"
    L56: "cluster.go:117: [addons] Applied essential addon: kube-proxy"
    L57: "cluster.go:117: "
    L58: "cluster.go:117: Your Kubernetes control-plane has initialized successfully!"
    L59: "cluster.go:117: "
    L60: "cluster.go:117: To start using your cluster, you need to run the following as a regular user:"
    L61: "cluster.go:117: "
    L62: "cluster.go:117:   mkdir -p $HOME/.kube"
    L63: "cluster.go:117:   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
    L64: "cluster.go:117:   sudo chown $(id -u):$(id -g) $HOME/.kube/config"
    L65: "cluster.go:117: "
    L66: "cluster.go:117: Alternatively, if you are the root user, you can run:"
    L67: "cluster.go:117: "
    L68: "cluster.go:117:   export KUBECONFIG=/etc/kubernetes/admin.conf"
    L69: "cluster.go:117: "
    L70: "cluster.go:117: You should now deploy a pod network to the cluster."
    L71: "cluster.go:117: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
    L72: "cluster.go:117:   https://kubernetes.io/docs/concepts/cluster-administration/addons/"
    L73: "cluster.go:117: "
    L74: "cluster.go:117: Then you can join any number of worker nodes by running the following on each as root:"
    L75: "cluster.go:117: "
    L76: "cluster.go:117: kubeadm join 10.0.0.48:6443 --token oh6n45.zpdblo50yrjpebz0 _"
    L77: "cluster.go:117:  --discovery-token-ca-cert-hash sha256:e17168522f48cfb860b8610504c1f5a0a3f61811b402b81ff9fd9a052a667ad7 "
    L78: "cluster.go:117: i  Using Cilium version 1.12.5"
    L79: "cluster.go:117: ? Auto-detected cluster name: kubernetes"
    L80: "cluster.go:117: ? Auto-detected datapath mode: tunnel"
    L81: "cluster.go:117: ? Auto-detected kube-proxy has been installed"
    L82: "cluster.go:117: i  helm template --namespace kube-system cilium cilium/cilium --version 1.12.5 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,extraConfig.cluster-pool-ipv4-?cidr=192.168.0.0/17,extraConfig.enable-endpoint-routes=true,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vx?lan"
    L83: "cluster.go:117: i  Storing helm values file in kube-system/cilium-cli-helm-values Secret"
    L84: "cluster.go:117: ? Created CA in secret cilium-ca"
    L85: "cluster.go:117: ? Generating certificates for Hubble..."
    L86: "cluster.go:117: ? Creating Service accounts..."
    L87: "cluster.go:117: ? Creating Cluster roles..."
    L88: "cluster.go:117: ? Creating ConfigMap for Cilium version 1.12.5..."
    L89: "cluster.go:117: i  Manual overwrite in ConfigMap: enable-endpoint-routes=true"
    L90: "cluster.go:117: i  Manual overwrite in ConfigMap: cluster-pool-ipv4-cidr=192.168.0.0/17"
    L91: "cluster.go:117: ? Creating Agent DaemonSet..."
    L92: "cluster.go:117: ? Creating Operator Deployment..."
    L93: "cluster.go:117: ? Waiting for Cilium to be installed and ready..."
    L94: "cluster.go:117: ? Cilium was successfully installed! Run _cilium status_ to view installation health"
    L95: "cluster.go:117: daemonset.apps/cilium patched"
    L96: "cluster.go:117: ?[33m    /??_"
    L97: "cluster.go:117: ?[36m /???[33m___/?[32m??_?[0m    Cilium:         ?[32mOK?[0m"
    L98: "cluster.go:117: ?[36m ___?[31m/??_?[32m__/?[0m    Operator:       ?[32mOK?[0m"
    L99: "cluster.go:117: ?[32m /???[31m___/?[35m??_?[0m    Hubble:         ?[36mdisabled?[0m"
    L100: "cluster.go:117: ?[32m ___?[34m/??_?[35m__/?[0m    ClusterMesh:    ?[36mdisabled?[0m"
    L101: "cluster.go:117: ?[34m    ___/"
    L102: "cluster.go:117: ?[0m"
    L103: "cluster.go:117: DaemonSet        cilium             "
    L104: "cluster.go:117: Deployment       cilium-operator    "
    L105: "cluster.go:117: Containers:      cilium-operator    "
    L106: "cluster.go:117:                  cilium             "
    L107: "cluster.go:117: Cluster Pods:    0/0 managed by Cilium"
    L108: "cluster.go:117: Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service ??? /etc/systemd/system/kubelet.service."
    L109: "--- FAIL: kubeadm.v1.27.2.cilium.base/node_readiness (91.87s)"
    L110: "kubeadm.go:295: nodes are not ready: ready nodes should be equal to 2: 1_"
    L111: " "
    L112: "  "

kubeadm.v1.27.2.flannel.base 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

linux.nfs.v3 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

linux.nfs.v4 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (2) ❌ Failed: qemu_uefi-arm64 (1)

Diagnostic output for qemu_uefi-arm64, run 1

    L1: "  "
    L2: " Error: _nfs.go:81: Cluster.NewMachine: machine __3e5a580c-3c7d-4706-9b52-cea19b698aed__ failed to start: ssh journalctl failed: time limit exceeded: dial tcp 10.0.0.127:22: connect: no route to host_"
    L3: " "

linux.ntp 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

packages 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

systemd.journal.remote 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

systemd.journal.user 🟢 Succeeded: qemu_uefi-amd64 (2); qemu_uefi-arm64 (1) ❌ Failed: qemu_uefi-amd64 (1)

Diagnostic output for qemu_uefi-amd64, run 1

    L1: " Error: _cluster.go:130: __journalctl --user --unit hello.service | grep Foo__ failed: output , status Process exited with status 1_"
    L2: " "
    L3: "  "

systemd.sysext.custom-docker 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

systemd.sysext.custom-oem 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

systemd.sysext.simple 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

systemd.sysusers.gshadow 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

torcx.enable-service 🟢 Succeeded: qemu_uefi-amd64 (1); qemu_uefi-arm64 (1)

dongsupark · 2023-07-18T07:47:27Z

cluster.go:117: http: invalid Host header

Already known issue in the upstream docker community.
moby/moby#45935
moby/moby#45942

dongsupark · 2023-07-20T07:27:59Z

Finally CI passed.
Let me sort out commits and rerun CI.

Reset to the state according to the state of Gentoo upstream, to commit 964117a, to create Flatcar patches on top of that.

Apply Flatcar patches, based on commit 9d6af12.

Docker client and daemon started to fail at sending or handling most local connections when being compiled with Go 1.19.11, which addresses CVE-2023-29406 by blocking invalid host headers of HTTP/1. As a workaround, Docker started to define a dummy host header, and to use it for local connections. Backport the fixes to Flatcar to fix the runtime failures. See also moby/moby#45935, moby/moby#45942.

Reset to the state according to the state of Gentoo upstream, to commit b93160f. to create Flatcar patches on top of that.

Apply Flatcar patches, based on commit 817ad4c.

…valid header To fix invalid header issue that started to happen when being built with Go 1.19.11+, it is necessary for the docker cli repo to vendor the new docker client part of github.com/docker/docker. Based on docker/cli@5d4e44df90bb.

Flatcar Buildbot added 2 commits July 17, 2023 07:26

dev-lang/go: Update from 1.19.10 to 1.19.11

1b03f99

dev-lang/go: Update from 1.20.5 to 1.20.6

dd25c1c

flatcar-infra added the main label Jul 17, 2023

flatcar-infra temporarily deployed to development July 17, 2023 07:26 — with GitHub Actions Inactive

flatcar-infra temporarily deployed to development July 17, 2023 07:39 — with GitHub Actions Inactive

changelog: add security changelog for Go 1.19.11, 1.20.6

7ae05d5

dongsupark temporarily deployed to development July 17, 2023 08:32 — with GitHub Actions Inactive

dongsupark added the on-hold label Jul 18, 2023

dongsupark had a problem deploying to development July 18, 2023 14:48 — with GitHub Actions Error

dongsupark force-pushed the go-1.19.11-and-1.20.6-main branch from 8aef9c8 to bdf1d11 Compare July 18, 2023 14:49

dongsupark temporarily deployed to development July 18, 2023 14:49 — with GitHub Actions Inactive

dongsupark added on-hold and removed on-hold labels Jul 18, 2023

dongsupark temporarily deployed to development July 19, 2023 12:45 — with GitHub Actions Inactive

dongsupark removed the on-hold label Jul 20, 2023

dongsupark added 6 commits July 20, 2023 10:11

overlay app-containers/docker: reset to Gentoo

c7209aa

Reset to the state according to the state of Gentoo upstream, to commit 964117a, to create Flatcar patches on top of that.

overlay app-containers/docker: Apply Flatcar modifications

3d001c5

Apply Flatcar patches, based on commit 9d6af12.

overlay app-containers/docker-cli: reset to Gentoo

82045f0

Reset to the state according to the state of Gentoo upstream, to commit b93160f. to create Flatcar patches on top of that.

overlay app-containers/docker-cli: Apply Flatcar modifications

3d1c3a6

Apply Flatcar patches, based on commit 817ad4c.

dongsupark force-pushed the go-1.19.11-and-1.20.6-main branch from 3e7390e to f36ae06 Compare July 20, 2023 08:24

dongsupark temporarily deployed to development July 20, 2023 08:24 — with GitHub Actions Inactive

dongsupark requested a review from a team July 21, 2023 06:30

krnowak approved these changes Jul 24, 2023

View reviewed changes

dongsupark merged commit 027e1ca into main Jul 24, 2023
5 of 7 checks passed

dongsupark deleted the go-1.19.11-and-1.20.6-main branch July 24, 2023 15:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade Go from 1.19.10 and 1.20.5 to 1.19.11 and 1.20.6 #988

Upgrade Go from 1.19.10 and 1.20.5 to 1.19.11 and 1.20.6 #988

flatcar-infra commented Jul 17, 2023 •

edited by dongsupark

Loading

dongsupark commented Jul 18, 2023

github-actions bot commented Jul 18, 2023 •

edited

Loading

dongsupark commented Jul 18, 2023

dongsupark commented Jul 20, 2023

Upgrade Go from 1.19.10 and 1.20.5 to 1.19.11 and 1.20.6 #988

Upgrade Go from 1.19.10 and 1.20.5 to 1.19.11 and 1.20.6 #988

Conversation

flatcar-infra commented Jul 17, 2023 • edited by dongsupark Loading

dongsupark commented Jul 18, 2023

github-actions bot commented Jul 18, 2023 • edited Loading

Test report for 3665.0.0+nightly-20230714-2100-INTERMEDIATE / amd64 arm64

dongsupark commented Jul 18, 2023

dongsupark commented Jul 20, 2023

flatcar-infra commented Jul 17, 2023 •

edited by dongsupark

Loading

github-actions bot commented Jul 18, 2023 •

edited

Loading