none: cache images to local daemon if running instead of as tarballs on machine #7254
Comments
|
@eoinreilly93 : the cache is disabled for the "none" driver, automatically The (dubious) theory is that you will cache them in docker, rather than on disk. It would be possible to change the code to pull them in docker instead, for the Instead of saving as "tarball" in the cache directory, it would save them to "daemon" in docker |
afbjorklund
added
co/none-driver
kind/feature
|
As a workaround, you can load them yourself from the cache (using |
|
Interesting that |
tstromberg
changed the title
tstromberg
added
the
kind/documentation
|
Thanks for the response. I have tried what you suggested and loaded in the docker images and re-ran the minikube start command, but now I am getting a different error relating to kubelet I believe. I ran The exact command to reproduce the issue: The full output of the command that failed: [root@SR-SVR-206 kubernetesui]# minikube start --vm-driver=none --kubernetes-version=1.16.3 ! minikube v1.8.2 on Centos 7.7.1908
X Error starting cluster: init failed. output: "-- stdout --\n[init] Using Kubernetes version: v1.16.3\n[preflight] Running pre-flight checks\n[preflight] Pulling images required for setting up a Kubernetes cluster\n[preflight] This might take a minute or two, depending on the speed of your internet connection\n[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'\n[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"\n[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"\n[kubelet-start] Activating the kubelet service\n[certs] Using certificateDir folder "/var/lib/minikube/certs"\n[certs] Using existing ca certificate authority\n[certs] Using existing apiserver certificate and key on disk\n[certs] Generating "apiserver-kubelet-client" certificate and key\n[certs] Generating "front-proxy-ca" certificate and key\n[certs] Generating "front-proxy-client" certificate and key\n[certs] Generating "etcd/ca" certificate and key\n[certs] Generating "etcd/server" certificate and key\n[certs] etcd/server serving cert is signed for DNS names [sr-svr-206.dev.os.net localhost] and IPs [15.6.1.206 127.0.0.1 ::1]\n[certs] Generating "etcd/peer" certificate and key\n[certs] etcd/peer serving cert is signed for DNS names [sr-svr-206.dev.os.net localhost] and IPs [15.6.1.206 127.0.0.1 ::1]\n[certs] Generating "etcd/healthcheck-client" certificate and key\n[certs] Generating "apiserver-etcd-client" certificate and key\n[certs] Generating "sa" key and public key\n[kubeconfig] Using kubeconfig folder "/etc/kubernetes"\n[kubeconfig] Writing "admin.conf" kubeconfig file\n[kubeconfig] Writing "kubelet.conf" kubeconfig file\n[kubeconfig] Writing "controller-manager.conf" kubeconfig file\n[kubeconfig] Writing "scheduler.conf" kubeconfig file\n[control-plane] Using manifest folder "/etc/kubernetes/manifests"\n[control-plane] Creating static Pod manifest for "kube-apiserver"\n[control-plane] Creating static Pod manifest for "kube-controller-manager"\n[control-plane] Creating static Pod manifest for "kube-scheduler"\n[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"\n[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s\n[kubelet-check] Initial timeout of 40s passed.\n\nUnfortunately, an error has occurred:\n\ttimed out waiting for the condition\n\nThis error is likely caused by:\n\t- The kubelet is not running\n\t- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)\n\nIf you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:\n\t- 'systemctl status kubelet'\n\t- 'journalctl -xeu kubelet'\n\nAdditionally, a control plane component may have crashed or exited when started by the container runtime.\nTo troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.\nHere is one example how you may list all Kubernetes containers running in docker:\n\t- 'docker ps -a | grep kube | grep -v pause'\n\tOnce you have found the failing container, you can inspect its logs with:\n\t- 'docker logs CONTAINERID'\n\n-- /stdout --\n** stderr ** \n\t[WARNING Firewalld]: firewalld is active, please ensure ports [8443 10250] are open or your cluster may not function correctly\n\t[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/\n\t[WARNING Swap]: running with swap on is not supported. Please disable swap\n\t[WARNING FileExisting-socat]: socat not found in system path\n\t[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.3. Latest validated version: 18.09\n\t[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'\nerror execution phase wait-control-plane: couldn't initialize a Kubernetes cluster\nTo see the stack trace of this error execute with --v=5 or higher\n\n** /stderr **": /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.16.3:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification": exit status 1 Unfortunately, an error has occurred: This error is likely caused by: If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands: Additionally, a control plane component may have crashed or exited when started by the container runtime. stderr:
The output of the [root@SR-SVR-206 kubernetesui]# minikube logs
The output of the [root@SR-SVR-206 kubernetesui]# systemctl status kubelet Mar 27 10:52:00 SR-SVR-206.dev.os.net kubelet[6155]: E0327 10:52:00.226896 6155 kubelet.go:2267] node "sr-svr-206.dev.os.net" not found |
|
Seems like a timeout. If you haven't yet disabled SELinux, you need to do that. https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/ |
|
Thanks! That seems to have solved that problem. When I run Do you know why this might be? I installed minikube as root using |
|
Hey @eoinreilly93, what is the output of |
|
You need to install This might change in a future release, to use the built-in |
afbjorklund
added
the
priority/important-longterm
|
@afbjorklund Thanks! I've installed it separately which looks to have solved the issue. I have other problems starting the dashboard, but I've raised them under a new ticket now. |
|
@priyawadhwa |
priyawadhwa
removed
the
kind/documentation
priyawadhwa
changed the title
|
@eoinreilly93 is anything else needed here? Seems like the issues for this issue have been resolved. |
|
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
k8s-ci-robot
added
the
lifecycle/stale
|
Stale issues rot after 30d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
k8s-ci-robot
added
lifecycle/rotten
|
Rotten issues close after 30d of inactivity. Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
|
@fejta-bot: Closing this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
I am trying to install Minikube 1.8.2 on a work machine that has has no access to the internet. To do this, I installed minikube locally on my Windows 10 laptop which worked fine except for this warning message I received about not being able to access 'k8s.gcr.io', but I was told in this ticket I raised that that is should not cause any issues. I then brought across the '~/.minikube/cache' directory to the other environment, which is a VM running CentOS 7.7 and Docker 19.03.3. According to the cache documentation, minikube should use the data in the cache if it is present instead of trying to download dependencies. However when I run the start command, it errors out with the message below, saying it timed out trying to pull images.
I have tried changing permissions on the cache directory to be 777, but this has not helped. Please see my cache structure below:
As you can see all the images it is trying to pull exist in the cache, so I do not understand why it still attempting to pull them from the docker registry.
The exact command to reproduce the issue:
minikube start --kubernetes-version=1.16.3 --driver-none
The full output of the command that failed:
! minikube v1.8.2 on Centos 7.7.1908
! Node may be unable to resolve external DNS records
! VM is unable to access k8s.gcr.io, you may need to configure a proxy or set --image-repository
X Error starting cluster: init failed. output: "-- stdout --\n[init] Using Kubernetes version: v1.16.3\n[preflight] Running pre-flight checks\n[preflight] Pulling image s required for setting up a Kubernetes cluster\n[preflight] This might take a minute or two, depending on the speed of your internet connection\n[preflight] You can als o perform this action in beforehand using 'kubeadm config images pull'\n\n-- /stdout --\n** stderr ** \n\t[WARNING Firewalld]: firewalld is active, please ensure ports [8443 10250] are open or your cluster may not function correctly\n\t[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended d river is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/\n\t[WARNING Swap]: running with swap on is not supported. Please disable swap\n\t [WARNING FileExisting-socat]: socat not found in system path\n\t[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.3. Lat est validated version: 18.09\n\t[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'\nerror execution phase prefligh t: [preflight] Some fatal errors occurred:\n\t[ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-apiserver:v1.16.3: output: Error response from daemon: Get https:/ /k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)\n, error: exit status 1\n\t[ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-controller-manager:v1.16.3: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waitin g for connection (Client.Timeout exceeded while awaiting headers)\n, error: exit status 1\n\t[ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-scheduler:v1.16.3: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) \n, error: exit status 1\n\t[ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-proxy:v1.16.3: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/h ttp: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)\n, error: exit status 1\n\t[ERROR ImagePull]: failed to pull image k 8s.gcr.io/pause:3.1: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded wh ile awaiting headers)\n, error: exit status 1\n\t[ERROR ImagePull]: failed to pull image k8s.gcr.io/etcd:3.3.15-0: output: Error response from daemon: Get https://k8s.g cr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)\n, error: exit status 1\n\t[ERROR ImagePull]: failed to pull image k8s.gcr.io/coredns:1.6.2: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client .Timeout exceeded while awaiting headers)\n, error: exit status 1\n[preflight] If you know what you are doing, you can make a check non-fatal with
--ignore-preflight-e rrors=...\nTo see the stack trace of this error execute with --v=5 or higher\n\n** /stderr **": /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.16.3:$PATH ku beadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-li b-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-ma nifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification": exit status 1stdout:
[init] Using Kubernetes version: v1.16.3
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
stderr:
[WARNING Firewalld]: firewalld is active, please ensure ports [8443 10250] are open or your cluster may not function correctly
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kuberne tes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING FileExisting-socat]: socat not found in system path
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.3. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-apiserver:v1.16.3: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request can celed while waiting for connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1
[ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-controller-manager:v1.16.3: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: re quest canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1
[ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-scheduler:v1.16.3: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request can celed while waiting for connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1
[ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-proxy:v1.16.3: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request cancele d while waiting for connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1
[ERROR ImagePull]: failed to pull image k8s.gcr.io/pause:3.1: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while w aiting for connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1
[ERROR ImagePull]: failed to pull image k8s.gcr.io/etcd:3.3.15-0: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled whi le waiting for connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1
[ERROR ImagePull]: failed to pull image k8s.gcr.io/coredns:1.6.2: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled whi le waiting for connection (Client.Timeout exceeded while awaiting headers)
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with
--ignore-preflight-errors=...To see the stack trace of this error execute with --v=5 or higher
The output of the
minikube logscommand:The operating system version:
CentOS 7.71908
Docker 19.03.3
Minikube 1.8.2
The text was updated successfully, but these errors were encountered: