-
|
kubernetes noob here, I've been doing Docker for more than 5 years now and I tried to get k3s with embedded etcd DBs working. As a starting point I wanted to set up a simple replicated NTP server on a simulated environment, for this I decided to use Vagrant and spin up this config: Vagrant.configure("2") do |config|
# Define the Alpine boxes
config.vm.define "alpine1" do |alpine1|
alpine1.vm.box = "generic/alpine318"
alpine1.vm.hostname = "alpine1"
alpine1.vm.network :private_network, ip: "10.0.0.11"
alpine1.vm.provider "virtualbox" do |vb|
vb.memory = "1024"
end
end
config.vm.define "alpine2" do |alpine2|
alpine2.vm.box = "generic/alpine318"
alpine2.vm.hostname = "alpine2"
alpine2.vm.network :private_network, ip: "10.0.0.12"
alpine2.vm.provider "virtualbox" do |vb|
vb.memory = "1024"
end
end
config.vm.define "alpine3" do |alpine3|
alpine3.vm.box = "generic/alpine318"
alpine3.vm.hostname = "alpine3"
alpine3.vm.network :private_network, ip: "10.0.0.13"
alpine3.vm.provider "virtualbox" do |vb|
vb.memory = "1024"
end
end
endVMs go up without any issues, and they can ping each other. Here's the output on everything seems to go without any issues so far. Now for apparently something went wrong with the service, all here are some logs: and the last errors in the log: Any ideas of what could be going wrong? I've basically taken these commands from the docs and after trying several other variations I wasn't able to make the etcd embedded db setup work. |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 4 replies
-
|
There is no issue with multi master nodes or any other setups Your error is clear etcd cluster join failed: context deadline exceeded by the way , first try and Vagrant sometimes acts up specially networking part |
Beta Was this translation helpful? Give feedback.
-
|
At least for port 6443 and pings the two VMs can indeed reach each other |
Beta Was this translation helpful? Give feedback.
-
|
This is a vagrant networking issue. The internal network is always configured as a secondary interface, because vagrant deploys a dedicated local network for ssh key setup as the primary. You need to tell flannel to use that interface. See our examples in our E2E testing. https://github.com/k3s-io/k3s/blob/master/tests/e2e/validatecluster/Vagrantfile#L49-L50 Also, since you are using vagrant, I would be remiss if I didn't mentions our vagrant-k3s plugin. |
Beta Was this translation helpful? Give feedback.
-
|
I'm guessing there are things about kubernetes and k3s that I yet don't understand, but I did not expect at all for this software to require one exclusive network interface for communication, and for it to be picked automatically. You'd expect for a software designed to be resilient against failures to be able to use more than one network interface in case one drops. Maybe it's worth adding a note or warning about this behavior on the install steps? After doing some research it turns out that not only do you need to set As for vagrant-k3s, I only used vagrant to open this discussion and have something that could be replicated and shared, so I won't be needing it now that this is resolved, thanks for mentioning it though, it could come in handy in the future. |
Beta Was this translation helpful? Give feedback.
-
Kubernetes is a complex piece of software, built on top of many other complicated pieces of software. You can configure it to do many things, but absent any specific configuration it assumes reasonable defaults. One of those reasonable defaults is "I should use the interface that has a default route". It also only supports a single private node IP of each address family, so if you're expecting native multihomed network support, you're expecting too much. |
Beta Was this translation helpful? Give feedback.
-
|
It says "I should exclusively use the interface that has a default route" which is really the problem here. |
Beta Was this translation helpful? Give feedback.
This is a vagrant networking issue. The internal network is always configured as a secondary interface, because vagrant deploys a dedicated local network for ssh key setup as the primary. You need to tell flannel to use that interface.
See our examples in our E2E testing. https://github.com/k3s-io/k3s/blob/master/tests/e2e/validatecluster/Vagrantfile#L49-L50
Also, since you are using vagrant, I would be remiss if I didn't mentions our vagrant-k3s plugin.