Zookeeper - KV connection error: Failed to test KV store connection

[raclepoulpe]

Hi,
Traefik (in a docker) can't discuss with my zookeeper (in a docker too).
This is my traefik.toml :

traefikLogsFile = "/tmp/traefik.log"
accessLogsFile = "/tmp/access.log"
logLevel = "DEBUG"

[web]
address = ":8081"

[docker]
endpoint = "unix:///var/run/docker.sock"
domain = "docker.localhost"
watch = true

[zookeeper]
endpoint = "127.0.0.1:2181"
watch = true
prefix = "/traefik"

And the docker run command :
docker run -d -p 8081:8081 -p 80:80 -h traefik -v ~/ContainersLocalFiles/traefik/traefik.toml:/etc/traefik/traefik.toml -v /var/run/docker.sock:/var/run/docker.sock -v /tmp/traefik.log:/tmp/traefik.log --name traefik traefik

I use and run the jplock/zookeeper as this :
docker run -d -p 2181:2181 -v ~/ContainersLocalFiles/zookeeper:/tmp/zookeeper --name zookeeper jplock/zookeeper start-foreground

and I have created sample datas as describe here

And this the logs when I start traefik :

time="2016-06-13T09:55:35Z" level=info msg="Traefik version v1.0.0-rc2 built on 2016-06-08_12:35:41PM" 
time="2016-06-13T09:55:35Z" level=info msg="Using TOML configuration file /etc/traefik/traefik.toml" 
time="2016-06-13T09:55:35Z" level=debug msg="Global configuration loaded {\"GraceTimeOut\":10,\"Debug\":false,\"AccessLogsFile\":\"/tmp/access.log\",\"TraefikLogsFile\":\"/tmp/traefik.log\",\"LogLevel\":\"DEBUG\",\"EntryPoints\":{\"http\":{\"Network\":\"\",\"Address\":\":80\",\"TLS\":null,\"Redirect\":null}},\"Constraints\":[],\"ACME\":null,\"DefaultEntryPoints\":[\"http\"],\"ProvidersThrottleDuration\":2000000000,\"MaxIdleConnsPerHost\":200,\"Retry\":null,\"Docker\":{\"Watch\":true,\"Filename\":\"\",\"Constraints\":null,\"Endpoint\":\"unix:///var/run/docker.sock\",\"Domain\":\"docker.localhost\",\"TLS\":null},\"File\":null,\"Web\":{\"Address\":\":8081\",\"CertFile\":\"\",\"KeyFile\":\"\",\"ReadOnly\":false},\"Marathon\":null,\"Consul\":null,\"ConsulCatalog\":null,\"Etcd\":null,\"Zookeeper\":{\"Watch\":true,\"Filename\":\"\",\"Constraints\":[],\"Endpoint\":\"127.0.0.1:2181\",\"Prefix\":\"/traefik\",\"TLS\":null},\"Boltdb\":null,\"Kubernetes\":null}" 
time="2016-06-13T09:55:35Z" level=info msg="Preparing server http &{Network: Address::80 TLS:<nil> Redirect:<nil>}" 
time="2016-06-13T09:55:35Z" level=info msg="Starting provider _provider.Docker {\"Watch\":true,\"Filename\":\"\",\"Constraints\":null,\"Endpoint\":\"unix:///var/run/docker.sock\",\"Domain\":\"docker.localhost\",\"TLS\":null}" 
time="2016-06-13T09:55:35Z" level=info msg="Starting provider *main.WebProvider {\"Address\":\":8081\",\"CertFile\":\"\",\"KeyFile\":\"\",\"ReadOnly\":false}" 
time="2016-06-13T09:55:35Z" level=info msg="Starting provider *provider.Zookepper {\"Watch\":true,\"Filename\":\"\",\"Constraints\":[],\"Endpoint\":\"127.0.0.1:2181\",\"Prefix\":\"/traefik\",\"TLS\":null}" 
time="2016-06-13T09:55:35Z" level=info msg="Starting server on :80" 
*_time="2016-06-13T09:55:35Z" level=error msg="KV connection error: Failed to test KV store connection: zk: could not connect to a server, retrying in 682.242338ms"*\* 
time="2016-06-13T09:55:35Z" level=debug msg="Docker connection established with docker 1.11.1 (API 1.23)" 
time="2016-06-13T09:55:35Z" level=debug msg="Filtering container with more than 1 port and no traefik.port label /traefik" 
time="2016-06-13T09:55:35Z" level=debug msg="Filtering container with more than 1 port and no traefik.port label /zookeeper" 
time="2016-06-13T09:55:35Z" level=debug msg="Configuration received from provider docker: {\"backends\":{\"backend-mongo\":{\"servers\":{\"server-mongo\":{\"url\":\"http://172.17.0.2:27017\",\"weight\":1}}},\"backend-vertxjs\":{\"servers\":{\"server-vertxjs\":{\"url\":\"http://172.17.0.5:8082\",\"weight\":1}}}},\"frontends\":{\"frontend-Host-mongo-docker-localhost\":{\"backend\":\"backend-mongo\",\"routes\":{\"route-frontend-Host-mongo-docker-localhost\":{\"rule\":\"Host:mongo.docker.localhost\"}},\"passHostHeader\":true,\"priority\":0},\"frontend-Host-vertxjs-docker-localhost\":{\"backend\":\"backend-vertxjs\",\"routes\":{\"route-frontend-Host-vertxjs-docker-localhost\":{\"rule\":\"Host:vertxjs.docker.localhost\"}},\"passHostHeader\":true,\"priority\":0}}}" 
time="2016-06-13T09:55:35Z" level=debug msg="Last docker config received more than 2s, OK" 
time="2016-06-13T09:55:35Z" level=debug msg="Creating frontend frontend-Host-mongo-docker-localhost" 
time="2016-06-13T09:55:35Z" level=debug msg="Wiring frontend frontend-Host-mongo-docker-localhost to entryPoint http" 
time="2016-06-13T09:55:35Z" level=debug msg="Creating route route-frontend-Host-mongo-docker-localhost Host:mongo.docker.localhost" 
time="2016-06-13T09:55:35Z" level=debug msg="Creating backend backend-mongo" 
time="2016-06-13T09:55:35Z" level=debug msg="Creating load-balancer wrr" 
time="2016-06-13T09:55:35Z" level=debug msg="Creating server server-mongo at http://172.17.0.2:27017 with weight 1" 
time="2016-06-13T09:55:35Z" level=debug msg="Creating frontend frontend-Host-vertxjs-docker-localhost" 
time="2016-06-13T09:55:35Z" level=debug msg="Wiring frontend frontend-Host-vertxjs-docker-localhost to entryPoint http" 
time="2016-06-13T09:55:35Z" level=debug msg="Creating route route-frontend-Host-vertxjs-docker-localhost Host:vertxjs.docker.localhost" 
time="2016-06-13T09:55:35Z" level=debug msg="Creating backend backend-vertxjs" 
time="2016-06-13T09:55:35Z" level=debug msg="Creating load-balancer wrr" 
time="2016-06-13T09:55:35Z" level=debug msg="Creating server server-vertxjs at http://172.17.0.5:8082 with weight 1" 
time="2016-06-13T09:55:35Z" level=info msg="Server configuration reloaded on :80" 
**time="2016-06-13T09:55:36Z" level=error msg="KV connection error: Failed to test KV store connection: zk: could not connect to a server, retrying in 741.756464ms" 
time="2016-06-13T09:55:37Z" level=error msg="KV connection error: Failed to test KV store connection: zk: could not connect to a server, retrying in 708.825532ms" 
time="2016-06-13T09:55:37Z" level=error msg="KV connection error: Failed to test KV store connection: zk: could not connect to a server, retrying in 1.002564428s"**

Etc..

Note : I can connect to ZK and managing datas from any hosts.

Thx

[raclepoulpe]

Anybody ? :)

[emilevauge]

Hi, it seems traefik cannot connect to zookeeper. Are you sure this is not a network issue?

[raclepoulpe]

Hi,
the 2 dockers are running on localhost, so I'm pretty sure it's not a network issue :)

[emilevauge]

@raclepoulpe you configured traefik to talk to Zookeeper on 127.0.0.1:2181. As you launched traefik container in bridged mode, 127.0.0.1 will not ends on zookeeper container.
This issue comes from a bad networking configuration of your containers.
You can create a link between these two containers to fix that.

[raclepoulpe]

@emilevauge thx.
I plugged traefik on an another zk, and the connection is Ok.
This issue can be solved