How to register external consul agent with consul server running on K8

[ervikrant06]

Deployed consul using helm chart on minikube. By default consul-server is headless service.

 kubectl get svc
NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                                                   AGE
consul-dns      ClusterIP   10.103.165.156   <none>        53/TCP,53/UDP                                                             4h
consul-server   ClusterIP   None             <none>        8500/TCP,8301/TCP,8301/UDP,8302/TCP,8302/UDP,8300/TCP,8600/TCP,8600/UDP   4h
consul-ui       NodePort    10.103.15.241    <none>        80:30613/TCP                                                              4h
kubernetes      ClusterIP   10.96.0.1        <none>        443/TCP                                                                   5h
my-nginx        NodePort    10.102.190.41    <none>        80:32513/TCP                                                              1h

If I am running a consul agent outside the K8 cluster how can I register it with consul server running on K8.

Tried the options like port-forward but it's listening on the loopback address only which will not work in my scenario.

I am running minikube (192.168.99.100) on my mac. I started docker-machine (192.168.99.101) on which I wanted to start the consul-agent and connect it with consul server running on K8.

$ kubectl port-forward consul-server-0 8500:8500
Forwarding from 127.0.0.1:8500 -> 8500

Running consul-agent on docker-machine.

docker@consul1:~$ docker run -d --rm --net=host consul agent --retry-join=192.168.99.100:8500 -bind=192.168.99.101
f82b6b20e004b214ca1db5d497d3b056c35d4571c36096c88061fca6b90feda0

docker@consul1:~$ docker logs f82b6b20e004
==> Starting Consul agent...
==> Consul agent running!
           Version: 'v1.2.3'
           Node ID: '36bda601-4cb8-5125-d765-5419ff6a1144'
         Node name: 'consul1'
        Datacenter: 'dc1' (Segment: '')
            Server: false (Bootstrap: false)
       Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, DNS: 8600)
      Cluster Addr: 192.168.99.101 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false

==> Log data will now stream in as it occurs:

    2018/10/05 18:50:13 [INFO] serf: EventMemberJoin: consul1 192.168.99.101
    2018/10/05 18:50:13 [INFO] agent: Started DNS server 127.0.0.1:8600 (udp)
    2018/10/05 18:50:13 [INFO] agent: Started DNS server 127.0.0.1:8600 (tcp)
    2018/10/05 18:50:13 [INFO] agent: Started HTTP server on 127.0.0.1:8500 (tcp)
    2018/10/05 18:50:13 [INFO] agent: started state syncer
    2018/10/05 18:50:13 [INFO] agent: Retry join LAN is supported for: aliyun aws azure digitalocean gce k8s os packet scaleway softlayer triton vsphere
    2018/10/05 18:50:13 [INFO] agent: Joining LAN cluster...
    2018/10/05 18:50:13 [INFO] agent: (LAN) joining: [192.168.99.100:8500]
    2018/10/05 18:50:13 [WARN] manager: No servers available
    2018/10/05 18:50:13 [ERR] agent: failed to sync remote state: No known Consul servers
    2018/10/05 18:50:23 [INFO] agent: (LAN) joined: 0 Err: 1 error(s) occurred:

* Failed to join 192.168.99.100: read tcp 192.168.99.101:52520->192.168.99.100:8500: i/o timeout
    2018/10/05 18:50:23 [WARN] agent: Join LAN failed: <nil>, retrying in 30s

How can I make this work?

[mitchellh]

These issues shouldn't be used for general Q&A, please ask this question on the Consul mailing list. We try to keep issues for actual feature requests or bug reports for the particular project.

I'll post one response here but if you want follow-ups please pick it back up on the ML. The recommended approach for an agent to join a server running in K8S is documented here: https://www.consul.io/docs/platform/k8s/out-of-cluster-nodes.html As documented there, a certain amount of network connectivity between the agents is necessary. Some other options are given there if that isn't possible, but its pretty limiting.