Inconsistencies when host has multiple IP addresses

[n11c]

Our docker hosts have 2 IP adresses that Dockers might get ports forwarded from : internal (LAN) and public.
So far we were only using the LAN IPs and since our Consul agents are bound to those IPs as well, registrator did its job perfectly.
However, if we map Dockers ports from the public IPs, the services are registered in Consul with the IP of the agent, which is the LAN one.
This is because Registrator uses consulapi.AgentServiceRegistration, which registers the service as one of an existing Agent(node).
With Consul K/V or ETCD (I guess, not tested) the correct IP is registered.

Obviously, Consul DNS resolution for those services is incorrect.
I see 3 ways to address this:

• Completely skip service registration for the container based on an environment variable (SERVICE_IGNORE). This doesn't really solve the problem, but it might be useful for other purposes and is quite easy to implement. I'll be submitting a PR shortly.
• Don't register services for port mappings with a service.IP that's different from the host's IP. Again, it doesn't completely solve the issue, but at least we would be able to keep registering other services that bind to the LAN IP for the same container.
• Register services mapped to an IP that's different from the host's IP as External Services (http://www.consul.io/docs/guides/external.html)

I'll try and submit a PR for the second option as well.
Just need to dive into Golang a bit more.

The third option would be the best one, but it implies the creation of a new node.
I'm not sure how it could be named, so that it could be reused for other services bound to that same IP.
Perhaps it could simply be named after the IP (eg. 8-8-8-8).

I'm open to suggestions.

[progrium]

The real solution is that Consul Services should allow custom IPs. That's what I'd rather push for.

Until then, SERVICE_IGNORE sounds reasonable. I already had that in mind for other obvious reasons.

I don't actually like the second option because it's confusing and sometimes you're ok a different IP.

That last option is interesting, I've thought about it, but you lose certain capabilities there that would be the point of using Consul Services in the first point. Again, I'd rather we push them to support adding services with custom IPs ... that would actually be an easy PR and they've already told me they'd take it.

[n11c]

You're right.
I guess patching Consul looked too daunting given my coding skills :)
I'll have a look at their code to get my head around it anyway.
Thanks for the quick reply.

[n11c]

Worth pointing out that the SERVICE_IGNORE introduced in PR #14 can also be used to skip registration of a single port using SERVICE_XX_IGNORE, since each port mapping is treated like a separate service.

[progrium]

Tracking this issue in Consul:
hashicorp/consul#229

[sfitts]

Any visibility into when this might be addressed? I agree that the Consul based solution would be preferable, but I can see how to address it here as well, so maybe providing the option for registering external services in the meantime? Seems related to #59

[andyshinn]

Support for this is in Consul 0.5.0 (hashicorp/consul#570) and registrator master branch (#91).

[endzyme]

+1 for registering external services (using the IP of the container itself)