Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

force cgo resolver for name resolution #753

Closed
wants to merge 1 commit into from
Closed

force cgo resolver for name resolution #753

wants to merge 1 commit into from

Conversation

andrewhsu
Copy link
Contributor

By default, the pure Go resolver is used which makes direct DNS
requests first to resolve a hostname before checking /etc/hosts.
If a host on the network has the same name as the linked container,
the host on the network will be used instead of the linked container.

I had a machine on the network with hostname mysql and bringing up the containers with docker-compose had the server and signer linked containers trying to connect to to it...

$ sudo docker-compose up
...
mysql_1  | 2016-05-19 19:33:29 140622360905664 [Note] mysqld: ready for connections.
mysql_1  | Version: '10.1.10-MariaDB-1~jessie'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
server_1 | waiting for notarymysql to come up.
signer_1 | waiting for notarymysql to come up.
server_1 | waiting for notarymysql to come up.
signer_1 | waiting for notarymysql to come up.
server_1 | waiting for notarymysql to come up.
...

Forcing cgo resolver will use C library routines that will honor
values in /etc/hosts first and then fall back on DNS.

From the docs: https://golang.org/pkg/net/#hdr-Name_Resolution

Signed-off-by: Andrew Hsu andrewhsu@acm.org (github: andrewhsu)

@docker-jenkins
Copy link

Can one of the admins verify this patch?

By default, the pure Go resolver is used which makes direct DNS
requests first to resolve a hostname before checking /etc/hosts.
If a host on the network has the same name as the linked container,
the host on the network will be used instead of the linked container.

Forcing cgo resolver will use C library routines that will honor
values in /etc/hosts first and then fall back on DNS.

From the docs: https://golang.org/pkg/net/#hdr-Name_Resolution

Signed-off-by: Andrew Hsu <andrewhsu@acm.org>
@andrewhsu
Copy link
Contributor Author

Sorry about the multiple force pushes...trying to get signature requirement squared away. I think it's good to go now.

@cyli
Copy link
Contributor

cyli commented May 19, 2016

jenkins, test this please

@andrewhsu
Copy link
Contributor Author

@cyli umm...i think jenkins is a bot. that's all he can say: https://github.com/docker-jenkins?tab=activity

@cyli
Copy link
Contributor

cyli commented May 20, 2016

@andrewhsu Yes it is :). We gate PRs on code coverage results (which wait for results from Jenkins and CircleCI) as well as CI results - this Jenkins server runs our yubikey tests, but doesn't automatically run them for PRs from authors not in the org. The comment I left basically lets it know that it's ok to run tests for this PR, else the codecov check would never finish.

We were wondering if converting the compose file to v2 and using a network definition would be sufficient? That would not put anything in /etc/hosts, and the DNS resolution would hopefully work correctly.

@andrewhsu
Copy link
Contributor Author

@cyli I tried as you suggested, to convert the docker compose files to v2 but it still had the same issue. If I add the environment variable GODEBUG=netdns=cgo then everything works. So, I created #755 as an alternative to this PR with the docker compose v2 format change.

@endophage
Copy link
Contributor

@andrewhsu I may be misunderstanding the docs but they don't appear to indicate the behaviour you describe. We think what you might be encountering is a compose issue (previously described here: distribution/distribution#1362 ) and while it can be solved with the cgo change, we really don't want to go that route of hacking every other project to work around an issue elsewhere. If you think it's the same issue could you file a ticket on the docker-compose repository?

@andrewhsu
Copy link
Contributor Author

@endophage I understand. I've adjusted #755 to remove the netdns workaround. I'll close out this PR.

@cyli
Copy link
Contributor

cyli commented Jun 3, 2016

@andrewhsu Just checking as a possible reason for the issue - did you happen to have more than 2 DNS servers on your host when the other machine took precedence over the linked mysql container?

@andrewhsu
Copy link
Contributor Author

andrewhsu commented Jul 7, 2016

@cyli no i did not have 2 dns servers on the host at the time. Outside of docker containers on the host I had entries something like this in /etc/resolv.conf:

search internal.example.com
nameserver 10.0.0.1
nameserver 10.0.0.2

And running our own DNS servers resolves hostnames like this:

bash$ host mysql
mysql.internal.example.com has address 10.10.10.5

In any case, I tried to replicate the issue again after moving my entire development environment from Kitematic to the new Docker for Mac OS X 1.12.0-rc2-beta16 and I was not able to get it to fail like before so I'm not going to spend any more time on it. Stuff works.

@cyli
Copy link
Contributor

cyli commented Jul 7, 2016

Ah ok, thanks for clarifying and trying to replicate!

On Thursday, July 7, 2016, Andrew Hsu notifications@github.com wrote:

@cyli https://github.com/cyli no i did not have 2 dns servers on the
host at the time. Outside of docker containers on the host I had entries
something like this in /etc/resolv.conf:

search internal.example.com
nameserver 10.0.0.1
nameserver 10.0.0.2

And running our own DNS servers resolves hostnames like this:

bash$ host mysqlmysql.internal.example.com has address 10.10.10.5

In any case, I tried to replicate the issue again after moving my entire development environment from Kitematic to the new Docker for Mac OS X 1.12.0-rc2-beta16 and I was not able to get it to fail like before so I'm not going to spend any more time on it. Stuff works.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#753 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AANn4PBau7RBlOc9LdGAHhpg1_Rh7Ms_ks5qTRjygaJpZM4IimJO
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants