Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow DNS resolution #5944

Open
Symbianx opened this issue Nov 8, 2023 · 8 comments
Open

Slow DNS resolution #5944

Symbianx opened this issue Nov 8, 2023 · 8 comments
Assignees
Labels
area/dns kind/bug Something isn't working platform/macos triage/next-candidate Discuss if it should be moved to "Next" milestone

Comments

@Symbianx
Copy link

Symbianx commented Nov 8, 2023

Actual Behavior

DNS queries take 1 second to resolve inside the Rancher Desktop VM.

This shows in very slow docker pulls.

Enabling/Disabling VZ/Rosetta does not change the behaviour.

Steps to Reproduce

Run a DNS query. E.g.:
rdctl shell time nslookup google.com

Result

Initial answer is instant, but for some reason it waits for another answer and stops after 1 second.

Server:		192.168.5.3
Address:	192.168.5.3:53

Non-authoritative answer:
Name:	google.com
Address: 142.251.39.110

Non-authoritative answer:

real	0m 1.00s
user	0m 0.00s
sys	0m 0.00s

Expected Behavior

A single, instant answer should be returned by nslookup.

Additional Information

Running the same nslookup command on the host machine results in an instant result.

I don't know how useful this info is but running the nslookup on a VM created by colima does result in an instant result.

Rancher Desktop Version

1.11.0

Rancher Desktop K8s Version

Not enabled

Which container engine are you using?

moby (docker cli)

What operating system are you using?

macOS

Operating System / Build Version

macOS 14.1

What CPU architecture are you using?

arm64 (Apple Silicon)

Linux only: what package format did you use to install Rancher Desktop?

None

Windows User Only

No response

@Symbianx Symbianx added the kind/bug Something isn't working label Nov 8, 2023
@taras-prosvirov
Copy link

Faced exactly the same issue... plus 1 second for DNS resolution from inside container.

host-machine# time curl http://example.com
... ~300 ms (ok from host machine)

host-machine# docker run -it --rm amazonlinux:2.0.20231020.1 bash
bash-4.2# time curl -H "Host: example.com" http://93.184.216.34
... ~300 ms (ok from container using IP address)
bash-4.2# time curl http://example.com
... ~1300 ms (slow from container using domain name)

@jandubois jandubois added this to the 1.13 milestone Nov 18, 2023
@jandubois
Copy link
Member

Tentatively adding this to the 1.13 milestone, with the expectation that this will be improved by switching to the gVisor based user-v2 networking stack. Might close as a duplicate of #4258.

@Symbianx
Copy link
Author

I don't know what changed but the DNS responses are instant now.
Still getting 2 "Non-authoritative answer"s but way faster now.

Still running rancher desktop 1.11.0 but I did uninstall and reinstall a couple of times. Maybe some upstream dependency changed and fixed the issue?

@andrey-tarasov-wiley
Copy link

andrey-tarasov-wiley commented Jan 27, 2024

I had the same issue, in my case it's gone after enabling networking tunnel:
image
Seems, that issue is in "host-resolver" binary which adds delay.
When "Enable networking tunnel" is disabled then bundled with rancher host-resolver is used in WSL distro and responses are slow.
But when it's enabled - host resolver isn't used and responses are fast.
Location of host resolver on Windows is "C:\Program Files\Rancher Desktop\resources\resources\linux\internal\host-resolver"

UPDATE: Enabling networking tunnel makes rt mapping work very unstable (connection reset from mapped port on host machine) with Rancher Dresktop Version: 1.12.2 on Windows. So, it isn't solution now.

UPDATE: Found, that slow network response is from ipv6, for ipv4 response is quick:
image

@jandubois jandubois removed this from the 1.13 milestone Feb 13, 2024
@alex-ioma
Copy link

I confirm this is still an issue on Rancher Desktop 1.13.1. This should be address as it makes the overall user experience very bad.

M1 Max with RD vs. M1 8GB with Podman. Podman was already up and running by the time I finished pulling the first layer of mongo.

@alex-ioma
Copy link

Hi there @jandubois. It seems this topic might be moving towards a stale status, although the issue is very much alive.

Currently needing minutes every time I pull an image on Rancher 1.14.1 on macOS 14.5.

Is there any resolution plan in place?

Thanks!

@jandubois
Copy link
Member

Is there any resolution plan in place?

There is no "resolution plan" as the root cause is still unknown.

We did not have time to prioritize this yet, as we had a lot of work to do on Windows to make the new networking stack the default (and it will be the only option starting in 1.15.0). Since this is about to wrap up, we'll hopefully get around to looking into macOS networking again, but I still can't promise any timeline; there are a lot of competing demands on our time...

@jandubois jandubois added the triage/next-candidate Discuss if it should be moved to "Next" milestone label Jun 17, 2024
@alex-ioma
Copy link

Hi there @jandubois. I've seen you labelled this as triage. Were you able to find the root cause of this issue. Still having the same behavior after updating to v1.16 - although not critical, it is very annoying in everyday usage.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/dns kind/bug Something isn't working platform/macos triage/next-candidate Discuss if it should be moved to "Next" milestone
Projects
None yet
Development

No branches or pull requests

6 participants