net: "cannot unmarshal DNS message" when using netdns=go under Windows/WSL2 running ubuntu 20.04 #44135

ncw · 2021-02-06T12:01:07Z

What version of Go are you using (`go version`)?

$ go version
go version go1.15.7 linux/amd64

Does this issue reproduce with the latest release?

Haven't tried go1.16-rc1 or tip

What operating system and processor architecture are you using (`go env`)?

The problem has been reproduced with Microsoft Windows [Version 10.0.19042.746], WSL2 with Ubuntu 20.04

What did you do?

Run this program on WSL2 with Ubuntu 20.04. It is a very simple program, essentially just calling net.LookupHost

fmt.Printf("Looking up %q\n", host)
addrs, err := net.LookupHost(host)
if err != nil {
	fmt.Printf("- Failed with error: %v\n", err)
} else {
	for _, addr := range addrs {
		fmt.Printf("- Addr: %v\n", addr)
	}
}

What did you expect to see?

I expected it to resolve names correctly

What did you see instead?

$ ./host  prevasonline.sharepoint.com
Looking up "prevasonline.sharepoint.com"
- Failed with error: lookup prevasonline.sharepoint.com on 172.29.224.1:53: cannot unmarshal DNS message

$ GODEBUG=netdns=go ./host prevasonline.sharepoint.com
Looking up "prevasonline.sharepoint.com"
- Failed with error: lookup prevasonline.sharepoint.com on 172.29.224.1:53: cannot unmarshal DNS message

$ GODEBUG=netdns=cgo ./host prevasonline.sharepoint.com
Looking up "prevasonline.sharepoint.com"
- Addr: 13.107.136.9

This issue was originally from rclone/rclone#4984 and @hstaugaard and @black-snow have reported it. I (@ncw) haven't verified it personally since I don't have Windows with WSL2.

This issue is superficially similar to #37362 and #36718 but the adddress in question doesn't have any SRV records and we aren't doing an SRV query, just HostLookup

The text was updated successfully, but these errors were encountered:

seankhliao · 2021-02-06T22:39:54Z

can this be run with GODEBUG=netdns=go+2 and GODEBUG=netdns=cgo+2, also what if the DNS servers are changed (eg. to 8.8.8.8)?

hstaugaard · 2021-02-08T06:49:01Z

can this be run with GODEBUG=netdns=go+2 and GODEBUG=netdns=cgo+2, also what if the DNS servers are changed (eg. to 8.8.8.8)?

$ GODEBUG=netdns=go+2 ./host prevasonline.sharepoint.com
Looking up "prevasonline.sharepoint.com"
go package net: GODEBUG setting forcing use of Go's resolver
go package net: hostLookupOrder(prevasonline.sharepoint.com) = files,dns
- Failed with error: lookup prevasonline.sharepoint.com on 172.29.128.1:53: cannot unmarshal DNS message

$ GODEBUG=netdns=cgo+2 ./host prevasonline.sharepoint.com
Looking up "prevasonline.sharepoint.com"
go package net: using cgo DNS resolver
go package net: hostLookupOrder(prevasonline.sharepoint.com) = cgo
- Addr: 13.107.136.9

Can't test Googles DNS, no public DNS on work network.

jryan128 · 2021-02-10T17:41:31Z

Same issues with a different go program. When I run with +2. I am using a VPN and WSL2. The server is on the VPN.

go package net: built with netgo build tag; using Go's DNS resolver
go package net: hostLookupOrder(****) = files,dns

gopherbot · 2021-03-06T22:45:22Z

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

moxli · 2021-03-24T16:04:52Z

I tested it with 1.16.2 and got the same results.

mdorner@lt01858 ~/code/debug/rclone_wsl2_dns_go_debug> go version
go version go1.16.2 linux/amd64

mdorner@lt01858 ~/code/debug/rclone_wsl2_dns_go_debug> go run ./host.go prevasonline.sharepoint.com
Looking up "prevasonline.sharepoint.com"
Failed with error: lookup prevasonline.sharepoint.com on 172.30.192.1:53: cannot unmarshal DNS message

mdorner@lt01858 ~/code/debug/rclone_wsl2_dns_go_debug> GODEBUG=netdns=go go run ./host.go prevasonline.sharepoint.com
Looking up "prevasonline.sharepoint.com"
Failed with error: lookup prevasonline.sharepoint.com on 172.30.192.1:53: cannot unmarshal DNS message

mdorner@lt01858 ~/code/debug/rclone_wsl2_dns_go_debug> GODEBUG=netdns=cgo+2 go run ./host.go prevasonline.sharepoint.com
Looking up "prevasonline.sharepoint.com"
go package net: using cgo DNS resolver
go package net: hostLookupOrder(prevasonline.sharepoint.com) = cgo
- Addr: 40.108.168.35`

ncw · 2021-03-24T17:33:27Z

@seankhliao @gopherbot please re-open this - the info was provided above and the issue isn't fixed - thanks!

eric-engberg · 2022-02-09T18:22:13Z

I just started experiencing this issue myself with terraform 0.13.5... i haven't updated terraform so no idea why it suddenly started.

ianlancetaylor · 2022-02-10T20:36:35Z

I guess this didn't get reopened, but #51127 has more information about this problem.

gopherbot · 2022-02-11T04:18:29Z

Change https://go.dev/cl/385035 mentions this issue: net: send EDNS(0) packet length in DNS query

We used to only accept up to 512 bytes in a DNS packet, per RFC 1035. Increase the size we accept to 1232 bytes, per https://dnsflagday.net/2020/, and advertise that larger limit in a EDNS(0) OPT record. Fixes #6464 Fixes #21160 Fixes #44135 Fixes #51127 Change-Id: I496a294e9a8015de4161cbc1825b0dc5b4e9f5d8 Reviewed-on: https://go-review.googlesource.com/c/go/+/385035 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>

gopherbot · 2022-02-12T05:50:17Z

Change https://go.dev/cl/385374 mentions this issue: [release-branch.go1.17] net: send EDNS(0) packet length in DNS query

gopherbot · 2022-02-12T05:56:05Z

Change https://go.dev/cl/385375 mentions this issue: [release-branch.go1.16] net: send EDNS(0) packet length in DNS query

gopherbot · 2022-02-15T21:54:15Z

Change https://go.dev/cl/386015 mentions this issue: net: increase maximum accepted DNS packet to 1232 bytes

gopherbot · 2022-02-15T21:54:18Z

Change https://go.dev/cl/386016 mentions this issue: net: send EDNS(0) packet length in DNS query

gopherbot · 2022-02-15T21:54:21Z

Change https://go.dev/cl/386014 mentions this issue: Revert "net: send EDNS(0) packet length in DNS query"

This reverts https://go.dev/cl/385035. For 1.18 we will use a simple change to increase the accepted DNS packet size, to handle what appear to be broken resolvers that don't honor the 512 byte limit. For 1.19 we will restore CL 385035 to make a proper EDNS request, so that it has more testing time before it goes out in a release. For #6464 For #21160 For #44135 For #51127 For #51153 Change-Id: Ie4a0eb85ca0a6a73bee5cd4cfc6b7d2a15ef259f Reviewed-on: https://go-review.googlesource.com/c/go/+/386014 Trust: Ian Lance Taylor <iant@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Damien Neil <dneil@google.com>

gopherbot · 2022-02-15T22:31:50Z

Change https://go.dev/cl/386034 mentions this issue: [release-branch.go1.16] net: increase maximum accepted DNS packet to 1232 bytes

gopherbot · 2022-02-15T22:33:18Z

Change https://go.dev/cl/386035 mentions this issue: [release-branch.go1.17] net: increase maximum accepted DNS packet to 1232 bytes

The existing value of 512 bytes as is specified by RFC 1035. However, the WSL resolver reportedly sends larger packets without setting the truncation bit, which breaks using the Go resolver. For 1.18 and backports, just increase the accepted packet size. This is what GNU glibc does (they use 65536 bytes). For 1.19 we plan to use EDNS to set the accepted packet size. That will give us more time to test whether that causes any problems. No test because I'm not sure how to write one and it wouldn't really be useful anyhow. Fixes #6464 Fixes #21160 Fixes #44135 Fixes #51127 For #51153 Change-Id: I0243f274a06e010ebb714e138a65386086aecf17 Reviewed-on: https://go-review.googlesource.com/c/go/+/386015 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>

…1232 bytes The existing value of 512 bytes as is specified by RFC 1035. However, the WSL resolver reportedly sends larger packets without setting the truncation bit, which breaks using the Go resolver. For 1.18 and backports, just increase the accepted packet size. This is what GNU glibc does (they use 65536 bytes). For 1.19 we plan to use EDNS to set the accepted packet size. That will give us more time to test whether that causes any problems. No test because I'm not sure how to write one and it wouldn't really be useful anyhow. For #6464 For #21160 For #44135 For #51127 For #51153 Fixes #51162 Change-Id: I0243f274a06e010ebb714e138a65386086aecf17 Reviewed-on: https://go-review.googlesource.com/c/go/+/386015 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> (cherry picked from commit 6e82ff8) Reviewed-on: https://go-review.googlesource.com/c/go/+/386035 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

…1232 bytes The existing value of 512 bytes as is specified by RFC 1035. However, the WSL resolver reportedly sends larger packets without setting the truncation bit, which breaks using the Go resolver. For 1.18 and backports, just increase the accepted packet size. This is what GNU glibc does (they use 65536 bytes). For 1.19 we plan to use EDNS to set the accepted packet size. That will give us more time to test whether that causes any problems. No test because I'm not sure how to write one and it wouldn't really be useful anyhow. For #6464 For #21160 For #44135 For #51127 For #51153 Fixes #51161 Change-Id: I0243f274a06e010ebb714e138a65386086aecf17 Reviewed-on: https://go-review.googlesource.com/c/go/+/386015 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> (cherry picked from commit 6e82ff8) Reviewed-on: https://go-review.googlesource.com/c/go/+/386034 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

Advertise to DNS resolvers that we are willing and able to accept up to 1232 bytes in a DNS packet. The value 1232 was chosen based on https://dnsflagday.net/2020/. For #6464 For #21160 For #44135 For #51127 Fixes #51153 Change-Id: If9182d5210bfe047cf0a4d46163effc6812ab677 Reviewed-on: https://go-review.googlesource.com/c/go/+/386016 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>

ncw mentioned this issue Feb 6, 2021

Windows WSL: "cannot unmarshal DNS message" - sync fails rclone/rclone#4984

Closed

seankhliao added OS-Windows WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Feb 6, 2021

dmitshur added this to the Backlog milestone Feb 6, 2021

gopherbot closed this as completed Mar 6, 2021

benc-uk mentioned this issue Apr 5, 2021

Installing porter fails using https://cdn.porter.sh/latest/install-linux.sh getporter/porter#1521

Open

ivandeex mentioned this issue Oct 1, 2021

Provide official CGO/libfuse build of rclone for Linux rclone/rclone#5090

Open

untoldone mentioned this issue Oct 28, 2021

Failed to resolve domain/ silently failed to track within WSL2 microsoft/ApplicationInsights-Go#56

Open

jboelter mentioned this issue Nov 5, 2021

WSL Internet Connection Sharing DNS resolver does not adhere to 512 byte UDP limit microsoft/WSL#7642

Closed

2 tasks

AaronFriel mentioned this issue Feb 10, 2022

net: 512 byte DNS response size limit causes "cannot unmarshal DNS" error #51127

Closed

This comment was marked as off-topic.

Sign in to view

seumasdunlop mentioned this issue Feb 17, 2022

Trying to connect to Azure through terraform hashicorp/terraform-provider-azurerm#15487

Closed

apparentlymart mentioned this issue Oct 5, 2022

Terraform 1.3.1 (and 1.3.0) forcing and failing DNS resolution on IPv6 hashicorp/terraform#31935

Closed

AaronFriel mentioned this issue Oct 21, 2022

net: Support the /etc/resolver DNS resolution configuration hierarchy on OS X when cgo is disabled #12524

Closed

golang locked and limited conversation to collaborators Feb 15, 2023

gopherbot added the FrozenDueToAge label Feb 15, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

net: "cannot unmarshal DNS message" when using netdns=go under Windows/WSL2 running ubuntu 20.04 #44135

net: "cannot unmarshal DNS message" when using netdns=go under Windows/WSL2 running ubuntu 20.04 #44135

ncw commented Feb 6, 2021

seankhliao commented Feb 6, 2021

hstaugaard commented Feb 8, 2021

jryan128 commented Feb 10, 2021 •

edited

Loading

gopherbot commented Mar 6, 2021

moxli commented Mar 24, 2021

ncw commented Mar 24, 2021

eric-engberg commented Feb 9, 2022

ianlancetaylor commented Feb 10, 2022

gopherbot commented Feb 11, 2022

This comment was marked as off-topic.

This comment was marked as off-topic.

gopherbot commented Feb 12, 2022

gopherbot commented Feb 12, 2022

gopherbot commented Feb 15, 2022

gopherbot commented Feb 15, 2022

gopherbot commented Feb 15, 2022

gopherbot commented Feb 15, 2022

gopherbot commented Feb 15, 2022

net: "cannot unmarshal DNS message" when using netdns=go under Windows/WSL2 running ubuntu 20.04 #44135

net: "cannot unmarshal DNS message" when using netdns=go under Windows/WSL2 running ubuntu 20.04 #44135

Comments

ncw commented Feb 6, 2021

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

seankhliao commented Feb 6, 2021

hstaugaard commented Feb 8, 2021

jryan128 commented Feb 10, 2021 • edited Loading

gopherbot commented Mar 6, 2021

moxli commented Mar 24, 2021

ncw commented Mar 24, 2021

eric-engberg commented Feb 9, 2022

ianlancetaylor commented Feb 10, 2022

gopherbot commented Feb 11, 2022

This comment was marked as off-topic.

This comment was marked as off-topic.

gopherbot commented Feb 12, 2022

gopherbot commented Feb 12, 2022

gopherbot commented Feb 15, 2022

gopherbot commented Feb 15, 2022

gopherbot commented Feb 15, 2022

gopherbot commented Feb 15, 2022

gopherbot commented Feb 15, 2022

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

jryan128 commented Feb 10, 2021 •

edited

Loading