net: DNS Resolution functions return truncated results when UDP responses are fragmented

[LeeBrotherston]

Go version

go version go1.23.1 darwin/arm64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/lee/Library/Caches/go-build'
GOENV='/Users/lee/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/lee/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/lee/go'
GOPRIVATE=''
GOPROXY='direct'
GOROOT='/opt/homebrew/Cellar/go/1.23.1/libexec'
GOSUMDB='off'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/opt/homebrew/Cellar/go/1.23.1/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.23.1'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/Users/lee/Library/Application Support/go/telemetry'
GCCGO='gccgo'
GOARM64='v8.0'
AR='ar'
CC='cc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='/Users/lee/tmp/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/j3/qv7s4b3j6jdd5h58n8jncnz80000gp/T/go-build1232499662=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

Select a DNS query which will return a response larger than the current MTU on your default interface. To illustrate I am using nat.travisci.net is it returns 79 A records, with a packet size of just over 1300.

If you run the code snippet below, whilst connected via a standard ethernet connection (probably 1500 MTU), you should get 79 responses, which can be verified by comparing the output of the same query using nslookup, host, dig, etc.

If you now use a connection with a lower MTU, for example using a VPN lowers my MTU to 1280, and run the same tests you will see that the code snippet below only returns 29 results, while nslookup, host, dig, etc still return the full set of results.

If we use some sort of packetsniffer (e.g. wireshark, tcpdump, etc) to analyze the data on the wire, we can see that the response is fragmented due to being larger than the MTU, as we would expect. As far as I can tell the other tools successfully reconstruct the packet from the fragments, however go seems to truncate the response, presumably due to the fragmentation.

This seems to happen on Linux as well as macOS, with and without CGO.

Snippet for testing:

package main                                                
                                                            
import (                                                    
        "fmt"                                               
        "net"                                               
)                                                           
                                                            
func main() {                                               
        ipList, err := net.LookupIP("nat.travisci.net")     
        if err != nil {                                     
                fmt.Printf("there was a problem: %s\n", err)
                return                                      
        }                                                   
        fmt.Printf("len: %d\n", len(ipList))                
        for x, y := range ipList {                          
                fmt.Printf("%d => %v\n", x, y)              
        }                                                   
        return                                              
}                                                           

What did you see happen?

I saw the go lookup return truncated results whilst on a lower MTU connection, while nslookup, etc continued to return the full results.

What did you expect to see?

I expected to see the same results between the sample code and nslookup, etc no matter which connection I was using.

[gabyhelp]

Related Issues and Documentation

• net: dns lookup should retry on TCP for truncated response #5686 (closed)
• net: netgo dns resolver does not return addresses on TCP truncated responses #64896 (closed)
• net: BSD: DNS go resolver loses first UDP outbound packet if onlink target not in neighbor cache #43398
• net: issue with DNS response > 512 bytes (cannot unmarshal DNS message) #21160 (closed)
• net: EDNS(0) records may cause dns dropped by some gateway #65197 (closed)
• net: DNS Lookups Very Slow (netgo) #49253 (closed)
• affected/package: net #57506 (closed)
• net: netgo resolver cannot lookup 254 characters FQDN #54285 (closed)
• net: LookupCNAME returns empty string if entry from /etc/hosts is used #44741 (closed)
• net: go DNS resolver fails to connect to local DNS server #67925 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[mateusz834]

In the go resolver we advertise support up to 1232 Bytes for UDP packets as per DNS flag day 2020 through EDNS(0).

go/src/net/dnsclient_unix.go

Lines 37 to 39 in ca1123f

	// Maximum DNS packet size.
	// Value taken from https://dnsflagday.net/2020/.
	maxDNSPacketSize = 1232

Quote from http://www.dnsflagday.net/2020/:

An EDNS buffer size of 1232 bytes will avoid fragmentation on nearly all current networks. This is based on an MTU of 1280, which is required by the IPv6 specification, minus 48 bytes for the IPv6 and UDP headers and the aforementioned research.

@LeeBrotherston can you post the entire output of:

dig nat.travisci.net +qr 

For both of the links you were testing, so the 1280 MTU (VPN) and the normal one (1500)?

On my side i get a TC reply in the UDP DNS query while using dig and the query is retired with TCP.

; <<>> DiG 9.20.1 <<>> nat.travisci.net +qr
;; global options: +cmd
;; Sending:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8105
;; flags: rd ad; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 81ce0f6cc2b7b06f
;; QUESTION SECTION:
;nat.travisci.net.		IN	A

;; QUERY SIZE: 57

;; Truncated, retrying in TCP mode.
;; Sending:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30406
;; flags: rd ad; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 81ce0f6cc2b7b06f
;; QUESTION SECTION:
;nat.travisci.net.		IN	A

;; QUERY SIZE: 57

;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30406
;; flags: qr rd ra; QUERY: 1, ANSWER: 79, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;nat.travisci.net.		IN	A

;; ANSWER SECTION:
nat.travisci.net.	297	IN	A	35.193.14.140
nat.travisci.net.	297	IN	A	34.74.91.53
nat.travisci.net.	297	IN	A	35.192.10.37
nat.travisci.net.	297	IN	A	34.74.16.120
nat.travisci.net.	297	IN	A	104.154.113.151
nat.travisci.net.	297	IN	A	35.184.226.236
nat.travisci.net.	297	IN	A	104.196.134.93
nat.travisci.net.	297	IN	A	34.73.66.97
nat.travisci.net.	297	IN	A	34.66.50.208
nat.travisci.net.	297	IN	A	34.73.41.95
nat.travisci.net.	297	IN	A	35.188.15.155
nat.travisci.net.	297	IN	A	34.74.15.71
nat.travisci.net.	297	IN	A	35.192.91.101
nat.travisci.net.	297	IN	A	34.68.144.114
nat.travisci.net.	297	IN	A	34.66.178.120
nat.travisci.net.	297	IN	A	35.231.17.119
nat.travisci.net.	297	IN	A	35.185.100.193
nat.travisci.net.	297	IN	A	104.196.57.92
nat.travisci.net.	297	IN	A	207.254.16.37
nat.travisci.net.	297	IN	A	35.231.60.105
nat.travisci.net.	297	IN	A	35.192.85.2
nat.travisci.net.	297	IN	A	104.197.122.201
nat.travisci.net.	297	IN	A	104.154.182.187
nat.travisci.net.	297	IN	A	34.66.200.49
nat.travisci.net.	297	IN	A	34.122.173.211
nat.travisci.net.	297	IN	A	104.154.120.187
nat.travisci.net.	297	IN	A	35.231.58.0
nat.travisci.net.	297	IN	A	34.74.79.111
nat.travisci.net.	297	IN	A	35.202.145.110
nat.travisci.net.	297	IN	A	104.196.213.122
nat.travisci.net.	297	IN	A	35.185.97.135
nat.travisci.net.	297	IN	A	34.75.51.254
nat.travisci.net.	297	IN	A	35.243.226.204
nat.travisci.net.	297	IN	A	34.148.176.122
nat.travisci.net.	297	IN	A	207.254.16.36
nat.travisci.net.	297	IN	A	34.74.231.179
nat.travisci.net.	297	IN	A	207.254.16.35
nat.travisci.net.	297	IN	A	34.74.31.114
nat.travisci.net.	297	IN	A	35.237.48.193
nat.travisci.net.	297	IN	A	35.237.8.208
nat.travisci.net.	297	IN	A	207.254.16.39
nat.travisci.net.	297	IN	A	35.229.88.230
nat.travisci.net.	297	IN	A	35.227.58.83
nat.travisci.net.	297	IN	A	35.222.7.205
nat.travisci.net.	297	IN	A	35.188.1.99
nat.travisci.net.	297	IN	A	34.73.65.1
nat.travisci.net.	297	IN	A	35.196.98.161
nat.travisci.net.	297	IN	A	35.193.7.13
nat.travisci.net.	297	IN	A	35.192.187.174
nat.travisci.net.	297	IN	A	35.229.37.196
nat.travisci.net.	297	IN	A	207.254.16.38
nat.travisci.net.	297	IN	A	35.237.212.185
nat.travisci.net.	297	IN	A	35.193.184.18
nat.travisci.net.	297	IN	A	35.224.112.202
nat.travisci.net.	297	IN	A	34.66.25.221
nat.travisci.net.	297	IN	A	34.75.144.248
nat.travisci.net.	297	IN	A	35.243.167.131
nat.travisci.net.	297	IN	A	35.188.73.34
nat.travisci.net.	297	IN	A	35.196.99.99
nat.travisci.net.	297	IN	A	35.202.245.105
nat.travisci.net.	297	IN	A	35.192.136.167
nat.travisci.net.	297	IN	A	34.75.228.231
nat.travisci.net.	297	IN	A	34.139.146.111
nat.travisci.net.	297	IN	A	35.229.115.143
nat.travisci.net.	297	IN	A	35.237.56.208
nat.travisci.net.	297	IN	A	35.196.72.151
nat.travisci.net.	297	IN	A	34.74.122.24
nat.travisci.net.	297	IN	A	34.74.253.255
nat.travisci.net.	297	IN	A	35.227.97.188
nat.travisci.net.	297	IN	A	35.196.82.30
nat.travisci.net.	297	IN	A	104.198.131.58
nat.travisci.net.	297	IN	A	34.73.34.132
nat.travisci.net.	297	IN	A	35.231.37.215
nat.travisci.net.	297	IN	A	104.196.53.161
nat.travisci.net.	297	IN	A	34.74.74.27
nat.travisci.net.	297	IN	A	34.122.208.80
nat.travisci.net.	297	IN	A	35.196.158.85
nat.travisci.net.	297	IN	A	34.139.57.23
nat.travisci.net.	297	IN	A	35.196.218.48

;; Query time: 3 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (TCP)
;; WHEN: Thu Sep 26 19:47:02 CEST 2024
;; MSG SIZE  rcvd: 1309

[LeeBrotherston]

@mateusz834 thankyou! I was just coming back to close this ticket as I just came across the comment about this exact thing in the source. I now see that my VPN connection changes the nameserver to one within the VPN network, and which sends larger sized responses.