Fix DNS proxying with large/truncated packets #239
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problems
The DNS proxy only forwards to upstreams over UDP.
It also passes the EDNS0 max message size option on unchanged, as received from the client.
This means that if you query a domain and record type with a big response (e.g. TXT cloudflare.com, especially with +dnssec) which doesn't fit in your advertised bufsize or exceeds the PMTU, the proxy receives a truncated response from the upstream. It returns it to the client, which then retries the query using TCP. However the proxy will either return a cached, truncated response, or ask the upstreams over UDP again and receive it truncated again.
Note that in some instances I could only reproduce this by querying the proxy over IPv6, for IPv4 I assume fragmentation along the path helps to hide the issues.
Changes
Set outgoing EDNS0 UDP buffer size to 1232 byte
Instead of passing on what clients send us, we set the recommended value of 1232 bytes explicitly. This ensures upstreams don't try to send us bigger responses which may get dropped (silently), but instead return a reply with
tc=1
so we can try again over TCP.For modifying the bufsize value the query message needs to be copied, so the original client message isn't manipulated.
Additionally when storing and loading messages from the cache they are copied now as well to prevent accidental changes to the cached message. This means two or three allocations of a few hundred bytes per query now; hopefully this does not regress performance for bigger deployments. If it does we may have to use
sync.Pool
or similar.Retry over TCP for truncated packets from upstreams
Now the proxy has its own retry logic for truncated replies and immediately retries over TCP, using a separate, dedicated TCP client. This happens transparently.
Truncate response to client if required
Since the proxy may now receive a response over TCP that is too big to send to the client, the proxy truncates the message if required and sets
tc=1
. If the transport between client<->proxy is TCP, the limit is 65535 bytes, if it's UDP w/ EDNS0 it's whatever the client told us, if it's UDP w/o EDNS0 it's 512 bytes.And I've added a workflow to run
go test
, for each "supported" Go version (we should keep the matrix up to date to include all versions upwards from what we say ingo.mod
).