Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DNS keep retrying different qtypes on hard error(s) from DNS server #10182

Closed
1 task done
komapa opened this issue Jan 27, 2023 · 3 comments · Fixed by #11386
Closed
1 task done

DNS keep retrying different qtypes on hard error(s) from DNS server #10182

komapa opened this issue Jan 27, 2023 · 3 comments · Fixed by #11386
Assignees
@hanshuebner
Labels
bug dns

Comments

@komapa
Copy link

komapa commented Jan 27, 2023 •
edited
Loading

Is there an existing issue for this?

  • I have searched the existing issues

Kong version ($ kong version)

Version 3.0.2

Current Behavior

By looking at the source code and also the end behavior, when DNS server is hard down, settings like RES_OPTIONS = "timeout:1 attempts:2" fail the request after 4s instead of 2s. This is because you try SRV first, that fails, and then you try A and that fails but each of these take 2s to fails, so total of 4s.

Expected Behavior

I would expect that if you get a hard error from DNS during SRV resolution, you should stop right there because even if the A query succeeds, you are actually doing the wrong thing because we want the SRV to be respected first. The A record will not work as intended and if you were to cache and try to serve that, you will be returning different type of errors that will be hard to debug.

Steps To Reproduce

No response

Anything else?

No response
@hanshuebner
Copy link
Contributor

@komapa Thank you for your bug report and sorry for not responding to this earlier. We're tracking this issue internally (KAG-2300) and will let you know once a fix is available.

@komapa
Copy link
Author

komapa commented Aug 10, 2023

Thank you!

@hanshuebner hanshuebner mentioned this issue Aug 10, 2023
Merged
3 tasks
hanshuebner added a commit that referenced this issue Aug 11, 2023
@hanshuebner
fix(dns): fix retry and timeout handling
b5662d9 
 - Stop retrying in dns/client.lua, let the resolver handle this.  This
   change also makes it possible to disable retries, which previously
   was not possible
 - Be more faithful to the timeouts set by the user.  Previously, the
   timeout configured was used only for the ultimate request sent to
   the DNS server, but asynchronous requests allowed longer timeouts
   which was not transparent.
 - When the DNS server fails, stop trying other query types.  Previously,
   the behavior was such that after an (intermediate) failure to query
   for one record type (say "SRV"), the client would try the next record
   type (say "A") and succeed with that.  It would then return the
   contents of the "A" record even if the "SRV" record pointed to a
   different address.
 - Change domain names used for testing the DNS client into the
   kong-gateway-testing.link zone, which is controlled by the Kong Gateway
   team.

#10182
KAG-2300
hanshuebner added a commit that referenced this issue Aug 11, 2023
@hanshuebner
fix(dns): fix retry and timeout handling (#11386)
9294874 
- Stop retrying in dns/client.lua, let the resolver handle this.  This
   change also makes it possible to disable retries, which previously
   was not possible
 - Be more faithful to the timeouts set by the user.  Previously, the
   timeout configured was used only for the ultimate request sent to
   the DNS server, but asynchronous requests allowed longer timeouts
   which was not transparent.
 - When the DNS server fails, stop trying other query types.  Previously,
   the behavior was such that after an (intermediate) failure to query
   for one record type (say "SRV"), the client would try the next record
   type (say "A") and succeed with that.  It would then return the
   contents of the "A" record even if the "SRV" record pointed to a
   different address.
 - Change domain names used for testing the DNS client into the
   kong-gateway-testing.link zone, which is controlled by the Kong Gateway
   team.

Fixes #10182
KAG-2300
@hanshuebner
Copy link
Contributor

Please let us know if the fix is OK.

@chobits chobits mentioned this issue Nov 16, 2023
Merged
3 tasks
chobits pushed a commit that referenced this issue Nov 16, 2023
@hanshuebner @chobits
fix(dns): fix retry and timeout handling (#11386)
4c7b011 
- Stop retrying in dns/client.lua, let the resolver handle this.  This
   change also makes it possible to disable retries, which previously
   was not possible
 - Be more faithful to the timeouts set by the user.  Previously, the
   timeout configured was used only for the ultimate request sent to
   the DNS server, but asynchronous requests allowed longer timeouts
   which was not transparent.
 - When the DNS server fails, stop trying other query types.  Previously,
   the behavior was such that after an (intermediate) failure to query
   for one record type (say "SRV"), the client would try the next record
   type (say "A") and succeed with that.  It would then return the
   contents of the "A" record even if the "SRV" record pointed to a
   different address.
 - Change domain names used for testing the DNS client into the
   kong-gateway-testing.link zone, which is controlled by the Kong Gateway
   team.

Fixes #10182
KAG-2300
chobits pushed a commit that referenced this issue Nov 16, 2023
@hanshuebner @chobits
fix(dns): fix retry and timeout handling (#11386)
4c80884 
- Stop retrying in dns/client.lua, let the resolver handle this.  This
   change also makes it possible to disable retries, which previously
   was not possible
 - Be more faithful to the timeouts set by the user.  Previously, the
   timeout configured was used only for the ultimate request sent to
   the DNS server, but asynchronous requests allowed longer timeouts
   which was not transparent.
 - When the DNS server fails, stop trying other query types.  Previously,
   the behavior was such that after an (intermediate) failure to query
   for one record type (say "SRV"), the client would try the next record
   type (say "A") and succeed with that.  It would then return the
   contents of the "A" record even if the "SRV" record pointed to a
   different address.
 - Change domain names used for testing the DNS client into the
   kong-gateway-testing.link zone, which is controlled by the Kong Gateway
   team.

Fixes #10182
KAG-2300

Signed-off-by: Xiaochen Wang <wangxiaochen0@gmail.com>
bungle pushed a commit that referenced this issue Nov 16, 2023
@hanshuebner @bungle
fix(dns): fix retry and timeout handling (#11386)
1ab8980 
- Stop retrying in dns/client.lua, let the resolver handle this.  This
   change also makes it possible to disable retries, which previously
   was not possible
 - Be more faithful to the timeouts set by the user.  Previously, the
   timeout configured was used only for the ultimate request sent to
   the DNS server, but asynchronous requests allowed longer timeouts
   which was not transparent.
 - When the DNS server fails, stop trying other query types.  Previously,
   the behavior was such that after an (intermediate) failure to query
   for one record type (say "SRV"), the client would try the next record
   type (say "A") and succeed with that.  It would then return the
   contents of the "A" record even if the "SRV" record pointed to a
   different address.
 - Change domain names used for testing the DNS client into the
   kong-gateway-testing.link zone, which is controlled by the Kong Gateway
   team.

Fixes #10182
KAG-2300
dndx pushed a commit that referenced this issue Nov 24, 2023
@hanshuebner @dndx
fix(dns): fix retry and timeout handling (#11386)
7315817 
- Stop retrying in dns/client.lua, let the resolver handle this.  This
   change also makes it possible to disable retries, which previously
   was not possible
 - Be more faithful to the timeouts set by the user.  Previously, the
   timeout configured was used only for the ultimate request sent to
   the DNS server, but asynchronous requests allowed longer timeouts
   which was not transparent.
 - When the DNS server fails, stop trying other query types.  Previously,
   the behavior was such that after an (intermediate) failure to query
   for one record type (say "SRV"), the client would try the next record
   type (say "A") and succeed with that.  It would then return the
   contents of the "A" record even if the "SRV" record pointed to a
   different address.
 - Change domain names used for testing the DNS client into the
   kong-gateway-testing.link zone, which is controlled by the Kong Gateway
   team.

Fixes #10182
KAG-2300

Signed-off-by: Xiaochen Wang <wangxiaochen0@gmail.com>
chobits pushed a commit that referenced this issue Nov 29, 2023
@hanshuebner @chobits
fix(dns): fix retry and timeout handling (#11386)
e137d99 
- Stop retrying in dns/client.lua, let the resolver handle this.  This
   change also makes it possible to disable retries, which previously
   was not possible
 - Be more faithful to the timeouts set by the user.  Previously, the
   timeout configured was used only for the ultimate request sent to
   the DNS server, but asynchronous requests allowed longer timeouts
   which was not transparent.
 - When the DNS server fails, stop trying other query types.  Previously,
   the behavior was such that after an (intermediate) failure to query
   for one record type (say "SRV"), the client would try the next record
   type (say "A") and succeed with that.  It would then return the
   contents of the "A" record even if the "SRV" record pointed to a
   different address.
 - Change domain names used for testing the DNS client into the
   kong-gateway-testing.link zone, which is controlled by the Kong Gateway
   team.

Fixes #10182
KAG-2300
dndx pushed a commit that referenced this issue Nov 29, 2023
@hanshuebner @dndx
fix(dns): fix retry and timeout handling (#11386)
d4c7109 
- Stop retrying in dns/client.lua, let the resolver handle this.  This
   change also makes it possible to disable retries, which previously
   was not possible
 - Be more faithful to the timeouts set by the user.  Previously, the
   timeout configured was used only for the ultimate request sent to
   the DNS server, but asynchronous requests allowed longer timeouts
   which was not transparent.
 - When the DNS server fails, stop trying other query types.  Previously,
   the behavior was such that after an (intermediate) failure to query
   for one record type (say "SRV"), the client would try the next record
   type (say "A") and succeed with that.  It would then return the
   contents of the "A" record even if the "SRV" record pointed to a
   different address.
 - Change domain names used for testing the DNS client into the
   kong-gateway-testing.link zone, which is controlled by the Kong Gateway
   team.

Fixes #10182
KAG-2300
chobits pushed a commit that referenced this issue Nov 30, 2023
@hanshuebner @chobits
fix(dns): fix retry and timeout handling (#11386)
81fc4aa 
- Stop retrying in dns/client.lua, let the resolver handle this.  This
   change also makes it possible to disable retries, which previously
   was not possible
 - Be more faithful to the timeouts set by the user.  Previously, the
   timeout configured was used only for the ultimate request sent to
   the DNS server, but asynchronous requests allowed longer timeouts
   which was not transparent.
 - When the DNS server fails, stop trying other query types.  Previously,
   the behavior was such that after an (intermediate) failure to query
   for one record type (say "SRV"), the client would try the next record
   type (say "A") and succeed with that.  It would then return the
   contents of the "A" record even if the "SRV" record pointed to a
   different address.
 - Change domain names used for testing the DNS client into the
   kong-gateway-testing.link zone, which is controlled by the Kong Gateway
   team.

Fixes #10182
KAG-2300
dndx pushed a commit that referenced this issue Nov 30, 2023
@hanshuebner @dndx
fix(dns): fix retry and timeout handling (#11386)
e5fde8c 
- Stop retrying in dns/client.lua, let the resolver handle this.  This
   change also makes it possible to disable retries, which previously
   was not possible
 - Be more faithful to the timeouts set by the user.  Previously, the
   timeout configured was used only for the ultimate request sent to
   the DNS server, but asynchronous requests allowed longer timeouts
   which was not transparent.
 - When the DNS server fails, stop trying other query types.  Previously,
   the behavior was such that after an (intermediate) failure to query
   for one record type (say "SRV"), the client would try the next record
   type (say "A") and succeed with that.  It would then return the
   contents of the "A" record even if the "SRV" record pointed to a
   different address.
 - Change domain names used for testing the DNS client into the
   kong-gateway-testing.link zone, which is controlled by the Kong Gateway
   team.

Fixes #10182
KAG-2300
chobits pushed a commit that referenced this issue Nov 30, 2023
@hanshuebner @chobits
fix(dns): fix retry and timeout handling (#11386)
737b983 
- Stop retrying in dns/client.lua, let the resolver handle this.  This
   change also makes it possible to disable retries, which previously
   was not possible
 - Be more faithful to the timeouts set by the user.  Previously, the
   timeout configured was used only for the ultimate request sent to
   the DNS server, but asynchronous requests allowed longer timeouts
   which was not transparent.
 - When the DNS server fails, stop trying other query types.  Previously,
   the behavior was such that after an (intermediate) failure to query
   for one record type (say "SRV"), the client would try the next record
   type (say "A") and succeed with that.  It would then return the
   contents of the "A" record even if the "SRV" record pointed to a
   different address.
 - Change domain names used for testing the DNS client into the
   kong-gateway-testing.link zone, which is controlled by the Kong Gateway
   team.

Fixes #10182
KAG-2300
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hanshuebner hanshuebner

Labels
bug dns
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(dns): fix retry and timeout handling Kong/kong
2 participants
@komapa @hanshuebner