Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Local DNS zones and cached responses aren't served after the network lost #4825

Open
3 tasks done
EugeneOne1 opened this issue Aug 8, 2022 · 54 comments
Open
3 tasks done
Assignees
Labels
bug needs investigation Needs to be reproduced reliably.
Milestone

Comments

@EugeneOne1
Copy link
Member

Prerequisites

  • I have checked the Wiki and Discussions and found no answer

  • I have searched other issues and found no duplicates

  • I want to report a bug and not ask a question

Operating system type

Linux, Other (please mention the version in the description)

CPU architecture

64-bit ARM

Installation

Docker

Setup

On one machine

AdGuard Home version

v0.107.9

Description

This is a continuation of the thread started in #2657. The problem's first occurance was in v0.104.3 and has already been fixed a couple of times but still reported. We can't reproduce the issue on our machines. If you've faced it, please consider providing the following information:

  • the setup details (the OS, CPU architecture, installation type);
  • the environment details (other DNS servers, DHCP server);
  • the "General settings", "Cache" and "Encryption" configuration parts (any other details on AdGuard Home's configuration are appreciated);
  • the verbose log with the network loss moment captured.

The last two pieces of information (optionally anonymized) could be sent to devteam@adguard.com with this issue's number in the subject.

@EugeneOne1 EugeneOne1 added bug needs investigation Needs to be reproduced reliably. labels Aug 8, 2022
@EugeneOne1 EugeneOne1 added this to the v0.107.10 milestone Aug 8, 2022
@EugeneOne1 EugeneOne1 self-assigned this Aug 8, 2022
@EugeneOne1
Copy link
Member Author

EugeneOne1 commented Aug 8, 2022

Please, take a look at this, @handcoding, @conradseba, @abdalians, @s1lviu, @dinosoup1. I've mentioned you since you've reported the issue to the #2657. Could you please also help us with the investigation? Thanks.

@conradseba
Copy link

Same issue here since ever. My setup is:
Version: v0.108.0-b.11
Installed on PfSense 22.05, FreeBSD 12.3 (arm64) as a packet.
I'm using DOH, my FW encapsulates all traffic through OpenVPN, no encryption facing internal networks enabled, no DHCP on the AdGuard and no IPv6.

I really hope this is solved soon, since I'm suffering from this many times a day everyday (my Vodafone provider is the worst I've ever had).

Thank you!!

@abdalians
Copy link

@EugeneOne1 we just need the debug logs, right?

@abdalians
Copy link

abdalians commented Aug 8, 2022

Same issue here since ever. My setup is: Version: v0.108.0-b.11 Installed on PfSense 22.05, FreeBSD 12.3 (arm64) as a packet. I'm using DOH, my FW encapsulates all traffic through OpenVPN, no encryption facing internal networks enabled, no DHCP on the AdGuard and no IPv6.

I really hope this is solved soon, since I'm suffering from this many times a day everyday (my Vodafone provider is the worst I've ever had).

Thank you!!

@conradseba if your wan drop frequency is that bad, could you please capture the logs as requested in the other ticket? Save me from taking down the network for log capture. :)

@EugeneOne1
Copy link
Member Author

@abdalians, that's right, we call it "verbose".

@ainar-g ainar-g modified the milestones: v0.107.10, v0.107.11, v0.107.12 Aug 17, 2022
@abdalians
Copy link

abdalians commented Sep 2, 2022

Apologies for the delay in this I am finally in this broken state again and I am trying to collect as much Information as I can will post shortly.

@abdalians
Copy link

abdalians commented Sep 2, 2022

adguard_logs_02Sep2022.tar.gz

To reiterate the point, this only happens when my primary internet (cable) fails over to secondary internet (dsl)

Please see investigation file attached.

  • adguard is running and listening on port 53
    Resolution:
    turning off Adguard PArental Control Web Service / Adguard borwsing securiy web service makes the queries work again.

Until the time that the primary internet connection is restored, then enabling the Adguard PArental Control Web Service / Adguard borwsing securiy web services makes Adguard work again.

adguard_investigation.txt

@handcoding
Copy link

handcoding commented Sep 5, 2022

Please, take a look at this, @handcoding, @conradseba, @abdalians, @s1lviu, @dinosoup1. I've mentioned you since you've reported the issue to the #2657. Could you please also help us with the investigation? Thanks.

@EugeneOne1 I haven’t personally run into this issue since the fix for #4317 landed on the main trunk. (But that’s just me.)

@kevindd992002
Copy link

Aha! I have the same issue and I posted about it just now:

#4969

What is the progress for this? My unifi network uses the FQDN of my unifi controller. When my Internet connection drops (it just did two days ago and it was out for 45 freaking hours!), I lose control over my local network because of AGH!

@EugeneOne1 EugeneOne1 modified the milestones: v0.107.15, v0.107.16 Oct 3, 2022
@abdalians
Copy link

@EugeneOne1 do you need more information the ticket? still says needs investigation and needs to be reproduced reliably. I can reproduce this every single time without failure. Also the milestones were set to 107.16 which is out now.. does that mean we have a potential fix?

@abdalians
Copy link

Version:
v0.107.16
still impacted by this.

@ve6rah
Copy link

ve6rah commented Dec 28, 2022

Version:
v0.107.17
still impacted by this.

@nonoMain
Copy link

Any updates on the matter?
I stopped using it for now..

@Palleri
Copy link

Palleri commented Oct 1, 2024

Still a problem Version: v0.107.52

@fuomag9
Copy link

fuomag9 commented Oct 1, 2024

Still a problem Version: v0.107.52

Can confirm as well, even the suggested fixes do not work for me

@blakeusblade
Copy link

Still a problem Version: v0.107.52

OS Type: GLi-Net 4.6.8 / LuCI openwrt-21.02
Hardware: GL-MT6000 Flint2
CPU: ARM
AdGuard Home Version: v0.107.52

Can confirm as well... Issue arrose after upgrading to v0.107.52.

Turning off AdGaurd restores local lan name resolution, and turning it back on again breaks it.

@GentleHoneyLover
Copy link

This workaround doesn't work for me on v0.107.54 (running in Docker on x86 machine):

My home internet is currently down. Wasn't able to access my network via local DNS. If I disabled AGH protection, local DNS works. My solution was to add @@||mydomain.tld^ to the custom filtering rules. Immediately started resolving again.

@RedFoxy
Copy link

RedFoxy commented Nov 21, 2024

This workaround doesn't work for me on v0.107.54 (running in Docker on x86 machine):

My home internet is currently down. Wasn't able to access my network via local DNS. If I disabled AGH protection, local DNS works. My solution was to add @@||mydomain.tld^ to the custom filtering rules. Immediately started resolving again.

This is my configuration and it goes when the connection goes offline, I've 192.168.0.x LAN and my router 192.168.0.1 give me static dns for LAN device

https://github.com/RedFoxy/HA-MyConf/blob/main/AdGuardHome/AdGuardHome.yaml

I hope that can help you

@kevindd992002
Copy link

This workaround doesn't work for me on v0.107.54 (running in Docker on x86 machine):

My home internet is currently down. Wasn't able to access my network via local DNS. If I disabled AGH protection, local DNS works. My solution was to add @@||mydomain.tld^ to the custom filtering rules. Immediately started resolving again.

This is my configuration and it goes when the connection goes offline, I've 192.168.0.x LAN and my router 192.168.0.1 give me static dns for LAN device

https://github.com/RedFoxy/HA-MyConf/blob/main/AdGuardHome/AdGuardHome.yaml

I hope that can help you

What?

@RedFoxy
Copy link

RedFoxy commented Nov 21, 2024

This workaround doesn't work for me on v0.107.54 (running in Docker on x86 machine):

My home internet is currently down. Wasn't able to access my network via local DNS. If I disabled AGH protection, local DNS works. My solution was to add @@||mydomain.tld^ to the custom filtering rules. Immediately started resolving again.

This is my configuration and it goes when the connection goes offline, I've 192.168.0.x LAN and my router 192.168.0.1 give me static dns for LAN device
https://github.com/RedFoxy/HA-MyConf/blob/main/AdGuardHome/AdGuardHome.yaml
I hope that can help you

What?

With my configuration I can access the DNS of the local names provided by the gateway and ADGuard's DNS cache, also when the internet becomes available again ADGuard comes back to work completely without any problems

@kevindd992002
Copy link

This workaround doesn't work for me on v0.107.54 (running in Docker on x86 machine):

My home internet is currently down. Wasn't able to access my network via local DNS. If I disabled AGH protection, local DNS works. My solution was to add @@||mydomain.tld^ to the custom filtering rules. Immediately started resolving again.

This is my configuration and it goes when the connection goes offline, I've 192.168.0.x LAN and my router 192.168.0.1 give me static dns for LAN device
https://github.com/RedFoxy/HA-MyConf/blob/main/AdGuardHome/AdGuardHome.yaml
I hope that can help you

What?

With my configuration I can access the DNS of the local names provided by the gateway and ADGuard's DNS cache, also when the internet becomes available again ADGuard comes back to work completely without any problems

Right. So do you know which specific setting in your config is fixing this?

@RedFoxy
Copy link

RedFoxy commented Nov 22, 2024

Right. So do you know which specific setting in your config is fixing this?

Simply in my configuration I planned to have a separate DNS server for the LAN, in my case it is a service provided by my gateway with ip 192.168.0.1, that would act as a CACHE and DNS for the local static DNS, so in “Settings -> DNS settings” under “Upstream DNS servers” I added the rules for which the gateway DNS server should be used:

[/.local/]192.168.0.1
[/
.mydomain.com/]192.168.0.1

Basically for all DNS requests that end in .local or .mydomain.com instead of ADGuard responding the DNS server 192.168.0.1 is queried

On the same page I activated the item “Use private reverse DNS resolvers”.

After that under “Filters -> Custom filtering rules,” just in case, I added the local domains not to be blocked:

@@||local^
@@||eth.local^
@@||wifi.local^
@@||mydomain.com^

Doing so solved the problem of internet drops and AdGuard Home not responding once the internet came back

@ve6rah
Copy link

ve6rah commented Nov 22, 2024

While I suppose that is a workaround, it also doesn't make any sense. Adguard home is supposed to function as a caching DNS server. What you've done is add another DNS server to your network. What this issue is about is the fact that you shouldn't need another DNS server because adguard home should fill that role.

@kevindd992002
Copy link

While I suppose that is a workaround, it also doesn't make any sense. Adguard home is supposed to function as a caching DNS server. What you've done is add another DNS server to your network. What this issue is about is the fact that you shouldn't need another DNS server because adguard home should fill that role.

But his configuration is expected if your upstream device is a firewall router like pfsense/opnsense. These have unbound in them and you point AGH to that as it is also the DHCP server of the network. This is a supported config.

@RedFoxy , I have the same config as you do, at least for the DNS servers part. But why do you still have .local there? I only have my local domain listed there.

Also, why the need to put them in the whitelist? This part is what I don't have.

@ve6rah
Copy link

ve6rah commented Nov 22, 2024

I have to strongly disagree with this. If you are running a DNS server on your router, then you should be doing ad filtering at that level as well. Adding extra DNS servers along the way. Just slows down DNS lookups and adds extra points of failure and extra complication.

@kevindd992002
Copy link

I have to strongly disagree with this. If you are running a DNS server on your router, then you should be doing ad filtering at that level as well. Adding extra DNS servers along the way. Just slows down DNS lookups and adds extra points of failure and extra complication.

I get your point but that latency is negligible fora home network. To be fair, I have AGH installed on my opnsense router itself and is pointed to itself (localhost), and I get an average processing time of 6ms.

@ve6rah
Copy link

ve6rah commented Nov 22, 2024

But you were still adding ridiculous unnecessary extra complication. Just point the adguard instance at the real upstream DNS server, instead of adding another one in the middle. If you're trying to pretend that this is a workaround for this bug. You might as well say just don't use adguard. Because the whole point to this bug is that adguard doesn't work if it can't access an upstream server. When you add an extra upstream server within your own house, all you've done is move the bug. One more layer. You'll still have the exact same issue if that server within your house goes down.

@RedFoxy
Copy link

RedFoxy commented Nov 22, 2024

@RedFoxy , I have the same config as you do, at least for the DNS servers part. But why do you still have .local there? I only have my local domain listed there.

Also, why the need to put them in the whitelist? This part is what I don't have.

Maybe I confused you for a moment, DHCP is handled by my gateway (a Mikrotik) and among other services it provides, it also has a DNS server for all my local .local names like pve.local or frigate.pve. local, but also to override the names of my external domain mydomain.com, this is because if I go to frigate.mydomain.com with my cell phone and I am connected to WiFi at home he resolves it as 192.168.0.10 if I am away from home he resolves it with my external ip.

In the local network I do NOT directly use any DNS server other than AdGuard Home, while ADG uses my gateway as upstream, so when I ask frigate.pve.local to resolve me I ask ADG which in turn asks the gateway.

Why do I do this?

I realized that ADG when it does not reach the external DNS it crashes and does not always come back to work when the external DNS becomes available again, if I provide it with an always working DNS, such as my gateway's DNS, it never crashes and always resolves my local DNS and the ones it has cached.

@RedFoxy
Copy link

RedFoxy commented Nov 22, 2024

But you were still adding ridiculous unnecessary extra complication. Just point the adguard instance at the real upstream DNS server, instead of adding another one in the middle. If you're trying to pretend that this is a workaround for this bug. You might as well say just don't use adguard. Because the whole point to this bug is that adguard doesn't work if it can't access an upstream server. When you add an extra upstream server within your own house, all you've done is move the bug. One more layer. You'll still have the exact same issue if that server within your house goes down.

I completely understand what you mean, but unfortunately I have an unstable line and internet drops easily every time it rains, even for a few seconds, the fact that ADG would always crash forcing me to restart its service in order to surf take advantage of the network again, I preferred to use this system, I don't feel all this lag in the resolution and since I use it I don't have network problems anymore, while before I was very tempted to uninstall ADG

@ve6rah
Copy link

ve6rah commented Nov 22, 2024

But then why are you commenting on this bug, if your whole point is to just not use adguard because your line is unstable?
This bug is an attempt to get the fact that adguard goes down when your line does fixed! Telling us to just skip adguard to solve the problem does not add anything to the conversation about fixing the bug in adguard in the first place.

@RedFoxy
Copy link

RedFoxy commented Nov 22, 2024

But then why are you commenting on this bug, if your whole point is to just not use adguard because your line is unstable? This bug is an attempt to get the fact that adguard goes down when your line does fixed! Telling us to just skip adguard to solve the problem does not add anything to the conversation about fixing the bug in adguard in the first place.

I use EVERYTIME ADGuard! why do you say that I don't use ADG? The trouble is when the internet goes offline and ADG doesn't goes! but with that workaround you'll continue to use ADG when you are offline or when you switch from land cable to mobile hotspot etc...

@ve6rah
Copy link

ve6rah commented Nov 22, 2024

No, you specifically stated that you do not use adguard, you only use adguard as a relay to your other DNS server. This bug is about those of us who are trying to use adguard as a DNS server.

@RedFoxy
Copy link

RedFoxy commented Nov 22, 2024

No, you specifically stated that you do not use adguard, you only use adguard as a relay to your other DNS server. This bug is about those of us who are trying to use adguard as a DNS server.

Where did I say this? I said I use ADGuard as my only DNS, the upstream DNS that ADGuard uses when it doesn't know what to resolve is my gateway where google and cloudflare DNS are also set, but all my queries go through ADG first!

Devices -> DNS REquest -> ADG -> Gateway -> Other external DNS

@ve6rah
Copy link

ve6rah commented Nov 22, 2024

Exactly, you are not using adguard as a DNS, you are using adguard simply as a relay to your real DNS server that is on your router.
Some of us are trying to use adguard as a DNS server, not simply as a relay. And that's who this bug is for. People who want to use adguard as a DNS server.

@RedFoxy
Copy link

RedFoxy commented Nov 22, 2024

Exactly, you are not using adguard as a DNS, you are using adguard simply as a relay to your real DNS server that is on your router. Some of us are trying to use adguard as a DNS server, not simply as a relay. And that's who this bug is for. People who want to use adguard as a DNS server.

excuse me, my wronge, but ADG doesn't need an external dns to resolve names?

@ve6rah
Copy link

ve6rah commented Nov 22, 2024

The key word here is external. And external depends on the internet. We are looking to allow adguard to serve local DNS without an internet connection. That's the whole bug. Adguard currently refuses to serve local DNS when it cannot access an external DNS server. That's wrong. It should be able to do so.

This whole thing doesn't apply to your situation because you don't have adguard pointing to an external DNS. And you don't have it trying to serve local DNS at all, you have your other DNS server serving local DNS.

Adguard is a DNS server, and as such should not require an upstream resolver to resolve addresses it already knows, in this case local ones.

@RedFoxy
Copy link

RedFoxy commented Nov 22, 2024

The key word here is external. And external depends on the internet. We are looking to allow adguard to serve local DNS without an internet connection. That's the whole bug. Adguard currently refuses to serve local DNS when it cannot access an external DNS server. That's wrong. It should be able to do so.

This whole thing doesn't apply to your situation because you don't have adguard pointing to an external DNS. And you don't have it trying to serve local DNS at all, you have your other DNS server serving local DNS.

Adguard is a DNS server, and as such should not require an upstream resolver to resolve addresses it already knows, in this case local ones.

I used to have local dns on ADG but the fact that it doesn't work when the internet goes down was blocking me too much.

I'm sorry to have bothered you

@kevindd992002
Copy link

But you were still adding ridiculous unnecessary extra complication. Just point the adguard instance at the real upstream DNS server, instead of adding another one in the middle. If you're trying to pretend that this is a workaround for this bug. You might as well say just don't use adguard. Because the whole point to this bug is that adguard doesn't work if it can't access an upstream server. When you add an extra upstream server within your own house, all you've done is move the bug. One more layer. You'll still have the exact same issue if that server within your house goes down.

I point it to my own dns server because I want to use my own DNS resolver. Nothing is complicated with that. We're all here for tinkering with things. If you don't want that setup, good for you. This is just like the question of virtualizing your router. There's always two sides in a coin. And mind you, I still have this bug even if my setup is the same as RedFoxy's.

@kevindd992002
Copy link

The key word here is external. And external depends on the internet. We are looking to allow adguard to serve local DNS without an internet connection. That's the whole bug. Adguard currently refuses to serve local DNS when it cannot access an external DNS server. That's wrong. It should be able to do so.

This whole thing doesn't apply to your situation because you don't have adguard pointing to an external DNS. And you don't have it trying to serve local DNS at all, you have your other DNS server serving local DNS.

Adguard is a DNS server, and as such should not require an upstream resolver to resolve addresses it already knows, in this case local ones.

For me, I do have AGH pointed to my router's DNS AND external DNS servers. So for local records, it queries my unbound DNS server. Again, nothing wrong with that. And I don't know why you consider that "ridiculously complicated". It isn't. It's so simple.

@GentleHoneyLover
Copy link

While I understand @kevindd992002‘s point I agree with @RedFoxy — local name resolution is a job of a DNS server — which is what AGH is. It should work independently of the WAN connection and there’s no reason it shouldn’t do DNS rewrites without an upstream server.

@kevindd992002
Copy link

While I understand @kevindd992002‘s point I agree with @RedFoxy — local name resolution is a job of a DNS server — which is what AGH is. It should work independently of the WAN connection and there’s no reason it shouldn’t do DNS rewrites without an upstream server.

You mean you agree with @ve6rah?

Just to be clear, I think we're saying the same thing here. My setup is like so:

Client -> AGH -> Unbound in opnsense (which is also my Internet gateway)

I agree with you that AGH should be able to resolve local DNS entries even if the WAN connection is down. That'a the whole point of this bug. It should still be able to use the upstream servers that are in its local subnet, or as long as it has a route to wherever those upstream DNS servers are.

Regardless of the difference in setup here, we all are pointing to the same bug. AGH's cache and upstream servers (except external servers) should work even if the WAN connection goes down.

@GentleHoneyLover
Copy link

Yes, I meant @ve6rah and yes, I think regardless of the upstream dns chosen, AGH should work independently of the internet connection 👍 I just thought it was important to clarify this :) Thanks!

@RedFoxy
Copy link

RedFoxy commented Nov 23, 2024

Yes, I meant @ve6rah and yes, I think regardless of the upstream dns chosen, AGH should work independently of the internet connection 👍 I just thought it was important to clarify this :) Thanks!

when that bug will fixed I'll remove my gateway DNS server... I've it only to able to use my DNS when I'm offline!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug needs investigation Needs to be reproduced reliably.
Projects
None yet
Development

No branches or pull requests