Takes a lot of time to start and fails to load list of public resolvers

[sryze]

Hello,

I'm using dnscrypt-proxy 2.1.1 on a router. Recently I began experiencing issues with DNS and upon digging I found that dnscrypt-proxy is to blame (I think).

When I run dnscrypt-proxy manually with my config file, I get an error:

[2023-02-15 00:28:47] [NOTICE] dnscrypt-proxy 2.1.1
[2023-02-15 00:28:48] [NOTICE] Network connectivity detected
[2023-02-15 00:28:48] [NOTICE] Now listening to 127.0.0.1:53 [UDP]
[2023-02-15 00:28:48] [NOTICE] Now listening to 127.0.0.1:53 [TCP]
[2023-02-15 00:28:48] [NOTICE] Now listening to 10.1.1.1:53 [UDP]
[2023-02-15 00:28:48] [NOTICE] Now listening to 10.1.1.1:53 [TCP]
[2023-02-15 00:28:48] [INFO] Source [public-resolvers] loading from URL [https://ipv6.download.dnscrypt.info/resolvers-list/v3/public-resolvers.md]
[2023-02-15 00:28:48] [INFO] Source [public-resolvers] loading from URL [https://raw.githubusercontent.com/DNSCrypt/dnscrypt-resolvers/master/v3/public-resolvers.md]
[2023-02-15 00:29:18] [INFO] Source [public-resolvers] loading from URL [https://download.dnscrypt.info/resolvers-list/v3/public-resolvers.md]
[2023-02-15 00:29:48] [INFO] Source [public-resolvers] loading from URL [https://download.dnscrypt.net/resolvers-list/v3/public-resolvers.md]
[2023-02-15 00:30:18] [CRITICAL] Unable to retrieve source [public-resolvers]: [Get "https://download.dnscrypt.net/resolvers-list/v3/public-resolvers.md": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)]
[2023-02-15 00:30:18] [FATAL] Get "https://download.dnscrypt.net/resolvers-list/v3/public-resolvers.md": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

It takes like a minute to load all the public-resolvers.md files.

Question: is this by design or it's a bug? Also why is it not using the cached versions of public-resolvers.md while it's downloading the new ones? I noticed that when I run nslookup google.com 127.0.0.1 (dnscrypt-proxy is running locally) it just times out.

My config (changed .toml to .txt because GitHub won't let me upload otherwise):
dnscrypt-proxy.toml.txt

[lifenjoiner]

At starting up, there will be a gap that is out of service until "valid" source files are loaded successfully, especially when the network is bad. The duration depends on the downloading progress.

Definition for "valid":

1. no existing cached files, newly downloaded and validated;
2. cached files exist:
   2.1. unexpired by refresh_delay;
   2.2. expired, and then downloading failed, it becomes temporarily valid.

[2023-02-15 00:30:18] [CRITICAL] Unable to retrieve source [public-resolvers]

Shows there is none cached public-resolvers.md:

dnscrypt-proxy/dnscrypt-proxy/config.go

Lines 896 to 902 in acc25fc

	if err != nil {
	if len(source.in) <= 0 {
	dlog.Criticalf("Unable to retrieve source [%s]: [%s]", cfgSourceName, err)
	return err
	}
	dlog.Infof("Downloading [%s] failed: %v, using cache file to startup", source.name, err)
	}

You'd better use the latest release, if possible.

[sryze]

@lifenjoiner Does dnscrypt-proxy use bootstrap_resolvers when it downloads the public resolvers?

[lifenjoiner]

Yes, that is the initial purpose of bootstrap_resolvers, I think.

[sryze]

@lifenjoiner Sorry for bothering you again, but I'm running into a strange issue on startup:

[2023-02-16 21:34:19] [ERROR] Get "https://dns.cloudflare.com/dns-query?dns=some_long_text_here": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
[2023-02-16 21:34:19] [NOTICE] dnscrypt-proxy is waiting for at least one server to be reachable

However, I'm able to fetch the same URL using curl and one of the bootstrap resolvers (8.8.8.8). And it also works well with the same config on my Windows PC.
Any idea why it would time out?
Version: 2.1.4

[lifenjoiner]

If you have moved on to the latest release, that's hard to say at the moment.

• Update your config file too.
• If you specify very few servers by server_names, there are less backups but more chances to fail. Reasons could be network quality or software bugs.

Maybe more logs, details and research could help.

Append:

(Client.Timeout exceeded while awaiting headers)

It is an error message of net/http, called from

dnscrypt-proxy/dnscrypt-proxy/xtransport.go

Line 476 in 96ffb21

resp, err := client.Do(req)

So, it is less likely a fault of dnscrypt-proxy, but network quality is suspected.

[57382]

Of course, in my option, loading existing cache files first could be a better choice for bad networks.

Append a new case for "valid":

3. It is people's illusion: cached files will be valid.
   In fact, the cached files could be not very old, and there are sdns URLs available. This is the most case, I guess. But, there could be the worst case, the cached files are too old to have even one sdns URL available.

For loading the cached files first, in the ideal/normal case, it will help; in the worst case, it will take a long time to validate all the sdns URLs, still out of service.

Forcing, forcing to refresh the SDNS cache is an extremely stupid design in my opinion, it's like Windows 10 forcing automatic updates. Ended up breaking a lot of stuff.
At least the user should be given an option to freely set the refresh cache. No restrictions. Users know what they are doing, never make decisions for users. no way. If the user wishes to disable, SDNS cache flushing must be disabled. This is Linux, this is freedom. @jedisct1
In Linux, if I want to shoot myself in the foot, I will be able to. You should warn me, but you shouldn't stop me.