Docker healthchecks introduced in v0.107.28 fail if "dns.bind_hosts" contains "0.0.0.0"

[piv-pav]

Prerequisites

• I have checked the Wiki and Discussions and found no answer

• I have searched other issues and found no duplicates

• I want to report a bug and not ask a question

Operating system type

Linux, Other (please mention the version in the description)

CPU architecture

AMD64

Installation

Docker

Setup

On one machine

AdGuard Home version

v0.107.28

Description

What did you do?

Run the container with following configuration piece in AdGuardHome.yaml config file:

dns:
  bind_hosts:
    - 0.0.0.0
  port: 53

Expected result

Container runs normally

Actual result

Docker reports container as Unhealthy because introduced healthcheck script fails with:

nslookup failed for 0.0.0.0:53

Additional information

After digging into healthcheck script code, it seems it builds the list of addresses DNS is listening to and then running nslookup command which suppose to request address resolution for healthcheck.adguardhome.test. against all IP addresses listed in dns.bind_hosts parameter.

In order to succeed with healthcheck DNS server must:

• have healthcheck.adguardhome.test. record (not guaranteed)
• be reachable by bind_hosts IP address (impossible with 0.0.0.0 address)

[piv-pav]

Current workaround is to disable health-checks by adding following config to docker-composer file for adguard/adguardhome:latest container:

healthcheck:
      disable: true

If you're running docker container from the CLI you might use --no-healthcheck docker parameter to disable healthcheck for now.

[ovizii]

Current workaround is to disable health-checks by adding following config to docker-composer file for adguard/adguardhome:latest container:

healthcheck:
      disable: true

If you're running docker container from the CLI you might use --no-healthcheck docker parameter to disable healthcheck for now.

THANKS! This was the hotfix I was looking for. Wondering what/were to find this "bind_hosts"?

[piv-pav]

Wondering what/were to find this "bind_hosts"?

It is the part of the /opt/adguardhome/conf/AdGuardHome.yaml config file.

General recommendation is to mount this config folder inside the container from local filesystem or as persistent volume, so please check your volumes configuration to find precise file location.

Also please don't confuse bind_host parameter with dns.bind_hosts, issue associated with dns.bind_hosts.

[ovizii]

Thanks, I found it and I see what you mean. Writing it down here in case anyone else finds this thread. Documentation to be found here: https://github.com/AdguardTeam/AdGuardHome/wiki/Configuration#configuration-file

bind_host: 0.0.0.0

vs

dns:                                                                                                                                                                                                                                                                                                                 
  bind_hosts:                                                                                                                                                                                                                                                                                              
    - 0.0.0.0

[EugeneOne1]

@piv-pav, hello and thanks for the report. We've actually added special handling for special subdomain healthcheck.adguardhome.test., here is the commit that contains it.

If it's not too much trouble, could you please check, if the localhost. server name works with nslookup inside the container? Something like nslookup -type=a healthcheck.adguardhome.test. localhost:53.

[piv-pav]

Weirdly enough, but I cannot reach DNS on the 127.0.0.1:53 or localhost:53 from the container

nslookup healthcheck.adguardhome.test. localhost:53
;; connection timed out; no servers could be reached

Although it works well if I'm using container's IP.

I'm using bridge network type and honestly speaking I'm a bit confused why those ports are unavailable from the container itself.

[EugeneOne1]

@piv-pav, the output also strangely different from the usual nslookup output. Anyway, I'm sorry for confusion, but it seems some distributions of nslookup doesn't support addresses in IP:Port format, so could you please retry with:

nslookup -type=a -port=53 healthcheck.adguardhome.test. 127.0.0.1
nslookup -type=a -port=53 healthcheck.adguardhome.test. localhost

If you're using some custom port for DNS, use it instead of 53. If this wouldn't work, I think we'll need to investigate the Docker documentation for bridge networking specifics.

[BobWs]

Is this problem also related when using AH with macvlan network?

[piv-pav]

@EugeneOne1 I don't think nslookup syntax is a problem because following works well:

# nslookup -type=a healthcheck.adguardhome.test. 172.17.0.4:53
Server:		172.17.0.4:53
Address:	172.17.0.4:53

Non-authoritative answer:

# echo $?
0

where 172.17.0.4 is the IP address of the container. Something is wrong with localhost or 127.0.0.1 access, but I don't yet have time to figure it out.

I have a feeling it might to do something with IPv6 because ports 53 are opened on :::53

# netstat -nlp | grep 53
tcp        0      0 :::53                   :::*                    LISTEN      7/AdGuardHome
udp        0      0 :::53                   :::*                                7/AdGuardHome

[EugeneOne1]

@piv-pav, hello again. We've tried to improve the situation by handling the unspecified addresses in a special way. It will now be considered failed if both :: and 127.0.0.1 failed. Could you please check the latest edge build?

[EugeneOne1]

@piv-pav, hello again. We've pushed one more fix related to unspecified address. It should fix the issue so we'll close it for now. Feel free to reopen it if the problem persists.

[BobWs]

Just updated to version 107.29 and now portainer is showing status unhealthy! So I guess it still isn't fixed.

[EugeneOne1]

@BobWs, see https://github.com/AdguardTeam/AdGuardHome/wiki/Docker#known-issues. Also, could you please share the exact message of unhealthy status?

Just noticed the version. The actual fix is available in the edge channel only currently.