Authoritative server exits on parse error while looking for NS addresses to notify

[gertvdijk]

• Program: Authoritative
• Issue type: Bug report

Short description

Very similar to #7493 (exit/crash with invalid zone data), but for regular non-SOA records on the BIND backend.

I do not expect a PowerDNS server to completely go down / exit upon a parser error.

Environment

• Operating system: Debian Stretch
• Software version: 4.1.8-1pdns.stretch & 4.2.0~rc1-1pdns.stretch
• Software source: PowerDNS repository

Steps to reproduce

1. Set up a master with the BIND backend.

2. Add a zone with zone file containing an invalid A record (example with quotes as invalid character, probably other cases to trigger this too?), e.g.:

   @       300     A       "1.2.3.4"
   

3. Increment the serial, reload the zone or await automatic reload.

4. Observe exit of server with log line:

   Zone 'mydomain.tld' (bind-zones/mydomain.tld.db) needs reloading
   Zone 'mydomain.tld' (bind-zones/mydomain.tld.db) reloaded
   1 domain for which we are master needs notifications
   Exiting because communicator thread died with STL error: Parsing record content (try 'pdnsutil check-zone'): while parsing IP address, expected digits at position 0 in '"1.2.3.4"'
   

Expected behaviour

Log error/warning about zone, but keep running for other valid zones loaded.

Actual behaviour

Complete server exiting.

Other information

I think it's very similar to #7493, but the bug appears only fixed for SOA record parser errors?

If I have another example error, in an MX record (data, again with quotes that are wrong: "10 myhost"), instead of an A record I get another issue on zone transfer (but not a full crash):

TCP Connection Thread died because of STL error: Parsing record content (try 'pdnsutil check-zone'): expected digits at position 0 in '"10 myhost".mydomain.tld'

^{I noticed this when I accidentally added quotes to records' data, not just the ones of type TXT.}

[omoerbeek]

Unable to reproduce using 4.1.8 (and master branch) on my development system.
Using 4.1.8 the broken record do produce a log entry when i retrieve them:

Apr 03 10:40:36 Exception building answer packet for example1.com/A (Parsing record content (try 'pdnsutil check-zone'): while parsing IP address, expected digits at position 0 in '"1.2.3.4"') sending out servfail

I do see an error message on notify:

Apr 03 10:26:33 TCP Connection Thread died because of STL error: Parsing record content (try 'pdnsutil check-zone'): while parsing IP address, expected digits at position 0 in '"1.2.3.4"'

Bu the process keep running in my test environment.
Will try to reproduce on a stretch system.

[gertvdijk]

I can still reliably reproduce this, both 4.1.8 and 4.2.0-rc1. This is the full configuration:

master pdns.conf (10.0.2.5) + entrypoint config see below:

setuid=pdns
setgid=pdns
server-id=ns1.mydomain.tld
version-string=powerdns

launch=bind
distributor-threads=1

bind-config=/etc/powerdns/bindbackend.conf
bind-dnssec-db=/var/lib/powerdns/bind-dnssec/bind-dnssec-db.sqlite3

bind-check-interval=5
slave-cycle-interval=5

master=yes
slave=no
disable-axfr=no
allow-axfr-ips=10.0.2.6,10.0.3.7
allow-notify-from=
only-notify=10.0.2.6,10.0.3.7
also-notify=10.0.2.6,10.0.3.7
default-soa-edit-signed=INCREMENT-WEEKS

master bindbackend.conf:

options {
    directory "bind-zones";
};

zone "mydomain.tld" {
    type master;
    file "mydomain.tld.db";
};

slave pdns.conf + entrypoint config see below:

setuid=pdns
setgid=pdns
server-id=ns2.mydomain.tld
version-string=powerdns

launch=bind
distributor-threads=1

bind-config=/etc/powerdns/bindbackend.conf
bind-dnssec-db=/var/lib/powerdns/bind-dnssec/bind-dnssec-db.sqlite3

master=no
slave=yes
disable-axfr=yes

allow-notify-from=10.0.2.5

On master, changing zone line

ns1                                     IN  A        123.45.67.89

to

ns1                                     IN  A        "123.45.67.89"

And this lives in a Docker environment with entrypoint

/usr/local/sbin/pdns-authoritative-entrypoint --guardian=no --daemon=no --disable-syslog --write-pid=no

in a "simple" Debian Stretch base image.

Perhaps I have made a configuration error with this that's contributing?
Anyway, if you cannot reproduce it, I could create a docker-compose.yml+Dockerfiles to demonstrate it reliably. Please let me know if that would be helpful.

HTH

[omoerbeek]

OK, can reproduce now (also on my dev system). The key thing is to break the NS record.

[gertvdijk]

OK, can reproduce now (also on my dev system). The key thing is to break the NS record.

Ah, good to hear that! didn't think of that specifically. I just got down to the first A-type record which always happens to be the NS record of course.

[omoerbeek]

(gdb) bt
#0  RecordTextReader::xfrIP (this=0xbb39ed6a750, val=@0xbb38426b9e0: 3755991007) at rcpgenerator.cc:108
#1  0x00000bb100a10d5a in ARecordContent::xfrPacket<RecordTextReader> (this=<optimized out>, conv=..., noDot=false)
    at dnsrecords.cc:90
#2  0x00000bb1009fcf01 in ARecordContent::ARecordContent (this=0xbb38426b9d8, zoneData=...) at dnsrecords.cc:90
#3  0x00000bb1009fcd30 in std::__1::__compressed_pair_elem<ARecordContent, 1, false>::__compressed_pair_elem<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, 0ul> (this=<optimized out>, __args=...)
    at /usr/include/c++/v1/memory:2139
#4  std::__1::__compressed_pair<std::__1::allocator<ARecordContent>, ARecordContent>::__compressed_pair<std::__1::allocator<ARecordContent>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&> (
    this=<optimized out>, __first_args=..., __second_args=..., __pc=...) at /usr/include/c++/v1/memory:2241
#5  std::__1::__shared_ptr_emplace<ARecordContent, std::__1::allocator<ARecordContent> >::__shared_ptr_emplace<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&> (this=0xbb38426b9c0, __args=..., __a=...)
    at /usr/include/c++/v1/memory:3654
#6  std::__1::shared_ptr<ARecordContent>::make_shared<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&> (__args=...) at /usr/include/c++/v1/memory:4313
#7  std::__1::make_shared<ARecordContent, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&> (__args=...) at /usr/include/c++/v1/memory:4692
#8  ARecordContent::make (zonedata=...) at dnsrecords.cc:90
#9  0x00000bb1009f1554 in DNSRecordContent::mastermake (qtype=<optimized out>, qclass=<optimized out>, content=...)
    at dnsparser.cc:145
#10 0x00000bb1009f1b52 in DNSRecord::DNSRecord (this=0xbb39ed6a940, rr=...) at dnsparser.cc:197
#11 0x00000bb1009db07a in DNSBackend::get (this=0xbb3ca188c00, dzr=...) at dnsbackend.cc:294
#12 0x00000bb100bd70ef in UeberBackend::handle::get (this=<optimized out>, r=...) at ueberbackend.cc:672
#13 UeberBackend::get (this=0xbb39ed6ad20, rr=...) at ueberbackend.cc:617
#14 0x00000bb100b48f2f in FindNS::lookup (this=<optimized out>, name=..., b=0xbb39ed6ad20) at ./communicator.hh:267
#15 0x00000bb100b45a16 in CommunicatorClass::queueNotifyDomain (this=0xbb100cbe448 <Communicator>, di=..., B=0xbb39ed6ad20)

[omoerbeek]

This can also happen on DB backends.

[gertvdijk]

This can also happen on DB backends.

Oh, yikes. If not properly validated before entering the database... boom. Security/DoS issue too? 😨