FTL v5.8.1

Pi-hole FTL hotfix release

• Don't try to terminate threads when we never launched them #1115
• Give threads a bit more time to shut down properly #1112
• Fix database update to version 7 reporting error #1108

FTL v5.8

New

• Automate IP blocking mode #965
  Until now, FTL's Full IP and IP (IPv6 NODATA) blocking modes sourced the IP to deliver on a blocked domain from the setupVars.conf values IPV4_ADDRESS and IPV6_ADDRESS. This is, however, quite a limitation, especially if the device running Pi-hole has more than one interface.
  This PR implements an automated IP blocking. Instead of reading the two addresses from setupVars.conf, we now determine the address of the interface a query arrived on. We then use this IP address in the blocked reply. This does not only reduce maintenance (IPV4_ADDRESS and IPV6_ADDRESS can now be removed from setupVars.conf) but also localizes blocked queries.
• Simplify handling of .lua and .db files #1086
  • pihole-FTL something.lua
    automaticlaly launches the embedded LUA engine
  • pihole-FTL something.db
    behaves the same way as sqlite3 something.db
  • Similar things like
    pihole-FTL something.db "SELECT * FROM abc;"
    are possible as well
• Update embedded dnsmasq to v2.85 #1071
  CHANGELOG:
  • Fix problem with DNS retries in 2.83/2.84.
    The new logic in 2.83/2.84 which merges distinct requests for the same domain causes problems with clients which do retries as distinct requests (differing IDs and/or source ports.) The retries just get piggy-backed on the first, failed, request.
    The logic is now changed so that distinct requests for repeated queries still get merged into a single ID/source port, but they now always trigger a re-try upstream.
  • Avoid treating a dhcp-host which has an IPv6 address as eligible for use with DHCPv4 on the grounds that it has no address, and vice-versa.
  • Add dynamic-host option
    A and AAAA records which take their network part from the network of a local interface. Useful for routers with dynamically prefixes.
  • Teach bogus-nxdomain and ignore-address to take an IPv4 subnet.
  • Use random source ports where possible if source addresses/interfaces in use. CVE-2021-3448 applies.
    It's possible to specify the source address or interface to be used when contacting upstream name servers: server=8.8.8.8@1.2.3.4 or server=8.8.8.8@1.2.3.4#66 or server=8.8.8.8@eth0, and all of these have, until now, used a single socket, bound to a fixed port. This was originally done to allow an error (non-existent interface, or non-local address) to be detected at start-up. This means that any upstream servers specified in such a way don't use random source ports, and are more susceptible to cache-poisoning attacks.
    We now use random ports where possible, even when the source is specified, so server=8.8.8.8@1.2.3.4 or server=8.8.8.8@eth0 will use random source ports. server=8.8.8.8@1.2.3.4#66 or any use of query-port will use the explicitly configured port, and should only be done with understanding of the security implications. Note that this change changes non-existing interface, or non-local source address errors from fatal to run-time. The error will be logged and communication with the server not possible.
  • Change the method of allocation of random source ports for DNS. Previously, without min-port or max-port configured, dnsmasq would default to the compiled in defaults for those, which are 1024 and 65535. Now, when neither are configured, it defaults instead to the kernel's ephemeral port range, which is typically 32768 to 60999 on Linux systems. This change eliminates the possibility that dnsmasq may be using a registered port > 1024 when a long-running daemon starts up and wishes to claim it. This change does likely slightly reduce the number of random ports and therefore the protection from reply spoofing. The older behaviour can be restored using the min-port and max-port config switches should that be a concern.
  • Scale the size of the DNS random-port pool based on the value of the dns-forward-max configuration.
  • TFTP tweak: Check sender of all received packets, as specified in RFC 1350 para 4.

Tweaks

• Add more regex warnings to message table #1092
• Circle CI: skip uploading build artifacts on forks #1093 (thanks @bershanskiy)
• Update SQLite to 3.35.4 #1083 #1089 #1097
• Various enhancements and a few memory-leak fixes #1084
• Resize shared memory only when locking #1072
  This is not really a functional change, however, it makes the code more read- and understandable in some places.
• Escape DHCP options if necessary #1070

Fixes

• Use MAXLOGAGE to control which queries get deleted by GC #1102
• Fix possible infinite loop when Pi-hole ecosystem is not present #1098
• Ensure FTL can be compiled from static tarballs #1091
• Use properly-sized buffer for format_time() #1088 (thanks @bershanskiy)
• Fix pihole-FTL test not terminating properly (noticed in a docker environment) #1067
• Fix incorrect "FATAL: Trying to access upstream ID -1" warning in the logs #1061

Pi-hole FTL v5.7

New

• Add per-client rate-limiting #1052

Tweaks

• Retain pseudoheader in blocked queries #1058
• Allow dhcp-discover to measure the actual lease time of the DHCP server #1055
• Prevent crashing when FTL is started multiple times #1054
• Add output of how much memory in /dev/shm is used by FTL itself #1053

Fixes

• Fix inconsistent counters on CNAME or exernally blocked queries #1057
• Fixes incorrect "Found unknown status 14 in long term database" warning #1048
• Fix incorrect "FATAL" error message during garbage collection #1047

Pi-hole FTL v5.6

Mainly, this release updates the embedded dnsmasq to v2.84
It also solves two cosmetic issues in the Query Log.

According to Simon Kelley, it looks like if two queries are combined (because they ask the same question) then dnsmasq can get confused when it comes to return the answer, the reply to the second query can be sent via the socket that the first one arrived on. That's normally OK, but if the first query arrives via IPv4 and the second via IPv6, for instance, then the bug is triggered.

In addition, FTL can get confused in a similar way because dnsmasq-v2.83+ forwards multiple queries to the same destination once and stores the other queries as duplicates. They do receive the answer later on, however, this is usually not logged (when log-queries=extra is enabled, there will be a warning about the duplicate). Commit 6bb025f in this PR handles such duplicates and introduces a new reply type 14 = "already forwarded" and fixes pi-hole/web#1713

Pi-hole FTL v5.5.1

This is a follow-up release on FTL v5.5 fixing an incorrect error message in the log that can quickly let your log grow out of control under certain circumstances.

---

Pi-hole FTL v5.5

New

• Update to dnsmasq v2.83

Fixes and Tweaks

• Fix for Query Log filtering by upstream server and memory optimizations #1032 (@DL6ER)
• Config option MAXDBDAYS improvements #1033

Pi-hole FTL v5.5

New

• Update to dnsmasq v2.83

Fixes and Tweaks

• Fix for Query Log filtering by upstream server and memory optimizations #1032 (@DL6ER)
• Config option MAXDBDAYS improvements #1033

Pihole FTL v5.4

New

• Implement support for displaying exact type instead of just OTHER #1013
  

• Add new query types SVCB and HTTPS #1010 (@DL6ER)
  These queries where shown as OTHER before. Now they can be distinguished on the query log pages and are included as separate items in the daily statistics.

• Detect and handle interface changes of clients with the same IP #999 (@DL6ER)

• Update SQLite3 to 3.34.0 #1006 (@DL6ER)
  We also expose a full-fledged SQLite3 shell interface through pihole-FTL sqlite3

Fixes

• Fix for errno not being set by posix_fallocate() #1023 (@DL6ER)
• Check for validity of iface pointer before dereferencing it #1018 (@DL6ER)
• Don't show retried queries when filtering for blocked queries #1022 (@Daxtorim)
• Do not record NXDOMAIN from DNS cache as "regex blocked" #1014 (@DL6ER)
• Added missing NS query type to getQueryTypes() #1007 (@Daxtorim)
• Do not return regex idx if privacy level > 0 #1016 (@DL6ER)

Pi-hole FTL v5.3.4

Fix

• Do not sync after executing regular expression on a domain #1001

Pi-hole FTL v5.3.3

New

• Immediately resolve host names of clients and upstream destinations #996
• Add UNKNOWN option for REFRESH_HOSTNAME #985
• Add lua and luac drop-in support #968
• Show BOOTP server and file strings used by TFTP in dhcp-discover #963

Tweaks

• Add DEBUG_EXTRA flag #994
• Improve compatibility with old (ancient) kernels #984 #969
• Add more debugging output for domain reloading (on receipt of SIGHUP) #970
• Implement interrupt-safe systemcalls #969

Fixes

• Escape spaces by ~ in domains and hostnames to fix API issues with such names #997
• Fix CNAME query blocking mode #995
• Fix crashing forks when regex structure changed #982
• Fix freeing regex pointers to free the global objects #971
• Fix possible crash in parse_FTLconf() #967
• Update dnsmasq version to 2.82 #964

Pi-hole FTL v5.3.2

This hotfix addresses a few minor issues found in the most recent release:

• Use case-insensitive comparison for MAC addresses #952
• Always try to resolve hostnames at least once #957
• Tweak code to restore compatibility with Gentoo gcc 10.2.0-r3 #954
• Fix for numerous hourly PTR requests seen in network with many (dead) IPv6 addresses (new config option REFRESH_HOSTNAMES) #953
• Use case-insensitive comparison for MAC addresses #952
• Fix compatibility with gcc version 10 (thanks @DDoSolitary) 4492562 + d61ddc3
• Do not warn about retried queries (query status 12 and 13) during history import 5709ce3
• Catch all real-time signals and handle them appropriately instead of terminating unexpectedly a8266a7
  This fixes unexpected terminations of FTL when requesting to re-resolve host names, ARP processing or alias-client reloading from the database.
• Check for memory allocation errors in parse_FTLconf() b04926a
  This should hopefully fix #947
• Respect RESOLVE_IPV4 and RESOLVE_IPv6 everywhere 267eb95