Add support for IDNs in HOSTS files.

[DL6ER]

By submitting this pull request, I confirm the following (please check boxes, eg [X]) Failure to fill the template will close your PR:

Please submit all pull requests against the development branch. Failure to do so will delay or deny your request

• I have read and understood the contributors guide.
• I have checked that another pull request for this purpose does not exist.
• I have considered, and confirmed that this submission will be valuable to others.
• I accept that this submission may not be used, and the pull request closed at the will of the maintainer.
• I give this submission freely, and claim no ownership to its content.

How familiar are you with the codebase?:

10

---

Use libidn. This will improve support for internationalized domain names (IDNs) by adding an automated conversion from locale's encoding to a legit ASCII domain name string. We do not use libidn2 as there are some bug reports and we found issues trying to compile statically against libidn2. Additionally, there seems to be no clear advantage compared to using libidn.

For this to work, we need to add libidn11-dev to our CI containers. Until then, this PR stays marked as [WIP]. Note that we have not added this before v5.0 as we were still loading the domains from HOSTS files. The automated conversion would have slowed down the initial startup of FTL by about one order of magnitude which would have been unacceptable on low-end hardware (one minute or more). It was always possible to enable IDN when compiling FTL from source as we described what needs to be done in the Makefile.

[dschaper]

This is waiting still on pi-hole/docker-base-images#13 ?

[DL6ER]

Yes. The CI jobs need to be restarted when the containers can build IDN stuff.

[dschaper]

Okay, I promise this will be done this weekend.

[dschaper]

Okay, I promise this will be done this weekend.

Well, had a few distractions this weekend so give me another day please.

[dschaper]

Containers should all be updated. Failed build on arm looks like it's the right result.

[dschaper]

Winner winner Chicken Dinner.

[DL6ER]

Note that this was intentionally sent to development as this does not need to be shipped with v5.0. More experiments are needed and international domain names should not be all that common in HOSTS files.

[pralor-bot]

This pull request has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/pi-hole-5-1-released/35577/1