Something is wrong with NextDNS processing Lightswitch05 - Ads & Tracking blocklist.

[crssi]

STR

1. Add Lightswitch05 - Ads & Tracking in the privacy section
2. Observe the blocklist's source from the NextDNS definition... there is no gstatic.com or fonts.gstatic.com defined.
3. Visit the fonts.gstatic.com and see in the NextDNS log saying its being blocked by Lightswitch05 - Ads & Tracking blocklist... which simply is not true.

What has happened here?

The problem started to happen from at about 4.8.2020 17:00 CET. (<- I hope I calculated the time right... but if not, there around +/- 2 hours).

Cheers

[romaincointepas]

We now treat all names (QNAMEs and CNAMEs) equally.

That list contains gstaticadssl.l.google.com, and that's a CNAME of fonts.gstatic.com, so it's blocked. The NextDNS logs should show that it actually matched on a CNAME.

Anyone using that list on Pihole/Adguard home should have the same issue, could you suggest them to exclude gstaticadssl.l.google.com?

[romaincointepas]

As a reference, all those blocklists are blocking gstaticadssl.l.google.com:

[crssi]

We now treat all names (QNAMEs and CNAMEs) equally.

Didn't know. This behavior is then new from yesterday on?

The NextDNS logs should show that it actually matched on a CNAME.

Do you have any ETA?
When done... can it show which CNAME is the cause, pretty please? 😄 ❤️

[crssi]

See the response lightswitch05/hosts#213 (comment)

I really don't know what is best for general use,
For sure the google fonts tracking is a big problem on Internet, but fonts not working also happens to be my problems by others in my house. 😄

I will allowlist the fonts.gstatic.com for now and sleep over.

Cheers

[badmojr]

We now treat all names (QNAMEs and CNAMEs) equally.

That list contains gstaticadssl.l.google.com, and that's a CNAME of fonts.gstatic.com, so it's blocked. The NextDNS logs should show that it actually matched on a CNAME.

Anyone using that list on Pihole/Adguard home should have the same issue, could you suggest them to exclude gstaticadssl.l.google.com?

The nature of CNAME blocking tends to be aggressive, so this should be optional...

e.g:- Since win 10 uses the domain www.msftconnecttest.com to check for connectivity, it reports that “No Internet, secured” ‘coz I’ve blocked msedge.net (the CNAME for www.msftconnecttest.com).

[crssi]

I like CNAME blocking option, but I do agree that it should be opt-in.
This came unannounced and it took quite an effort to see where is the problem.

[romaincointepas]

The reason we switched to treating all names equally is that many platforms do something called CNAME chasing, where they query each CNAME from the answer independently (so CNAMEs ends up appearing as QNAMEs). As far as we know, unbound, macOS and iOS do that.

We wanted to be ISO on all platforms, and deal with the few exceptions on a case by case basis. As both Pihole and adguard home added the CNAME blocking feature recently, blocklists maintainers are/should be adapting.

We will releasing a fix today so when a domain is blocked because of one of its CNAMEs, the CNAME is shown like this:

Please report any false positives you encounter following this change, it's hard to figure all of them out without the help of our community.

[crssi]

Cool. Thank you.
Cheers

[crssi]

Interestingly when I place fonts.gstatic.com to allowlist then in the logs doesn't show it with the expected green bar (sign of allowlisting) on the left.
Is this intentionally?

[badmojr]

Personally, I think this is a recipe for disaster.
Time is gonna tell.. so let’s wait!

[badmojr]

Here’s an example of an ad company posing as a legit CDN.
https://datacadamia.com/marketing/ad/ezoic

In order to access any site that uses such as a CDN, one has to allow g.ezoic.net (CNAME) i.e an adserver/tracker.
Hence, make it damn opt-in or atleast a faqin’ button to turn it off.

[romaincointepas]

@badmojr this behavior is already happening for all stub resolvers chasing CNAMEs (mDNS responder for iOS and macOS, and unbound, as far as we know).

Making it opt-in, aside from not catching CNAMEs that should be caught (like disguised trackers), would not solve this for this large percentage of users, for whom this is already happening and impossible to turn off.

Looking at the numerous issues related to fonts.gstatic.com on any blocklist repository makes this obvious. Even more now that both Pihole and Adguard now apply blocklists to CNAMEs (even if opt-in), even when the end stub resolvers doesn't.

The only approach we see viable long-term is to treat all names equally and make the blocking behavior ISO everywhere, and (hopefully) blocklists will adapt gracefully. There are already tons of domains not being blocked because they break things, this just potentially removes just a few more. Our findings tell us that most of the time, different subdomains are being used as CNAMEs and direct visible QNAMEs, in that case it's not an issue. For cases like g.ezoic.net it is.

Long story short, this is not a feature call that we made, it's a long-term choice we are taking to hopefully, by making blocking ISO for everyone, decrease the overall false positives.

[badmojr]

Good work on the CNAME indicator!
Atleast that’s a step in the right direction.

[crssi]

CNAME is a pest which must be dealt accordingly and I guess @romaincointepas is right to stand by his opinion and CNAME must be adopted ASAP... it is the future and future is changing constantly.

@romaincointepas:

Please report any false positives you encounter following this change, it's hard to figure all of them out without the help of our community.

I am thinking we should open a new issue to track/report those and close this one?

Cheers

[badmojr]

CNAME is a pest which must be dealt accordingly and I guess @romaincointepas is right to stand by his opinion and CNAME must be adopted ASAP... it is the future and future is changing constantly.

The thing I’ve begun to like about how this was implemented is unlike it’s counterparts, NextDNS’ logs section doesn’t intermix the names (QNAMEs & CNAMEs).

The above & the favicons makes it easy to "debug & fix" when breakages happen.
Good work Team!

[badmojr]

@romaincointepas
On a side note.
Suppose I’ve come across a false positive, say google.com.

As a list maintainer, I remove the above domain from the lists authored by me.

Then, I push an update; only to find out (from the NextDNS’ logs) that the above domain additionally has a CNAME.

Again, the CNAME gets removed & an update is pushed.

The entire process above could be simplied if every entry in the logs that has a CNAME was marked/shown as having so.
..........

Otherwise from now, I manually have to dig for a corresponding CNAME every time a false positive gets submitted/discovered.

Cheers!