Ublock 1.41.2 update ignores 'my filters'

[reaper666735]

Prerequisites

• I verified that this is not a filter issue (MUST be reported at filter issue tracker)
• This is not a support issue or a question
• I performed a cursory search of the issue tracker to avoid opening a duplicate issue
• The issue is not present after wholly disabling uBlock Origin ("uBO") in the browser
• I checked the documentation to understand that the issue I report is not a normal behavior

I tried to reproduce the issue when...

• uBO is the only extension
• uBO with default lists/settings
• using a new, unmodified browser profile

Description

since the latest update items i choose in the element picker mode (who end up in my filter list) are not being hidden anymore, reverting back to 1.40.8 fixes this issue

A specific URL where the issue occurs

https://outlook.live.com/mail/inbox

Steps to Reproduce

1. Use element picker mode
2. Add [ outlook.live.com##.f1vx9l62.f124akge.f1l02sjl.f1ctqxl6.f122n59.f22iagw.___lluazi0 ] and or [ outlook.live.com##._1elFdDtE02urszo_h9-MJw.full ] to the list (it's just a sidebar and banner frame)
3. After reload or browser restart they are back

Expected behavior

well i expected it to be hidden even after reload :p

Actual behavior

it seems to stay as long as you don't leave the page, otherwise it seems to not load from my filter list

uBlock Origin version

1.41.2

Browser name and version

Waterfox (G4.0.7)

Operating System and version

win 10 21h2

[gorhill]

• I verified that this is not a filter issue (MUST be reported at filter issue tracker)

How did you verify this?

[g0blin-NN]

@reaper666735 did you change user agent? return native and look.

[Nojuuu]

Are you sure you didn't use element zapper instead?

[reaper666735]

@g0blin-NN did not change anything except the update'd UB lol

@Nojuuu oh i'm very sure i used the "Element picker" ....

[gorhill]

Did you check that the element might have a randomly generated class at each reload? You need to make the case you know the issue is not a filter one, hence why I ask how did you rule out it's not a filter issue. Without this, the issue will be closed as invalid.

---

Also, "Waterfox G4.0.7" is not a supported browser, I have no idea on what version of Firefox it's based on and neither whether there are changes in it which breaks uBO.

[gorhill]

I use Firefox stable 97 and I could pick and remove the sidebar on the right using the filter:

outlook.live.com##._1dBL6mV6xRANqjh_Z8BjN6 > ._1fti_QgAzqGWPGlqh_FSvI

No issue upon reload of the page -- using default settings/lists.

[reaper666735]

it's based on Gecko from Firefox, release notes say it's on version 91.6 of that engine

[reaper666735]

also as i stated when i revert back to 1.40.8 i have no more issue ?, if it really was the random generated class as you suggested

[gorhill]

as i stated when i revert back to 1.40.8 i have no more issue ?

As stated, no issue with Firefox stable, so try with Firefox stable, Waterfox is not officially supported.

[gwarser]

G4.0.6 (64-bit) works fine here (does not update to .7). I tried hiding some stuff on YT - it works. Promoted posts on Reddit hidden too.

[anew742]

I'm having the same issue on both Firefox and Waterfox. My filters no longer work after updating uBO, but the default filter lists do still work.
EDIT: Downgrading to 1.40.8 fixes the issue

[gwarser]

Is "My filters" selected on "Filter lists" tab and shows how many filters are used?

[anew742]

Is "My filters" selected on "Filter lists" tab and shows how many filters are used?

Yes, both are true. But the weird part is that after updating back to 1.41.2, my filters now work and persist even after a restart. So whatever the problem is, it's fixed by downgrading then re-updating. No clue how or why that happens, but it works now so ¯_(ツ)_/¯

EDIT: now it's back to not working, so I'm sticking with the older version for now

[reaper666735]

in a weird way i'm happy i'm not the only one to report it now lol, i was feeling like it just had to be me ......

[Gamer60446]

I have the same issue with Firefox. In addition to custom filters not working, promoted content on Reddit and some YouTube ads will also start appearing when the bug is triggered, at least for me. I noticed that if I purge all caches and then update all filters, my custom filters will start working and all ads should be gone. However, they will randomly stop working again after some time and I don't know exactly what triggers it. I can confirm that downgrading to 1.40.8 fixes the issue.

[one-way-ticket]

FF 78.15 ESR on Windows 7 Pro SP1 x64.
UA: "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0"

After update uBO 1.40.8 to 1.41.2, the floating glitch also occurs - "My Rules" are ignored.
There are no records of the "My Rules" in Logger.
Temporarily fix - in settings turn off checkbox "My Rules" -> click "Apply Changes" -> turn on checkbox "My Rules" -> click "Apply Changes".
After that, "My Rules" begin to work and in the logger there are appropriate records.

Rollback to version 1.40.8 completely solves the problem.

[gorhill]

There are no records of the "My Rules" in Logger.

I gather you mean "My filters".

From what I read, this suggests a specific cosmetic filter is being problematic -- quite possibly a highly generic one. It would be useful to share your custom filters for those suffering the issue.

In uBO 1.41.2, highly generic cosmetic filters are wrapped into :is(...):not(html,body), where ... is the highly generic cosmetic filters, and this is done so as to fix #1692. Now I wonder whether a particular cosmetic filter is incompatible for being wrapped into :is(...):not(html,body) -- which by the way happens only for Firefox 84+.

So bottom line, I need information about your custom filters if this issue is to go forward.

[Nojuuu]

@gorhill DandelionSprout/adfilt#63 (comment) maybe related?

[uBlock-user]

@Nojuuu very bad regression, breaks uBO(my filters goes 0 out of 0) and blocks all network connections to any site alltogether(unable to browse any website). This warrants an emergency fix.

@gorhill

[gorhill]

This is not a regression, this has always been in there, it just needed a argument-less removeparam filter to expose the problem.

---

Actually, the conditions that had to be all reunited for the error to manifest:

• A block filter (exception filters are fine); AND
• An untokenizable pattern; AND
• An empty (catch-all) removeparam

I don't see any such instance of filter in either uBO filter lists, nor "Actually Legitimate URL Shortener Tool", nor "AdGuard URL Tracking filter". So although a bad issue warranting an emergency fix, it's not affecting majority of users.

---

Also, probably unrelated to the actual issue filed here.

[krystian3w]

Maybe poor filterlist used @@*$ehide and user don't saw this (no check commits or use logger).

Also may have enabled A/B Test with very unstable classes in dom tree/nodes - then filter no longer match after refresh.

[u-RraaLL]

@krystian3w that doesn't explain why the problem disappears when a user drops down to 1.40.8.

There was another instance on reddit just now: https://www.reddit.com/r/uBlockOrigin/comments/srmcy6/paradox_interactives_featured_content_antifeature/

[fluffymormegil]

Hi, I'm the reddit poster who raised the issue the user above just linked to.

The entirety of my ruleset is:

! 11/06/2019 https://forum.paradoxplaza.com
forum.paradoxplaza.com###\#pdx-cookie > .pdx-cookie--toggle.pdx-cookie-bottom-bar__toggle

! nuke paradox's stupid space-wasting featured content area
forum.paradoxplaza.com##.thfeature_featuredGrid
forum.paradoxplaza.com##h3.block-header

and I'm using Firefox ESR 78.12.0esr (64-bit) on Linux.

[krystian3w]

1.40.8 may also fail if page have enabled A/B Test to use new names after refresh.

[krystian3w]

-cookie

IMO paradox plaza need accept cookies instead only hide or remove by :remove().

[u-RraaLL]

IMO paradox plaza need accept cookies instead only hide or remove by :remove().

The issue was with the featured grid filter, though.

[krystian3w]

Better teach about GDPR breakges.

[fluffymormegil]

Yeah, the thing to permanently suppress their intrusive cookie button (the little bottom-of-window widget appears even if you're logged in and have accepted cookies!) worked fine.

[reaper666735]

and once again i'm on Waterfox but it could be somehow it might be related to that version more then we might realise

[gorhill]

i thought the latest update upped the min. to 68 only ?

Yes, but there seems to be an unforeseen issue, which is what we are trying to narrow.

[gorhill]

Looking at the code, there is a bit of a race condition in Firefox-based browsers, whereas uBO assumes the browser version to be 91, while waiting for the real version number to come through a promise. However I find it difficult that filter lists would be compiled before the promise is fulfilled, as it is one of the first thing uBO does at launch.

At this point I am contemplating seriously to revert gorhill/uBlock@7c8aec2 -- I have a strong suspicion the issue might be related to this -- and bring back the commit in a subsequent dev cycle with more testing being done (please use the dev build if you want such issues to be found before stable release).

---

I need to correct myself here -- this has nothing to do with compiling, but it's rather at injection time of highly generic cosmetic filters. In any case, it's even more unlikely that uBO wouldn't know the real browser version at that point.

[reaper666735]

i guess we will just have to wait and see (test) if that will fix it :p

[gorhill]

A commit which might be at the root of the issue here has been reverted, and the change is available in 1.41.5b1 or above. I will wait for feedback as to whether this makes a difference or not. If the issue does not appear to be there after installing 1.41.5b1+, probably best to use that dev build for a while before immediately concluding it's fixed.

[reaper666735]

just doing a quick test on several sites i have filters on does seem to show 1.41.5b1 to be working as it should again :), ofc i will keep testing wit this build and keep you informed should anything happen ;)

[one-way-ticket]

A commit which might be at the root of the issue here has been reverted, and the change is available in 1.41.5b1 or above. I will wait for feedback as to whether this makes a difference or not.

Install version 1.41.5b1 on FF 78.15 ESR.
The problem no longer arises. "My filters" works.
Thanks!

[coldimmune]

All my mobile devices use Firefox Android 68 and experienced the same issue -- the 1.41.5b1 fixed it.

It may be outdated and unsupported, but in my defense, Fenix should be classified as beneath us. Just a dogshit browser.

_Edit: Just came back here to thank you profusely for your work on Ublock Origin and having a fix for such an obscure issue on EOL browsers in only a few days. And for something that's free. My work can't do that for customers.

It's just such a Quality of Life improvement with something I use a hundred times a day. And you never screw around with the GUI or make major, powertripping changes. Like, the only step forward from here would be in 50 years when we're using augmented bionic eyes and you're still around, creating a tool to block affronts from our real vision._

[Candlemass]

I was having the same problem on Android mobile 68.x (Dont use the shitty new fenix verson). Reverting to 1.40.8 solved the problem but didnt have dark mode so I settled on 1.14.6 since it`s the latest stable version with that feature and so far seems to be resolved. I will report back if this changes for any reason.

Its more apparent than ever why a lot of users dont upgrade once they find something that works. Far too much breakage occurring these days of essential features in exchange of implementing new ones...

[HoldOffHunger]

SOLVED.

--Remove uBlock Origin
--Install uBlock Origin v. 1.18.17.16 from here https://www.crx4chrome.com/crx/149207/

I like how this problem is super easy to reproduce and demonstrate. But I like a lot better having the solution!

[e-t-l]

I was having the same issue on some sites, including Reddit, and updating to v1.44.5b6 did fix the issue with Element Picker-created filters not persisting across page refreshes.

@gorhill You might like to know, however, that after updating UBO I encountered a separate issue: ALL elements of Reddit were blocked (aka when the page finished loading, it was blank white) until I a) disabled cosmetic filtering on Reddit (not ideal), or b) checked Ignore Generic Cosmetic Filters in UBO settings (still problematic, because I'd like generic filters to apply to most sites).

[gorhill]

Just force a purge all cache then update now to ensure all your filters are recompiled with using the latest fixes.

[e-t-l]

@gorhill Now this is strange. I'm on v1.44.5b6, I purged and updated caches, but my Element Picker-created filter on Reddit is still not working! I'm no longer encountering a totally blank page, though. When I disable cosmetic filtering on Reddit, it still doesn't work (aka the element is not blocked). However, when I check Ignore Generic Cosmetic Filters in UBO settings, suddenly my filter does work, and the annoying element is blocked! I have no idea how to diagnose this further.

[gorhill]

Share troubleshooting information, and which filter you say is not working when "Ignore Generic Cosmetic Filters" is not checked.

[e-t-l]

Affected filter: www.reddit.com##.kEQVd8aneM1tVkcIKUyDT

Troubleshooting information:

uBlock Origin: 1.44.5b8
Firefox: 105
filterset (summary): 
  network: 388117
  cosmetic: 164875
  scriptlet: 39004
  html: 779
listset (total-discarded, last-updated): 
  added: 
    https://gitcdn.xyz/repo/NanoAdblocker/NanoFilters/master/NanoFilters/NanoBase.txt: 0-0, 19h.13m
    https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter.txt: 155490-2442, 19h.13m
    https://raw.githubusercontent.com/AFNIL-AntiFakeNewsInternationalList/AFNIL/master/AFNIL-ABP-uBO.txt: 5794-1562, 19h.13m
    https://raw.githubusercontent.com/NanoAdblocker/NanoFilters/master/NanoMirror/NanoDefender.txt: 2625-20, 19h.13m
    https://raw.githubusercontent.com/Spam404/lists/master/adblock-list.txt: 7252-143, 19h.13m
    https://raw.githubusercontent.com/austinheap/sophos-xg-block-lists/master/add-dead.txt: 20-0, 19h.13m
    https://raw.githubusercontent.com/austinheap/sophos-xg-block-lists/master/add-risk.txt: 2567-5, 19h.13m
    https://raw.githubusercontent.com/austinheap/sophos-xg-block-lists/master/add-spam.txt: 73-0, 19h.13m
    https://raw.githubusercontent.com/deathbybandaid/piholeparser/master/Subscribable-Lists/ParsedBlacklists/ABP-Clickbait-Blocklist.txt: 12-0, 19h.13m
    https://raw.githubusercontent.com/deathbybandaid/piholeparser/master/Subscribable-Lists/ParsedBlacklists/BreakingTheNews.txt: 1-0, 19h.13m
    https://raw.githubusercontent.com/marktron/fakenews/master/fakenews: 2197-2, 19h.13m
    https://raw.githubusercontent.com/ryanbr/fanboy-adblock/master/fake-news.txt: 584-242, 19h.13m
    https://raw.githubusercontent.com/yourduskquibbles/webannoyances/master/filters/newsletter_filters.txt: 2027-2, 19h.13m
    https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt: 2735-48, 19h.13m
    https://winhelp2002.mvps.org/hosts.txt: 8730-738, 19h.13m
    https://www.fanboy.co.nz/enhancedstats.txt: 2989-67, 19h.13m
    adguard-generic: 70138-5615, 19h.14m
    adguard-spyware: 26705-1614, 19h.14m
    curben-phishing: 75328-5, 19h.14m
    adguard-annoyance: 59684-5440, 19h.14m
    [8 lists not shown]: [too many]
  default: 
    user-filters: 49-4, never
    ublock-filters: 33694-379, 19h.15m
    ublock-badware: 4324-6, 19h.14m
    ublock-privacy: 273-1, 19h.14m
    ublock-abuse: 76-0, 19h.14m
    ublock-unbreak: 1875-2, 19h.14m
    easylist: 62254-18655, 19h.14m
    easyprivacy: 30638-14428, 19h.14m
    urlhaus-1: 7155-0, 19h.14m
    plowe-0: 3670-1800, 19h.13m
filterset (user): [array of 49 redacted]
trustedset: 
  added: [array of 73 redacted]
  removed: 
    edge-scheme
switchRuleset: 
  added: [array of 30 redacted]
hostRuleset: 
  added: [array of 15 redacted]
modifiedUserSettings: 
  advancedUserEnabled: true
  cloudStorageEnabled: true
  contextMenuEnabled: false
  ignoreGenericCosmeticFilters: true
  webrtcIPAddressHidden: true
modifiedHiddenSettings: 
  autoUpdateAssetFetchPeriod: 120
  autoUpdatePeriod: 7
  selfieAfter: 11
  userResourcesLocation: [redacted]
supportStats: 
  allReadyAfter: 68901 ms (selfie)
  maxAssetCacheWait: 19384 ms

[gorhill]

You have way too many filter lists for me to spend time on trying to reproduce. I suggest to disable a few filter lists at a time and see if this solve your issue. If so, please report which filter list seems to trigger the issue. Also, you are still subscribed to filter lists which have been abandoned since a long time, you need to clean up your config.

[e-t-l]

please report which filter list seems to trigger the issue

Can do. Sorry, didn't mean to act like I was dumping a huge task on you. You asked for troubleshooting info so I copied and pasted. I don't really know what I'm looking for - But why would another filter from a filter list cause mine not to work? Is there some order of precedence I'm not understanding (and if so, shouldn't it be reversed? Seems like the user's My Filters should override anything in an imported list, shouldn't it?)

still subscribed to filter lists which have been abandoned since a long time

How do I tell which ones are abandoned? And does abandoned mean none of the filters are relevant anymore?

[gorhill]

How do I tell which ones are abandoned?

Open the list in a tab. If it does not load, remove it. If it loads, check the last time it was updated. If it's more than a year, it's useless and may interfere negatively. When it comes to filter lists, more is not better. Some of these abandoned lists might be at the root of the long load time issue I can see in your troubleshooting information. Probably best to reset to default lists and add lists only when it actually do solve an actual issue you are trying to solve. Maybe it's the long load issue which cause you to misdiagnose a filter issue, it took over 1 minute for uBO to be ready (allReadyAfter).

[e-t-l]

Thanks.

Open the list in a tab. If it does not load, remove it. If it loads, check the last time it was updated.

Do I need to do this with ALL lists (including the ones uBlock ships with) or only for the lists I have added under the Custom heading?

Probably best to reset to default lists

Likewise, does "default" mean any/all of those non-Custom lists, or only the ones UBO has enabled upon fresh install?

[MasterKia]

or only for the lists I have added under the Custom heading?

This.

or only the ones UBO has enabled upon fresh install?

This.
You can use the "Reset to default settings" option in the first tab of options.

[gorhill]

@e-t-l Reset to default settings will cause your custom filters in My filters/My rules/Trusted sites to be wiped out so you may want to export them first.

[gorhill]

By the way, I would still like to know what list is causing your cosmetic filter to stop working, there has been changes in this area in the latest dev builds, and this might just help me fix a difficult to reproduce issue. Narrowing down to which list would help a lot. If it's not a specific list, it could be your own cosmetic filter, see if commenting them out make the affected filter working again.

[e-t-l]

Ah. Sorry. I had gone thru the lists and removed the ones that didn't load or were over a year old, but I didn't stop to refresh reddit and see if it was affected each time. At this point, my cosmetic filter is working again.

I only did this with Custom Lists and have not disabled any of the UBO-included lists, so we can safely say none of those are the issue.

I think I could re-add them incrementally, since I have their names in the troubleshooting output in my previous comment, and then test reddit to see if I can narrow it down.