Any reasons not to remove built-in ABP filtering and asks users to install µBlock?

[gorhill]

Discuss.

Here are my pros:

• One code base dedicated to ABP-compatible filtering engine.
• Dedicated means it does what it does very well, fast, efficiently, and with many features which are likely to never make it to HTTPSB.
• One code base to take care for an ABP-compatible filtering engine, no more duplicated work.
• Behavior would stay the same, i.e. ABP-filtering engine is separate from matrix filtering engine, which means a separate extension would result in same behavior as how currently HTTPSB behaves.
• Since both are authored by the same contributors, this means they are guarantee to work well together out of the box:
  • uBlock will never redirect or modify headers -- that's left to HTTPSB
  • So guarantee to never conflict
• Users who do not want ABP filtering would not be forced to still suffer the overhead of it in HTTPSB (whatever it might be)
• More focused code base means simpler logic, less bugs, etc
• Less work for me, better use of my time, hence more time to make better stuff.

[ghost]

It's been my secret wish that HTTPSB would basically be without the "Adblock" + premade lists portion, but mine is the block-all-allow-exceptionally use case which I realise is not for everyone.

So consider this a vote for splitting.

[kurtextrem]

👍

[mikhaelkh]

Split them!

[jankkm]

Does this mean that

• Parse and enforce Adblock+ complex filters. and
• Parse and enforce Adblock+ element hiding filters.

and the functionality in the background is removed but everything else will stay as it is?

[ghost]

Am I correct to assume that means µBlock will act as a ABP replacement. And httpsb will act as a no-script replacement for chrome/chromium browsers? I love this idea if that is the case.

[tailHey]

I must say I wouldn't like to see them splitted at all...
For now, I only see three reasons not to split them:

-µBlock not available in Opera Web Store -> You have Opera and you haven't heard of this new addon? Too bad, you won't find it! :S
-HTTPSB is some sort of AIO tool, removing one of the two engines would lessen this feeling. Moreover, I'd loathe to have to install two addons to have the same efficiency as having just one as it is right now!
-Using blocked hosts lists can prevent people who're still learning how to use HTTPSB wisely from doing some mistakes which could lead to ads being displayed ("This addon cannot block ads, let's remove it and go back to Adblock Plus!") or to users being tracked.

Some months ago, I was "dreaming" about a tool close to HTTPSB which would be adapted to beginners, I wasn't really thinking of "just another" adblocker like µBlock (don't take it wrong, I also appreciate his work on it! ^^)...

[ghost]

Interesting idea! Some thoughts/remarks to consider:

• One code base dedicated to ABP-compatible filtering engine.
  And what about the non-ABP ubiquituous lists? They exist in both
  extensions, so there will still be an overlap. They make sense in µBlock
  but I still want them to stay in HTTPSB (be it only for the reason that
  those blacklisted domains would still be visually separated from the
  others). So, if I enable them in HTTPSB - would HTTPSB communicate with
  µBlock to disable them there? At least, a corresponding hint would be
  required, IMHO.
• Dedicated means it does what it does very well, fast, efficiently,
  and with many features which are likely to never make it to HTTPSB.
  Which features are you thinking about?
• One code base to take care for an ABP-compatible filtering engine,
  no more duplicated work.
• Behavior would stay the same, i.e. ABP-filtering engine is separate
  from matrix filtering engine, which means a separate extension would
  result in same behavior as how currently HTTPSB behaves.
  Yes, with the exception of the ubiquituous lists, see above. And I
  expect one problem: If a site doesn't work properly I can find the
  reason rather easily in the statistics tab in HTTPSB. In the future it
  might be necessary quite often to look in both extensions in order
  to find the "culprit". Is this really an advantage for the user?
• Since both are authored by the same contributors, this means they
  are guarantee to work well together out of the box:
  o uBlock will never redirect or modify headers -- that's left to
  HTTPSB
  o So guarantee to never conflict
  On the other hand, if I think about Create Adblock Plus-like rules #357 and Surrogate scripts #359, we might have
  four extensions in the future. I agree with you that we should avoid
  that an extension becomes too bloated, but I think we don't want Chrome
  to become bloated by too many extensions, either ;-)
• More focused code base means simpler logic, less bugs, etc
  Yes, that's a clear advantage from a developer's perspective.

In sum, this idea has its benefits but there are also serious concerns
worth being considered. In any case, I'm curious about the outcome :-)

[jonvuri]

I am one of those who does use ABP filtering and I am most concerned about the added overhead of another separate extension, as well as another button in my toolbar. I'm very averse to both.

[gorhill]

@tlu1024 "Which features are you thinking about?"

Interactively creating filters with the mouse, disabling specific filters without disabling a whole list, having a choice of (say 4) filtering levels (instead of just on/off all) per site, etc.. There are many features specific to a blocker we can think of which would bloat HTTPSB, that would be just too much code and UI to fit in.

To all: many users will never install HTTPSB, too technical, so I do not have choice to keep uBlock going separately (it has already almost twice more users than HTTPSB). The only way at this point to minimize workload is to try to create a core shared by both. Porting some code to fix #360 was not fun. So in the end on my side this is hard work I have to deal with, not feelings. And that is aside being asked to port to Firefox and to add even more features to both. I have no intention whatsoever to make these projects more than a hobby, and as such the benefit to users is that these projects will never be tainted by any other agenda. Is that such a downside to have to install two extensions instead of one (which is already the case anyway for whoever doesn't use HTTPSB or uBlock).

Anyway, for now I will try to create a core which I could import as is in HTTPSB, and see how it goes.

[gorhill]

Anyway, for now I will try to create a core which I could import as is in HTTPSB, and see how it goes.

OK, already issues. What follows is technical but this is the reality of what I have to deal with. uBlock is able to hide blocked element on a page. To implement this, uBlock injects a content script in the web page. The content script works as follow: once the web page is loaded, or once a change occurs on the web page, the content script will ask the main uBlock process a list of all requests which were blocked, so that it can look-up which DOM elements have to be hidden.

What that does mean, is that the main HTTPSB process will also have to be able to send a list of blocked requests when asked. However, for reason of efficiency, HTTPSB does not keep of usable copy of all requests blocked (these are reduced to a two-character token), so it cannot return such a list. This means two options: get rid of HTTPSB code which was written with reducing memory footprint in mind, or to not have element hiding in HTTPSB (issue #355, #298).

See this is the kind of complications I have to deal with when trying to fit a lot of features in one package. That forces me to make choices which have negative side-effects, and which these are choices I would not have to make if it fitting all in one was not automatically perceived as a virtue.

(Actually I am now second-guessing whether it is worth tokenizing the request URLs in HTTPSB.. Sigh)

[ghost]

What is the best way to test out using µblock along side HTTPSB? Should I get µblock and keep using HTTPSB in Block all / allow exceptionally mode?

[gorhill]

What is the best way to test out using µblock along side HTTPSB?

Anyway you want, except that you would have to completely disable ABP filtering in HTTPSB (from the Ubiquitous rules tab) to get a real sense of how it would work as a complement to HTTPSB.

[gorhill]

In the future it might be necessary quite often to look in both extensions in order to find the "culprit". Is this really an advantage for the user?

Typically, a user using HTTPSB and uBlock would not enable redundant lists in uBlock. In such case, I expect that each extension's log of blocked requests won't overlap as a rule.

[gorhill]

And what about the non-ABP ubiquituous lists? They exist in both extensions, so there will still be an overlap ... So, if I enable them in HTTPSB - would HTTPSB communicate with µBlock to disable them there?

No, one extension will not disable parts of the other. There is a limit to what any piece of software can do to try to be "convenient", it can easily become annoying (Clippy syndrom). A user installing HTTPSB is a technical user, I expect them to quickly configure their extensions to avoid overlaps -- so for uBlock, that would mean unchecking Peter Lowe's and Malware domain lists. HTTPSB would stop supporting EasyList and such to extract domain names. The way it's currently done doesn't even follow the semantic of the ABP filters, which is to not block the root document of whatever hostname is being filtered, and anyways the count of hostnames used in the matrix is rather low compared to other hosts lists, 1,357 for EasyList + EasyPrivacy.

[gorhill]

@tailHey "µBlock not available in Opera Web Store"

I've just submitted a version complying with their requirements.

[ghost]

Typically, a user using HTTPSB and uBlock would not enable redundant
lists in uBlock. In such case, I expect that each extension log og
blocked requests won't overlap as a rule.

Okay, what I meant: Say, a site contains the script

http://www.google-analytics.com/ga.js

which is blocked in HTTPSB by default as it is in one of the hosts
files. However, even if it is blocked by HTTPSB, there is still an ABP
filter triggered in uBlock (because it doesn't "see" that GA is already
blocked in HTTPSB), namely:

||www.google-analytics.com^

If that website is one of the (rare) sites where you have to whitelist
GA in order to make it work properly, it's not enough to whitelist GA
for the corresponding domain- or site-level scope in HTTPSB - you also
have to remember to disable uBlock for that site after looking in its
statistics tab. This might be obvious in the case of GA but not
necessarily for other cases - and we all know that there are false
positives both in the hosts files and in the ABP filterlists. So this
makes things more complicated for the user, IMHO.

Of course, it would be presumably easier to disable all ubiquituous
lists in HTTPSB and only use the ones in uBlock - in other words: to
leave the blacklisting to uBlock and the greylisting to HTTPSB. But this
would mean that those adservers/trackers would no longer be separately
displayed/hidden from view in the lower part of the matrix (which is a
great feature). And if I enable the lists in both extensions, selective
whitelisting would become difficult, wouldn't it?

[gorhill]

Ok, I won't split (not because I agree with exactly everything above, I just don't have the energy to debate endlessly: I still think it was the way to go to keep the code base sane).

[jonvuri]

I still think it was the way to go to keep the code base sane

I definitely see where you're coming from - could this issue alone be solved by having both of them use a common library?

[gorhill]

could this issue alone be solved by having both of them use a common library?

Looks like there is no other way.

Aside code complication, I just didn't see ABP-filtering fitting with HTTPSB's core. In HTTPSB the ABP-filtering engine is really separate from matrix filtering engine, they do not fit together at all, and if the ABP filtering engine was taken out and installed as another extension, overall results would be just the same, because both engines are connected serially inside HTTPSB.

Add to this that cosmetic filtering is completely unrelated to security/privacy, which I want as core mission of HTTPSB. Whatever little details of "what-if" appeared kind of irrelevant to me when looking at the big picture: HTTPSB security/privacy, uBlock highly efficient mainstream blocker.

So a library I have no choice, but this doesn't mean less work, whenever I make change to the library, I have to test both extensions, and so far I am the only tester before releasing to stores.

[jonvuri]

Yeah, but it seems likely that a lot of people interested in the pure security/privacy part of HTTPSB will also be interested in blocking ads. And aside from the cosmetic filtering, the two aren't that far removed from a user perspective - for the most part, they both involve blocking network requests based on domains. The ABP filtering just tackles it a different way from the more control-oriented matrix style.

Making it a library would also make it very easy to leverage in a Firefox extension without having to make a separate proxy.

Also, I'd be happy to help out with anything I can. Maybe I can help with abstracting ublock into a library.

[gorhill]

Actually the more complicated one to "librarize" are the cosmetic filtering, because it needs separate scripts (two injected scripts, messaging back/forth with library in app). I've started looking at what needs to be done for ABP net filtering, this one is not too difficult. What scares me down the road is feature creep: start to support one thing, then users are going to expect a few more things to make that one thing better. Than whatever you add will lead to more. Etc. Feature creep is an excellent way to destroy a good product. I personally see focused and simpler as a virtue in software. Anyways, for now I will focus to librarize the parts. Moving on.

[jonvuri]

Is there any way that I can help with one or the other libraries (net/cosmetic)? Or would you rather focus on it without distraction?

[gorhill]

Thanks for the offer, I think I am more productive when I just plow without distraction. I wouldn't refuse any offer to port uBlock to Firefox though ;-)

[ghost]

Ok, I won't split (not because I agree with exactly everything above, I just don't have the energy to debate endlessly: I still think it was the way to go to keep the code base sane).

Hm - this makes me almost feel bad.... I really hope that you don't have the impression that I was trying to outtalk you! I presented some issues which I regard as problems from a user's perspective. If you think that you can solve these problems or prove those arguments invalid or nonsense, please do so. I surely don't want you to feel bad, either. Besides, HTTPSB and µBlock are your babies, not ours.

[jonvuri]

I wouldn't refuse any offer to port uBlock to Firefox though ;-)

If you are going to make the individual parts into libraries anyway, I'll wait till that's done, then I'd be happy to based on those.

[ghost]

I think that mental and code sanity for the developer should come first, as the current "I don't want to install another extension" argument seems like a weak reason to void both of those things.

However, I see the appeal in avoiding an endless argument as well. Thanks for putting up with us @gorhill

[Landpaddle]

I use both the matrix filtering engine and ABP-style blocking to remove adverts that are loaded as part of the CSS or as images.

If you can ensure both extensions function together without considerable overhead, you have my backing. I'd rather the code stay sane and maintainable myself.

[ghost]

I think that mental and code sanity for the developer should come first, as the current "I don't want to install another extension" argument seems like a weak reason to void both of those things.
However, I see the appeal in avoiding an endless argument as well. Thanks for putting up with us @gorhill

I can't agree with @maxrmp enough. I've been using HTTPSB with ublock for a while now and I don't see any slow-downs or glitches.

This is how I see it, does noscript on firefox block ads? Perhaps it can if you tweak it enough but it isn't the original purpose of noscript. Most users also use abp in unison on firefox to block both ads and scripts. I see HTTPSB as an alternate to noscript on google-chrome.

I can see the reason why people may be upset about ublock. As for my case I came across this very useful extension and immediately replaced scriptsafe and abp with HTTPSB. So all of the sudden one extension both blocked ads and scripts at once. This was very nice and useful for me.

Though I can see why some users may want things to stay the way it is. I would say that using ublock and HTTPSB and easing the pressure on the developer to do what we want to do is far more important than "I don't want to install an another extention".

Again thank you @gorhill for your hard work.

[ghost]

FWIW, I'd like to say that I've changed my mind. Raymond, considering what you wrote here and here, there is no overlap anymore (since µBlock 0.2.0.0) between both extensions (provided that all pattern-based lists are unchecked in HTTPSB, of course). This makes my objections pointless :-)