Old HTTPS Everywhere release still being referenced for rulesets

[Hainish]

Brave is still sourcing the old minor point release of HTTPS Everywhere, 5.2, for rulesets:

browser-laptop/js/constants/appConfig.js

Line 88 in 052277d

version: '5.2', // latest major point release from https://eff.org/https-everywhere

This is problematic as it has not been updated in quite some time. We are now using date-based versioning. The lastest version is 2017.9.12: https://github.com/EFForg/https-everywhere/tree/2017.9.12

Also, be aware that until recently, the rulesets were compiled into a default.rulesets XML file. This has changed, so that default.rulesets is now a JSON file. This should make your build process simpler, since it looks like you were adding an extra step to convert our rulesets to JSON.

cc @diracdeltas

[diracdeltas]

@Hainish thanks for bringing this to my attention. The comment is misleading; we are actually using the latest minor point release (give or take a couple months) but dropping the minor point number. So right now we are actually on 5.2.21; see https://github.com/brave/https-everywhere-builder for where the rulesets are built.

[Hainish]

@diracdeltas will this be updated to 2017.9.12? 5.2.21 is also a couple of months old (released July 18th).

[diracdeltas]

updating it now to 2017.10.24

[diracdeltas]

actually there's some breaking format changes between chrome/content/rulesets.json (what we currently use to derive our rules file) and webextension/rules/default.rulesets which is used in versions newer than 5.2.21. namely:

1. there is no attribute in default.rulesets which indicates the name of the XML file that each ruleset is from
2. it appears that instead of pre-building the mapping of targets to ruleset IDs and saving this in the rulesets file, this map is now built at runtime in webextension/rules.js

is there a reason that pre-building of the target map is no longer used? IIRC this was added to improve startup time. @Hainish

[Hainish]

@diracdeltas I'm not aware of the history of this feature. This may have been an improvement that never made it to the WebExtensions version of the addon, which we're using currently.

[Hainish]

Should we be filing this issue on https://github.com/brave/https-everywhere-builder?

Since chrome/content/rulesets.json is no longer used, does Brave have a roadmap for future inclusion based on the currently active chrome/content/default.rulesets mapping?

One thing to be aware of is that we plan to make the rulesets available via a separate file, which can be downloaded along with a signature for verification by downstream dependencies. See EFForg/https-everywhere#12606

[diracdeltas]

@Hainish this is on my todo list here: #11598. haven't gotten around to it yet.

will the separately-downloadable rulesets file have the same format (or be the same as) default.rulesets?

[Hainish]

@diracdeltas it will most likely be in the same format, plus gzipped and base64-encoded. So the filename will be default.rulesets.gz.base64.