Unify Adblock engines into 1 engine

[cuba]

Resolves brave/brave-browser#36035
Resolves brave/brave-browser#36539

Two issues are bundled here

1. Unify engines into two (standard/general and aggressive/additional)
2. Load from dat files during launch. Once unified the engines have to change how we load from cache. It is natural to do this using dat files instead of text files with very little additional work. Loading cached engines from text files would result in unneeded work and unneeded testing.

Submitter Checklist:

• I confirm that no security/privacy review is needed and no other type of reviews are needed, or that I have requested them
• There is a ticket for my issue
• Used Github auto-closing keywords in the PR description above
• Wrote a good PR/commit description
• Squashed any review feedback or "fixup" commits before merge, so that history is a record of what happened in the repo, not your PR
• Added appropriate labels (QA/Yes or QA/No; release-notes/include or release-notes/exclude; OS/...) to the associated issue
• Checked the PR locally:
  • npm run test -- brave_browser_tests, npm run test -- brave_unit_tests wiki
  • npm run presubmit wiki, npm run gn_check, npm run tslint
• Ran git rebase master (if needed)

Reviewer Checklist:

• A security review is not needed, or a link to one is included in the PR description
• New files have MPL-2.0 license header
• Adequate test coverage exists to prevent regressions
• Major classes, functions and non-trivial code blocks are well-commented
• Changes in component dependencies are properly reflected in gn
• Code follows the style guide
• Test plan is specified in PR before merging

After-merge Checklist:

• The associated issue milestone is set to the smallest version that the
  changes has landed on
• All relevant documentation has been updated, for instance:
  • https://github.com/brave/brave-browser/wiki/Deviations-from-Chromium-(features-we-disable-or-remove)
  • https://github.com/brave/brave-browser/wiki/Proxy-redirected-URLs
  • https://github.com/brave/brave-browser/wiki/Fingerprinting-Protections
  • https://github.com/brave/brave-browser/wiki/Brave%E2%80%99s-Use-of-Referral-Codes
  • https://github.com/brave/brave-browser/wiki/Web-Compatibility-Exceptions-in-Brave
  • https://github.com/brave/brave-browser/wiki/QA-Guide
  • https://github.com/brave/brave-browser/wiki/P3A

Test Plan:

1. Test the upgrade process ensures the engine is ready (i.e. youtube ad-blocking works immediately after upgrading). Note: Additional lists will not be ready right away after upgrading as it can cost up to 24s to perform this upgrade on slow devices. But future relaunches should see them ready.
2. Test that youtube adblocking works during relaunch and heading directly to youtube (including additional lists such as cookie consent notices)
3. Test https://dev-pages.brave.software/filtering/cosmetic-filtering.html
4. Test https://dev-pages.brave.software/filtering/scriptlets.html
5. Test https://dev-pages.brave.software/filtering/additional-lists.html
6. Test https://dev-pages.bravesoftware.com/filtering/text-ads.html
7. Test https://dev-pages.brave.software/filtering/scriptlets.html

Performance:

iPhone 8

	Previous	This
Launch	~11s	~3.3s (default) ~8s (all filter lists)
Post-launch	~2.2s (default) ~12s (all filter lists)	~0.2s
Screenshot

iPhone 15 pro

	Previous	This
Launch	~2s	~0.9s (default) ~2s (all filter lists)
Post-launch	~0.9s (default) ~4s (all filter lists)	~0.02s

[github-actions]

[puLL-Merge] - brave/brave-core@22127

Description

This PR makes changes to the ad-block engine management to support grouped filter lists and resources components. It introduces new classes AdBlockEngineManager and AdBlockGroupsManager to handle compilation and management of ad-block engines. The changes allow for better organization and performance of the ad-blocking system.

Changes

Changes

• AdBlockGroupsManager.swift: A new class that manages standard and aggressive ad-block engines, compiling them from grouped filter lists and resources. It interacts with AdBlockEngineManager, ContentBlockerManager and a SourceProvider to handle enabling/disabling sources and lazy-loading of engines.

• AdBlockEngineManager.swift: A new class that manages a single grouped ad-block engine. It handles adding/removing filter list info, checking if compilation is needed, loading engines from cache, and serializing/deserializing engines.

• GroupedAdBlockEngine.swift (renamed from CachedAdBlockEngine.swift): Updated to work with the new grouped engine format. Added EngineType enum to specify standard vs aggressive engines. Changed init and factory methods to take a FilterListGroup.

• FilterListResourceDownloader.swift, FilterListCustomURLDownloader.swift: Updated to interact with the new AdBlockGroupsManager when filter lists are downloaded or removed.

• Various files: Updated to use the new GroupedAdBlockEngine and AdBlockGroupsManager types. Removed usage of AdBlockStats which was replaced by the new managers.

• Tests: Added new tests for AdBlockEngineManager and AdBlockGroupsManager. Updated existing tests for the refactored types.

Security Hotspots

• The defaultCachedFolderName in the GroupedAdBlockEngine.EngineType extension uses hardcoded strings. Ensure these match up with intended cache folders. However, this doesn't pose a high security risk.

• AdBlockEngineManager writes compiled filter data to the cache folder. Ensure the serialized data is properly validated when loading. However, this is already being done, so risk is low.

• legacyCacheFiles(for:) in the default SourceProvider blindly returns any locally cached data as valid sources initially. However, this is just a temporary transitional method, so risk is limited.

Overall, the refactoring looks solid from a security standpoint. The main areas to validate are proper handling of the serialized engine cache, but no major issues stand out in the changeset. Nice work on the modernization of this code!