Cleanup brave shields tests and get rid of dead code.

[iefremov]

Submitter Checklist:

brave/brave-browser#6313

• Submitted a ticket for my issue if one did not already exist.
• Used Github auto-closing keywords in the commit message.
• Added/updated tests for this change (for new code or code which already has tests).
• Verified that these changes build without errors on
  • Android
  • iOS
  • Linux
  • macOS
  • Windows
• Verified that these changes pass automated tests (unit, browser, security tests) on
  • iOS
  • Linux
  • macOS
  • Windows
• Verified that all lint errors/warnings are resolved (npm run lint)
• Ran git rebase master (if needed).
• Ran git rebase -i to squash commits (if needed).
• Tagged reviewers and labelled the pull request as needed.
• Request a security/privacy review as needed.
• Add appropriate QA labels (QA/Yes or QA/No) to include the closed issue in milestone
• Public documentation has been updated as necessary. For instance:
  • https://github.com/brave/brave-browser/wiki/Deviations-from-Chromium-(features-we-disable-or-remove)
  • https://github.com/brave/brave-browser/wiki/Proxy-redirected-URLs
  • https://github.com/brave/brave-browser/wiki/Fingerprinting-Protection-Mode
  • https://github.com/brave/brave-browser/wiki/Brave%E2%80%99s-Use-of-Referral-Codes
  • https://github.com/brave/brave-browser/wiki/Custom-Headers
  • https://github.com/brave/brave-browser/wiki/Web-compatibility-issues-with-tracking-protection
  • https://github.com/brave/brave-browser/wiki/QA-Guide

Test Plan:

Reviewer Checklist:

• New files have MPL-2.0 license header.
• Request a security/privacy review as needed.
• Adequate test coverage exists to prevent regressions
• Verify test plan is specified in PR before merging to source

After-merge Checklist:

• The associated issue milestone is set to the smallest version that the
  changes has landed on.
• All relevant documentation has been updated.

[iefremov]

Please review! Can be easier to review per-commit

[bsclifton]

@iefremov can you give this a rebase?

Also - given that it's removing code, do you think it's critical for 0.70.x? (where Chromium 78 will land)

[iefremov]

@iefremov can you give this a rebase?

Also - given that it's removing code, do you think it's critical for 0.70.x? (where Chromium 78 will land)

@bsclifton Yeah, I don't think it's crucial for release branches, this is just a cleanup

[iefremov]

@bsclifton rebased

[iefremov]

@yrliou @fmarier @jumde PTAL

[jumde]

@iefremov - BraveSiteHacksNetworkDelegateHelperTest.QueryStringUntouched is failing

[jumde]

@iefremov - BraveSiteHacksNetworkDelegateHelperTest.QueryStringUntouched is failing

Looks like referrer_policy is not being set correctly. May be it needs to be set to CLEAR_REFERRER_ON_TRANSITION_FROM_SECURE_TO_INSECURE by default: https://cs.chromium.org/chromium/src/net/url_request/url_request.cc?l=565

[iefremov]

@jumde Can't see the failure - passes locally and there are two failures on CI (BraveRewardsBrowserTest.RecurringAndPartialAutoContribution and network audit)

[jumde]

@jumde Can't see the failure - passes locally and there are two failures on CI (BraveRewardsBrowserTest.RecurringAndPartialAutoContribution and network audit)

I see this crash on Debug:

[9753:18179:1016/165853.222279:1468800861779372:ERROR:kill_posix.cc(84)] Unable to terminate process group 9829: No such process (3)
[ RUN      ] BraveSiteHacksNetworkDelegateHelperTest.QueryStringFiltered
[9829:775:1016/165852.794779:1468800434285169:FATAL:referrer.cc(184)] Check failed: false.
0   libbase.dylib                       0x00000001352cdf5f base::debug::CollectStackTrace(void**, unsigned long) + 31
1   libbase.dylib                       0x0000000134f7742b base::debug::StackTrace::StackTrace(unsigned long) + 75
2   libbase.dylib                       0x0000000134f774ad base::debug::StackTrace::StackTrace(unsigned long) + 29
3   libbase.dylib                       0x0000000134f77488 base::debug::StackTrace::StackTrace() + 40
4   libbase.dylib                       0x0000000134fc1573 logging::LogMessage::~LogMessage() + 131
5   libbase.dylib                       0x0000000134fc02d5 logging::LogMessage::~LogMessage() + 21
6   libcontent.dylib                    0x0000000141a1d50d content::Referrer::NetReferrerPolicyToBlinkReferrerPolicy(net::URLRequest::ReferrerPolicy) + 285
7   brave_unit_tests                    0x000000010a8a6481 brave::(anonymous namespace)::ApplyPotentialReferrerBlock(std::__1::shared_ptr<brave::BraveRequestInfo>) + 353
8   brave_unit_tests                    0x000000010a8a62a0 brave::OnBeforeURLRequest_SiteHacksWork(base::RepeatingCallback<void ()> const&, std::__1::shared_ptr<brave::BraveRequestInfo>) + 48
9   brave_unit_tests                    0x00000001077086fc BraveSiteHacksNetworkDelegateHelperTest_QueryStringFiltered_Test::TestBody() + 924
10  brave_unit_tests                    0x0000000107c561d4 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) + 132
11  brave_unit_tests                    0x0000000107c3a0f2 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) + 114
12  brave_unit_tests                    0x0000000107c3a033 testing::Test::Run() + 195

[iefremov]

I see this crash on Debug:

Oh, yess, we didn't initialize the enum properly. Fixed!

[iefremov]

Fortunately it happens only in tests

[jumde]

lgtm