Manual test run on OS X for 0.56.x - Release Hotfix 2

[kjozwiak]

Per release specialty tests

• Upgrade to Chromium 70.0.3538.110.(#2200)

Installer

• Check that installer is close to the size of last release
• Check signature: If OS Run spctl --assess --verbose /Applications/Brave-Browser-Beta.app/ and make sure it returns accepted. If Windows right click on the brave_installer-x64.exe and go to Properties, go to the Digital Signatures tab and double click on the signature. Make sure it says "The digital signature is OK" in the popup window

Data(Upgrade from previous release)

• Make sure that data from the last version appears in the new version OK
• With data from the last version, verify that
  • bookmarks on the bookmark toolbar and bookmark folders can be opened
  • cookies are preserved
  • installed extensions are retained and work correctly
  • opened tabs can be reloaded
  • stored passwords are preserved
  • unpinned tabs can be pinned

Widevine

• Verify Widevine Notification is shown when you visit Netflix for the first time
• Test that you can stream on Netflix on a fresh profile after installing Widevine

Geolocation

• Check that https://developer.mozilla.org/en-US/docs/Web/API/Geolocation/Using_geolocation shows correct location
• Check that https://browserleaks.com/geo works and shows correct location
• Check that https://html5demos.com/geo/ works but doesn't require an accurate location

Crash Reporting

• Check that loading brave://crash causes the new tab to crash
• Check that brave://crashes lists all the crashes and includes both Crash Report ID & Local Crash ID
• Verify the crash ID matches the report on brave stats

Bravery settings

• Verify that HTTPS Everywhere works by loading http://https-everywhere.badssl.com/
• Turning HTTPS Everywhere off and shields off both disable the redirect to https://https-everywhere.badssl.com/
• Verify that toggling Ads and trackers blocked works as expected
• Visit https://testsafebrowsing.appspot.com/s/phishing.html, verify that Safe Browsing (via our Proxy) works for all the listed items
• Visit https://brianbondy.com/ and then turn on script blocking, page should not load. Allow it from the script blocking UI in the URL bar and it should load the page correctly
• Test that 3rd party storage results are blank at https://jsfiddle.net/7ke9r14a/9/ when 3rd party cookies are blocked and not blank when 3rd party cookies are unblocked

Fingerprint Tests

• Visit https://jsfiddle.net/bkf50r8v/13/, ensure 3 blocked items are listed in shields. Result window should show got canvas fingerprint 0 and got webgl fingerprint 00
• Test that audio fingerprint is blocked at https://audiofingerprint.openwpm.com/ only when Block all fingerprinting protection is on
• Test that Brave browser isn't detected on https://extensions.inrialpes.fr/brave/
• Test that https://diafygi.github.io/webrtc-ips/ doesn't leak IP address when Block all fingerprinting protection is on

Content tests

• Open a page with an input control and type some misspellings on a textbox, make sure they are underlined
• Make sure that right clicking on a word with suggestions gives a suggestion and that clicking on the suggestion replaces the text
• Test that https://mixed-script.badssl.com/ shows up as grey not red (no mixed content scripts are run)

Session storage

• Temporarily move away your browser profile and test that a new profile is created when browser is launched
  • macOS - ~/Library/Application\ Support/BraveSoftware/
  • Windows - %userprofile%\appdata\Local\BraveSoftware\
  • Linux(Ubuntu) - ~/.config/BraveSoftware/
• Test that windows and tabs restore when closed, including active tab
  • Ensure that the tabs in the above session are being lazy loaded when the session is restored

Chromium upgrade tests

Adblock

• Verify referrer blocking works properly for TLD+1. Visit https://technology.slashdot.org/ and verify adblock works properly similar to https://slashdot.org/

Components

• Delete Adblock folder from browser profile and restart browser. Visit brave://components and verify Brave Ad Block Updater downloads and update the component. Repeat for all Brave components