Url bar falsely showing insecure state in 1.62+

[iccub]

Test plan

Full discussion can be found here:
https://bravesoftware.slack.com/archives/C06UXF3KJ/p1707516746468709

1. Visit cnn.com
2. View a few pages. Also open some links in a new tab. Open another new tab and manually navigate to cnn.com (multiple tabs to same domain).
3. Ensure at any given time you do not get Not Secure in the URL bar
4. Visit Twitter.com and login
5. Click on a few posts. Also open some posts in a new tab. Open a another new tab and manually navigate to twitter.com (multiple tabs to same domain)
6. Ensure at any given time you do not get Not Secure in the URL bar
7. Go to search.brave.com and search for google docs
8. Follow the result for Google Docs
9. Ensure at any given time you do not get Not Secure in the URL bar

Description:

Let's monitor if this still happens in 1.62 and try to look for a fix

Steps to Reproduce

1. Browse the internet

Actual result:

URL bar says 'insecure'

Expected result:

Should not show it

Reproduces how often: [Easily reproduced, Intermittent Issue]

Brave Version:

• Can you reproduce this issue with the most recent build from TestFlight?
• Can you reproduce this issue with the previous version of the current build from TestFlight?
• Can you reproduce this issue with the current build from AppStore?

Device details:

Website problems only:

• Does the issue resolve itself when disabling Brave Shields?
• Is the issue reproducible on the latest version of Mobile Safari?

Additional Information

PRs merged related with the solution to the problem and recent PRs for URL Bar / Cert Validation

brave/brave-ios#8634
brave/brave-ios#8651
brave/brave-ios#8666
brave/brave-ios#8680
brave/brave-ios#8697
brave/brave-ios#8698
brave/brave-ios#8704

[ShivanKaul]

@pes10k and I have both seen it happen on twitter intermittently. @fmarier suggested looking at service workers, for that reason, which I think is a good suggestion.

[bsclifton]

Good discussion here:
https://bravesoftware.slack.com/archives/C06UXF3KJ/p1707516746468709

@soner-yuksel to help with next steps - adding a menu for Nightly

[soner-yuksel]

A PR is open for improving the logging for secure content state problem

#36281

[soner-yuksel]

A PR is made focusing on problems

The changes are

• Fix bugs with secureContentState being set in didFailProvisionalNavigation as URL is only ever updated in didCommit so there's a mismatch. This should NEVER be done.
• Fix external URLs not working due to inactive tab logic.
• Fix security certificate display showing when there's no cert at all.
• Remove serverPinningTrust because it can mismatch in didFailProvisionalNavigation when an AppStore URL is loaded on top of a already secure page URL, then the Appstore URL assumes the cert of the page which is wrong. Apple gives us no cert on purpose so we should not store the one from chain evaluation.

[kylehickinson]

Re-opening as it still seems to be an issue

[kylehickinson]

Opened a follow-up issue instead #36951