Double Ctrl-O required to go back on some sites

[sdothum]

Some sites seem to resist going back.. but I found that if I do the Ctrl-O twice in sequence, relatively quickly, vimb does go back. E.g.

http://imgur.com/a/d9Zvp

from

http://www.reddit.com/r/unixporn/comments/2vhn1m/what_are_your_favorite_beautiful_applications/

(the comment link "here" by redditErick to the imgur web page)

Pressing a single Ctrl-O does not go back. A quick double Ctrl-O does go back.

[fanglingsu]

I can reproduce this. On a first try I thought this is related to JavaScript. Because if JavaScript is disabled the click of the 'here' link leads to the imgur url and this redirects to a noscript variant of the page. The <C-O> brings vimb back to the imgur url which redirects immediately back to the noscript url http://imgur.com/a/d9Zvp/noscript. To go back I've to press <C-O> three times fast.
But the issue appears also if JavaScript is enabled, even if vimb goes back after two <C-O>.

[fanglingsu]

This issue seems to be related to HSTS (HTTP Strict Tranport Security) feature of vimb. If :set hsts=off the <C-O> leads direct back to the previous page.

The imgur page contains references to facebook which send the HSTS header and is put into the list of known hsts-hosts. But the URL for facebook within the page is set the http protocoll. Now vimb stops the http request to facebook and do a new frame load for the https variant of the previous stopped URL. This is a little hack to avoid a bug in webkitgtk (see also #146).

The problem is that there is no way to disable the back forward history for the ignored requests as I know. If the back forward list is disabled and enabled after the request the whole list is cleared. I'll see if there is another alternative to webkit_web_frame_load_uri that is not recorded in webkits back/forward list but forces webkit to return the right url in webkit_web_view_get_uri. A quick check showed that also webkit_web_frame_load_request records the request in the back/forward list and does therefor not help to fix the issue.

[sdothum]

Thanks Daniel! I can live with hsts disabled for now while you ponder the problem.. as there are enough sites whose url I don't remember which present this history navigation problem.

[semarie]

@fanglingsu I have followed your idea to reload the frame only if it is the main request, with some adaptations.

A test against soup_message_get_first_party would work with some code (it is the request as requested by user, so with "http://") So, in order to keep that simple, the code reload the page only if it is the main-frame (we have view and frame here: so just checking if frame is the mainframe is ok).

I have do some tests: it corrects #172, and seems to be ok for others cases (HSTS in simple case or after redirects). Please do some tests too: HSTS is a security feature that we shouldn't break :-)

[fanglingsu]

@semarie The two issue are solved. I've done some tests with included sources like images, and the url are properly rewritten to https (tested with wireshark). But I've seen cases where I requested a source via http that was rewritten to https but the certificate was considered invalid according to soup_message_get_flags that is used in webview_load_status_cb to color the statusbar to red.
The HSTS says that the request must be discarded if the certificate is untrusted. This logic exists already in hsts.c, but this uses soup_message_get_https_status to check this. Maybe we should use the same logic for the hsts and for the coloring of the statusbar, but I'm not sure which of them.

Any attempt to reproduce the invalid certificate failed. Seems to be a heisen-bug, it exists as long as you are are not viewing to it.

[fanglingsu]

In case I set the ca-bundle to an empty file the hsts prevents loading of the https resources like expected, independent from the strict-ssl option.

[shioyama]

Hmm... maybe it's just my config but turning hsts=off doesn't seem to work for me. Anybody else still having trouble with this?

[fanglingsu]

Why do you think it doesn not work? Are there still written entries into the $XDG_CONFIG_HOME/vimb/hsts file?

[shioyama]

Yes vimb still seems to be updating that file.

[fanglingsu]

The hsts file is also change if hsts is disabled in case the hsts was enabled before when vimb is closed. If vimb is closed the in memory hsts entries are rewritten to the hsts file. But if hsts is disabled the file should not change during browsing.

[shioyama]

Ok sorry, I cleared the hsts file and restarted vimb and it no longer updates the file, but I Ctrl-O still does not get me to the previous page... not sure what's going on, but I guess in my case at least hsts is not the cause.

[fanglingsu]

@shioyama Can you please provide some more information about the CTRL-O issue? Does it appear on all pages? Does it also appear if JavaScript is disabled?

[shioyama]

@fanglingsu Hmm not sure now, just tested on one of my two laptops with hsts=off and it seems to work now. Also just upgraded the system (but not vimb), although probably not related. I'll check the other machine and see if the issue persists, if so I'll report back with more details.

Thanks again (as always) for responding so quickly! My only major issue with vimb remains #235, otherwise it's running really smoothly.

[shioyama]

@fanglingsu sorry my comment above was wrong, the problem above persists and does seem to be related to javascript.

Here's an example, going from hackernews to e.g.: http://www.nytimes.com/2015/11/01/business/dealbook/arbitration-everywhere-stacking-the-deck-of-justice.html

With javascript on I can't go back with a single ctrl-O as expected.

[fanglingsu]

I don't think hsts is the reason for the issue. To be sure you could compile vimb without HSTS-Support and test it again. But I assume the issue will still be there even if I can't reproduce it with your URLs at the time. But you can use the webinspector (by 'gF') and look which pages are loaded when you open the nytimes page. It may be that you're redirected right after the page is loaded so that you have in fact visited two pages. If you press CTRL-O in such cases you will normally open the page that redirected you.