[Bug] History: Consecutive Duplicates should be filtered out

[cadeyrn]

Steps to reproduce

1. open history

Expected behavior

no duplicated entries, because the duplicates don't add much value

Actual behavior

duplicated entries

UNITO-UNDERSCORE!20190402-223933!

Device information

• Android device: HTC U11 / Android 8.0
• Fenix version: Fenix revision bd81e72

┆Issue is synchronized with this Jira Task

[pocmo]

Related AC issues:

• History: Ignore initial load of about:blank android-components#2606
• HistoryStorage: allow visitType filtering in getDetailedVisits android-components#2643

[ekager]

Those 2 AC issues are merged in and being used. We should have higher quality history now.

[lobontiumira]

Hi @cadeyrn,
I'm not completely sure if I understood correctly the expected behavior.
For example, I searched for YouTube and in History, I get http://youtube.com, and https://m.youtube.com/, although I typed "youtube" inside the tab, and I didn't select any video. Is this expected or not?
Thank you!

Please see the attached screenshot:

[ekager]

@grigoryk could you possibly weigh in on if the above screenshot is the "best" we can do with history filtering right now?

[sblatz]

Still need @grigoryk's input on this.

[thomcc]

We intentionally aren't filtering duplicates, but could easily add a parameter to do so.

[thomcc]

More broadly, there are a couple things you might want here:

1. Only show the (globally) most recent visit for a given URL. This is consistent with desktop's handling, and is easy for us to do, however it would mean that deeper into your history pages, the data might be noticably less accurate.
2. Within a given page, only show the most recent visit of a specific url. This is also easy for us to add, but the difference between adding it in rust and kotlin wouldn't be huge -- we'd just be doing the filtering after the query.
3. Other variants... I could imagine also a situation where only consecutive duplication is filtered...

Basically, I think if you want number 1, we should do it in the Rust. (It would be an optional flag though, since I can imagine cases where all pages are desired)

Otherwise, it probably is worth you doing it in Kotlin, so that you retain that flexibility to change it later without needing to go through us.

EDIT: OTOH it's worth noting that while passing data over the FFI isn't that expensive, it might be expensive enough that it's still worth it to do the filtering in rust first. (I'm not sure how many items would be excluded in practice)

[jdragojevic]

@vesta0 - Can you please consider the options that @thomcc outlined above for behaviours and let us know how you like it to be resolved?

[yoasif]

Other variants... I could imagine also a situation where only consecutive duplication is filtered...

I think consecutive duplication is the only one that is desirable for users. People on desktop already complain about behavior #1, and I don't see any reason why it is desirable to be consistent with this disliked behavior.

[vesta0]

@thomcc thanks for listing the options.

I don't think we should only show a most recent visit for a URL. Comparing how this works in Fennec vs Fenix, I think the problem is that Fenix is not pulling a correct/distinct page name all the time. For example, if you tap on different ideos on YouTube in Fenix they will all have the same name (m.youtube.com) even if the URL is different, whereas Fennec correctly displays the name of the video. Can we match this behaviour in Fenix?

Also what does the effort for option 3 (filtering out only consecutive dups) look like?

[jhugman]

So, threading the needle through all the above:

• we should filter out duplicate consecutive entries.
• I think we should do this in Kotlin, to give us some ongoing flexibility.

Other issues alluded to in this bug:

1. title of the page is not being saved

Fenix is not pulling a correct/distinct page name all the time. For example, if you tap on different ideos on YouTube in Fenix they will all have the same name (m.youtube.com) even if the URL is different, whereas Fennec correctly displays the name of the video.

1a. intermediate pages in a redirect should not be stored in history.

This has been filed before at ac#3027

2. filtering duplicates in history should be using title, or PSL/common prefixes trimmed URL.

Solving 2. by filtering by title would be relatively cheap, however, we cannot trust that the title for consecutive duplicates are actually the same, because 1. exists. Until then, we have garbage in, garbage out.

[st3fan]

This may be an unrelated problem but it is worth investigating:

Fenix is not pulling a correct/distinct page name all the time. For example, if you tap on different ideos on YouTube in Fenix they will all have the same name (m.youtube.com) even if the URL is different, whereas Fennec correctly displays the name of the video.

Is this unique to YouTube because it reloads the page not with actual reloads but by doing ajax requests and modifying the history stack? We've had this that problem in Firefox iOS where we did not get the proper notifications that the page changed.

This is not unique to YouTube - there are plenty of SPAs that use these techniques.

[jhugman]

@st3fan pushState may well be relevant here.

I'll file an ac issue to track that.