Duplicate content appearing in feed

[mattmcalister]

Sometimes the feed will show me two identical articles. In this example using Version 1.47.52 Chromium: 107.0.5304.91 (Official Build) nightly (arm64), I am not subscribed to any channels, only individual sources:

The duplicates do not appear as duplicates in the raw feed.

Steps to reproduce:

• Follow a built-in feed in region A, e.g. CNN
• Follow another built-in feed in region B, e.g. CNN World News
• Also follow something with presence in both region A and region B, e.g. TechCrunch

Expected:

• TechCrunch articles each only show once

Actual:

• TechCrunch articles each show twice

[petemill]

@mattmcalister Can you add a screenshot of your customize section and also more info about where in the feed the duplication is happening, relative to either the top of the feed if close, or another landmark like a list

[mattmcalister]

The first duplicate to appear is the 3rd item in my feed

And my Following list includes:

• Ars Technica
• Bleacher Report
• Brave Blog
• Breaking the News
• Formula One | The Guardian
• Gardening Know How
• Hacker News
• Space.com
• Techmeme
• The Atlantic
• The Isle Of Thanet News
• The Verge
• Timeout
• Toms Hardware
• Tottenham Hotspur | The Guardian
• Wikihow
• Wired
• XKCD

[mattmcalister]

STR

1. Open Brave News Discover panel.
2. Start with a clean slate. Unfollow all feeds.
3. Search for one feed and follow it. For example, The Atlantic or Wired.
4. Close Settings
5. Refresh NTP
6. Scroll down

[fallaciousreasoning]

I wonder if this is a bug in the fetching the feed in all locales code? What's in your Preferences file under today.sources and news.channels?

[mattmcalister]

"news":{"channels":{"":{"Top Sources":true},"en_US":{"Crypto":true,"Politics":true,"Top News":true,"Top Sources":true,"Weather":true,"World News":true}}}

[mattmcalister]

"sources":{"851807a03afb6804b83dd50a72a21afb0c90ce9be1035452d40601e47e4b02a0":false,"9267d9058a847bcb7d53de0e9ca1aa5946c6ca89c1f0ae2689ddf77cac3f0de9":false,"f3373c57bd23f617d991ef066a2645351eaf54859c37d8877c06f72749a0f573":false}

[fallaciousreasoning]

Ah, so I'm pretty sure that first one is the cause - I actually thought we fixed that a while back. Does it still happen with a new profile? If not, remove the "": { "Top Sources": true } from channels.

If it doesn't happen in a new profile then I think we're okay.

[mattmcalister]

fixed! well, maybe not. seems to have returned. am investigating further...

[mattmcalister]

latest update appears to have cleared it up. will keep an eye on it, but I'm going to close this issue now.

[petemill]

I added a test plan with the STR @fallaciousreasoning fixed with brave/brave-core#15984

[LaurenWags]

Requires 1.46.122 or higher to test.

[stephendonner]

Verified PASSED using

Brave	1.46.122 Chromium: 108.0.5359.48 (Official Build) beta (x86_64)
Revision	18ceeca0d99318e70c00d2e04d88aa55488b5c63-refs/branch-heads/5359@{#854}
OS	macOS Version 11.7.1 (Build 20G918)

Steps:

1. installed 1.46.122
2. launched Brave
3. opened a new-tab page
4. scrolled down
5. clicked on Customize
6. followed CNN, Techcrunch, and CNN - World News feeds
7. scrolled through my Brave News feed

Confirmed no feed/article duplicates

example	example	example