Articles marked as read are not correctly synced

[tobiasKaminsky]

As I have way too many news, I have a script that marks all articles older than 3 days as read:

#!/bin/bash

ids=$(mysql -unextcloud -pPASS -s -r -e "use nextcloud; SELECT id FROM oc_news_items where feed_id IN (SELECT id FROM oc_news_feeds where user_id like 'USER') and (unread = 1 and starred = 0) and pub_date <= UNIX_TIMESTAMP(DATE_SUB(CURDATE(), INTERVAL 3 DAY))" | grep -v id | sed -z s'#\n#,#g' | sed s'#,$##')

json="{ \"items\": [ $ids ] }"

curl --request PUT \
  --url https://nextcloud.kaminsky.me/apps/news/api/v1-2/items/read/multiple \
  --header 'Authorization: Basic TOKEN==' \
  --header 'Content-Type: application/json' \
  --data "$json"

As you can see i select id, then use API to mark them as read.
This works as intended.
On web ui: 64 items, oldest 3 days ago
On Android: 315, oldest 4 weeks ago

Somehow some articles are marked as read, some not.

[tobiasKaminsky]

I checked again and see that web/REST API is doing correct:
{{ base_url }}/apps/news/api/v1-3/items?type=3&getRead=false&batchSize=-1

This shows on web/REST: 18
But on Android 20, whereas the oldest 2 are more than one week old

@David-Development if you give me an hint, I can check what might be wrong :)

[David-Development]

Please reopen if it still exist. I think this should be fixed by now

[tobiasKaminsky]

Hi David,

this still happens.
I checked via DB that in DB all is correct, e.g. I have 280 unread articles from "Der Spiegel" and this is correctly shown in web ui.
But on Android News app it shows >500 items.
Searching for the oldest ones shows that they are not even anylonger in DB.

Resetting cache in news app and then syncing it again works.

What is strange is, that this right now only happens to http://www.spiegel.de/schlagzeilen/index.rss, but all other feeds are correctly updated…

If you need me to debug it, I am happy to assist.

[tobiasKaminsky]

news-android/News-Android-App/src/main/java/de/luhmer/owncloudnewsreader/reader/nextcloud/RssItemObservable.java

Line 171 in 79eed34

mNewsApi.updatedItems(lastModified, Integer.parseInt(FeedItemTags.ALL.toString()), 0)

I changed last modified to:
lastModified - (5 * 60 * 1000 * 1000) -> so that it fetches 5h old stuff.
And this seems to do the trick.

Right now I do not fully get why, but it seems to be a race condition between lastModified and marking articles as read by cronjob (which I do hourly).

I will monitor this and try to understand the reason.

[tobiasKaminsky]

5h does not help during night, when my emulator is off.
So it seems indeed to be a problem with this last modified.

[tobiasKaminsky]

During weekend my emulator was offline, so it now has a gap of ~80 articles.

[tobiasKaminsky]

I somehow do not get why this is happening

• news app syncs at .eg. timestamp 10 (small numbers for better readability)
• newest article on server is 9, oldest 2
• at 15:
  • cronjob marks article <=5 as read
  • updates all article to have last modification at 15
• at 17: news app syncs with last mod 10 -> should fetch updates at 15

[tobiasKaminsky]

I set "lastModified" to 0.
It then retrieves all 155 items from server, which is correct.
But News App still shows 359, as it only updates the ones received from server.
Those ~200 that are not returned by server will thus stay for eternity.

I now suspect that those 200 are already deleted on server, and thus cannot be given to news app.
This would mean that the read ones are purged too fast.

Indeed server app shows oldest read "der spiegel" article 2 days ago.
News app shows 1week old articles.

[tobiasKaminsky]

Changed to News app config:

            "enabled": "yes",
            "updateInterval": "600",
            "types": "",
            "installed_version": "24.0.0",
            "autoPurgeCount": "1000"
        },

[tobiasKaminsky]

increasing it to 1000 seems to help.
I will monitor it and then close

[tobiasKaminsky]

Increasing solved it.