Fetch more articles for current feed or category

[Alkarex]

In the GReader API, it is possible to fetch articles for a given feed or for a given category.
This is especially useful when there are many articles on the server (more than what the app is fetching) and the user wants to fetch older articles for a given feed or category.

So in the CapyReader app, it would be nice if the "pull to refresh" feature, when done from within a feed or a category, would fetch articles from that feed or category.

I can give more details if this is not clear enough, and do not hesitate to ask for help on back-end side (FreshRSS).

[jocmp]

@Alkarex I've added functionality for this in the latest dev release, https://github.com/jocmp/capyreader/releases/tag/2024.12.1080-dev. Let me know if this is working as expected!

[Alkarex]

Hello,
I am glad to see that you are progressing :-)
Are you sure it is the "Fetch more articles for current feed or category" feature, you have added?
Maybe confused with:

• Fetch read articles #564

Quick test on a feed that has plenty of entries on the server, but not so many recently, in view all articles modes.
I initially have two articles on that view, scroll to refresh, many (big) requests are made (could be improved), but no request that I can see is filtered on that feed. No new articles appear, unfortunately. Here is a log below, and let me know if you like more feedback.

2024-12-04T07:24:55.800064523Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:55 +0100] "GET /api/greader.php/reader/api/0/subscription/list?output=json HTTP/1.1" 200 9987 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:56.173118421Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:55 +0100] "GET /api/greader.php/reader/api/0/stream/items/ids?s=user%2F-%2Fstate%2Fcom.google%2Fstarred&n=10000&output=json HTTP/1.1" 200 14372 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:56.678484261Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:56 +0100] "GET /api/greader.php/reader/api/0/stream/items/ids?s=user%2F-%2Fstate%2Fcom.google%2Freading-list&n=10000&xt=user%2F-%2Fstate%2Fcom.google%2Fread&output=json HTTP/1.1" 200 59249 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:57.090059148Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:56 +0100] "GET /api/greader.php/reader/api/0/stream/items/ids?s=user%2F-%2Fstate%2Fcom.google%2Freading-list&ot=1733297062&n=100&output=json HTTP/1.1" 200 16 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:57.651534860Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:57 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 100107 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:57.659281396Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:57 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 114071 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:57.719098348Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:57 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 108591 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:57.758594976Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:57 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 56884 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:57.800327384Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:57 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 140060 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:58.212670999Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:57 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 62139 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:58.228826293Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:57 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 90874 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:58.387913569Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:57 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 108199 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:58.392909937Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:57 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 97157 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:58.407226480Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:57 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 86380 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:58.640276581Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:58 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 62427 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:58.878302391Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:58 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 96584 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:58.889764214Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:58 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 77906 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:58.987641530Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:58 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 78372 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:59.026547174Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:58 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 120494 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:59.047526236Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:58 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 42030 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:59.300856026Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:58 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 62758 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:59.318638320Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:59 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 81365 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"
2024-12-04T07:24:59.336580003Z 130.XXX.XXX.XXX - - [04/Dec/2024:08:24:58 +0100] "POST /api/greader.php/reader/api/0/stream/items/contents?output=json HTTP/1.1" 200 71952 "-" "CapyReader (RSS Reader https://capyreader.com/) okhttp/4.12.0"

[jocmp]

hey @Alkarex - I posted to the wrong thread, thanks for following up in the other one. I haven't started work on this "Fetch more" but I'll track it via #574.

In both this case and the "read" case the "Pull to Refresh" currently refreshes all feeds as you've noted. Initially I did this to avoid the complexity of passing conditional logic to the refresh task, but as you note that comes with a performance hit. I'll track that via #575.

[issteve]

It looks like there is no pull to refresh in the feeds overview, right? I have it on "refresh only manual" and it would be great to be able to refresh all feeds with one swipe from above in the feeds overview...

[jocmp]

@issteve yep there is - it's available on the top-level feeds, folders, and individual feeds as well.

Screen_recording_20250106_185645.mp4

[issteve]

@issteve yep there is - it's available on the top-level feeds, folders, and individual feeds as well.
Screen_recording_20250106_185645.mp4

Thank you for your response. I tried it again in the feeds overview (the one under the three lines having the "Feeds" headline and showing all the feeds) and it didn't show the behaviour in your video.
Why I need it there?
I have the unread filter always on and the "refresh only manual" option. So the feed I want to refresh is not listed and it doesn't get refreshed and I can't refresh it manually because I can't go into it. So I would have to switch the unread filter always back and forth.
I helped myself by keeping always one item in one feed to do the manual refresh there and hope it will update all the other feeds as well... But it's still somehow one annoying step to much if this feed doesn't have something new...

[jocmp]

@Alkarex this is included in the 2025.01.1093-dev release. Here's how it works

• On a top level feed, e.g. "All" or "Unread," only the latest read and unread articles are fetched based on the "max arrived at" timestamp. This is the timestamp of the last article that was received. This is unchanged from previous versions.
• On a category feed, e.g. a folder or an individual feed, the entire feed is refetched. This has some overhead but should pick up on any gaps including older articles.

[Alkarex]

Nice progress 👍🏻

• the entire feed is refetched

I suggest adding an option to define the max number of articles to retrieve (including high values such as 10k). It is very possible to have a category (or even a feed) with 100k+ articles, and fetching everything is a no-go

[jocmp]

Good point! I'll add the 10k max to the next update

• [FreshRSS] Only fetch 10k articles at most from categories #713