App enters in UnsuccesfulResponseException notification loop

[Zayon]

Issue details

Duplicate?

Have you searched the issues of this repository if your issue is already known? yes

Actual behaviour

I tried to add an entry through the app, but my wallabag server crashed (error 500) because it could not insert the entry in the database due to the entry containing emojis and the database charset not being properly configured.

The app enters in a notification loop:

Error: com.di77n.stuff.wallabag.apiwrapper.exceptions.UnsuccesfulResponseException: HTTP response: 500

Every few seconds the app retry to add the entry and re triggers the notification. This is really annoying.

The only way to make it stop is to force stop the app from the android settings.

The app becomes unusable as long as the server is not fixed.

Expected behaviour

There should be a way to cancel an entry.

Steps to reproduce the issue

1. Have a badly configured server (database_charset: utf8 instead of database_charset: utf8mb4)
2. Add an entry that crash the server (entry with emojis examle)

Environment details

• wallabag app version: 2.3.0
• wallabag app installation source (e.g. Gplay, F-Droid, manual): F-Droid
• Android OS version: 9
• Android ROM: OxygenOS 9.0.10
• Android hardware: OnePlus 5
• wallabag server version: 2.3.8
• Do you have Two-Factor-Authentication enabled?: yes/no

Logs

wallabag server

Please paste relevant wallabag server logs here (from file <wallabag>/var/logs/prod.log):

Just any CRITICAL error log.

Web server

Please paste relevant web server logs (e.g. from nginx, Apache, …) here:

172.18.0.9 - - [19/Jan/2020:15:56:59 +0100] "POST /api/entries.json HTTP/1.1" 500 83 "-" "okhttp/4.2.2"

Your experience with wallabag Android app

Have you had any luck using wallabag Android app before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)

Can't live without it, wallabag would be unusable on mobile without it.

[di72nn]

Hi, thanks for drawing attention to the problem.

To be honest I've always been reluctant to developing Android UI, I guess it's the main reason this is still not addressed.

Here are some thoughts for me or somebody else to implement:

1. Don't retry / block on failed items.
   Should be trivial for new URLs: simply add some field to QueueItem for filtering out the failed ones.
   A bit more complicated for other offline changes, since they may depend on each other. Maybe just mark all QueueItems that have the same articleId as failed. But I don't remember these ever fail, so not important.
2. Since these errors are pretty much always caused by misconfiguration or some unknown server-side errors, the app should notify the user regularly (not all the time of course). Maybe display a notification about problems after synchronizations (no more frequent than, say, once a day).
3. Some UI should be added. Ideally a list of all/failed unsynchronized items. But maybe something simpler would do, since this is a rarely needed thing.
4. Also, if the failed items no longer retried, the error messages should probably be stored somewhere.

For the 3rd point (the one that I really don't want to work on) I'm tempted to simply add two buttons: "copy failed items to clipboard" and "remove failed items". I wonder whether the current state of things warrants a solution even that poor.

[SansGuidon]

If that can help,

I faced the same error a few times in previous days, everytime I had to uninstall the app and reinstall it because I had no clue what was suddenly wrong.
This time I notice by checking the server and mobile app logs that Wallabag client is unable to add a big file https://biblionumericus.fr/wp-content/uploads/2022/05/Guide-3-non-tech-privacy.pdf

Server logs (running Wallabag from Cloudron) :

Jan 04 13:32:2746.179.157.33 - - [04/Jan/2024:12:32:25 +0000] "POST /api/entries.json HTTP/1.1" 500 135 "-" "okhttp/4.12.0"
Jan 04 13:32:27[Thu Jan 04 12:32:27.492637 2024] [php:error] [pid 154] [client 172.18.0.1:52266] PHP Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 39373376 bytes) in /app/code/wallabag/vendor/smalot/pdfparser/src/Smalot/PdfParser/RawData/FilterHelper.php on line 244
Jan 04 13:32:30- - - [04/Jan/2024:12:32:30 +0000] "GET / HTTP/1.1" 302 366 "-" "Mozilla (CloudronHealth)"
Jan 04 13:32:40- - - [04/Jan/2024:12:32:40 +0000] "GET / HTTP/1.1" 302 366 "-" "Mozilla (CloudronHealth)"

Mobile app (Android) logs :

01-04 13:32:24.509  8955  8955 E OplusBracketLog: [OplusViewMirrorManager] updateHostViewRootIfNeeded, not support android.view.ViewRootImpl@8687c80
01-04 13:32:24.509  8955  2754 D OfflineChangesSynchronizer: syncOfflineQueue() current QueueItem(1 out of 10): QueueItem{id=1, queueNumber=1, action=ADD_LINK, articleId=null, localArticleId=1496, extra='https://biblionumericus.fr/wp-content/uploads/2022/05/Guide-3-non-tech-privacy.pdf', extra2='null'}
01-04 13:32:24.509  8955  2754 D EventProcessor: onSyncQueueProgressEvent() started
01-04 13:32:24.511  8955  2754 D OfflineChangesSynchronizer: syncOfflineQueue() processing: queue item ID: 1, article ID: "null"
01-04 13:32:24.511  8955  2754 D OfflineChangesSynchronizer: addLink() link=https://biblionumericus.fr/wp-content/uploads/2022/05/Guide-3-non-tech-privacy.pdf, origin=null
01-04 13:32:24.515  8955  2754 D TrafficStats: tagSocket(3) with statsTag=0xffffffff, statsUid=-1
01-04 13:32:24.521  8955  8955 D BufferQueueConsumer: [](id:22fb00000006,api:0,p:-1,c:8955) connect: controlledByApp=false
01-04 13:32:24.521  8955  8955 E IPCThreadState: attemptIncStrongHandle(85): Not supported
01-04 13:32:24.521  8955  8955 E IPCThreadState: attemptIncStrongHandle(80): Not supported
...
01-04 13:32:24.556  8955  8955 D VRI[EditAddedArticleActivity]: handleWindowFocusChanged mWindowFocusChanged true mUpcomingWindowFocus true mAdded true
01-04 13:32:25.559  8955  8955 D OplusScrollToTopManager: fr.gaulupeau.apps.InThePoche/fr.gaulupeau.apps.Poche.ui.EditAddedArticleActivity,This DecorView@2404314[EditAddedArticleActivity] change focus to true
01-04 13:32:27.247  8955  2754 I *nRefreshingInterceptor: intercept() unsuccessful response; code: 500
01-04 13:32:27.249  8955  2754 W BaseNetworkWorker: syncOfflineQueue() wallabag.apiwrapper.exceptions.UnsuccessfulResponseException
01-04 13:32:27.249  8955  2754 W BaseNetworkWorker: wallabag.apiwrapper.exceptions.UnsuccessfulResponseException: HTTP response: 500 
01-04 13:32:27.249  8955  2754 W BaseNetworkWorker: 	at wallabag.apiwrapper.WallabagService.checkResponse(WallabagService.java:1525)
01-04 13:32:27.249  8955  2754 W BaseNetworkWorker: 	at wallabag.apiwrapper.WallabagService.checkResponseBody(WallabagService.java:1503)

[Strubbl]

Have you solved this by increasing the allowed memory size for php on server side?

[SansGuidon]

Hi @Strubbl I'm not sure how much memory is needed for Wallabag ? with about 1.5 links it seems 650 MB is not enough ? I've seen people struggling with Wallabag even with 2GB memory allocated.
For now my main way to tackle those blocking issues is by clearing the latest unsynced links from mobile app, but maybe increasing memory limit is indeed the only solution. I would love to get recommendations for tuning Wallabag for thousands of links ?

[Strubbl]

memory size of 268435456 bytes exhausted

Your server's memory limit for php seems to be at 256 MB. I have increased mine to 1024 MB. It is sufficient, but i almost never add PDFs to my wallabag.

[SansGuidon]

@Strubbl this seems unexpectedly low compared to what instructions are given to Cloudron for this app. Maybe the setting is for the whole image and not for PHP. The whole Cloudron template for this app is https://git.cloudron.io/cloudron/wallabag-app and I see in https://git.cloudron.io/cloudron/wallabag-app/-/blob/master/Dockerfile that the memory is hardcoded for PHP which could explain several of the memory issues I've encountered with this app from the beginning while I was always increasing the overall memory allocated to wallabag (the whole container).
Thanks for the pointer !

Regarding usage of Wallabag for bookmarking PDFs URLs, well, sometimes a document is only available as a PDF and you want to read it later and wallabag should be able to save just an URL to a PDF so I can read it later, at least that was one use case for me that I expect keeps being supported.

EDIT : case opened at https://forum.cloudron.io/topic/10824/app-ran-out-of-memory-related-to-php-settings

[SansGuidon]

Yet another case of URL that causes issues on server is http://sebsauvage.free.fr/favs.html , it seems to take forever to process. No idea why
Via curl it's going instant, but via Wallabag it takes ages. Given all kind of memory issues and slowness I face with Wallabag I'm wondering if it's really optimized for my cases (saving links to anything), maybe Shaarli would be more a fit for my needs.

[Zayon]

Well http://sebsauvage.free.fr/favs.html is huge, nearly 30k lines of html.

I managed to add it to my wallabag instance, using firefox's extension, not the android app. The http call timeout out but the backend process finished and the entry is available.

About your concern about wallabag not being suited to your needs, Wallabag is more than just a way to saving links, it keeps a local copy of the page's content allowing you to read it later, possibly offline, it's also a way to make a snapshot of a page in case it disappear.
If you really don't care about the read-it-later and snapshot part, then maybe another tool is more suited indeed.

Anyway, I've been using Wallabag both as a read-it-later and a "simple" link save for 8 years now and I almost never encountered cases where I could not add a link, I can't remember a specific one.

On a final note, this issue is about the android app being resilient from server errors, not the server errors themselves which should be issued in the main repository IMO.

[SansGuidon]

I do care about the read-it later and archiving parts as well as for the automated tagging and other nice features. But there are some features I lack and had to take care of via integrations like archiving to wayback machine.
I'm also recommended in Cloudron forum to increase memory of Wallabag to a few GB to keep it running, which is a warning flag for me.
The URLs i've tried (big html file, and big PDF file) are not correctly handled on my side so that forces me to use another app, which makes Wallabag less useful.
I probably should escalate those latest issues on the main repo, but I came here initially as I faced issues when using the mobile app, and when it happens, the mobile app becomes useless and I lose data without warning.

Thank you for your feedback @Zayon !