Can't delete huge libraries

[Eskuero]

Steps to reproduce

1. Have a very large library, I'm not sure what's the limit but I had 88k books in mine
2. Try to delete the library entirely

Expected behavior

The library should be gone.

Actual behavior

It fails because of SQLITE_TOOBIG:

I guess the deletion should be broken down in smaller statements to avoid failures.

Logs

komga  | 2025-01-26T23:07:44.170Z  INFO 1 --- [o-25600-exec-18] o.g.k.domain.service.SeriesLifecycle     : Delete series ids: [...]; 

komga  | 2025-01-27T07:07:47.673Z ERROR 1 --- [io-25600-exec-5] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.jooq.exception.DataException: SQL [select BOOK.ID, BOOK.CREATED_DATE, BOOK.LAST_MODIFIED_DATE, BOOK.FILE_LAST_MODIFIED, BOOK.NAME, BOOK.URL, BOOK.SERIES_ID, BOOK.FILE_SIZE, BOOK.NUMBER, BOOK.LIBRARY_ID, BOOK.FILE_HASH, BOOK.DELETED_DATE, BOOK.oneshot, BOOK.FILE_HASH_KOREADER from BOOK where BOOK.SERIES_ID in ('....') [SQLITE_TOOBIG] String or BLOB exceeds size limit (statement too long)] with root cause

Komga version

v1.19.0-master

Operating system

Arch Linux

Installation method

Docker

Other details

No response

Acknowledgements

• I have searched the existing issues (open AND closed) and this is a new ticket, NOT a duplicate or related to another open issue.
• I have written a short but informative title.
• I have checked the FAQ.
• I have updated the app to the latest version.
• I will fill out all of the requested information in this form.

[gotson]

can you provide the full log file with the exception ?

[gotson]

how many series do you have ?

[Eskuero]

As many series as books so 88k. Used "/" on the oneshot pattern to force it that way.

Here's the log. I truncated a portion of it because it literally prints every series ID when trying to delete so goes well beyond the limits of pastebin

https://pastebin.com/iKki0pjN

I must admit that was some testing I did once I now I have a more sane library size. Just wanted to get rid of the existing old one without nuking my entire database.

[gotson]

i've got a fair idea where the issue lies, trying to reproduce on my end

[gotson]

I managed to reproduce, a massive number of series is actually required to trigger this error. With 54k series it would not trigger, had to go up to 100K series to get it.

[gotson]

I've applied a quick fix. The Library deletion API is still synchronous, and thus the webui could hang for a while or even timeout while the deletion is happening.

In a future version the deletion should happen asynchronously as to not block the frontend.

[Eskuero]

Built a docker image from master, successfully deleted the library this time and then run a vacuum to recover the space without hiccups. Thanks a lot!

[root@remnant komga]# docker-compose down
[+] Running 2/2
 ✔ Container komga        Removed     2.2s 
 ✔ Network komga_default  Removed       0.1s 
[root@remnant komga]# du -sh config/database.sqlite
2.9G    config/database.sqlite
[root@remnant komga]# sqlite3 config/database.sqlite 
SQLite version 3.48.0 2025-01-14 11:05:00
Enter ".help" for usage hints.
sqlite> VACUUM;
sqlite> 

[root@remnant komga]# du -sh config/database.sqlite
111M    config/database.sqlite

[github-actions]

🎉 This issue has been resolved in 1.19.1 (Release Notes)