Uncaught exception: Error: Cannot create a string longer than 0x1fffffe8 characters

[siloportem]

Type: Bug Report

Your Environment

• Version used:
  • v4.6.1
  • 9eeff18
• Environment name and version:
  • Node.js version v16.7.0
  • npm version 7.21.1
  • Web browser irrelevant
• Operating system and version: Manjaro Linux aarch64
• Torrent client and version: rtorrent 0.9.8

Summary

Starting flood by calling npm run start -- --host 0.0.0.0 --port 3000 fails with the following exception:
Uncaught exception: Error: Cannot create a string longer than 0x1fffffe8 characters.

Full output is:

➜  npm run start -- --host 0.0.0.0 --port 3000

> flood@4.6.1 start
> node --use_strict dist/index.js "--host" "0.0.0.0" "--port" "3000"

Flood server 4.6.1 starting on http://0.0.0.0:3000

(node:297655) [DEP0111] DeprecationWarning: Access to process.binding('http_parser') is deprecated.
(Use 'node --trace-deprecation ...' to show where the warning was created)
FATAL internal error. Please open an issue.
Uncaught exception: Error: Cannot create a string longer than 0x1fffffe8 characters

Expected Behavior

flood starts succesfully

Steps to Reproduce

1. Download flood source tarball (tried it with v4.6.1 and 4.5.4)
2. npm install
3. npm run build
4. npm run start -- --host 0.0.0.0 --port 3000

Context

I was running flood successfully before. After upgrading to v4.6.1 flood fails to start. Trying to build and use a previously working version again fails with the same exception.

[jesec]

Hmm, 0x1fffffe8 is 512MB. May I ask how many torrents are there?

[jesec]

and are you using the vanilla rTorrent or the JSON-RPC version?

[siloportem]

There are currently 20 torrents active in rTorrent.

I use vanilla rTorrent and have not updated rTorrent or changed the rTorrent configuration for a long time.
Tomorrow I could try to build and run JSON-RPC rTorrent to see if I get the same error, if it helps?

[davmin]

Encountering the same issue after an upgrade to latest. Using docker-compose of jesec/flood and jesesc/rtorrent with 2144 torrents. Reverting to flood 4.5.3 did not resolve the issue.

[jesec]

Encountering the same issue after an upgrade to latest. Using docker-compose of jesec/flood and jesesc/rtorrent with 2144 torrents. Reverting to flood 4.5.3 did not resolve the issue.

Could you please run the debugging version (jesec/flood:master-dbg) ?

It should have more meaningful logs.

[jboiselle]

Seeing the same issue. Using jesec/flood:master-dbg on Docker 20.10.8.

> flood@0.0.0-master.3563344 start:development:server
> NODE_ENV=development TS_NODE_PROJECT=server/tsconfig.json ts-node-dev --transpile-only -r tsconfig-paths/register server/bin/start.ts "--host=0.0.0.0" "--port" "3000" "--allowedpath" "/data"
[INFO] 05:51:04 ts-node-dev ver. 1.1.8 (using ts-node ver. 9.1.1, typescript ver. 4.4.3)
Flood server 0.0.0-master.3563344 starting on http://0.0.0.0:3000
(node:24) [DEP0111] DeprecationWarning: Access to process.binding('http_parser') is deprecated.
(Use `node --trace-deprecation ...` to show where the warning was created)
Error: Cannot create a string longer than 0x1fffffe8 characters
    at Object.slice (node:buffer:593:37)
    at Buffer.toString (node:buffer:811:14)
    at FSReqCallback.readFileAfterClose [as oncomplete] (node:internal/fs/read_file_context:63:23)
[ERROR] 05:51:07 Error: Cannot create a string longer than 0x1fffffe8 characters

[slagiewka]

I'm experiencing same issues right now. Same for master-dbg as well as master-distroless and other versions released on Docker.

I tried downgrading, up to v4.0. Most versions on the way showed me something like this:

What you can see here is that there's a minifed code being printed out for some reason. It might be possible that the same happens for newer versions, but the buffer with code is somehow so big that it can't fit there.

The screenshot shows the end and the beginning of that output.

I've tested on two separate machines: Arch and Ubuntu Server installs, both running Docker (via compose like in the wiki).

[slagiewka]

Ok, seems that I have found the root cause: loading history.

I've previously stated that I had problems on two separate machines... but the new one had config rsynced from the old one.

Clean setup worked.

Digging deeper has shown me this:

~/dlconfig/.local/share/flood/db/b9Opj6gCUkwJkaD6/history$ ls -alh
total 896M
drwxr-xr-x 2 shimmy shimmy 4.0K Sep 15 16:40 .
drwxr-xr-x 4 shimmy shimmy 4.0K Sep 15 16:40 ..
-rw-r--r-- 1 shimmy shimmy    0 Sep 15 16:40 daySnapshot.db
-rw-r--r-- 1 shimmy shimmy 896M Aug  1 15:09 fiveMinSnapshot.db
-rw-r--r-- 1 shimmy shimmy    0 Sep 15 16:40 hourSnapshot.db
-rw-r--r-- 1 shimmy shimmy  35K Sep 15 16:40 monthSnapshot.db
-rw-r--r-- 1 shimmy shimmy    0 Sep 15 16:40 thirtyMinSnapshot.db
-rw-r--r-- 1 shimmy shimmy  59K Sep 15 16:40 weekSnapshot.db
-rw-r--r-- 1 shimmy shimmy 3.1K Sep 15 16:40 yearSnapshot.db

After I've deleted the 896 MB(!) snapshot, Flood started working as before.

[siloportem]

After I've deleted the 896 MB(!) snapshot, Flood started working as before.

Same for me. Deleted fiveMinSnapshot.db which had a size of more than 800MB. Now flood works again.

[Jogai]

For me it was in the path:

[whatever your flood path is]/db/_config/history

[TTR6214]

Same for me. My journalctl said

flood[7829]: Flood server 4.7.0 starting on http://127.0.0.1:3000
flood[7829]: (node:7829) [DEP0111] DeprecationWarning: Access to process.binding('http_parser') is deprecated.
flood[7829]: (Use `node --trace-deprecation ...` to show where the warning was created)
flood[7829]: FATAL internal error. Please open an issue.
flood[7829]: Uncaught exception: Error: Cannot create a string longer than 0x1fffffe8 characters

I deleted fiveMinSnapshot.db and now Flood start !

total 879228
-rw-r--r-- 1 rtorrent rtorrent         0 déc.  15 13:07 daySnapshot.db
-rw-r--r-- 1 rtorrent rtorrent 900167468 nov.   7 23:40 fiveMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent         0 déc.  15 13:07 hourSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent     53218 déc.  15 13:07 monthSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent         0 déc.  15 13:07 thirtyMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent     65437 déc.  15 13:07 weekSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent      5235 déc.  15 13:07 yearSnapshot.db

[Fire-Dragon-DoL]

Thanks for everyone involved in the debugging, I'm facing the same issue too, even on version 4.7.0.
The fix was to remove the fiveMinSnapshot.db in my case too.

[SiraVista]

Thanks for the help everyone. With Flood v4.7.0 this bug still persists as of June 6th, 2022.

At least one can manually delete fiveMinSnapshot.db as a temporary fix. Mine was showing at 530 MB !

[bbashy]

Found this issue as well today. Deleting the file that was almost a gig resolved it.

1017M Jul  8 21:34 fiveMinSnapshot.db

[JeremyMahieu]

Had this aswell, deleting the file instantly fixes it.

[itsthejoker]

Hit this myself just now. For anyone finding this, the fiveMinSnapshot.db file can be in multiple places in the flood tree, and the first one you find might not be the issue.

[HexPandaa]

I'm facing the same problem, however I don't have any snapshot file, only a users.db file.

[clintjedwards]

I'm facing the same problem, however I don't have any snapshot file, only a users.db file.

Next to the users.db file will be a directory with a string of characters that look like an ID(mine was 16 characters long), navigate into that directory and look for another directory called 'history'. It will be in there.

[elboulangero]

Clearly, there is something wrong with the fiveMinSnapshot.db file. It just grows forever, and not every 5 minutes, but all the time.

$ cd /home/rtorrent/.local/share/flood/db/<RANDOM-STRING>/history

$ systemctl stop flood@rtorrent.service
$ rm -fv fiveMinSnapshot.db 
removed 'fiveMinSnapshot.db'
$ systemctl start flood@rtorrent.service

$ while true; do ls -lh fiveMinSnapshot.db; sleep 5; done
-rw-r--r-- 1 rtorrent rtorrent 77 Aug 30 23:40 fiveMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent 338 Aug 30 23:40 fiveMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent 522 Aug 30 23:40 fiveMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent 783 Aug 30 23:40 fiveMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent 952 Aug 30 23:40 fiveMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent 1.2K Aug 30 23:41 fiveMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent 1.4K Aug 30 23:41 fiveMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent 1.7K Aug 30 23:41 fiveMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent 1.8K Aug 30 23:41 fiveMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent 2.1K Aug 30 23:41 fiveMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent 2.3K Aug 30 23:41 fiveMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent 2.5K Aug 30 23:41 fiveMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent 2.7K Aug 30 23:41 fiveMinSnapshot.db
-rw-r--r-- 1 rtorrent rtorrent 2.9K Aug 30 23:41 fiveMinSnapshot.db
...

[markan-io]

Same issue for me too. Just dropping this comment here to keep this alive.
Issue solved by stumbling on this thread.

In my case fiveMinSnapshot.db was almost a gig in size.
Worth noting that Flood was working fine for over a year running uninterrupted, but after server restart, it didn't wanna start anymore.

[stephen304]

I had the same issue, but didn't find the fiveMinSnapshot anywhere in the /config folder. But I did find a 4+GB transmission.log file, and I think that might have done the trick, since /config is the only folder persisted outside my container (the same path is mounted as transmission's config dir) and recreating the container didn't work.

[CircuitCoder]

For those unfortunate souls stumbling on this issue:
v4.8.0 (af8de75) removed the on-disk persistent storage of the history snapshot, so updating to >=v4.8.0 should solve any problems with fiveMinSnapshot.db . In fact, after upgrading, the entire history directory should be gone.

#834 enabled compaction for other data stores. This should solve issues caused by other data stores.