Javascript heap out of memory

[alxndrsn]

Environment

• better-sqlite3: v7.5.3
• journal_mode = WAL
• busy_timeout = 5000
• synchronous = NORMAL
• nodejs: v16.17.1

Description

I'm seeing occasional Javascript heap out of memory errors when running a project's test suite. Frequency is about 1 in 100 times (estimated - I haven't yet measured this).

I have verbose logging enabled, and if the logging from that is output before the bad db interaction, the crash seems to happen at two specific points in the test suite:

1. running db.exec('DELETE FROM a;DELETE FROM b')
2. running db.prepare('SELECT ...').get([ param1, param2 ]); this is selecting 7 columns from a table containing less than 10 rows.

Before the crash, the process appears to hang for some time. I haven't yet timed this, but I'd guess in the order of 10s of seconds.

Is this a known issue? I could find a couple of similar previous issues, but neither seems directly relevant:

• Cannot run benchmarks #20 (user permissions)
• memory crash with iterate() after 10M records #280 (excessive logging)

Example log

<--- Last few GCs --->

[35318:0x5fd48e0]    78415 ms: Scavenge 2033.0 (2063.2) -> 2029.4 (2063.5) MB, 15.4 / 0.0 ms  (average mu = 0.213, current mu = 0.179) allocation failure 
[35318:0x5fd48e0]    78432 ms: Scavenge 2033.2 (2063.5) -> 2029.6 (2063.7) MB, 5.6 / 0.0 ms  (average mu = 0.213, current mu = 0.179) allocation failure 
[35318:0x5fd48e0]    78455 ms: Scavenge 2033.4 (2063.7) -> 2029.8 (2072.0) MB, 16.7 / 0.0 ms  (average mu = 0.213, current mu = 0.179) allocation failure 


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb02960 node::Abort() [~/.nvm/versions/node/v16.17.1/bin/node]
 2: 0xa18149 node::FatalError(char const*, char const*) [~/.nvm/versions/node/v16.17.1/bin/node]
 3: 0xcdd22e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [~/.nvm/versions/node/v16.17.1/bin/node]
 4: 0xcdd5a7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [~/.nvm/versions/node/v16.17.1/bin/node]
 5: 0xe94c15  [~/.nvm/versions/node/v16.17.1/bin/node]
 6: 0xe956f6  [~/.nvm/versions/node/v16.17.1/bin/node]
 7: 0xea3c1e  [~/.nvm/versions/node/v16.17.1/bin/node]
 8: 0xea4660 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [~/.nvm/versions/node/v16.17.1/bin/node]
 9: 0xea7555 v8::internal::Heap::HandleGCRequest() [~/.nvm/versions/node/v16.17.1/bin/node]
10: 0xe34bf7 v8::internal::StackGuard::HandleInterrupts() [~/.nvm/versions/node/v16.17.1/bin/node]
11: 0x11e0fd5 v8::internal::Runtime_StackGuard(int, unsigned long*, v8::internal::Isolate*) [~/.nvm/versions/node/v16.17.1/bin/node]
12: 0x15d54f9  [~/.nvm/versions/node/v16.17.1/bin/node]

[alxndrsn]

🤦 very unrelated to better-sqlite3 - sorry for opening this here.