Out of memory in parseStack.

[cveilleux]

My node app starts with ~250MB ram usage.

When an error occur, the ram usage spikes over 2GB and the node process crashes because it runs out of the default RAM limit (~1.75GB).

Upon investigation, it is the Raven code that consumes all that ram.

The utils.parseStack() implements a simple cache to avoid parsing the same javascript file multiple times if there is more than one stack frame on the same filename for the error.

It performs an fs.readFile() which is an async call, and adds the result to the cache on success.

My app is a webpack bundle on the server-side, and it results in an out of memory error because the javascript bundle is about 9mb and is being parsed over 20 times concurrently.

It seems like the cache is not working, the file should be parsed only once.

What happens is that all the frames will generate a cache-miss, and then populate the same cache key on success... rendering the cache useless.

Relevant piece of code is here:
https://github.com/getsentry/raven-node/blob/master/lib/utils.js#L204-L213

You can see the cache key is set only on success.

A possible fix would be to split the code into 2 parts: First, loop over the frames and load all files, fully populating the cache.
Wait for all cache entries to be loaded, then perform the parseLines() on each stack frame.

[cveilleux]

This is how node js crashes when it runs out of memory:

<--- Last few GCs --->

   30250 ms: Mark-sweep 1388.2 (1428.0) -> 1379.5 (1424.3) MB, 43.3 / 0 ms (+ 6.4 ms in 112 steps since start of marking, biggest step 0.2 ms) [allocation failure] [GC in old space requested].
   30293 ms: Mark-sweep 1379.5 (1424.3) -> 1379.5 (1430.3) MB, 42.9 / 0 ms [allocation failure] [GC in old space requested].
   30343 ms: Mark-sweep 1379.5 (1430.3) -> 1379.5 (1426.3) MB, 50.4 / 0 ms [last resort gc].
   30383 ms: Mark-sweep 1379.5 (1426.3) -> 1379.5 (1426.3) MB, 40.3 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3a80e82c9fa9 <JS Object>
    1: toString [buffer.js:523] [pc=0x71d5d43d01c] (this=0xcf8a89fc3b1 <an Uint8Array with map 0x473fe705549>)
    2: /* anonymous */(aka /* anonymous */) [/home/xxx/git/xxx/node_modules/raven/lib/utils.js:206] [pc=0x71d5d22ef03] (this=0x3a80e8204189 <undefined>,_err=0x3a80e8204101 <null>,file=0xcf8a89fc3b1 <an Uint8Array with map 0x473fe705549>)
    3: oncomplete(aka readFileAfter...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

notice how it crashes at raven/lib/utils.js:206

[LewisJEllis]

Thanks for the report, @cveilleux. If you have a bundle of that size, I can definitely see this happening, as I described in my second comment on #318, which (great timing) implements roughly the fix you've described here. I'll get a patch release out tomorrow with that change to fix this, and I'll test with:

• a unit test using a spy to make sure we don't read the same file repeatedly
• a case in our manual memory tests to confirm that it cuts down on the memory expansion

[LewisJEllis]

@cveilleux I just published version 2.0.1 with this fix.

[lxcid]

The memory usage is still fairly significant it seems. I have a 512mb memory limit, my app average at 100mb. It spike over the limit of 512mb and overshot to 650mb which I believe at this point Heroku is hard throttling the memory usage. The error never get sent to sentry.

The error is just a simple throw new Error('Broke!'), similar to the one taught by the tutorial.

It's a showstopper for me as it took so much memory though.

I also get bad decoding and timeout error but on my local machine setup.

[LewisJEllis]

@lxcid do you have any more details I could use to try to reproduce? I added tests for memory usage during capture/parsing with the PR that fixed this issue, and they don't show any spike like what you're describing, even on a massive file with a massive stack trace. What version of raven are you using? Have you narrowed the increased memory usage to being "when an exception is captured" or just to "when raven is added to my application"?

Could you paste the exact error(s) you get locally regarding bad decoding and timeout error?

[lxcid]

Hi! I'm using raven@2.0.2, I'll try to setup a clean setup and try it out again and file you a repo that can be deploy to heroku for monitoring.

Also, I been facing 413 undefined, 400 bad memory and timeout when sending from raven node, but I didn't see anyone facing similar issue. I'll hope to setup such that I can reproduce the issue and send a ticket your way! Thanks for responding!