-
Notifications
You must be signed in to change notification settings - Fork 29.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Core dump on process.stdout.write() or console.log() for large files #5131
Comments
Including node & host version information $ node --version |
This is not unexpected. You read faster than you write, and eventually run out of memory. |
Writing to stdout and stderr is async and as @vkurchatkin said, the output to the terminal is being increasingly buffered in memory because writing is async and it takes longer for those writes to complete than to read data from the file. |
Let's close this. There is no back-pressure in the OP's example so there is no way for node.js to know it needs to slow down reading from that file. Caveat emptor. I checked the documentation and it looks like I forgot to update |
Mea culpa, looks like I forgot to update console.markdown in commit dac1d38 ("doc: stdout/stderr can block when directed to file"). This commit rectifies that. Refs: nodejs#5131 PR-URL: nodejs#5133 Reviewed-By: Brian White <mscdex@mscdex.net> Reviewed-By: Evan Lucas <evanlucas@me.com>
Mea culpa, looks like I forgot to update console.markdown in commit dac1d38 ("doc: stdout/stderr can block when directed to file"). This commit rectifies that. Refs: nodejs#5131 PR-URL: nodejs#5133 Reviewed-By: Brian White <mscdex@mscdex.net> Reviewed-By: Evan Lucas <evanlucas@me.com>
Using filesystem stream read a large file for size ~6GB having random contents, write the read contents to the console using console.log() or process.stdout.write(), node hangs for about a minute and dumps core.
No issues noted while piping the read stream to fs.writable stream() & file written successful.
btw, executing "cat " on the same large file completes successfully.
Node script:
<--- Last few GCs --->
<--- JS stacktrace --->
[bnoordhuis - fixed formatting]
The text was updated successfully, but these errors were encountered: