Add highwatermark to readable in buffer.tostream

[rclark]

I ran into this issue with recursive nextTick calls again and thought I'd just throw in a PR that helps: see #158 (comment). Again, while this problem does represent a problem in the node.js stream api, adding this is a very simple workaround.

p.s. this time I encountered the error when uploading a ~25MB zip file to lambda.

[lsegal]

I'm not sure we can pull this in. This change causes some of our progress event tests to fail because adding a HWM causes Node.js to stop emitting granular progress events, and based on the documentation, it looks like this change would cause Node.js to buffer 5mb chunks into memory, which would potentially incur a significant memory performance regression into the SDK.

Out of curiosity, are you running into this on a small EC2 instance? We've noticed this behavior on t2.micro type instances because of the known issue in 0.10.x that you mentioned in #158. I think the better solution here might be to upgrade to 0.12.x now that it is an officially supported version, which should resolve your nextTick issue.

Let me know what you think.

[rclark]

I think the better solution here might be to upgrade to 0.12.x

This sounds fine to me. We encounter this problem rarely and can usually find a way around it. I've seen it crop up on all kinds of EC2s. I've never run a t2 instance so I'm afraid I can't speak to that.

[lsegal]

@rclark in that case I'm going to mark this PR as closed, but thanks for the extra info!

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.