Assertion `parser->current_buffer_.IsEmpty()' Exception

[omkadiri]

Version: 12.0.0
Platform: UNIX AWS EC2

Processing lot of json file from S3 using worker_thread node module in an EC2(m4.xlarge)

Process step
(1)Got message for SQS
(2)Use message info to list bucket using listObjectV2
(3)Get each json from the list api call and process it(merge the data into one file).
(4)write back to S3.

This error only occurs when the number of json file increase to hundred of thousand

Exception message from run
/.nvm/versions/node/v12.0.0/bin/node[3884]: ../src/node_http_parser_impl.h:500:static void node::{anonymous}::Parser::Finish(const v8::FunctionCallbackInfov8::Value&): Assertion `parser->current_buffer_.IsEmpty()' failed.
1: 0x97a1c0 node::Abort() [/.nvm/versions/node/v12.0.0/bin/node]
2: 0x97a247 [/.nvm/versions/node/v12.0.0/bin/node]
3: 0x991492 [/.nvm/versions/node/v12.0.0/bin/node]
4: 0xb7a2b6 [/.nvm/versions/node/v12.0.0/bin/node]
5: 0xb7c1c9 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/.nvm/versions/node/v12.0.0/bin/node]
6: 0x1a5c1a2 [/.nvm/versions/node/v12.0.0/bin/node]

This file processing run successfully 99% of the time.

Node should catch this exception and print a detail message for why this exception occurs.

[addaleax]

Is there any chance you can verify that this also happens on the latest v12.x release? v12.0.0 was over a year ago, and lots of bug fixes have gone into it since then.

[omkadiri]

I got the same exception running with version 12.18.1

/.nvm/versions/node/v12.18.1/bin/node[3921]: ../src/node_http_parser_impl.h:509:static void node::{anonymous}::Parser::Finish(const v8::FunctionCallbackInfov8::Value&): Assertion `parser->current_buffer_.IsEmpty()' failed.
1: 0xa08900 node::Abort() [/.nvm/versions/node/v12.18.1/bin/node]
2: 0xa0897e [/.nvm/versions/node/v12.18.1/bin/node]
3: 0xa20302 [/.nvm/versions/node/v12.18.1/bin/node]
4: 0xbeaee9 [/.nvm/versions/node/v12.18.1/bin/node]
5: 0xbeccd7 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/.nvm/versions/node/v12.18.1/bin/node]
6: 0x13c9919 [/.nvm/versions/node/v12.18.1/bin/node]

[bnoordhuis]

I haven't been able to reproduce but I can see the general shape of the bug. The call chain probably looks like this:

• Parser::Execute() -> llhttp_execute() -> Parser::on_body() -> parserOnBody() -> ... -> parser.finish()

I think it should be safe to remove the CHECK and reset the current_buffer_ fields like this:

diff --git a/src/node_http_parser.cc b/src/node_http_parser.cc
index c7a3df8d06..f9729f3f51 100644
--- a/src/node_http_parser.cc
+++ b/src/node_http_parser.cc
@@ -514,7 +514,9 @@ class Parser : public AsyncWrap, public StreamListener {
     Parser* parser;
     ASSIGN_OR_RETURN_UNWRAP(&parser, args.Holder());
 
-    CHECK(parser->current_buffer_.IsEmpty());
+    parser->current_buffer_.Clear();
+    parser->current_buffer_len_ = 0;
+    parser->current_buffer_data_ = nullptr;
     Local<Value> ret = parser->Execute(nullptr, 0);
 
     if (!ret.IsEmpty())

Opinions? Parser::Execute() performs the same reset after calling llhttp_execute().

@omkadiri If you want you can compile Node from source with that patch applied until this issue is resolved.