src: unconsume stream fix

[Kasher]

When emitting a connection event on a httpServer, the function connectionListener is called.
Then, a new parser is created, and consume method is called on the
socket's externalStream. However, if this stream was already consumed and unconsumed,
the process crashes with a cpp assert from the Consume method in
stream_base.h. This commit makes sure that no SIGABRT will be raised
and the process will stay alive (after emitting the socket).

Checklist

• make -j4 test (UNIX), or vcbuild test (Windows) passes
• tests and/or benchmarks are included
• commit message follows commit guidelines

Affected core subsystem(s)

src

Issue: #11017

[Fishrock123]

R= @indutny probably?

[indutny]

I think we'd rather want to handle it from JS. We should mark stream as previously consumed and fallback to slow .on('data') case.

Does this make sense?

[Kasher]

@indutny - Thanks for your response.
May I ask why is it better to handle this from JS?
I guess it would add some state on the stream itself, and it may be easily misused in the future (if someone uses stream in other scenarios)...... I believe that when calling to "unconsume" and resetting the stream's callbacks (in node_http_parser.cc), it should reset the stream's entire state, and make it ready to be re-consumed.
But, again, I'm not too familiar with the considerations or difficulties you had while writing this code, so please correct me if I'm wrong 👍

[indutny]

@Kasher this C++ API needs some serious renovation and this is a separate question, in my opinion.

I suggested to do it in JS, because this bug will still manifest itself if anyone would try to emit('connection', req.connection). Fixing it in JS will make your test case work as well as many other possible variations.

[Kasher]

@indutny - Cool. I got it.
Just one last question before I'm implementing this, in order to make sure I got it correctly.
I saw in node_http_parser.cc::Consume that in addition to calling "Consume" on the stream, some callbacks are being set (allocation callback and read callback). Those callbacks are then called with a context, which is the instance of the parser object.
If we remove the call to Consume in _http_server.js::connectionListener, those callbacks won't ever be set with the right context. If, however, we call to Consume, the process will crash because of the assertion in StreamBase.

How should I implement your suggestion? Should I keep the same parser for the socket in _http_server.js, and never free\unconsume it (which means some changes in the function onParserExecuteCommon in _http_server.js)?
Or, should I just skip the call to Consume in _http_server.js::connectionListener if the stream was consumed already, and as a result the allocation callback and the read callback will be called with the wrong context (an instance of a different parser)?
Or, should we add a third c++ method that only sets that callbacks and doesn't call to StreamBase::Consume, and we'll call this method in _http_server.js::connectionListener if we detect that the stream was consumed?

Thanks a lot!!

[indutny]

@Kasher there is a fallback mode there when it checks for _externalStream:

node/lib/_http_server.js

Lines 331 to 335 in a67a04d

	var external = socket._handle._externalStream;
	if (external) {
	parser._consumed = true;
	parser.consume(external);
	}

. It would be best to check if the socket was ever consumed and never enter that clause if it was.

This won't affect context of callbacks as the attached http parser (if any) will still be alive until unconsume is called.

[Kasher]

@indutny Thanks again for your help.
Maybe I'm missing something, but I'm having trouble understanding how will your suggestion work.
As I wrote before, and as you can see from the following image:

My 'connect' handler is being called from the function onParserExecuteCommon in _http_server.js :

node/lib/_http_server.js

Line 466 in a67a04d

server.emit(eventName, req, socket, bodyHead);

A few lines before my handler is called, there is a call to unconsume on the attached http-parser:

node/lib/_http_server.js

Line 453 in a67a04d

unconsume(parser, socket);

Hence, when my connect handler is called, there is no http-parser attached to the socket. If I remove the call to consume as you suggested, no parser will be attached at all, and no callbacks will be set to the socket's _externalStream.
Do you suggest to remove the call to unconsume as well, so the parser will still be attached? This will result in a refactor in the onParserExecuteCommon method, since I guess we shouldn't execute the following lines as well:

node/lib/_http_server.js

Lines 450 to 456 in a67a04d

	socket.removeListener('data', state.onData);
	socket.removeListener('end', state.onEnd);
	socket.removeListener('close', state.onClose);
	unconsume(parser, socket);
	parser.finish();
	freeParser(parser, req, null);
	parser = null;

Did I miss something?

Thanks a lot, your help is really appreciated!!!

[indutny]

@kosher consume() is just one way to attach socket, the fallback is to attach socketOnData event listener. I suggest that consume() should not be called if the socket was ever previously consumed (even if it is not consumed right now).

[Kasher]

@indutny Thanks again.
I've updated my commit, and I would really appreciate any other comments 👍

[indutny]

Few style nits, otherwise looks good!

[indutny]

Minor style nit:

var external = socket._handle._externalStream;
if (!socket._handle._consumed && external) {
  ...
}

Less indent - awesome 😉

[joyeecheung]

Looks like it's ready for a CI: https://ci.nodejs.org/job/node-test-pull-request/6103/

[Kasher]

@indutny @joyeecheung

The linux tester failed for fedora24 - 32bit on the test sequential/test-child-process-pass-fd
Do you think it is related to my changes?
I've tried to run this test like 10 times, and it always passes (I'm testing it on ubuntu 14.04).
This test did pass for fedora24 - 64bit though.

So, just to verify - is this by any chance a known issue? If it isn't - I'll download the appropriate iso and create a VM to test this.

Thanks.

[joyeecheung]

Yeah it's a known issue(#11041), also the arm failure is actually not a failure, it's related to some infra issue so it's actually green in the webpage. I'll say CI is green for this PR.

[Kasher]

@joyeecheung Awesome, thanks!

[jasnell]

Landed in 3e3bfc5