Unexpected socket hang up triggered by http.ClientRequest

[c4milo]

I've been bending HTTP a bit to implement a bidirectional communication that works within the scope of RFC2616 without using websockets nor the known polling techniques. While doing that in node 0.6.11, I ran into an unexpected socket hang up, triggered by http.ClientRequest. I've been wondering whether it's something that has to be fixed in http.ClientRequest or I definitely would have to use the net module to write my own client.

I wrote some code that reproduces the issue in https://gist.github.com/1889610

FWIW, I had it working before in nodejs 0.4.12, with similar code, without any issues.

Any help is very much appreciated.

[c4milo]

BTW, this is the output if I run:
$ NODE_DEBUG="net http" node client.js

NET: connect: find host localhost
HTTP: write ret = false
NET: afterConnect
HTTP: write ret = true
NET: Drain the connect queue
NET: destroy
NET: close
{ [Error: socket hang up] code: 'ECONNRESET' }
HTTP: HTTP socket close
socket closed

I don't understand why the socket is getting destroyed since I am not closing or destroying the connection in client.js.

[bnoordhuis]

Your example is working as I would expect it to in v0.6.x. You need to pass a custom http.Agent object to http.request(), the default one closes the connection when there are no requests queued up.

[OrangeDog]

Where does one get a custom http.Agent from? The documentation does not say.

[bnoordhuis]

http://nodejs.org/docs/latest/api/http.html#http.Agent

Subclass it, then pass it in the options argument to http.request().

[OrangeDog]

Yes, but you shouldn't have to look through the source code to see what you can override.
Also, new http.Agent({closeIdleConnections: true, etc.}) would be a better interface than writing a subclass.

[bnoordhuis]

Yes, but you shouldn't have to look through the source code to see what you can override.

Documentation patches are welcome.

[c4milo]

Hey @bnoordhuis, why if I put the first write, on socket connection, within an interval function of zero ms, it does work as expected without a custom Agent? See please client.js gist .

[c4milo]

I also disagree with the current default behavior since it does not honor rfc2616.

[bnoordhuis]

Hey @bnoordhuis, why if I put the first write, on socket connection, within an interval function of zero ms, it does work as expected without a custom Agent?

Because the connection is still open at that point, it's closed at the next tick of the event loop.

[c4milo]

hmm, I am still not convinced. If you change the 0 for 1000 ms it will still work as expected. As I said, it does not honor rfc2616, http.ClientRequest should not close a keep-alive connection at all, unless the user directly invokes .end(). It's like if a browser closes the connection deliberately while getting the chunked response from the server because there are not more requests queued up.

[c4milo]

@mikeal any opinions here?

[tamzinblake]

@c4milo long rfc is long. Maybe it would help folks if you pointed to the bit that's being violated.

[c4milo]

http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html > 8.1.2.1 Negotiation

[tamzinblake]

@c4milo No violation there. There is only one 'MUST' in that section, and it does not pertain to your complaint. Also, there are 2 'SHOULD's and 1 'SHOULD NOT', and those don't seem to be violated either. As far as I can see, servers can throw away keep-alives to their hearts' content.

Remember that 'MAY' is there specifically so that vendors can establish reasonable behavior that works for their unique situations. So it's entirely up to Node to decide whether this is a reasonable default, without in any way failing to honor the RFC.