socket hang up error when using with got and http-proxy

[knolleary]

Node: v14.16.0

I have set up an http proxy using the http-proxy module and verified it is working using curl.

I am now trying to use hpagent with got to send a request via the proxy. However I am getting a socket hang up error back from got.

My best guess is something to do with how hpagent is creating the connection - hence why I'm raising the issue here. I hope someone can either point out the mistake I have made, or some suggestions on how to track this down further.

I've added a bunch of debug to http-proxy, hpagent and got, but everything looks 'normal'. On the proxy, I do see a socket get created, but the socket gets closed before it reaches any of the proxy code. As far as I can tell, that's happening within the core node.js code - somewhat harder to debug if you're not set up to do so.

To reproduce, the following code creates an HTTP server on port 9000 and the proxy on 8000.

var http = require('http'), httpProxy = require('http-proxy');
httpProxy.createProxyServer({target:'http://localhost:9000'}).listen(8000);

http.createServer(function (req, res) {
  res.writeHead(200, { 'Content-Type': 'text/plain' });
  res.write('request successfully proxied!' + '\n' + JSON.stringify(req.headers, true, 2));
  res.end();
}).listen(9000);

The client code is:

const got = require("got");
const { HttpProxyAgent, HttpsProxyAgent } = require('hpagent');
let options = {
    retry: 0,
    agent: {
        http: new HttpProxyAgent({proxy:"http://localhost:8000" })
    }
}
got("http://localhost:9000", options).then(res => { console.log(res.body)}).catch(err => { console.log(err.toString())})

As you can see, it's a fairly default configuration.

With the proxy code running, you can verify that part is working using curl:

[~]$ export http_proxy=http://localhost:8000
[~]$ curl http://localhost:9000
request successfully proxied!
{
  "proxy-connection": "Keep-Alive",
  "accept": "*/*",
  "user-agent": "curl/7.64.1",
  "host": "localhost:9000",
  "connection": "close"
}%

But running the got code which uses hpagent, you get:

[~]$ node client.js
RequestError: socket hang up

[ghost]

happens to me, but only after ~30 minutes (i do periodically disconnect the proxy by doing .destroy but it still happens)

[szmarczak]

Can confirm, got maintainer here. Those proxies don't support the CONNECT protocol. They're usual web proxies.

[aleclarson]

I have the same problem. Tested with multiple proxy agents (hpagent, https-proxy-agent, and simple-proxy-agent) and two request libraries (axios and Node https). Every time, I hit the same issue, but only with a specific domain. I tested the exact same request with HTTPie and it worked fine,

...so that leads me to believe it's a Node.js bug. I've tested on Node v17 and v15.

edit: This issue (nodejs/node#35784) mentions proxy-agent not working in Node v15, yet it reportedly works in Node v14.