Agent never establishes connections simultaneously

[yannooo]

I was trying to open 100 concurrent connections to an http server by setting
the agent maxSockets property to 100. During my tests I was unable to open more
than 10 concurrent connections to the server.
Looking at the agent's code in the _cycle method I discovered the agent won't open
a new connection if it can find a connecting socket. I believe this is wrong because
it won't try to establish multiple connections in parallel.

By removing this line in the _cycle command, new connection will be established even
if one is pending.
if (socket._httpConnecting) haveConnectingSocket = true;

I've made a small node program that shows that no more than one socket will be created if I create an http server that don't respond to http requests.
https://gist.github.com/903338

I'm using latest node version v0.4.5
Let me know if you need anything else.

[tenorviol]

Me too. Why is this a feature and not a bug? Particularly since there is a cached agent per host:port anyways.

[koichik]

You should understand async I/O. Node works fine 100 concurrent connections.

var http = require('http');

var agent = http.getAgent('127.0.0.1', 8124);
agent.maxSockets = 100;

var server = http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.write('Hello World\n');
  //res.end();
});

server.listen(8124, "127.0.0.1", function() {
  for (var i = 0; i < 100; i++) {
    var options = {
      host: '127.0.0.1',
      port: 8124,
      path: '/',
    }
    http.get(options, function(res) {
    });
  }
});

agent.on('connect', function() {
  console.log('Socket: ' + agent.sockets.length + '/' + agent.maxSockets +  ' queued: '+ agent.queue.length);
});

http.get() does not connect immediately. It is an asynchronous.

[tenorviol]

Koichik,

The problem is not a misunderstanding of async I/O. The problem is that when you're trying to spam a server with many simultaneous http requests, the http Agent doesn't let you do it. That's because of the way the agent processes requests. When you make 100 http.get calls to the same server, the Agent queues all of those requests. Then it takes the first request and starts connecting. While it's waiting for a connection to the server, all 99 other requests are waiting in the queue. Once the first connection occurs, that transaction will do its thing and the second queued get will start connecting. Now you have one transacting connection, one pending connection, and 98 requests waiting in the queue. When the time it takes to do the transaction outweighs the time it takes to connect, the Agent will never reach the maxSockets.

There is a way around this, by setting the { agent: false } in the http.connect options. Doing that does away with the inconvenience of the agent. But it seems the agent is supposed to do something important involving governing the flood of http requests. So yannooo's solution seems reasonable although I don't fully understand the full ramifications of that change.

Chris

[koichik]

Thanks, Christopher.

Okay, I understand.
The title is wrong because Agent finally reaches maxSockets.
It should be "Agent never establishes connections simultaneously" shouldn't it?

[yannooo]

Just renamed the issue title to your proposition. It is more accurate.

[davepacheco]

I'm hitting this (or something with the same symptoms) with v0.4.8. I'm trying to write an HTTP load generator in node. The "agent: false" workaround works, as does setting the "connection: keep-alive" header with each request.

[murvinlai]

After reading lots of post, I find out:

1. If I set the headers with connection = 'keep-alive', the performance is good and can go up to maxSocket = 1024 (which is my linux setting).

var options1 = {
                  host: 'www.google.com',
                  port: 80,
                  path: '/',
                  method: 'GET',
    headers: {
            'Connection':'keep-alive'
    }
                };

If I set it to "Connection":"close", the response time would be 100 times slower.

Funny things happened here:

1. in EC2, when I first test with Connection:keep-alive, it will take about 20-30 ms. Then if I change to Connection:Close OR set Agent:false, the response time will slow down to 300 ms. WIHTOUT restarting the server, if I change to Connection:keep-alive again, the response time will slow down even further to 4000ms. Either I have to restart server or wait for a while to get back my 20-30ms lighting speed response.

2. If I run it with agent:false, at first, the response time will slow down to 300ms. But then, it will get faster again and back to "normal".

My guess is connection pooling still in effect even you set agent:false. However, if you keep connection:keep-alive, then it will be fast for sure. just don't switch it.

[murvinlai]

Additional:

for httpfix, it works. However, compare to just keep-alive, it is slower. e.g. around 200ms call. while doing keep-alive requires 60ms. However, when I look into keep-alive log, actually, it takes a longer time (200-300 ms) in the very beginning then speed up to 60ms. httpfix is steadily 190-200ms.

In addition, i find out with both approach, if I run a 100concurrent x 100 loop test, it usually fails the call at 9000 something andn waiting for some callbacks. If one wait, the subsequent response will be on hold.

[murvinlai]

In addition:
What I find out is, if set agent:false with connection:keep-alive, the speed is consistent but a bit slower. If agent is set with connection:keep-alive, sometimes, the first couple requests will be slower but then it will be very quick. (faster than agent:false). if connection not set, or connection:close, then it will be a disaster.