Dgram reuseAddr not working with cluster

[hit9]

Hi, it seems that reuseAddr is not working well with cluster module, when we bind dgram socket in
a worker process. The following case runs well, I could run this program for two or more instances without an EADDRINUSE error, this is the except behavior.

var dgram = require('dgram');
dgram.createSocket({type: 'udp4', reuseAddr: true}).bind(9527);

But this EADDRINUSE error occurred when the socket was created and bound in a cluster worker:

var dgram = require('dgram');
var cluster = require('cluster');

if (cluster.isMaster) {
  cluster.fork();
} else {
  dgram.createSocket({type: 'udp4', reuseAddr: true}).bind(9527);
}

Tested with node 0.12.2 and iojs 2.0.2

Is this a bug or a feature?

[mscdex]

I just tried that code on master on Linux and it works for me.

[hit9]

@mscdex Have you run two or more instance the same time?

[mscdex]

I see what you mean now. I have a fix incoming...

[hit9]

Thanks @mscdex , and I have some questions:

1. Is reuseAddr the SO_REUSEADDR or SO_REUSEPORT in nodejs?
2. If I start 2 udp server instances which share the same port (reuseAddr is true), how is the data handled? Will linux dispatch data to the 2 processes via round-robin ?

Thanks a lot.

[mscdex]

@hit9

1. Yes.
2. UDP packet distribution is left up to the OS.

[hit9]

1. Both them will be set if reuseAdd is true ?
2. It seems that data will always be sent to the instance which bind the port first. Maybe the OS shards packets by data's incoming address.. I just wonder if the option reuseAddr can be used as a way of load balancing. It seems that nodejs just bind only 1 udp socket on cluster mode, the socket can be too busy when incoming data stream is very fast. I am finding a way to make "1 socket to N sockets".

[mscdex]

1. No, either one or the other, depending on the operating system. Currently SO_REUSEPORT is used on non-Linux systems that support it, otherwise SO_REUSEADDR is used for those systems that support it.
2. I believe I read somewhere once that Linux uses a hash with [srcIP, srcPort, dstIP, dstPort] as the key, but I could be wrong on that (for UDP anyway). However that could easily be different on other operating systems. As far as I know there is no technical barrier to allowing the usage of cluster's built-in round robin scheduler for UDP sockets.

[hit9]

Thanks.

1. I just found the following code in libuv (https://github.com/libuv/libuv/blob/v1.x/src/unix/udp.c#L253):

   /* On the BSDs, SO_REUSEPORT implies SO_REUSEADDR but with some additional
   * refinements for programs that use multicast.
   *
   * Linux as of 3.9 has a SO_REUSEPORT socket option but with semantics that
   * are different from the BSDs: it _shares_ the port rather than steal it
   * from the current listener.  While useful, it's not something we can emulate
   * on other platforms so we don't enable it.
   */
   static int uv__set_reuse(int fd) {
   int yes;
   
   #if defined(SO_REUSEPORT) && !defined(__linux__)
   yes = 1;
   if (setsockopt(fd, SOL_SOCKET, SO_REUSEPORT, &yes, sizeof(yes)))
     return -errno;
   #else
   yes = 1;
   if (setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes)))
     return -errno;
   #endif
   
   return 0;
   }

   That means that the REUSEPORT in nodejs is currently disabled on Linux...

2. The cluster is actually serving the only one socket, isn't it ? It's different with tcp, where epoll works well with multiple client sockets, the udp server talks to clients on only one socket, that makes the bottleneck (packet loss).

[hit9]

@mscdex

[mscdex]

@hit9

1. Right, but SO_REUSEADDR would be used on Linux.
2. I dunno, I haven't really worked with the cluster module at all before this. You might create an issue as a feature request if there is new functionality that you would like to see and is doable.

[mscdex]

Fixed by c7be08c.