Restarted process in cluster mode can't bind to a port

[sosloow]

What's going wrong?

In version >= 3.1 if a script is ran in cluster mode (in my case 2 processes) on restart one of the processes fails to bind to a port:

Error: listen EADDRINUSE 0.0.0.0:8080
    at Server.setupListenHandle [as _listen2] (net.js:1286:14)
    at listenInCluster (net.js:1334:12)
    at doListen (net.js:1460:7)
    at args.(anonymous function) (/usr/lib/node_modules/pm2/node_modules/event-loop-inspector/index.js:138:29)

In earlier versions (e.g. 3.0.0) restart works alright.

How could we reproduce this issue?

• create config for a script in cluster mode that binds to a port
• pm2 start script.json
• pm2 restart script.json
• one of the copies in cluster will keep restarting

Supporting information

$ pm2 report # for 3.2.0
===============================================================================
--- PM2 REPORT (Thu Oct 25 2018 19:56:02 GMT+0300 (Moscow Standard Time)) -----
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 3.2.0
node version         : 10.12.0
node path            : /usr/bin/pm2
argv                 : /usr/bin/node,/usr/lib/node_modules/pm2/lib/Daemon.js
argv0                : node
user                 : user
uid                  : 1000
gid                  : 1000
uptime               : 5min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 3.2.0
node version         : 10.12.0
node path            : /usr/bin/pm2
argv                 : /usr/bin/node,/usr/bin/pm2,report
argv0                : node
user                 : user
uid                  : 1000
gid                  : 1000
===============================================================================
--- System info --------------------------------------------
arch                 : x64
platform             : linux
type                 : Linux
cpus                 : Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz
cpus nb              : 2
freemem              : 784125952
totalmem             : 3705360384
home                 : /home/user
===============================================================================

$ pm2 report # for 3.0.2
===============================================================================
--- PM2 REPORT (Thu Oct 25 2018 20:08:51 GMT+0300 (Moscow Standard Time)) -----
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 3.0.4
node version         : 10.12.0
node path            : /usr/bin/pm2
argv                 : /usr/bin/node,/usr/lib/node_modules/pm2/lib/Daemon.js
argv0                : node
user                 : user
uid                  : 1000
gid                  : 1000
uptime               : 5min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 3.0.4
node version         : 10.12.0
node path            : /usr/bin/pm2
argv                 : /usr/bin/node,/usr/bin/pm2,report
argv0                : node
user                 : user
uid                  : 1000
gid                  : 1000
===============================================================================
--- System info --------------------------------------------
arch                 : x64
platform             : linux
type                 : Linux
cpus                 : Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz
cpus nb              : 2
freemem              : 531365888
totalmem             : 3705360384
home                 : /home/user
===============================================================================

[ajimix]

I'm facing exactly the same issue.
Also pm2 reload is affected

Downgrading to 3.0.3 makes the restarts/reloads work again

[kermitology]

Looks like the same as this: #3943

[sosloow]

Unfortunately, upgrading to 3.2.2 didn't work for me. And the latest version that worked is 3.0.4 - not 3.1. So maybe it's a different issue.

[djbobbydrake]

Same for me. I only saw this issue when we upgraded from node 8.x to 10.13.0. Now the reload doesn't work. I have to pm2 kill. And then pm2 start process.json.

[SaKKo]

i fix by running

pm2 update

i speculated that my problem was my deployment folder being symlinked to ./current
when it gets deleted and relinked, it sometime cause pm2 to stop functioning. mainly because i saw process.cwd() keep failing

[pankleks]

Same here, running latest pm2 3.2.2

This is serious issue, currently cluster functionallity is not usable. Issue is confirmed by multiple people.

I'm supprised that there is no response from the team for so long.

[suyu0925]

Agree with @pankleks, it's serious issue, why there is no response for so long.

Here is a temporary soluation:

running

pm2 reload appname

instead of

pm2 reload config.json

[pankleks]

@suyu0925 thanks, but your solution does not work for me - same result.

[ajimix]

@suyu0925 same here, doesn't work for me. I'm still facing this problem with latest version. I'm using 3.0.3 on production which is the one without this issue.

[appximus]

@ajimix @pankleks
have you tried pm2 update ?

[pankleks]

I'm using 3.2.2 which is latest at this time.

[ajimix]

@appximus as reported initially, the issue is affecting version >= 3.1

[kermitology]

Has anyone tried changing exec_mode to "cluster_mode"?

One of the closed issues references cluster being deprecated (or removed in this case)

#3996

[ajimix]

@kermitology I just tried and seems like it fixes the issue with pm2 3.2.2 using exec_mode as cluster_mode instead of cluster
@sosloow @pankleks can you also check to see if it fixes for you?

[pankleks]

@ajimix thanks works for me too.

Although it's hard to not comment on legitimacy of this change. I have multiple enviroments running old and new version of pm2.

At least some backward compablility should be implemented.

Anyway - thank you for help!

[sosloow]

Can confirm, it works for 3.2.2. Although I think, if the "cluster" option is deprecated, pm2 should prompt a deprecation warning or something when it is used.

[sosloow]

Closing the issue. @kermitology pointed to the right solution.

[Unitech]

inspecting

[Unitech]

Ok managed to reproduce, I'm working on it now

[Unitech]

pm2 published with the fix

$ npm install pm2@latest -g
$ pm2 update

Please let me know if you see anything wrong

[zsolt-dev]

Thank you @Unitech,

It seems to be working, but you have to first delete all apps first.. Just doing pm2 update did not help.

pm2 delete all

[t0lkman]

so what we need to use in ecosystem.config.js?
exec_mode: 'cluster_mode' or exec_mode: 'cluster'?