Skip to content
This repository has been archived by the owner on Sep 25, 2019. It is now read-only.

Resource leak that brings down the Hub #54

Closed
Splaktar opened this issue Sep 18, 2015 · 4 comments · Fixed by #57
Closed

Resource leak that brings down the Hub #54

Splaktar opened this issue Sep 18, 2015 · 4 comments · Fixed by #57
Labels
bug
Milestone
v0.1.0

Comments

@Splaktar
Copy link
Contributor

Last night, the Hub stopped responding to all requests. Antonio Bertucci let us know about it in the Gitter chat about 5 hours ago.

When I logged into the console for the Hub and attached to the tmux session, I found the following issue:
nodejs/node-v0.x-archive#9409

There is a work around in that thread, but I haven't had time to try it. Also we've only seen this issue once, so it may be a fluke. If it happens again, and the work around fixes it, we can be a bit more confident.

There is a PR for the issue but it was never merged and now appears to be targetted at an abandoned repo of Node.js.
@Splaktar Splaktar added the bug label Sep 18, 2015
@Splaktar Splaktar added this to the v0.1.0 milestone Sep 18, 2015
@Splaktar
Copy link
Contributor Author

Happened again last night:

[10298] GET / - 17524 ms - 2296
assert.js:86
throw new assert.AssertionError({
        ^
AssertionError: Resource leak detected.
at removeWorker (cluster.js:346:9)
at ChildProcess.<anonymous> (cluster.js:366:34)
at ChildProcess.g (events.js:199:16)
at ChildProcess.emit (events.js:110:17)
at Process.ChildProcess._handle.onexit (child_process.js:1074:12)

@Splaktar
Copy link
Contributor Author

Had to restart the hub again tonight.

Splaktar added a commit that referenced this issue Sep 28, 2015
@Splaktar
fix(cluster): fix server outages due to cluster resource leakage
227f519 
Add initWorker() to server which should clean up worker resources on both disconnect and exit.
Reference nodejs/node-v0.x-archive#9409
Update JSHint for server and fix some issues including removing unused requires.
Add key entries for frisbee keys.
Remove some newRelic cruft.

Fixes #54.
@Splaktar
Copy link
Contributor Author

Went down again a couple hours ago, working on a fix...

@Splaktar Splaktar mentioned this issue Sep 28, 2015
Merged
@tasomaniac
Copy link
Member

I happened 3 times in 10 days. Waiting for 4 or more days makes sense.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
v0.1.0
Development

Successfully merging a pull request may close this issue.

fix(cluster): fix server outages due to cluster resource leakage gdg-x/hub
2 participants
@tasomaniac @Splaktar