Hystrix ThreadPools are not shutdown, system hangs. #102
Comments
|
At a minimum I'll add a shutdown method if there isn't something cleaner. Interesting that it doesn't allow shutdown as I would have thought it would also prevent unit tests from shutting down. and I've never had that issue. JUnit must force a System.exit. |
ghost
assigned 
|
Thanks A shutdown of com.netflix.hystrix.HystrixThreadPool.Factory#threadPools would be great. Will be looking forward to the next release. |
|
Please take a look at the pull request and let me know if that would solve this. If so I'll merge and release. |
|
I am modifying my approach to one that is more generic so that it supports resetting anything that has state and supports possible future work that could involve other resources besides thread-pools that need to be cleaned up if an app wants a graceful shutdown rather than just being terminated. The new command will be: Here is my test case: package com.netflix.hystrix;
public class TestHystrixApp {
public static void main(String args[]) {
System.out.println(new TestCommand().execute());
System.out.println("reset");
Hystrix.reset();
// the reset allows us to start using Hystrix again
System.out.println(new TestCommand().execute());
System.out.println("ending");
// or it allows a clean shutdown
Hystrix.reset();
}
private static class TestCommand extends HystrixCommand<String> {
public TestCommand() {
super(HystrixCommandGroupKey.Factory.asKey("test"));
}
@Override
protected String run() throws Exception {
return "hello";
}
}
}It results in this with the reset() function being called: WIthout the reset() it does not exit unless System.exit() is called (or the process is killed). |
I'm changing the design from the previous commits so it's more abstract and can handle any type of resources needing cleanup, not just threadpools. ReactiveX/RxJava#45
|
I merged the code which adds I'll release the code shortly. |
|
Released in 1.2.8 which is making its way to Maven Central now. https://github.com/Netflix/Hystrix/blob/master/CHANGES.md#version-128-maven-central |
|
I also ran into this, is there a reason you are not using deamon threads for the thread pool which would not have this issue, seems odd to have to explicitly close down the thread pool when exiting. |
|
Good question - is there any specific reason for the threads not being daemon threads??? |
Hi,
We are having an issue with Hystrix where the ThreadPools are not shutdown, so when our process tries to terminate it can't due to treads still in waiting state. According to the logs 2 HystrixCommands ran with an exception happened in run so fallback got executed. What we had to do is to keep track of Hystrix Thread Pools and when the main process terminates (hook in via a Listener) we shutdown all Hystrix TreadPools. via com.netflix.hystrix.HystrixThreadPool.Factory.getInstance() (unfortunately Factory class is package-private which forces us to create our class in com.netflix package). Can you suggest a better way of dealing with this issue or perhaps provide a better way to gracefully shutdown all ThreadPools, would settle fo making com.netflix.hystrix.HystrixThreadPool.Factory public?
Here is the full thread dumb
Regards
Denis
The text was updated successfully, but these errors were encountered: