Attempt to handle KeyboardInterrupt exceptions #50
Conversation
…worker processes
Traces would look something like:
```
Traceback (most recent call last):
File "/usr/lib64/python2.6/multiprocessing/process.py", line 232, in _bootstrap
self.run()
File "/usr/lib64/python2.6/multiprocessing/process.py", line 88, in run
self._target(*self._args, **self._kwargs)
File "/home/thjackso/src/pythonfutures/concurrent/futures/process.py", line 122, in _process_worker
call_item = call_queue.get(block=True)
File "/usr/lib64/python2.6/multiprocessing/queues.py", line 91, in get
res = self._recv()
```
|
After looking some it seems that python3 has the same issues (unhandled keyboardinterrupt and no terminate() method), so if we think this is a valid fix I can start the ball rolling getting this into python3 as well, but from the issue thread this bug is less of an issue in python3. |
|
As this is a backport, I only accept fixes that are either backport specific or have been applied upstream as well. API changes are a problem in that regard. And not all upstream fixes are even applicable due to some capabilities being unavailable in older Pythons. But if you get your patch accepted upstream, I'm willing to take another look at it and integrate it into the backport if possible. Would that first commit work on its own? The KeyboardInterrupt catch I mean? |
|
Yes it would, but as my comment states its also a bug upstream. Sounds like
|
|
Upstream issue: http://bugs.python.org/issue25908 |
For #25 there are effectively two problems. (1) the worker process doesn't handle the keyboard interrupt and as such the process pool cannot shutdown properly. c2ae8dc fixes this issue, such that the worker processes handle the exception and continue working. (2) There is no way to terminate a process pool. In the event that a keyboard interrupt was received and you do in fact want to exit-- you have no choice but to wait for the tasks to complete (assuming you have the first fix I mentioned). the addition of a terminate() method (f5727bd) allows you to do a more forcible shutdown of the pool-- clearing all unstarted jobs and forcibly killing all inflight processes.
So to recap c2ae8dc makes it so you can handle the keyboard interrupt from the caller and f5727bd gives you a mechanism to stop the pool.
Potential fix for #25