Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qcluster command doesn't handle interrupts. #56

Closed
carltongibson opened this issue Sep 9, 2015 · 9 comments
Closed

qcluster command doesn't handle interrupts. #56

carltongibson opened this issue Sep 9, 2015 · 9 comments

Comments

@carltongibson
Copy link

The management command is intercepting interrupts but then not handling them.

Thus I get an error and can't exit...

  File "/Users/carlton/ve/nws/lib/python2.7/site-packages/django_q/cluster.py", line 520, in set_cpu_affinity
    p.cpu_affinity(affinity)
AttributeError: 'Process' object has no attribute 'cpu_affinity'
^C^C^C^C^C^C^C^C^C^C^C

I have to kill the shell and start over. It would be nice if ^C worked here.

@carltongibson
Copy link
Author

^C works after startup — the problem here is with an error before then.

@Koed00
Copy link
Owner

Koed00 commented Sep 9, 2015

The cluster command does explicitly handle interrupts, it's just that the child processes ignore interrupts and only look to the parent process for a stop signal. Which doesn't really work well when that process just crashed. I'll experiment a little to see what can be improved there.

More interesting is your error. I assume you installed ``psutil. So`cpu_affinity` is either not available on your platform or something is happening with the freshly spawned workers.

@carltongibson
Copy link
Author

I assume you installed psutil. So cpu_affinity is either not available on your platform or something is happening with the freshly spawned workers.

Indeed, yes, I installed psutil. I'm on OS X (10.10.5)

@Koed00
Copy link
Owner

Koed00 commented Sep 9, 2015

Unfortunately it seems that cpu_affinity is just not supported on OS X by psutil.
Apparently the API for this on OS X didn't exist until 10.5.

All I can do is add a clause to catch this exception.

@carltongibson
Copy link
Author

OK. That's interesting. 10.5 was a while ago... is there a link to an issue somewhere?

@Koed00
Copy link
Owner

Koed00 commented Sep 9, 2015

There is this giampaolo/psutil#238 , but it got closed since no one replied to the OS X portion of it.

By the way, if you are just planning on developing on OS X and deploying on a different platform that does support cpu_affinity. You can set the testing option in the Q_CLUSTER dict to True or DEBUG and it will use all the code right up to the point that the actual affinity is set. I added this cause the Travis containers don't like cpu affinity either, but I didn't want it to affect my coverage too much.

@carltongibson
Copy link
Author

OK. Cool. Thanks. (Side note: the testing key isn't documented, as yet.)

@Koed00
Copy link
Owner

Koed00 commented Sep 9, 2015

Is there any chance you could try the Dev branch for me?
I have no access to OSX and Travis does have the cpu_affinity attribute but just crashes on it.

I made it so it will warn you in the log that cpu affinity is not supported and then continue the code as if testing was set. So you can forgo setting that manually.

@carltongibson
Copy link
Author

Is there any chance you could try the Dev branch for me?

No problem. Will report back later on.

Koed00 added a commit that referenced this issue Sep 9, 2015
#56 cpu_affinity not supported on some platforms
@Koed00 Koed00 closed this as completed Sep 9, 2015
msabatier pushed a commit to msabatier/django-q that referenced this issue Jan 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants