Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add usable_cpu_count() function #1122

Open
giampaolo opened this issue Sep 5, 2017 · 5 comments
Open

Add usable_cpu_count() function #1122

giampaolo opened this issue Sep 5, 2017 · 5 comments

Comments

@giampaolo
Copy link
Owner

Recently os.cpu_count() on Windows has been fixed in order to take process groups into account and return the number of all available CPUs:
http://bugs.python.org/issue30581
The fix will be included in psutil too (#771) but that's another story.

This made me realize that psutil.cpu_count() does not return the number of usable CPUs, which could possibly represent a better default value for things like multiprocessing and process pools.
It is currently possible to retrieve this info on UNIX with len(os.sched_getaffinity(0)) which takes CPU affinity and (I think) Linux cgroups into account, but it's not possible to do the same on Windows which is supposed to provide this value via GetActiveProcessorCount() API.

@giampaolo
Copy link
Owner Author

psutil.cpu_usable_count() would probably be a better name as right now all system functions with the exception of memory-related ones use a prefix which makes sense (cpu_*, disk_*, net_*, etc.).

@giampaolo
Copy link
Owner Author

Actually I'm confused: I now realize it should already be possible to get usable CPU count on Windows with len(psutil.Process().cpu_affinity()). Internally it uses GetProcessAffinityMask but I'm not sure how it differs from GetActiveProcessorCount().

@KOLANICH
Copy link

KOLANICH commented Dec 30, 2017

On Linux the both ways return the number of currently enabled cpu cores.
The correct value can be obtained from the files in /sys/devices/system/cpu/

Also these files can be used to brign up/down cores and check their statuses.

@tiran
Copy link

tiran commented Apr 9, 2019

It's actually more complicated. cgroups can limit the amount of usable CPUs, see https://bugs.python.org/issue36054

@dhirschfeld
Copy link

Windows Containers use Job objects to limit the number of CPUs so to work inside a container would require querying this object

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants