Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect use of cores vs threads #100

Open
Bastian-Eisenmann opened this issue Dec 17, 2024 · 0 comments
Open

Incorrect use of cores vs threads #100

Bastian-Eisenmann opened this issue Dec 17, 2024 · 0 comments

Comments

@Bastian-Eisenmann
Copy link
Collaborator

The plugin will currently may

Depending on the CPU architecture a CPU often has several physical cores (e.g. 4) which are later split into multiple logical threads. (2 per core -> 8)

The used cpu capacity is reported by nextflow on a thread level.
That means, that a cpu utility of 200% reflects the utilization of 2*1 Threads which refers to one CPU core.
Given a sample system with 4 cpus and 8 logical threads nextflow can report an utilization up to 800%.

The energy consumption is currently based on a TDP per core level. That means: A cpu with a TDP of 100 and 4 cores will have a TDP of 25 per core.

A full utilized system with 8 Threads, 4 CPU and a TDP per core of 25 will afterwards be measured with a overall TDP of 200 (800% * 25 = 200)
As a solution the calculation parameter could be changed from "TDP per CPU" to "TDP per Thread" instead.

Some additional side nodes:

  • The hypervisor under the used virtualization level will may distribute tasks in it´s own way. E.g. docker on OSX will try to distribute the load on different threads (if the application does support it) rather than utilize one thread/core complete.
  • The power consumption of a score does not scale perfect linear based on its utilization level but it´s a good approximation.
  • The nextflow cpu boundaries chosen in the workflow configuration (e.g. cpu: 2) will may be ignored by the target system. (Tested with OSX running docker)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant