Skip to content

Commit

Permalink
Allow MultiprocessingEvaluator to initialize with cpu_count minus N
Browse files Browse the repository at this point in the history
This commit allows inputting negative integers for n_processes in the MultiprocessingEvaluator defined in evaluators.py. This way the number of logical CPU cores minus that negative number would be used. For example, on a 12 thread processor, -2 results in using 10 threads. There is a guard around the function that always at least one process.

The main advantage is that when inputting a negative number, there is always some CPU power available for system and background tasks, which means the system would lag way less. When working in teams or on different devices, you can just leave it to -1 or -2 and always have some computing power to spare.
  • Loading branch information
EwoutH committed May 31, 2022
1 parent d4464a3 commit 904b287
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion ema_workbench/em_framework/evaluators.py
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,8 @@ class MultiprocessingEvaluator(BaseEvaluator):
----------
msis : collection of models
n_processes : int (optional)
A negative number can be inputted to use the number of logical cores minus the negative cores.
For example, on a 12 thread processor, -2 results in using 10 threads.
max_tasks : int (optional)
"""
Expand Down Expand Up @@ -347,6 +349,10 @@ def initialize(self):
random_part = "".join(random_part)
self.root_dir = os.path.abspath("tmp" + random_part)
os.makedirs(self.root_dir)

# Calcuate n_processes if negative value is inputted
if self.n_processes < 0:
self.n_processes = max(multiprocessing.cpu_count() + self.n_processes, 1)

self._pool = multiprocessing.Pool(
self.n_processes,
Expand Down Expand Up @@ -794,4 +800,4 @@ def robust_optimize(
convergence_freq,
logging_freq,
**kwargs,
)
)

0 comments on commit 904b287

Please sign in to comment.