Implementing Multiprocessing

Python has a build in library called multiprocessing. This can be used to run multiple processes at once. This can be useful to run multiple genomes at once when evaluating their fitness.

First, we import the library.

import multiprocessing as mp

Then, we make a queue of all our outputs.

output = mp.Queue()

Then we define the processes we want to run at once. In this case, genomes is a list of genomes to be evaluated, i is a counter, self.par is a variables that defines how many genomes to run at once. So, we can slice the genomes list to get the genomes we want to run and make a list of all the processes. Our function, in this case self._fitness_func(), has to have an extra parameter for the output Queue, so we can retrieve the outputs. We can also choose not to return anything and set the fitness in the function itself, but to learn how outputs are done while using multiprocessing, I decided to use a queue.

processes = [mp.Process(target=self._fitness_func, args=(genome, config, output)) for genome in
                         genomes[i:i + self.par]]

Then we start and join the processes. I used list comprehension for speed and for the code to look cleaner.

[p.start() for p in processes]
[p.join() for p in processes]

Then, we can get the results out of the queue and set the fitness accordingly.

results = [output.get() for p in processes]
for n, r in enumerate(results):
    genomes[i + n].fitness = r