How to set n_jobs = -1 from tpot

[lesshaste]

I am trying out the simple MNIST digit recognition example from https://rhiever.github.io/tpot/examples/ .

I noticed that only one core was being used despite many of the sklearn classifiers supporting n_jobs.

Context of the issue

Is there a simple way to add n_jobs to those classifiers that support it through tpot?

Process to reproduce the issue

Run the simple MNIST digit recognition example from https://rhiever.github.io/tpot/examples/
Run top. Press 1 to see number of cores being used.

Expected result

It would be great if all the cores were used.

Current result

Only one core is used.

Possible fix

Tpot could pass n_jobs = -1 to those classifiers that support it.

[weixuanfu]

Please check TPOT API. TPOT allows parallel computing during optimization process with n_jobs parameter.

[lesshaste]

Thank you and I apologize for not having seen that before.

[rhiever]

There are two ways to accomplish this task. You can do what @weixuanfu suggested and set TPOT's n_jobs parameter, which will parallelize TPOT's pipeline evaluation process. You can also make your own custom TPOT configuration to set all of the n_jobs parameters of the individual algorithms. However, I recommend using TPOT's parallelization rather than parallelizing the individual algorithms.

[HamedMP]

I try to set n_jobs to any value other than 1, like -1, 2, 4, ..., the cpu core usage goes to 100% but no progress is shown on the notebook. And no result get returned after waiting for hours, tested with latest version of tpot, both on mac and ubuntu I have the same problem. It would be good to parallelize training.
Found related to #645

[shaunstoltz]

Dosnt work, use something else.