Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can cpnest be made to accept a pool of processes? #41

Open
cdcapano opened this issue Jan 25, 2019 · 1 comment
Open

Can cpnest be made to accept a pool of processes? #41

cdcapano opened this issue Jan 25, 2019 · 1 comment

Comments

@cdcapano
Copy link

The fact that CPNest currently tries to handle parallel processing on its own presents it from being able to use MPI (as noted in Issue #33), and also makes it a pain to work with. Most (all?) other samplers that I've countered allow a preallocated pool of processes to be passed to them, which gives the user greater freedom to control the parallel environment in the manner that best fits their environment. For example, in PyCBC Inference, we create a pool of processes ourselves (which allows us to side step pickling issues when using python's multiprocessing), then give that pool to the sampler. As a result, we immediately can use either multiprocessing or MPI for parallelization for all other samplers.

If CPNest used a pool instead of trying to manage parallelization on its own, then it would make it easier both on our end to use it, and on the CPNest developers' end to support parallelization, since now you would not need to add code to deal with different types of communication schemes, negating issues like #33.

@wdpozzo
Copy link
Collaborator

wdpozzo commented May 15, 2019

Any suggestion/code snippet would be greatly appreciated. I personally have little experience with parallelisation and the current way cpnest works is the result of a lot of trial and error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants