Multiprocessing failure on MacOS and python3.8+

[johnveitch]

CPNest currently doesn't work on MacOS with python 3.8 or above.
The failure may be related to the change to the way multiprocessing creates new processes on this platform, which changed in v3.8. See the issue here.

The error I see is

  File "/opt/anaconda3/envs/cpnest_py38/lib/python3.8/site-packages/cpnest/sampler.py", line 116, in reset
    for n in tqdm(range(self.poolsize), desc='SMPLR {} init draw'.format(self.thread_id),
AttributeError: 'MetropolisHastingsSampler' object has no attribute 'thread_id'

[wdpozzo]

I managed to have a look at the issue, and I came to the conclusion that the getstate and setstate functions that we wrote for the checkpointing of the sampler class is interfering with the serialisation necessary for the multiprocessing module. Something changed under the hood, and I cannot really track down what it is. Hence, I think we either rewrite the code or move on to a different architecture for the parallelism as in massively_parallel.

[dylaghi]

I have reproduced this error several times on different Apple M1 laptops, where it is not possible to directly install python 3.7 (see https://stackoverflow.com/questions/70205633/cannot-install-python-3-7-on-osx-arm64).

A workaround that worked for me is to install cpnest in a conda environment that simulates the x86_64 architecture, as detailed in the link above.

[SimonMaenaut]

From my understanding, it would be recommended to adapt the code in order for CPNest to work with the spawn start method.

In Python 3.14 the default start method will also change for Linux from fork to spawn, which is generally considered safer.

As mentioned in the Python 3.12 documentation on Multiprocessing specifically:

The default start method will change away from fork in Python 3.14. Code that requires fork should explicitly specify that via get_context() or set_start_method().

Changed in version 3.12: If Python is able to detect that your process has multiple threads, the os.fork() function that this start method calls internally will raise a DeprecationWarning. Use a different start method. See the os.fork() documentation for further explanation.