Issue #18999: Make multiprocessing use context objects.

This allows different parts of a program to use different methods for
starting processes without interfering with each other.

Author: shibturn

Doc/library/multiprocessing.rst

@@ -98,8 +98,8 @@ necessary, see :ref:`multiprocessing-programming`.
 
 
 
-Start methods
-~~~~~~~~~~~~~
+Contexts and start methods
+~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Depending on the platform, :mod:`multiprocessing` supports three ways
 to start a process.  These *start methods* are
@@ -132,7 +132,7 @@ to start a process.  These *start methods* are
     unnecessary resources are inherited.
 
     Available on Unix platforms which support passing file descriptors
-    over unix pipes.
+    over Unix pipes.
 
 Before Python 3.4 *fork* was the only option available on Unix.  Also,
 prior to Python 3.4, child processes would inherit all the parents
@@ -153,18 +153,46 @@ example::
 
        import multiprocessing as mp
 
-       def foo():
-           print('hello')
+       def foo(q):
+           q.put('hello')
 
        if __name__ == '__main__':
            mp.set_start_method('spawn')
-           p = mp.Process(target=foo)
+           q = mp.Queue()
+           p = mp.Process(target=foo, args=(q,))
            p.start()
+           print(q.get())
            p.join()
 
 :func:`set_start_method` should not be used more than once in the
 program.
 
+Alternatively, you can use :func:`get_context` to obtain a context
+object.  Context objects have the same API as the multiprocessing
+module, and allow one to use multiple start methods in the same
+program. ::
+
+       import multiprocessing as mp
+
+       def foo(q):
+           q.put('hello')
+
+       if __name__ == '__main__':
+           ctx = mp.get_context('spawn')
+           q = ctx.Queue()
+           p = ctx.Process(target=foo, args=(q,))
+           p.start()
+           print(q.get())
+           p.join()
+
+Note that objects related to one context may not be compatible with
+processes for a different context.  In particular, locks created using
+the *fork* context cannot be passed to a processes started using the
+*spawn* or *forkserver* start methods.
+
+A library which wants to use a particular start method should probably
+use :func:`get_context` to avoid interfering with the choice of the
+library user.
 
 
 Exchanging objects between processes
@@ -859,11 +887,30 @@ Miscellaneous
 
    .. versionadded:: 3.4
 
-.. function:: get_start_method()
+.. function:: get_context(method=None)
+
+   Return a context object which has the same attributes as the
+   :mod:`multiprocessing` module.
 
-   Return the current start method.  This can be ``'fork'``,
-   ``'spawn'`` or ``'forkserver'``.  ``'fork'`` is the default on
-   Unix, while ``'spawn'`` is the default on Windows.
+   If *method* is *None* then the default context is returned.
+   Otherwise *method* should be ``'fork'``, ``'spawn'``,
+   ``'forkserver'``.  :exc:`ValueError` is raised if the specified
+   start method is not available.
+
+   .. versionadded:: 3.4
+
+.. function:: get_start_method(allow_none=False)
+
+   Return the name of start method used for starting processes.
+
+   If the start method has not been fixed and *allow_none* is false,
+   then the start method is fixed to the default and the name is
+   returned.  If the start method has not been fixed and *allow_none*
+   is true then *None* is returned.
+
+   The return value can be ``'fork'``, ``'spawn'``, ``'forkserver'``
+   or *None*.  ``'fork'`` is the default on Unix, while ``'spawn'`` is
+   the default on Windows.
 
    .. versionadded:: 3.4
 
@@ -1785,7 +1832,7 @@ Process Pools
 One can create a pool of processes which will carry out tasks submitted to it
 with the :class:`Pool` class.
 
-.. class:: Pool([processes[, initializer[, initargs[, maxtasksperchild]]]])
+.. class:: Pool([processes[, initializer[, initargs[, maxtasksperchild [, context]]]]])
 
    A process pool object which controls a pool of worker processes to which jobs
    can be submitted.  It supports asynchronous results with timeouts and
@@ -1805,6 +1852,13 @@ with the :class:`Pool` class.
       unused resources to be freed. The default *maxtasksperchild* is None, which
       means worker processes will live as long as the pool.
 
+   .. versionadded:: 3.4
+      *context* can be used to specify the context used for starting
+      the worker processes.  Usually a pool is created using the
+      function :func:`multiprocessing.Pool` or the :meth:`Pool` method
+      of a context object.  In both cases *context* is set
+      appropriately.
+
    .. note::
 
       Worker processes within a :class:`Pool` typically live for the complete