Issue #309: Add Python 3.9 support to spawnv_passfds()

[vstinner]

Python 3.9 added (umask, user, group, extra_groups) parameters to
_posixsubprocess.fork_exec().

[vstinner]

Without this change "tox -e py39" fails with:

____________________________ test_spawn.test_start _____________________________

self = <t.unit.test_spawn.test_spawn object at 0x7f53727ca760>

    def test_start(self):
        ctx = get_context('spawn')
    
        p = ctx.Process(target=task_from_process, args=('opa',))
>       p.start()

t/unit/test_spawn.py:17: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
billiard/process.py:124: in start
    self._popen = self._Popen(self)
billiard/context.py:341: in _Popen
    return Popen(process_obj)
billiard/popen_spawn_posix.py:39: in __init__
    super(Popen, self).__init__(process_obj)
billiard/popen_fork.py:24: in __init__
    self._launch(process_obj)
billiard/popen_spawn_posix.py:49: in _launch
    tracker_fd = semaphore_tracker.getfd()
billiard/semaphore_tracker.py:38: in getfd
    self.ensure_running()
billiard/semaphore_tracker.py:62: in ensure_running
    spawnv_passfds(exe, args, fds_to_pass)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

path = '/home/vstinner/billiard/.tox/py39/bin/python'
args = ['/home/vstinner/billiard/.tox/py39/bin/python', '-c', 'from billiard.semaphore_tracker import main;main(30)']
passfds = [10, 30]

    def spawnv_passfds(path, args, passfds):
        passfds = sorted(passfds)
        errpipe_read, errpipe_write = os.pipe()
        try:
>           return _posixsubprocess.fork_exec(
                args, [fsencode(path)], True, tuple(passfds), None, None,
                -1, -1, -1, -1, -1, -1, errpipe_read, errpipe_write,
                False, False, None)
E               TypeError: fork_exec() takes exactly 21 arguments (17 given)

billiard/compat.py:223: TypeError
=========================== short test summary info ============================
FAILED t/unit/test_spawn.py::test_spawn::test_start - TypeError: fork_exec() ...

[vstinner]

pypy job of Travis CI failed on installing psutil:

Collecting psutil (from -r /home/travis/build/celery/billiard/requirements/test.txt (line 3))

  Using cached https://files.pythonhosted.org/packages/c4/b8/3512f0e93e0db23a71d82485ba256071ebef99b227351f0f5540f744af41/psutil-5.7.0.tar.gz

    Complete output from command python setup.py egg_info:

    Traceback (most recent call last):

      File "<module>", line 1, in <module>

      File "/tmp/pip-install-TiPPEX/psutil/setup.py", line 46, in <module>

        from _compat import PY3  # NOQA

      File "/tmp/pip-install-TiPPEX/psutil/psutil/_compat.py", line 120, in <module>

        "broken / incompatible Python implementation, see: "

    RuntimeError: broken / incompatible Python implementation, see: https://github.com/giampaolo/psutil/issues/1659

It doesn't seem to be related to my PR.

[vstinner]

For the psutil, in my pyperf project, I used environment markers on the psutil dependency to workaround this PyPy issue:

psutil; platform_python_implementation != "PyPy"

https://github.com/vstinner/pyperf/blob/master/test-requirements.txt

[vstinner]

Hello. @auvipy approved my PR but it's not merged yet. How can I help to get his PR merged?

[vstinner]

I proposed a solution to skip psutil on PyPy in a previous comment.

[auvipy]

OK

[vstinner]

FYI this issue is tracked in Fedora as https://bugzilla.redhat.com/show_bug.cgi?id=1792056 (my commit has been backported manually)

[auvipy]

there will be a new release soon