PyPy 2.7 unable to handle 3.X exceptions

[graingert]

Platform

• { Linux2 } (also add appropriate OS issue label (linux, windows, ...))
• { psutil==5.6.7 } (use "pip show psutil")

Bug description
child_proc.wait(3) raising OSError.errno = errno.ECHILD
even though it should be caught here:

psutil/psutil/_psposix.py

Line 87 in 78c8669

except ChildProcessError:

t/unit/test_spawn.py:29: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.tox/pypy/site-packages/psutil/__init__.py:1383: in wait
    return self._proc.wait(timeout)
.tox/pypy/site-packages/psutil/_pslinux.py:1517: in wrapper
    return fun(self, *args, **kwargs)
.tox/pypy/site-packages/psutil/_pslinux.py:1725: in wait
    return _psposix.wait_pid(self.pid, timeout, self._name)
.tox/pypy/site-packages/psutil/_psposix.py:84: in wait_pid
    retpid, status = waitcall()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    def waitcall():
>       return os.waitpid(pid, os.WNOHANG)
E       OSError: [Errno 10] No child processes

https://github.com/graingert/billiard/blob/351a7d5546841e7a34ec78076a6bf5d41ba7faac/t/unit/test_spawn.py#L29

[graingert]

here's a standalone repro: https://github.com/graingert/psutil-pypy-echild

[graingert]

an even smaller repro, this works on cpython2 or pypy3 but fails on pypy2 Python 2.7.13 (7.1.1+dfsg-1, Aug 09 2019, 05:11:07)

import errno

from psutil._compat import ChildProcessError

class MyPretendChildProcessError(EnvironmentError):
    errno = errno.ECHILD


try:
    raise MyPretendChildProcessError
except ChildProcessError:
    pass

I think reverting https://github.com/giampaolo/psutil/pull/1544/files will fix it

[graingert]

@giampaolo here's the error in pypy: https://bitbucket.org/pypy/pypy/issues/3149/compatability-excpeptions-using-a

edit: heptapod link: https://foss.heptapod.net/pypy/pypy/-/issues/3149

[graingert]

@giampaolo the fix is now in pypy waiting a release

[giampaolo]

Another repo:

import os
from psutil._compat import FileNotFoundError
try:
   raise os.listdir('???')
except FileNotFoundError:
   pass

the fix is now in pypy waiting a release

What do you mean?

[giampaolo]

May be related to this: https://doc.pypy.org/en/latest/cpython_differences.html#subclasses-of-built-in-types

[graingert]

Another repo:

import os
from psutil._compat import FileNotFoundError
try:
   raise os.listdir('???')
except FileNotFoundError:
   pass

the fix is now in pypy waiting a release

What do you mean?

The fix is here: https://bitbucket.org/pypy/pypy/commits/11a00b95fd59

edit: heptapod link: https://foss.heptapod.net/pypy/pypy/-/commit/c0d1321538200285f9307eb9c8b806af0f360af6

[giampaolo]

Does this mean current psutil code will work with next version of pypy? If not, do you know a workaround (in psutil) to make it work with current/older pypy versions? If option 2 is not possible I'm keen on going with option 1 (wait for new pypy release) and return error in setup.py telling to upgrade pypy.

[graingert]

Does this mean current psutil code will work with next version of pypy?

There will be a future release of pypy2 with the fix in it

do you know a workaround (in psutil) to make it work with current/older pypy versions?
use:

except EnvironmentError as e:
    if e.errno == errno...:
        ...
    ...

If option 2 is not possible I'm keen on going with option 1 (wait for new pypy release) and return error in setup.py telling to upgrade pypy.

Tbh I think the subclasshook is a rather ugly hack and the fact that it was removed intentionally in cpython3 makes me think it should not be used. I'd prefer it if you revert to using e.errno==errno... in psutil

[giampaolo]

This change was introduced not long ago: #1544. Latest working version is psutil 5.6.3. Sorry but I prefer to leave things as is. cPython 2 is deprecated and so should be pypy2, especially a bugged version which will be fixed and released soon anyway. The gain of #1544 in terms of code maintainability is high and reverting #1544 manually would be a bit tricky too. I can add a more detailed error message though.

[graingert]

cPython 2 is deprecated and so should be pypy2

pypy2 isn't going anywhere

…

On Fri, 14 Feb 2020, 00:14 giampaolo, ***@***.***> wrote: Closed #1659 <#1659>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1659?email_source=notifications&email_token=AADFATBLJQ2UEWYBKK3AD33RCXO5BA5CNFSM4KEXBU7KYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOWUCMS2Q#event-3036989802>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADFATEJ4ZSZ5BFFQFWNIALRCXO5BANCNFSM4KEXBU7A> .

[giampaolo]

pypy2 isn't going anywhere

Also cPython 2. BTW, I don't plan on dropping Python 2 support any time soon (will happen years from now). I'm just giving up on a specific broken PyPy2 version which hopefully will be fixed soon.

[kashirin-alex]

one thing for sure, The exception subclass FileExistsError throw for OSError is only for PY3
without arguing py2 is going nowhere and just staying

so, it will be appropriate to have PY3 and platform.python_implementation() for the platform sanity check at

psutil/psutil/_compat.py

Line 113 in 4d6a086

if platform.python_implementation() != "CPython":

meanwhile work around is:

wget https://github.com/giampaolo/psutil/archive/release-5.7.0.tar.gz; 
tar -xf release-5.7.0.tar.gz;
sed -i 's/platform.python_implementation()/PY3 and platform.python_implementation()/g' \
 psutil-release-5.7.0/psutil/_compat.py;
pypy -m pip install psutil-release-5.7.0/

-- while issue will be the Exceptions handling of os.kill(PID, 0) , expecting py3 Exceptions.
++ in such case, (considering py2 out from future releases) than the python_implementation() != "CPython" needs to be removed and the Sanity check be applied as well to python2 (as it is a broken install for python2) !