Can't pickle subclass of OSError

[lukeyeager]

Bug report

I use a library which creates a subclass of ConnectionError. When I use that library in a subprocess (multiprocessing) and it raises that exception, Python is unable to unpickle the error in the main process. I have narrowed it down to the following minimal repro:

import pickle

class C(OSError):
    def __init__(self, message):
        self.message = message
        super().__init__()

x = C('message')
s = pickle.dumps(x)
x = pickle.loads(s)

$ python main.py
Traceback (most recent call last):
  File "/tmp/tmp.tqfN6CedbQ/main.py", line 11, in <module>
    x = pickle.loads(s)
TypeError: C.__init__() missing 1 required positional argument: 'message'

I suspect what's going on is that the implementation of OSError.__reduce__ doesn't properly account for child classes.

But this could also be user-error since the mismatched __init__() signatures is not "good design":

Good design dictates that such implementations have the same calling signature in every case (because the order of calls is determined at runtime, because that order adapts to changes in the class hierarchy, and because that order can include sibling classes that are unknown prior to runtime).
https://docs.python.org/3.10/library/functions.html#super

If this is not a CPython bug, I would appreciate a pointer as to how I can fix the library's exception class. I tried to implement __newargs__ but it doesn't look like OSError's __reduce__ calls it. And I tried setting __reduce__ = object.__reduce__ for the subclass but it results in TypeError: cannot pickle 'C' object.

Your environment

• Arch linux kernel 6.1.4-arch1-1
• python package 3.10.9-1

[lukeyeager]

I see the same error with CPython 3.11.1 and 2.7.18, and I see a similar error with PyPy 3.9.16.

And yet I still feel like OSError.__reduce__ should at least call self.__newargs__.

I did find a succinct workaround to monkey-patch the library, at least:

C.__reduce__ = lambda self: (C, (self.message,))

[sobolevn]

This works for me:

>>> import pickle
>>> 
>>> class C(OSError):
...     def __init__(self, message):
...         self.message = message
...         super().__init__()
...     def __reduce__(self):
...         return (C, (self.message,))
... 
>>> x = C('message')
>>> s = pickle.dumps(x)
>>> x = pickle.loads(s)

Related: #76877

I think, we need to update the docs for this use-case, since users report this again and again :)

[lukeyeager]

Ah, thanks for the link! I failed to find these similar issues earlier: #73652, #74191, and #87626. So this is a known issue with no real plan to address it.

I think, we need to update the docs for this use-case, since users report this again and again :)

I agree, if this isn't going to get fixed then I would appreciate a warning in the docs, at least. I read through a lot of documentation related to pickling and exceptions yesterday and didn't find anything pointing me to these discussions.

I'll go ahead and close this as a duplicate.