bpo-35900: Add a state_setter arg to save_reduce #12588
Conversation
|
ping @pitrou |
|
I fixed a couple of small error message issues, and a typo in the news entry, and this otherwise looks OK to me. However, I'll leave the final review to @pitrou :) |
Misc/NEWS.d/next/Library/2019-03-27-15-09-00.bpo-35900.fh56UU.rst
Outdated
Show resolved
Hide resolved
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
|
I have made the requested changes; please review again |
|
Thanks for making the requested changes! @pitrou: please review the changes made to this pull request. |
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
|
I can reproduce the cert failures on git master, so they're unrelated. |
|
The test failure was fixed on master, can you merge / rebase? |
pierreglaser
force-pushed
the
give-a-state-setter-to-save-reduce
branch
from
1539f68 to
3f2fdb2
Compare
|
Thanks a lot @pitrou! |
Second PR of bpo-35900.
What
The aim of this PR is to allow users to specify custom state setting methods for types they have no control on. This is done via adding a
state_setterkeyword argument inPickler.save_reduce, expected to be a callable. This callable will be called at unpickling time with the following signaturestate_setter(obj, state, slotstate).Why
A practical use case is the one of cloudpickle that provides extended serialization procedures, especially for functions and classes. In
cloudpickle, functions and classes are reconstructed from scratch at unpickling time. As for other built-in types reconstructed from scratch (list, dicts...), circular-references are handled by initalizing a skeleton-object, and then updating it in a state-setting phase. This state-setting phase currently only relies on potential__getstate__methods, or fallbacks to a default procedure, and this is simply not enough to properly reconstruct functions and classes, hence this proposal.See cloudpipe/cloudpickle#253 for more details.
Notes
dependencies
This PR comes alongside #12499 . Without it, leveraging #12499 changes won't work for
functionsandclasses.cross-protocol/version usage
An interrogation that remains is whether using this new API with a protocol < 5 should raise an error at pickling time. If not,
unpicklingwill not succeed if the pickle string is loaded using apython < 3.8. This may be surprising for users as one can assume that the point of using low protocols is to ensure retro-compatibility with older python versions. On the other side, this API will probably be of interested for advanced user only, that would understand why using a new API with an old protocol. As of today, this is still an open question.https://bugs.python.org/issue35900