Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

Fix #4434: support pickle in serializer #4552

Merged
merged 20 commits into from
Mar 3, 2022
Merged

Conversation

ultmaster
Copy link
Contributor

@ultmaster ultmaster commented Feb 15, 2022

Description

The issue that decorator and pickler doesn't play well together is well known. The workaround solution is:

  1. On linux: to use a fool class (_pickling_object) which pretends to be the pickled object. We then put the original type, as well as args and kwargs in its __new__ argument.
  2. On Win/Mac: to replace the __init__ in-place.

I suspect that their could still be problems when things get complex, e.g., the wrapped class has a custom pickling or __new__. But it can't be worse because the previous pickle doesn't work at all.

This PR also fixes an issue with @nni.trace on super-class.

Fixes #4434.

Checklist

  • test case: need to check whether Windows/MAC pipeline works as expected.
  • doc

How to test

Test dataloader with traced dataset on Windows, or DataParallel with pl.Classification.

@ultmaster ultmaster linked an issue Feb 15, 2022 that may be closed by this pull request
@ultmaster ultmaster marked this pull request as ready for review February 17, 2022 15:00
@ultmaster ultmaster mentioned this pull request Mar 3, 2022
2 tasks
@ultmaster ultmaster merged commit 21abc28 into microsoft:master Mar 3, 2022
@J-shang J-shang mentioned this pull request Mar 23, 2022
51 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

nni.trace + pickle don't play well together under non-linux envs
3 participants