Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ShardedDDP] Expose 'module' property from ShardedDDP #646

Closed
apsdehal opened this issue May 3, 2021 · 2 comments · Fixed by #647
Closed

[ShardedDDP] Expose 'module' property from ShardedDDP #646

apsdehal opened this issue May 3, 2021 · 2 comments · Fixed by #647
Labels
SDP ShardedDataParallel (zero-2)

Comments

@apsdehal
Copy link

apsdehal commented May 3, 2021

🚀 Feature

Expose module property from ShardedDDP again which points to underlying wrapped model.

Motivation

Recent PR #611 made a lot of class members in ShardedDDP private. Unfortunately, this behavior is now different from DDP which exposes module property to expose the underlying model directly. In MMF's checkpointing system, we use this property to extract out the underlying model to allow checkpoints to work agnostic of distributed system or not.

Pitch

I can open up a PR to expose it as a property if possible.

Alternatives

I can hack around in our code to check _module but the change in itself is backwards incompatible from at least 3.3 to 3.6 version of fairscale.

@apsdehal apsdehal changed the title [ShardedDDP] Expose 'module [ShardedDDP] Expose 'module' property from ShardedDDP May 3, 2021
@min-xu-ai min-xu-ai added the SDP ShardedDataParallel (zero-2) label May 3, 2021
@blefaudeux
Copy link
Contributor

ah crap, there was a unit test on that, how did this slip through :/ Thanks for the report @apsdehal , fixing that ASAP

@blefaudeux
Copy link
Contributor

let me know if you face any issues @apsdehal, this auto closed because the issue was linked, I believe that should be fine by now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
SDP ShardedDataParallel (zero-2)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants