Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DataPipe] Skip FullSync operation when world_size == 1 #1065

Closed
wants to merge 7 commits into from

Conversation

NivekT
Copy link
Contributor

@NivekT NivekT commented Mar 2, 2023

NivekT added a commit that referenced this pull request Mar 2, 2023
ghstack-source-id: 7547f29c1bf8bdf1943285c15d89187b3cda7dad
Pull Request resolved: #1065
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 2, 2023
@NivekT
Copy link
Contributor Author

NivekT commented Mar 2, 2023

@NivekT has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Copy link
Contributor

@ejguan ejguan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve with one comment

torchdata/datapipes/iter/util/distributed.py Outdated Show resolved Hide resolved
torchdata/datapipes/iter/util/distributed.py Outdated Show resolved Hide resolved
NivekT added a commit that referenced this pull request Mar 3, 2023
ghstack-source-id: 18f4cdb035fea1e74592d953eb8cecc7141e3f2c
Pull Request resolved: #1065
@NivekT
Copy link
Contributor Author

NivekT commented Mar 3, 2023

@NivekT has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@NivekT
Copy link
Contributor Author

NivekT commented Mar 7, 2023

@NivekT has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@ejguan ejguan added this to the 0.6.1 milestone Mar 21, 2023
NivekT added a commit that referenced this pull request Apr 10, 2023
ghstack-source-id: d5bd3db7115c387351e65ee31295542783386f0b
Pull Request resolved: #1065
@NivekT
Copy link
Contributor Author

NivekT commented Apr 10, 2023

@NivekT has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

NivekT added a commit that referenced this pull request Apr 14, 2023
ghstack-source-id: 9f27159e84c823bb706c1054c49c2a8c6556f0e1
Pull Request resolved: #1065
@NivekT NivekT requested a review from ejguan April 14, 2023 21:57
@NivekT
Copy link
Contributor Author

NivekT commented Apr 14, 2023

@NivekT has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Copy link
Contributor

@ejguan ejguan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test?

torchdata/datapipes/iter/util/distributed.py Show resolved Hide resolved
NivekT added a commit that referenced this pull request Apr 17, 2023
ghstack-source-id: f22a775ea7c4bf8743f281f7bcc1637823fad426
Pull Request resolved: #1065
@NivekT
Copy link
Contributor Author

NivekT commented Apr 17, 2023

@NivekT has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

NivekT added a commit that referenced this pull request Apr 19, 2023
ghstack-source-id: de637603e941e347f24e615e475851e85b802bd7
Pull Request resolved: #1065
@NivekT
Copy link
Contributor Author

NivekT commented Apr 19, 2023

@NivekT has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@NivekT merged this pull request in 2093503.

NivekT added a commit that referenced this pull request Apr 19, 2023
Summary: Pull Request resolved: #1065

Test Plan: Imported from OSS

Reviewed By: ejguan

Differential Revision: D43751382

Pulled By: NivekT

fbshipit-source-id: cd85e6dc360494289b82facb0cee34502a178c27
NivekT added a commit that referenced this pull request Apr 19, 2023
Summary: Pull Request resolved: #1065

Test Plan: Imported from OSS

Reviewed By: ejguan

Differential Revision: D43751382

Pulled By: NivekT

fbshipit-source-id: cd85e6dc360494289b82facb0cee34502a178c27
ejguan pushed a commit that referenced this pull request Apr 20, 2023
Summary: Pull Request resolved: #1065

Test Plan: Imported from OSS

Reviewed By: ejguan

Differential Revision: D43751382

Pulled By: NivekT

fbshipit-source-id: cd85e6dc360494289b82facb0cee34502a178c27
@facebook-github-bot facebook-github-bot deleted the gh/NivekT/110/head branch April 23, 2023 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants