Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] Fix iter bug when resuming checkpoint in distributed train #866

Merged
merged 2 commits into from
Sep 11, 2021
Merged

[Fix] Fix iter bug when resuming checkpoint in distributed train #866

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
57c69ec
[Fix] Fix iter bug when resuming checkpoint in distributed train
FreyWang Sep 10, 2021
02ad416
fix lint error
FreyWang Sep 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion tools/train.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

import mmcv
import torch
from mmcv.runner import init_dist
from mmcv.runner import get_dist_info, init_dist
from mmcv.utils import Config, DictAction, get_git_hash

from mmseg import __version__
Expand Down Expand Up @@ -94,6 +94,9 @@ def main():
else:
distributed = True
init_dist(args.launcher, **cfg.dist_params)
# gpu_ids is used to calculate iter when resuming checkpoint,
_, world_size = get_dist_info()
cfg.gpu_ids = range(world_size)

# create work_dir
mmcv.mkdir_or_exist(osp.abspath(cfg.work_dir))
Expand Down