Skip to content

Commit

Permalink
Merge commit 'refs/pull/622/head' of https://github.com/open-mmlab/mm…
Browse files Browse the repository at this point in the history
  • Loading branch information
wangjiangben-hw committed Oct 19, 2022
2 parents 64d6d39 + 915d00c commit 6b963f0
Show file tree
Hide file tree
Showing 10 changed files with 102 additions and 9 deletions.
3 changes: 2 additions & 1 deletion .github/workflows/merge_stage_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,8 @@ jobs:
with:
python-version: ${{ matrix.python-version }}
- name: Upgrade pip
run: pip install pip --upgrade
# Windows CI could fail If we call `pip install pip --upgrade` directly.
run: python -m pip install pip --upgrade
- name: Install PyTorch
run: pip install torch==1.8.1+${{matrix.platform}} torchvision==0.9.1+${{matrix.platform}} -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
- name: Build MMEngine from source
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/pr_stage_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,8 @@ jobs:
with:
python-version: ${{ matrix.python-version }}
- name: Upgrade pip
run: pip install pip --upgrade
# Windows CI could fail If we call `pip install pip --upgrade` directly.
run: python -m pip install pip --upgrade
- name: Install PyTorch
run: pip install torch==1.8.1+${{matrix.platform}} torchvision==0.9.1+${{matrix.platform}} -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
- name: Build MMEngine from source
Expand Down
13 changes: 12 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,17 @@ Major features:
- Provides rich components and strategies.
- Complete controls on the training process with different levels of APIs.

## What's New

v0.2.0 was released in 2022-10-11.

1. Refactor FileIO to provide a more unified interface without BC breaking.
2. Add the base model for test time augmentation.
3. Support converting `BN` to `SyncBN` during distributed data-parallel training.
4. Add SMDDP backend and support running on AWS.

Read [Changelog](./docs/en/notes/changelog.md#v0.2.0 "11/10/2022") for more details.

## Installation

Before installing MMEngine, please ensure that PyTorch has been successfully installed following the [official guide](https://pytorch.org/get-started/locally/).
Expand Down Expand Up @@ -145,7 +156,7 @@ val_dataloader = DataLoader(batch_size=32,
<details>
<summary>Build Metrics</summary>

To validate and test the model, we need to define a **Metric** called accuracy to evaluate the model. This metric needs inherit from `BaseMetric` and implements the `process` and `compute_metrics` methods.
To validate and test the model, we need to define a **Metric** called accuracy to evaluate the model. This metric needs to inherit from `BaseMetric` and implements the `process` and `compute_metrics` methods.

```python
from mmengine.evaluator import BaseMetric
Expand Down
11 changes: 11 additions & 0 deletions README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,17 @@ MMEngine 是一个基于 PyTorch 用于深度学习模型训练的基础库,
- 对神经网络模型中各个层的优化配置进行细粒度调整
- 混合精度训练的灵活控制

## 最近进展

最新版本 v0.2.0 在 2022.10.11 发布。

1. 重构 FileIO 以提供更加易用的接口并保持向下兼容。
2. 新增 Test time augmentation 模型基类。
3. 分布式训练时,支持将 BN 自动转化为 SyncBN。
4. 新增了 SMDDP 后端并支持在 AWS 进行分布式训练。

如果想了解更多版本更新细节和历史信息,请阅读[更新日志](./docs/en/notes/changelog.md#v0.2.0 "11/10/2022")

## 安装

在安装 MMengine 之前,请确保 PyTorch 已成功安装在环境中,可以参考 [PyTorch 官方安装文档](https://pytorch.org/get-started/locally/)
Expand Down
6 changes: 6 additions & 0 deletions docs/en/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,12 @@ You can switch between Chinese and English documents in the lower-left corner of
mmengine.utils <api/utils>
mmengine.utils.dl_utils <api/utils.dl_utils>

.. toctree::
:maxdepth: 2
:caption: Notes

notes/changelog.md

.. toctree::
:caption: Switch Language

Expand Down
57 changes: 57 additions & 0 deletions docs/en/notes/changelog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Changelog of v0.x

## v0.2.0 (11/10/2022)

### New Features & Enhancements

- Add SMDDP backend and support running on AWS by @austinmw in https://github.com/open-mmlab/mmengine/pull/579
- Refactor `FileIO` but without breaking bc by @zhouzaida in https://github.com/open-mmlab/mmengine/pull/533
- Add test time augmentation base model by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/538
- Use `torch.lerp\_()` to speed up EMA by @RangiLyu in https://github.com/open-mmlab/mmengine/pull/519
- Support converting `BN` to `SyncBN` by config by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/506
- Support defining metric name in wandb backend by @okotaku in https://github.com/open-mmlab/mmengine/pull/509
- Add dockerfile by @zhouzaida in https://github.com/open-mmlab/mmengine/pull/347

### Docs

- Fix API files of English documentation by @zhouzaida in https://github.com/open-mmlab/mmengine/pull/525
- Fix typo in `instance_data.py` by @Dai-Wenxun in https://github.com/open-mmlab/mmengine/pull/530
- Fix the docstring of the model sub-package by @zhouzaida in https://github.com/open-mmlab/mmengine/pull/573
- Fix a spelling error in docs/zh_cn by @cxiang26 in https://github.com/open-mmlab/mmengine/pull/548
- Fix typo in docstring by @MengzhangLI in https://github.com/open-mmlab/mmengine/pull/527
- Update `config.md` by @Zhengfei-0311 in https://github.com/open-mmlab/mmengine/pull/562

### Bug Fixes

- Fix `LogProcessor` does not smooth loss if the name of loss doesn't start with `loss` by @liuyanyi in
https://github.com/open-mmlab/mmengine/pull/539
- Fix failed to enable `detect_anomalous_params` in `MMSeparateDistributedDataParallel` by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/588
- Fix CheckpointHook behavior unexpected if given `filename_tmpl` argument by @C1rN09 in https://github.com/open-mmlab/mmengine/pull/518
- Fix error argument sequence in `FSDP` by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/520
- Fix uploading image in wandb backend @okotaku in https://github.com/open-mmlab/mmengine/pull/510
- Fix loading state dictionary in `EMAHook` by @okotaku in https://github.com/open-mmlab/mmengine/pull/507
- Fix circle import in `EMAHook` by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/523
- Fix unit test could fail caused by `MultiProcessTestCase` by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/535
- Remove unnecessary "if statement" in `Registry` by @MambaWong in https://github.com/open-mmlab/mmengine/pull/536
- Fix `_save_to_state_dict` by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/542
- Support comparing NumPy array dataset meta in `Runner.resume` by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/511
- Use `get` instead of `pop` to dump `runner_type` in `build_runner_from_cfg` by @nijkah in https://github.com/open-mmlab/mmengine/pull/549
- Upgrade pre-commit hooks by @zhouzaida in https://github.com/open-mmlab/mmengine/pull/576
- Delete the error comment in `registry.md` by @vansin in https://github.com/open-mmlab/mmengine/pull/514
- Fix Some out-of-date unit tests by @C1rN09 in https://github.com/open-mmlab/mmengine/pull/586
- Fix typo in `MMFullyShardedDataParallel` by @yhna940 in https://github.com/open-mmlab/mmengine/pull/569
- Update Github Action CI and CircleCI by @zhouzaida in https://github.com/open-mmlab/mmengine/pull/512
- Fix unit test in windows by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/515
- Fix merge ci & multiprocessing unit test by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/529

### New Contributors

- @okotaku made their first contribution in https://github.com/open-mmlab/mmengine/pull/510
- @MengzhangLI made their first contribution in https://github.com/open-mmlab/mmengine/pull/527
- @MambaWong made their first contribution in https://github.com/open-mmlab/mmengine/pull/536
- @cxiang26 made their first contribution in https://github.com/open-mmlab/mmengine/pull/548
- @nijkah made their first contribution in https://github.com/open-mmlab/mmengine/pull/549
- @Zhengfei-0311 made their first contribution in https://github.com/open-mmlab/mmengine/pull/562
- @austinmw made their first contribution in https://github.com/open-mmlab/mmengine/pull/579
- @yhna940 made their first contribution in https://github.com/open-mmlab/mmengine/pull/569
- @liuyanyi made their first contribution in https://github.com/open-mmlab/mmengine/pull/539
6 changes: 6 additions & 0 deletions docs/zh_cn/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,12 @@
mmengine.utils <api/utils>
mmengine.utils.dl_utils <api/utils.dl_utils>

.. toctree::
:maxdepth: 2
:caption: Notes

notes/changelog.md

.. toctree::
:caption: 语言切换

Expand Down
8 changes: 4 additions & 4 deletions docs/zh_cn/tutorials/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -308,15 +308,15 @@ print(cfg.a)
{'type': 'MobileNet', 'depth': 50}
```

解析后发现,`a` 的 type 变成了 `MobileNet`
解析后发现,`a` 的 type 变成了 `MobileNet`

## 配置文件的导出

在启动训练脚本时,用户可能通过传参的方式来修改配置文件的部分字段,为此我们提供了 `dump`
接口来导出更改后的配置文件。与读取配置文件类似,用户可以通过 `cfg.dump('config.xxx')` 来选择导出文件的格式。`dump`
同样可以导出有继承关系的配置文件,导出的文件可以被独立使用,不再依赖于 `_base_` 中定义的文件。

基于继承一节定义的 `resnet50.py`,我们将其加载后导出:
基于继承一节定义的 `resnet50.py`我们将其加载后导出

```python
cfg = Config.fromfile('resnet50.py')
Expand Down Expand Up @@ -394,14 +394,14 @@ print(cfg.work_dir)
- `{{fileDirname}}` - 当前文件的目录名,例如 `/home/your-username/your-project/folder`
- `{{fileBasename}}` - 当前文件的文件名,例如 `file.py`
- `{{fileBasenameNoExtension}}` - 当前文件不包含扩展名的文件名,例如 file
- `{{fileBasenameNoExtension}}` - 当前文件不包含扩展名的文件名,例如 `file`
- `{{fileExtname}}` - 当前文件的扩展名,例如 `.py`
### 命令行修改配置
有时候我们只希望修改部分配置,而不想修改配置文件本身,例如实验过程中想更换学习率,但是又不想重新写一个配置文件,常用的做法是在命令行传入参数来覆盖相关配置。考虑到我们想修改的配置通常是一些内层参数,如优化器的学习率、模型卷积层的通道数等,因此 MMEngine 提供了一套标准的流程,让我们能够在命令行里轻松修改配置文件中任意层级的参数。
1. 使用 `argparser` 解析脚本运行的参数
1. 使用 `argparse` 解析脚本运行的参数
2. 使用 `argparse.ArgumentParser.add_argument` 方法时,让 `action` 参数的值为 [DictAction](mmengine.config.DictAction),用它来进一步解析命令行参数中用于修改配置文件的参数
3. 使用配置类的 `merge_from_dict` 方法来更新配置
Expand Down
2 changes: 1 addition & 1 deletion docs/zh_cn/tutorials/model.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ from torchvision import datasets
from torchvision.transforms import ToTensor
from mmengine.model import BaseModel
from mmengine.evaluator import BaseMetric
from mmengine import Runner
from mmengine.runner import Runner


training_data = datasets.FashionMNIST(
Expand Down
2 changes: 1 addition & 1 deletion mmengine/version.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Copyright (c) OpenMMLab. All rights reserved.

__version__ = '0.1.0'
__version__ = '0.2.0'


def parse_version_info(version_str):
Expand Down

0 comments on commit 6b963f0

Please sign in to comment.