add fuse_bn_add_act_ops args #4864
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zhangting2020
force-pushed
the
fuse_bn_add_act
branch
from
79de711 to
cb673c1
Compare
wzzju
approved these changes
Sep 23, 2020
qingqing01
approved these changes
Oct 15, 2020
zhangting2020
added a commit
that referenced
this pull request
Oct 22, 2020
This reverts commit 294ff30.
zhangting2020
added a commit
that referenced
this pull request
Oct 27, 2020
* Revert "add fuse_bn_add_act_ops args (#4864)"
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
为AMP训练增了了fuse_bn_add_act op，目前通过python接口使用这个fuse op，因此对应修改了模型：增加add fuse_bn_add_act_ops args。
TODO：后续添加了fuse_bn_add_act的pass后，通过build_strategy开启，就不再需要对resnet.py和build_model.py进行修改了。因此待pass添加后，这两个文件会改回去。
性能收益：
V100, CUDA 10.1, cuDNN 7.6, single card, BS=128
收敛性验证在PR27230中已验证无问题