Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix advance indexing bug and zeros_like sbp bug #7238

Merged
merged 32 commits into from
Jan 26, 2022

Conversation

wyg1997
Copy link
Contributor

@wyg1997 wyg1997 commented Jan 12, 2022

修复在跑 swin-transformer eager consistent 2d sbp 时遇到的两个问题:

  1. AdvanceIndexing 在多维 placement 时,sbp 设置错误;
  2. zeros_like op(ones_like 也一样) 结果的 sbp 和输入的 sbp 不一致(这里是由于它走了 sbp 推导的流程,而 B -> ? 的代价都是 0,所以可能会拿到错误的 sbp),重写了这个 UserOp 的 NdSbpInferFn 就可以了。

@wyg1997 wyg1997 force-pushed the fix-zeros_like_sbp branch from bbacd74 to 76f61dc Compare January 12, 2022 06:22
@wyg1997 wyg1997 requested review from oneflow-ci-bot and removed request for oneflow-ci-bot January 12, 2022 08:08
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot January 12, 2022 08:49
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot January 12, 2022 10:16
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot January 12, 2022 14:06
@github-actions
Copy link
Contributor

CI failed when running job: cuda-module. PR label automerge has been removed

@oneflow-ci-bot oneflow-ci-bot removed their request for review January 12, 2022 16:46
@oneflow-ci-bot oneflow-ci-bot removed their request for review January 22, 2022 07:36
@github-actions
Copy link
Contributor

CI failed when running job: cuda-speed-test. PR label automerge has been removed

@oneflow-ci-bot oneflow-ci-bot removed their request for review January 22, 2022 08:59
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot January 24, 2022 02:10
@github-actions
Copy link
Contributor

CI failed when running job: cuda-module. PR label automerge has been removed

@oneflow-ci-bot oneflow-ci-bot removed their request for review January 24, 2022 04:10
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot January 26, 2022 07:53
@github-actions
Copy link
Contributor

Speed stats:
GPU Name: GeForce GTX 1080 

OneFlow resnet50 time: 136.6ms (= 13661.7ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 139.8ms (= 13977.5ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.02 (= 139.8ms / 136.6ms)

OneFlow resnet50 time: 78.5ms (= 7846.8ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 85.5ms (= 8547.7ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.09 (= 85.5ms / 78.5ms)

OneFlow resnet50 time: 52.2ms (= 10445.4ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 57.6ms (= 11515.0ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.10 (= 57.6ms / 52.2ms)

OneFlow resnet50 time: 39.7ms (= 7945.6ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 48.1ms (= 9626.6ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.21 (= 48.1ms / 39.7ms)

OneFlow resnet50 time: 42.2ms (= 8446.8ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.4ms (= 7880.6ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 0.93 (= 39.4ms / 42.2ms)

OneFlow resnet50 time: 149.8ms (= 14977.6ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 160.2ms (= 16020.3ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.07 (= 160.2ms / 149.8ms)

OneFlow resnet50 time: 89.7ms (= 8965.5ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 101.7ms (= 10165.2ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.13 (= 101.7ms / 89.7ms)

OneFlow resnet50 time: 64.7ms (= 12931.4ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 71.6ms (= 14327.6ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.11 (= 71.6ms / 64.7ms)

OneFlow resnet50 time: 57.1ms (= 11412.5ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 61.6ms (= 12325.1ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.08 (= 61.6ms / 57.1ms)

OneFlow resnet50 time: 53.9ms (= 10770.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 58.1ms (= 11620.8ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.08 (= 58.1ms / 53.9ms)

@oneflow-ci-bot oneflow-ci-bot removed their request for review January 26, 2022 08:50
@oneflow-ci-bot oneflow-ci-bot merged commit 115186a into master Jan 26, 2022
@oneflow-ci-bot oneflow-ci-bot deleted the fix-zeros_like_sbp branch January 26, 2022 08:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants