add double grad for div #8877

pingzhuu · 2022-08-08T12:33:36Z

添加 div 二阶导算子

…_div

…d' into add_double_grad_for_div

python/oneflow/test/modules/test_global_higher_derivative_div.py

python/oneflow/test/modules/test_higher_derivative_div.py

Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>

…ation

github-actions · 2022-08-15T14:56:00Z

CI failed when running job: cpu-module. PR label automerge has been removed

github-actions · 2022-08-15T14:57:08Z

Speed stats:

…Inc/oneflow into add_double_grad_for_div

github-actions · 2022-08-18T19:13:00Z

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8877/

github-actions · 2022-08-18T19:14:48Z

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.4ms (= 12844.4ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 143.0ms (= 14301.4ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.11 (= 143.0ms / 128.4ms)

OneFlow resnet50 time: 75.3ms (= 7532.8ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 84.5ms (= 8449.1ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.12 (= 84.5ms / 75.3ms)

OneFlow resnet50 time: 48.6ms (= 9720.3ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 58.6ms (= 11724.8ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.21 (= 58.6ms / 48.6ms)

OneFlow resnet50 time: 36.1ms (= 7226.0ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 50.0ms (= 9992.8ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.38 (= 50.0ms / 36.1ms)

OneFlow resnet50 time: 28.3ms (= 5668.8ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 35.9ms (= 7170.5ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.26 (= 35.9ms / 28.3ms)

OneFlow swin dataloader time: 0.255s (= 50.919s / 200, num_workers=1)
PyTorch swin dataloader time: 0.149s (= 29.819s / 200, num_workers=1)
Relative speed: 0.586 (= 0.149s / 0.255s)

OneFlow swin dataloader time: 0.068s (= 13.688s / 200, num_workers=4)
PyTorch swin dataloader time: 0.040s (= 7.935s / 200, num_workers=4)
Relative speed: 0.580 (= 0.040s / 0.068s)

OneFlow swin dataloader time: 0.039s (= 7.811s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.419s / 200, num_workers=8)
Relative speed: 0.566 (= 0.022s / 0.039s)

❌ OneFlow resnet50 time: 136.8ms (= 13681.9ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 161.3ms (= 16133.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.18 (= 161.3ms / 136.8ms)

OneFlow resnet50 time: 85.1ms (= 8505.4ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 101.6ms (= 10164.0ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.19 (= 101.6ms / 85.1ms)

OneFlow resnet50 time: 58.3ms (= 11655.5ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 78.8ms (= 15751.4ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.35 (= 78.8ms / 58.3ms)

OneFlow resnet50 time: 45.7ms (= 9133.3ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 77.6ms (= 15525.7ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.70 (= 77.6ms / 45.7ms)

OneFlow resnet50 time: 39.0ms (= 7800.5ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 68.2ms (= 13649.2ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.75 (= 68.2ms / 39.0ms)

github-actions · 2022-08-19T23:14:28Z

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.6ms (= 12860.2ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 141.4ms (= 14141.9ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.10 (= 141.4ms / 128.6ms)

OneFlow resnet50 time: 75.3ms (= 7533.4ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 84.4ms (= 8436.6ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.12 (= 84.4ms / 75.3ms)

OneFlow resnet50 time: 48.5ms (= 9696.3ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 59.6ms (= 11925.7ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.23 (= 59.6ms / 48.5ms)

OneFlow resnet50 time: 36.1ms (= 7217.2ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 47.3ms (= 9459.7ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.31 (= 47.3ms / 36.1ms)

OneFlow resnet50 time: 28.1ms (= 5628.4ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 35.4ms (= 7082.4ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.26 (= 35.4ms / 28.1ms)

OneFlow swin dataloader time: 0.417s (= 83.410s / 200, num_workers=1)
PyTorch swin dataloader time: 0.152s (= 30.443s / 200, num_workers=1)
Relative speed: 0.365 (= 0.152s / 0.417s)

OneFlow swin dataloader time: 0.069s (= 13.870s / 200, num_workers=4)
PyTorch swin dataloader time: 0.043s (= 8.506s / 200, num_workers=4)
Relative speed: 0.613 (= 0.043s / 0.069s)

OneFlow swin dataloader time: 0.040s (= 8.036s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.479s / 200, num_workers=8)
Relative speed: 0.557 (= 0.022s / 0.040s)

❌ OneFlow resnet50 time: 136.9ms (= 13688.0ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 165.4ms (= 16538.0ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.21 (= 165.4ms / 136.9ms)

OneFlow resnet50 time: 84.6ms (= 8460.7ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 101.9ms (= 10191.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.20 (= 101.9ms / 84.6ms)

OneFlow resnet50 time: 57.8ms (= 11551.1ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 78.3ms (= 15653.3ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.36 (= 78.3ms / 57.8ms)

OneFlow resnet50 time: 45.3ms (= 9065.6ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 70.3ms (= 14057.6ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.55 (= 70.3ms / 45.3ms)

OneFlow resnet50 time: 39.2ms (= 7847.1ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 68.0ms (= 13606.0ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.73 (= 68.0ms / 39.2ms)

github-actions · 2022-08-19T23:14:49Z

CI failed when running job: cuda-module. PR label automerge has been removed

github-actions · 2022-08-20T06:06:19Z

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8877/

github-actions · 2022-08-20T06:10:35Z

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.5ms (= 12851.9ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 143.2ms (= 14315.4ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.11 (= 143.2ms / 128.5ms)

OneFlow resnet50 time: 75.6ms (= 7559.1ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 86.7ms (= 8670.0ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.15 (= 86.7ms / 75.6ms)

OneFlow resnet50 time: 49.5ms (= 9906.7ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 56.7ms (= 11336.3ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.14 (= 56.7ms / 49.5ms)

OneFlow resnet50 time: 36.6ms (= 7322.2ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 41.8ms (= 8368.9ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.14 (= 41.8ms / 36.6ms)

OneFlow resnet50 time: 28.5ms (= 5698.1ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 36.4ms (= 7288.0ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.28 (= 36.4ms / 28.5ms)

OneFlow swin dataloader time: 0.265s (= 52.926s / 200, num_workers=1)
PyTorch swin dataloader time: 0.150s (= 30.083s / 200, num_workers=1)
Relative speed: 0.568 (= 0.150s / 0.265s)

OneFlow swin dataloader time: 0.115s (= 22.981s / 200, num_workers=4)
PyTorch swin dataloader time: 0.041s (= 8.216s / 200, num_workers=4)
Relative speed: 0.358 (= 0.041s / 0.115s)

OneFlow swin dataloader time: 0.042s (= 8.336s / 200, num_workers=8)
PyTorch swin dataloader time: 0.023s (= 4.519s / 200, num_workers=8)
Relative speed: 0.542 (= 0.023s / 0.042s)

❌ OneFlow resnet50 time: 136.9ms (= 13687.4ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 161.8ms (= 16182.5ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.18 (= 161.8ms / 136.9ms)

OneFlow resnet50 time: 86.0ms (= 8604.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 107.4ms (= 10743.2ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.25 (= 107.4ms / 86.0ms)

OneFlow resnet50 time: 58.5ms (= 11694.7ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 85.4ms (= 17083.5ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.46 (= 85.4ms / 58.5ms)

OneFlow resnet50 time: 46.2ms (= 9243.3ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 71.5ms (= 14304.9ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.55 (= 71.5ms / 46.2ms)

OneFlow resnet50 time: 39.0ms (= 7804.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 69.5ms (= 13893.4ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.78 (= 69.5ms / 39.0ms)

github-actions · 2022-08-20T06:10:59Z

CI failed when running job: cpu-misc. PR label automerge has been removed

github-actions · 2022-08-20T06:42:30Z

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8877/

github-actions · 2022-08-20T06:43:07Z

Speed stats:

GPU Name: NVIDIA GeForce GTX 1080 

❌ OneFlow resnet50 time: 130.2ms (= 13015.2ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 143.5ms (= 14349.9ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.10 (= 143.5ms / 130.2ms)

OneFlow resnet50 time: 77.2ms (= 7724.2ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 85.4ms (= 8542.3ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.11 (= 85.4ms / 77.2ms)

OneFlow resnet50 time: 50.3ms (= 10065.2ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 55.9ms (= 11173.4ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.11 (= 55.9ms / 50.3ms)

OneFlow resnet50 time: 38.3ms (= 7650.8ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 46.3ms (= 9252.3ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.21 (= 46.3ms / 38.3ms)

OneFlow resnet50 time: 29.7ms (= 5933.5ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 42.0ms (= 8394.4ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.41 (= 42.0ms / 29.7ms)

OneFlow swin dataloader time: 0.275s (= 54.931s / 200, num_workers=1)
PyTorch swin dataloader time: 0.152s (= 30.352s / 200, num_workers=1)
Relative speed: 0.553 (= 0.152s / 0.275s)

OneFlow swin dataloader time: 0.073s (= 14.678s / 200, num_workers=4)
PyTorch swin dataloader time: 0.041s (= 8.229s / 200, num_workers=4)
Relative speed: 0.561 (= 0.041s / 0.073s)

OneFlow swin dataloader time: 0.039s (= 7.884s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.392s / 200, num_workers=8)
Relative speed: 0.557 (= 0.022s / 0.039s)

❌ OneFlow resnet50 time: 147.1ms (= 14714.3ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 166.2ms (= 16618.8ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.13 (= 166.2ms / 147.1ms)

OneFlow resnet50 time: 98.1ms (= 9812.7ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 113.7ms (= 11369.6ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 113.7ms / 98.1ms)

OneFlow resnet50 time: 70.2ms (= 14031.1ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 88.7ms (= 17736.7ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.26 (= 88.7ms / 70.2ms)

OneFlow resnet50 time: 58.0ms (= 11596.7ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 73.8ms (= 14768.3ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.27 (= 73.8ms / 58.0ms)

OneFlow resnet50 time: 51.4ms (= 10286.9ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 69.7ms (= 13937.2ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.35 (= 69.7ms / 51.4ms)

add double grad for div

b1b1cf2

pingzhuu added enhancement op test labels Aug 8, 2022

pingzhuu added 5 commits August 10, 2022 12:18

register reduce_sum_like backward function and add testcase

1204b47

refine testcase

eafc628

Merge remote-tracking branch 'origin/master' into add_double_grad_for…

53e8113

…_div

Merge remote-tracking branch 'origin/register_reduce_sum_like_backwar…

db60573

…d' into add_double_grad_for_div

refine testcase

0040734

pingzhuu marked this pull request as ready for review August 10, 2022 08:46

pingzhuu requested review from hjchen2, BBuf, jackalcooper, chengtbf, strint and daquexian as code owners August 10, 2022 08:46

pingzhuu requested a review from wyg1997 August 10, 2022 08:46

Merge branch 'master' into add_double_grad_for_div

54b07b2

wyg1997 approved these changes Aug 15, 2022

View reviewed changes

python/oneflow/test/modules/test_global_higher_derivative_div.py Outdated Show resolved Hide resolved

python/oneflow/test/modules/test_global_higher_derivative_div.py Outdated Show resolved Hide resolved

python/oneflow/test/modules/test_higher_derivative_div.py Outdated Show resolved Hide resolved

pingzhuu and others added 2 commits August 15, 2022 12:41

Update python/oneflow/test/modules/test_global_higher_derivative_div.py

65ac525

Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>

refine testcase and checkout error msg changes about high order deriv…

c49fe31

…ation

hjchen2 approved these changes Aug 15, 2022

View reviewed changes

pingzhuu added the automerge label Aug 15, 2022

pingzhuu requested a review from oneflow-ci-bot August 15, 2022 10:40

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 15, 2022 14:05

github-actions bot removed the automerge label Aug 15, 2022

fix div precision error

f3389d5

mergify bot and others added 3 commits August 18, 2022 13:08

Merge branch 'master' into add_double_grad_for_div

4dd99ca

refine precision

7599e65

Merge branch 'add_double_grad_for_div' of https://github.com/Oneflow-…

057a087

…Inc/oneflow into add_double_grad_for_div

Merge branch 'master' into add_double_grad_for_div

539ffe4

pingzhuu requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 19, 2022 07:25

mergify bot and others added 5 commits August 19, 2022 09:45

Merge branch 'master' into add_double_grad_for_div

fcd54a9

Merge branch 'master' into add_double_grad_for_div

945028d

Merge branch 'master' into add_double_grad_for_div

1893591

Merge branch 'master' into add_double_grad_for_div

b5c9eba

Merge branch 'master' into add_double_grad_for_div

06fc0dd

github-actions bot removed the automerge label Aug 19, 2022

pingzhuu added the automerge label Aug 20, 2022

mergify bot added 3 commits August 20, 2022 02:10

Merge branch 'master' into add_double_grad_for_div

9e03520

Merge branch 'master' into add_double_grad_for_div

4bd9f2b

Merge branch 'master' into add_double_grad_for_div

8d7eeb0

github-actions bot removed the automerge label Aug 20, 2022

pingzhuu added the automerge label Aug 20, 2022

mergify bot merged commit 68f932c into master Aug 20, 2022

mergify bot deleted the add_double_grad_for_div branch August 20, 2022 07:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add double grad for div #8877

add double grad for div #8877

pingzhuu commented Aug 8, 2022 •

edited

Loading

github-actions bot commented Aug 15, 2022

github-actions bot commented Aug 15, 2022

github-actions bot commented Aug 18, 2022

github-actions bot commented Aug 18, 2022

github-actions bot commented Aug 19, 2022

github-actions bot commented Aug 19, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

add double grad for div #8877

add double grad for div #8877

Conversation

pingzhuu commented Aug 8, 2022 • edited Loading

github-actions bot commented Aug 15, 2022

github-actions bot commented Aug 15, 2022

github-actions bot commented Aug 18, 2022

github-actions bot commented Aug 18, 2022

github-actions bot commented Aug 19, 2022

github-actions bot commented Aug 19, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

pingzhuu commented Aug 8, 2022 •

edited

Loading