Add stack series api #8901

marigoold · 2022-08-10T12:21:02Z

背景：https://github.com/Oneflow-Inc/OneTeam/issues/1519
概述：缺少 column_stack api
实现：本PR实现了以下接口及其单测、文档

column_stack
row_stack
vstack
hstack
dstack
atleast_1d
atleast_2d
atleast_3d

文档：

BBuf · 2022-08-10T15:55:32Z

python/oneflow/framework/docstr/array_ops.py

@@ -268,6 +268,94 @@
    """,
 )

+add_docstr(
+    oneflow.atleast_1d,
+    r"""Returns a 1-D view of each input tensor with 0 dimensions. Tensors with dimensions greater than 0 will be returned directly.


这个语意应该是说低于1维的扩展为1维，高于1维的输入tensor直接返回，这里的描述可以refine一下，第一看没看懂。

BBuf · 2022-08-10T15:56:38Z

python/oneflow/test/modules/test_atleast.py

+
+@flow.unittest.skip_unless_1n1d()
+class TestAtLeast(flow.unittest.TestCase):
+    @autotest(check_graph=True)


Suggested change

@autotest(check_graph=True)

@autotest(n=5)

BBuf

顺便把global的测试也补了吧

BBuf · 2022-08-10T15:56:56Z

python/oneflow/test/modules/test_stack.py

@@ -44,6 +44,91 @@ def test_stack_bool_with_random_data(test_case):
        out = torch.stack((x, y), dim=random(low=1, high=4).to(int))
        return out

+    @autotest(check_graph=True)


Suggested change

@autotest(check_graph=True)

@autotest(n=5)

下同

BBuf · 2022-08-10T15:57:41Z

python/oneflow/framework/docstr/array_ops.py

@@ -299,6 +387,151 @@
    """,
 )

+add_docstr(
+    oneflow.hstack,
+    r"""Stack tensors in :attr:`tensors` horizontally (column wise).


按规范贴一下pytorch文档链接

laoliu97 · 2022-08-11T08:49:02Z

python/oneflow/framework/docstr/array_ops.py

@@ -268,6 +268,101 @@
    """,
 )

+add_docstr(
+    oneflow.atleast_1d,
+    r"""Returns a 1-dimensional view of each input tensor with zero dimensions. Input tensors with one or more dimensions are returned as-is.


在docstr中应增加对接口传入参数、参数类型的对应描述，对齐pytorch
如：下图torch.atleast_1d( *tensors ) 括号中的内容

下面的接口也是如此

github-actions · 2022-08-19T05:47:13Z

Speed stats:

github-actions · 2022-08-20T05:07:39Z

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8901/

github-actions · 2022-08-20T05:08:14Z

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.5ms (= 12851.0ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 144.0ms (= 14398.9ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.12 (= 144.0ms / 128.5ms)

OneFlow resnet50 time: 75.6ms (= 7557.0ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 83.4ms (= 8343.3ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.10 (= 83.4ms / 75.6ms)

OneFlow resnet50 time: 49.1ms (= 9819.0ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 57.1ms (= 11426.4ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.16 (= 57.1ms / 49.1ms)

OneFlow resnet50 time: 36.5ms (= 7299.0ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 42.3ms (= 8462.2ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.16 (= 42.3ms / 36.5ms)

OneFlow resnet50 time: 28.4ms (= 5671.5ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.1ms (= 7812.4ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.38 (= 39.1ms / 28.4ms)

OneFlow swin dataloader time: 0.268s (= 53.645s / 200, num_workers=1)
PyTorch swin dataloader time: 0.152s (= 30.415s / 200, num_workers=1)
Relative speed: 0.567 (= 0.152s / 0.268s)

OneFlow swin dataloader time: 0.072s (= 14.446s / 200, num_workers=4)
PyTorch swin dataloader time: 0.043s (= 8.611s / 200, num_workers=4)
Relative speed: 0.596 (= 0.043s / 0.072s)

OneFlow swin dataloader time: 0.039s (= 7.855s / 200, num_workers=8)
PyTorch swin dataloader time: 0.021s (= 4.230s / 200, num_workers=8)
Relative speed: 0.539 (= 0.021s / 0.039s)

❌ OneFlow resnet50 time: 136.9ms (= 13690.6ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 161.0ms (= 16098.2ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.18 (= 161.0ms / 136.9ms)

OneFlow resnet50 time: 85.1ms (= 8514.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 102.7ms (= 10272.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.21 (= 102.7ms / 85.1ms)

OneFlow resnet50 time: 58.4ms (= 11673.3ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 78.0ms (= 15603.5ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.34 (= 78.0ms / 58.4ms)

OneFlow resnet50 time: 45.6ms (= 9112.4ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 76.6ms (= 15323.0ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.68 (= 76.6ms / 45.6ms)

OneFlow resnet50 time: 39.1ms (= 7829.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 72.1ms (= 14413.4ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.84 (= 72.1ms / 39.1ms)

github-actions · 2022-08-20T05:50:53Z

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8901/

github-actions · 2022-08-20T05:52:18Z

CI failed when running job: cuda-module. PR label automerge has been removed

github-actions · 2022-08-20T05:52:53Z

Speed stats:

github-actions · 2022-08-20T09:10:47Z

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.5ms (= 12846.3ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 140.7ms (= 14073.2ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.10 (= 140.7ms / 128.5ms)

OneFlow resnet50 time: 75.7ms (= 7572.4ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 85.0ms (= 8499.1ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.12 (= 85.0ms / 75.7ms)

OneFlow resnet50 time: 49.7ms (= 9932.8ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 55.6ms (= 11125.0ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.12 (= 55.6ms / 49.7ms)

OneFlow resnet50 time: 36.9ms (= 7383.2ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 41.5ms (= 8290.0ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.12 (= 41.5ms / 36.9ms)

OneFlow resnet50 time: 28.5ms (= 5706.8ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.2ms (= 7832.3ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.37 (= 39.2ms / 28.5ms)

OneFlow swin dataloader time: 0.417s (= 83.439s / 200, num_workers=1)
PyTorch swin dataloader time: 0.151s (= 30.151s / 200, num_workers=1)
Relative speed: 0.361 (= 0.151s / 0.417s)

OneFlow swin dataloader time: 0.072s (= 14.402s / 200, num_workers=4)
PyTorch swin dataloader time: 0.042s (= 8.304s / 200, num_workers=4)
Relative speed: 0.577 (= 0.042s / 0.072s)

OneFlow swin dataloader time: 0.039s (= 7.852s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.467s / 200, num_workers=8)
Relative speed: 0.569 (= 0.022s / 0.039s)

❌ OneFlow resnet50 time: 136.6ms (= 13661.4ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 162.2ms (= 16219.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.19 (= 162.2ms / 136.6ms)

OneFlow resnet50 time: 85.1ms (= 8513.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 111.8ms (= 11183.6ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.31 (= 111.8ms / 85.1ms)

OneFlow resnet50 time: 58.6ms (= 11726.0ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 77.9ms (= 15587.3ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.33 (= 77.9ms / 58.6ms)

OneFlow resnet50 time: 46.2ms (= 9236.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 70.6ms (= 14129.0ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.53 (= 70.6ms / 46.2ms)

OneFlow resnet50 time: 39.4ms (= 7880.1ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 68.3ms (= 13657.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.73 (= 68.3ms / 39.4ms)

github-actions · 2022-08-20T09:12:20Z

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8901/

github-actions · 2022-08-20T13:06:19Z

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.5ms (= 12847.8ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 141.1ms (= 14111.8ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.10 (= 141.1ms / 128.5ms)

OneFlow resnet50 time: 75.5ms (= 7552.9ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 87.1ms (= 8709.8ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.15 (= 87.1ms / 75.5ms)

OneFlow resnet50 time: 49.0ms (= 9798.2ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 57.6ms (= 11512.6ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.17 (= 57.6ms / 49.0ms)

OneFlow resnet50 time: 36.6ms (= 7312.8ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 44.7ms (= 8931.4ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.22 (= 44.7ms / 36.6ms)

OneFlow resnet50 time: 28.5ms (= 5701.4ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.5ms (= 7905.2ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.39 (= 39.5ms / 28.5ms)

OneFlow swin dataloader time: 0.268s (= 53.633s / 200, num_workers=1)
PyTorch swin dataloader time: 0.151s (= 30.273s / 200, num_workers=1)
Relative speed: 0.564 (= 0.151s / 0.268s)

OneFlow swin dataloader time: 0.071s (= 14.178s / 200, num_workers=4)
PyTorch swin dataloader time: 0.041s (= 8.294s / 200, num_workers=4)
Relative speed: 0.585 (= 0.041s / 0.071s)

OneFlow swin dataloader time: 0.040s (= 8.003s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.461s / 200, num_workers=8)
Relative speed: 0.557 (= 0.022s / 0.040s)

❌ OneFlow resnet50 time: 136.8ms (= 13675.0ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 163.7ms (= 16372.2ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.20 (= 163.7ms / 136.8ms)

OneFlow resnet50 time: 85.6ms (= 8560.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 103.1ms (= 10309.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.20 (= 103.1ms / 85.6ms)

OneFlow resnet50 time: 58.3ms (= 11662.6ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 78.2ms (= 15632.2ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.34 (= 78.2ms / 58.3ms)

OneFlow resnet50 time: 45.4ms (= 9082.7ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 79.7ms (= 15942.5ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.76 (= 79.7ms / 45.4ms)

OneFlow resnet50 time: 38.6ms (= 7712.9ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 69.5ms (= 13896.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.80 (= 69.5ms / 38.6ms)

marigoold added 5 commits August 10, 2022 17:45

add h/v/dstack, add atleast_1/2/3d, add column/row_stack api

786fccc

add h/v/dstack doc

b86a1fc

add column/row_stack doc

d22ad60

add atleast_1/2/3d doc

2692b8e

refine docs, match param names with torch

d44621a

marigoold requested review from doombeaker, jackalcooper, hjchen2, BBuf and daquexian as code owners August 10, 2022 12:21

marigoold added enhancement documentation api python labels Aug 10, 2022

BBuf reviewed Aug 10, 2022

View reviewed changes

BBuf approved these changes Aug 10, 2022

View reviewed changes

marigoold added 6 commits August 11, 2022 11:43

add global test and refine unittest

4ed65d8

refine docs, add reference to torch

246e2ce

add unittest for single input

04ce797

fix bug of duplicated register

bd4d770

refine docs

36aed37

refine docs

c1ac649

laoliu97 reviewed Aug 11, 2022

View reviewed changes

refine docs, add params

3e69152

laoliu97 approved these changes Aug 12, 2022

View reviewed changes

marigoold requested a review from oneflow-ci-bot August 12, 2022 13:18

marigoold added the automerge label Aug 12, 2022

Merge branch 'master' into dev_add_xxxstack_api

eb0edaa

marigoold enabled auto-merge (squash) August 12, 2022 13:19

fix global test max_dim bug

f3dfae4

marigoold added the automerge label Aug 19, 2022

marigoold and others added 2 commits August 19, 2022 15:53

fix typo in docstr

79c40b2

Merge branch 'master' into dev_add_xxxstack_api

fb13e96

marigoold requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 19, 2022 11:46

mergify bot added 6 commits August 19, 2022 12:53

Merge branch 'master' into dev_add_xxxstack_api

8492919

Merge branch 'master' into dev_add_xxxstack_api

9b65394

Merge branch 'master' into dev_add_xxxstack_api

399c880

Merge branch 'master' into dev_add_xxxstack_api

84da90b

Merge branch 'master' into dev_add_xxxstack_api

7d9d15a

Merge branch 'master' into dev_add_xxxstack_api

3ae8c4a

Merge branch 'master' into dev_add_xxxstack_api

7479ef7

github-actions bot removed the automerge label Aug 20, 2022

marigoold added the automerge label Aug 20, 2022

mergify bot and others added 2 commits August 20, 2022 09:59

Merge branch 'master' into dev_add_xxxstack_api

cf4eab3

Merge branch 'master' into dev_add_xxxstack_api

836a5e3

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 20, 2022 17:34

marigoold merged commit b0369f9 into master Aug 20, 2022

marigoold deleted the dev_add_xxxstack_api branch August 20, 2022 18:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add stack series api #8901

Add stack series api #8901

marigoold commented Aug 10, 2022 •

edited

Loading

BBuf Aug 10, 2022

BBuf Aug 10, 2022

BBuf left a comment

BBuf Aug 10, 2022

BBuf Aug 10, 2022

laoliu97 Aug 11, 2022

github-actions bot commented Aug 19, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

Add stack series api #8901

Add stack series api #8901

Conversation

marigoold commented Aug 10, 2022 • edited Loading

BBuf Aug 10, 2022

Choose a reason for hiding this comment

BBuf Aug 10, 2022

Choose a reason for hiding this comment

BBuf left a comment

Choose a reason for hiding this comment

BBuf Aug 10, 2022

Choose a reason for hiding this comment

BBuf Aug 10, 2022

Choose a reason for hiding this comment

laoliu97 Aug 11, 2022

Choose a reason for hiding this comment

github-actions bot commented Aug 19, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

github-actions bot commented Aug 20, 2022

marigoold commented Aug 10, 2022 •

edited

Loading