Develop_OneFlow_Normal_Distribution_Synced_With_PyTorch_Tensor

[ccssu]

• signature: "Tensor (Tensor x, Float mean=0.0, Float std=1.0, Generator generator=None) => Normal_"
  "Tensor (Tensor mean, Tensor std, *, Tensor out=None,
• Generator generator=None, Bool requires_grad=False) => TensorTensorNormal",
  "Tensor (Tensor mean, Float std=1.0, *, Tensor out=None,
• Generator generator=None, Bool requires_grad=False) => TensorScalarNormal",
  "Tensor (Float mean, Tensor std, *, Tensor out=None,
• Generator generator=None, Bool requires_grad=False) => ScalarTensorNormal",

[github-actions]

Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally.

[github-actions]

Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally.

[github-actions]

Static analysis with clang failed. PR label automerge has been removed

[github-actions]

Speed stats:

[github-actions]

CI failed when running job: cpu-misc. PR label automerge has been removed

[github-actions]

Speed stats:

[github-actions]

Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally.

[github-actions]

Speed stats:

[github-actions]

Speed stats:

[github-actions]

Speed stats:

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/10185/

[github-actions]

Speed stats:

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/10185/

[github-actions]

Speed stats:

GPU Name: NVIDIA GeForce RTX 3090 

❌ OneFlow resnet50 time: 42.7ms (= 4266.7ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 57.6ms (= 5759.9ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.35 (= 57.6ms / 42.7ms)

OneFlow resnet50 time: 26.2ms (= 2623.6ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 37.5ms (= 3745.4ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.43 (= 37.5ms / 26.2ms)

OneFlow resnet50 time: 18.3ms (= 3665.8ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 35.7ms (= 7145.5ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.95 (= 35.7ms / 18.3ms)

OneFlow resnet50 time: 17.0ms (= 3400.7ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 30.7ms (= 6147.1ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.81 (= 30.7ms / 17.0ms)

OneFlow resnet50 time: 16.7ms (= 3339.8ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 29.5ms (= 5896.2ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.77 (= 29.5ms / 16.7ms)

OneFlow swin dataloader time: 0.201s (= 40.106s / 200, num_workers=1)
PyTorch swin dataloader time: 0.128s (= 25.608s / 200, num_workers=1)
Relative speed: 0.639 (= 0.128s / 0.201s)

OneFlow swin dataloader time: 0.055s (= 10.999s / 200, num_workers=4)
PyTorch swin dataloader time: 0.033s (= 6.651s / 200, num_workers=4)
Relative speed: 0.605 (= 0.033s / 0.055s)

OneFlow swin dataloader time: 0.031s (= 6.176s / 200, num_workers=8)
PyTorch swin dataloader time: 0.017s (= 3.390s / 200, num_workers=8)
Relative speed: 0.549 (= 0.017s / 0.031s)

❌ OneFlow resnet50 time: 48.7ms (= 4873.7ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 65.8ms (= 6580.6ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.35 (= 65.8ms / 48.7ms)

OneFlow resnet50 time: 36.2ms (= 3619.3ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 45.8ms (= 4584.6ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.27 (= 45.8ms / 36.2ms)

OneFlow resnet50 time: 28.7ms (= 5732.4ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 43.0ms (= 8607.0ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.50 (= 43.0ms / 28.7ms)

OneFlow resnet50 time: 25.6ms (= 5118.8ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 39.4ms (= 7882.8ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.54 (= 39.4ms / 25.6ms)

OneFlow resnet50 time: 24.2ms (= 4844.8ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 37.4ms (= 7485.8ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.55 (= 37.4ms / 24.2ms)