Fix global mode recursive call

[lixiang007666]

This PR is done:

Fix: https://github.com/Oneflow-Inc/OneTeam/issues/1942

test with：

import oneflow as flow
from oneflow.utils.global_view import global_mode

a = flow.randn(
    (1, 8),
    sbp=flow.sbp.broadcast,
    placement=flow.placement("cuda", ranks=[0])
)

with global_mode(True, placement=flow.placement(type="cuda", ranks=[0,1,2,3]),sbp=flow.sbp.broadcast):
    print(a)

output:

[github-actions]

Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally.

[github-actions]

Speed stats:

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

❌ OneFlow resnet50 time: 141.1ms (= 14114.8ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 143.0ms (= 14304.2ms / 100, input_shape=[16, 3, 224, 224])
❌ Relative speed: 1.01 (= 143.0ms / 141.1ms)

OneFlow resnet50 time: 82.1ms (= 8208.5ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 88.6ms (= 8856.3ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.08 (= 88.6ms / 82.1ms)

OneFlow resnet50 time: 51.4ms (= 10277.2ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 62.4ms (= 12479.1ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.21 (= 62.4ms / 51.4ms)

OneFlow resnet50 time: 34.3ms (= 6865.2ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 44.9ms (= 8987.2ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.31 (= 44.9ms / 34.3ms)

OneFlow resnet50 time: 26.3ms (= 5263.7ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 42.5ms (= 8497.3ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.61 (= 42.5ms / 26.3ms)

OneFlow swin dataloader time: 0.246s (= 49.101s / 200, num_workers=1)
PyTorch swin dataloader time: 0.163s (= 32.606s / 200, num_workers=1)
Relative speed: 0.664 (= 0.163s / 0.246s)

OneFlow swin dataloader time: 0.069s (= 13.889s / 200, num_workers=4)
PyTorch swin dataloader time: 0.040s (= 8.036s / 200, num_workers=4)
Relative speed: 0.579 (= 0.040s / 0.069s)

OneFlow swin dataloader time: 0.040s (= 8.052s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.389s / 200, num_workers=8)
Relative speed: 0.545 (= 0.022s / 0.040s)

❌ OneFlow resnet50 time: 153.6ms (= 15355.7ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 163.6ms (= 16361.5ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
❌ Relative speed: 1.07 (= 163.6ms / 153.6ms)

OneFlow resnet50 time: 93.1ms (= 9305.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 104.1ms (= 10407.0ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.12 (= 104.1ms / 93.1ms)

OneFlow resnet50 time: 60.9ms (= 12187.0ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 79.3ms (= 15856.0ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.30 (= 79.3ms / 60.9ms)

OneFlow resnet50 time: 43.1ms (= 8612.7ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 71.8ms (= 14356.8ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.67 (= 71.8ms / 43.1ms)

OneFlow resnet50 time: 36.9ms (= 7375.5ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 68.8ms (= 13757.9ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.87 (= 68.8ms / 36.9ms)

[github-actions]

CI failed when running job: cuda-module. PR label automerge has been removed

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

❌ OneFlow resnet50 time: 141.1ms (= 14110.6ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 141.5ms (= 14147.2ms / 100, input_shape=[16, 3, 224, 224])
❌ Relative speed: 1.00 (= 141.5ms / 141.1ms)

OneFlow resnet50 time: 82.0ms (= 8200.3ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 84.2ms (= 8424.5ms / 100, input_shape=[8, 3, 224, 224])
❌ Relative speed: 1.03 (= 84.2ms / 82.0ms)

OneFlow resnet50 time: 50.9ms (= 10187.1ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 54.7ms (= 10933.5ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.07 (= 54.7ms / 50.9ms)

OneFlow resnet50 time: 33.9ms (= 6775.6ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 42.6ms (= 8528.1ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.26 (= 42.6ms / 33.9ms)

OneFlow resnet50 time: 26.2ms (= 5247.9ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 38.2ms (= 7639.9ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.46 (= 38.2ms / 26.2ms)

OneFlow swin dataloader time: 0.235s (= 47.053s / 200, num_workers=1)
PyTorch swin dataloader time: 0.152s (= 30.368s / 200, num_workers=1)
Relative speed: 0.645 (= 0.152s / 0.235s)

OneFlow swin dataloader time: 0.068s (= 13.617s / 200, num_workers=4)
PyTorch swin dataloader time: 0.039s (= 7.875s / 200, num_workers=4)
Relative speed: 0.578 (= 0.039s / 0.068s)

OneFlow swin dataloader time: 0.041s (= 8.213s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.463s / 200, num_workers=8)
Relative speed: 0.543 (= 0.022s / 0.041s)

❌ OneFlow resnet50 time: 153.2ms (= 15320.4ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 165.0ms (= 16498.5ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
❌ Relative speed: 1.08 (= 165.0ms / 153.2ms)

OneFlow resnet50 time: 92.9ms (= 9291.1ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 105.1ms (= 10511.3ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.13 (= 105.1ms / 92.9ms)

OneFlow resnet50 time: 60.8ms (= 12168.5ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 79.7ms (= 15949.1ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.31 (= 79.7ms / 60.8ms)

OneFlow resnet50 time: 42.7ms (= 8531.1ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 69.6ms (= 13925.6ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.63 (= 69.6ms / 42.7ms)

OneFlow resnet50 time: 36.3ms (= 7251.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 68.0ms (= 13604.9ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.88 (= 68.0ms / 36.3ms)

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/10056/