Remove ActorMsg::user_data_

[liujuncheng]

原来的实现中，IBVerbs 的 token 是嵌入到 ActorMsg 里面的，影响了 ActorMsg 的 size，这个PR中改成IBVerbsActorMsgWrapper 封装 ActorMsg 和 token，以减少 ActorMsg 的大小

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 









❌ OneFlow resnet50 time: 139.7ms (= 13971.8ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 160.0ms (= 16000.2ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.15 (= 160.0ms / 139.7ms)

OneFlow resnet50 time: 85.0ms (= 8495.1ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 113.0ms (= 11297.6ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.33 (= 113.0ms / 85.0ms)

OneFlow resnet50 time: 57.3ms (= 11452.1ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 77.5ms (= 15509.0ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.35 (= 77.5ms / 57.3ms)

OneFlow resnet50 time: 44.6ms (= 8920.7ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 71.0ms (= 14208.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.59 (= 71.0ms / 44.6ms)

OneFlow resnet50 time: 40.7ms (= 8148.5ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 74.2ms (= 14842.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.82 (= 74.2ms / 40.7ms)

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/9762/

[ouyangyu]

2机共16卡测试(25/27，NVIDIA_GeForce_RTX_3090)

export NCCL_DEBUG=INFO ，详情见下表日志链接

case	oneflow@master	oneflow@dev_refactor_actor_msg
LibAI_bert_graph_FP16_acfalse_DP16_mbs4_gbs64_acc1_2n8g	16159 MiB / 2524.32 samples/s	16650 MiB / 2496 samples/s
LibAI_bert_FP16_actrue_DP16_mbs32_gbs512_acc1_2n8g	8714 MiB / 7904 samples/s	8714 MiB / 7904 samples/s

loss对比

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 









❌ OneFlow resnet50 time: 139.9ms (= 13994.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 163.2ms (= 16320.5ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.17 (= 163.2ms / 139.9ms)

OneFlow resnet50 time: 85.4ms (= 8539.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 101.3ms (= 10133.7ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.19 (= 101.3ms / 85.4ms)

OneFlow resnet50 time: 57.5ms (= 11501.8ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 78.7ms (= 15732.7ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.37 (= 78.7ms / 57.5ms)

OneFlow resnet50 time: 45.5ms (= 9092.9ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 77.5ms (= 15496.1ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.70 (= 77.5ms / 45.5ms)

OneFlow resnet50 time: 42.0ms (= 8401.4ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 68.3ms (= 13660.3ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.63 (= 68.3ms / 42.0ms)

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/9762/

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 









❌ OneFlow resnet50 time: 139.9ms (= 13991.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 160.7ms (= 16069.9ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.15 (= 160.7ms / 139.9ms)

OneFlow resnet50 time: 85.0ms (= 8495.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 101.3ms (= 10130.2ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.19 (= 101.3ms / 85.0ms)

OneFlow resnet50 time: 57.7ms (= 11549.0ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 78.3ms (= 15661.7ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.36 (= 78.3ms / 57.7ms)

OneFlow resnet50 time: 44.7ms (= 8930.9ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 81.4ms (= 16288.8ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.82 (= 81.4ms / 44.7ms)

OneFlow resnet50 time: 42.0ms (= 8394.3ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 68.5ms (= 13702.8ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.63 (= 68.5ms / 42.0ms)

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/9762/