How to export static model with other batch_size?

[VJoer]

Now export the default batchsize for the static model is 1,how do I export a static model with the batchsize other than 1?

[irexyc]

@VJoer

What backend do you wnat?

For tensorrt, you can use dynamic config but set same shape for min/opt/max shape in deploy config.

For onnxruntime, I think dynamic shape will satisfy you need.

[mydhui]

@irexyc
Hi, I tried to deploy RTMDet_ins-s into tensorrt engine, and the batch 1 deploy is working well.

but when I tried to use batch 4, there are errors reporting dim inconsistency, [1,3,640,640] not matching [4,3,640,640].

After that I found another way to make it work which is using dynamic config like this

but when inference with the exported trt engine with 4 image tensor of [4,3,640,640], but the output just returned the first image, the following 3 image are empty result. Do you have any idea about why this happen?

BTW, I also have 2 QA questions to ask, please help is you can.

1. Can I export trt engine in an A100 GPU and deploy the trt engine in another env e.g. 3070. do I have to export trt engine within 3070 env to make it work ?
2. Will larger workspace_size (e.g. in A100 platform, max_workspace_size=1 << 34)) help accelerate inference speed?

[irexyc]

@mydhui

What is the version of mmdepoy? Could you print the output of python tools/check_env.py?

---

Can I export trt engine in an A100 GPU and deploy the trt engine in another env e.g. 3070. do I have to export trt engine within 3070 env to make it work ?

Yes, you can as they have same architecture.

Will larger workspace_size (e.g. in A100 platform, max_workspace_size=1 << 34)) help accelerate inference speed

the workspace is build option and have no effect for inference.