trt-engine-explorer plot circle in graph of an model engine

[roachsinai]

This engine was built in fp16 mode. And tensorrt infer results is right.

Just wonder how tensorrt know when to stop the infer as ther is a circle in below img?

By the way, why there are different colors for edges between nodes, what's the meaning of different color?

Thanks a lot!

[lix19937]

green means int8 dataflow, yellow means fp16 dataflow, red means fp32 dataflow.

this is not circle, you can upload your onnx.

[roachsinai]

@lix19937 Thanks for your reply! Before I don't know the meaning of red and yellow colors. I think there is a circle ignore edge color.

But even with the infomation you said, why the node output1 could accept two diffferent data type which are coming in different time.

How tensorrt know which inputs should be the truely output of node output1?

[lix19937]

@roachsinai you can upload the subgraph(onnx) match the above #4249 (comment) ops .

[roachsinai]

@lix19937 sorry I'm not a English native speaker. Let me restate my question again.

Firstly, I thought there is no circle as the inference result of this engine is right (no circle in neither the onnx model nor the trt plan).

Sencondly, the svg created by trt-engine-explorer confused me, as don't know why add an red edge from Reformatting CopyNode for Output Tensor 0 to output1 to output1. I thought there is a path output1 -> Reformatting CopyNode for Output Tensor 0 to output1 then Reformatting CopyNode for Output Tensor 0 to output1 -> output1, isn't it a circle?

After I check the json file which the above svg file is created by, maybe the red edge is just a shortcut that no need draw the truely output.

Thanks.

[lix19937]

TRT only support DAG, what is your trex version ? Can you try to use the latest vesrion ?

[roachsinai]

trex version: 0.1.6.

checkout below json, I copy it from corresponding .onnx.engine.graph.json:

},{
  "Name": "output2",
  "LayerType": "Shuffle",
  "Inputs": [
  {
    "Name": "Reformatted Input Tensor 0 to output2",
    "Location": "Device",
    "Dimensions": [1,25,625],
    "Format/Datatype": "Row major linear FP32"
  }],
  "Outputs": [
  {
    "Name": "output2",
    "Location": "Device",
    "Dimensions": [1,625,25],
    "Format/Datatype": "Row major linear FP32"
  }],
  "ParameterType": "Shuffle",
  "FirstTranspose": [0,2,1],
  "Reshape": "nbDims=-1",
  "SecondTranspose": [0,1,2],
  "ZeroIsPlaceholder": 1,
  "TacticValue": "0x0000000000000000",
  "StreamId": 0,
  "Metadata": "[ONNX Layer: output2]"
},{

which leads to below small circle (at bottom of below img):