[Bug] [FRONTEND][ONNX] fails to convert a valid onnx model due to the wrong process of DepthToSpace operator: Check failed: (expr->struct_info_.defined()) is false: The struct_info is not populated, check if you have normalized the expr

[coffezhou]

Expected behavior

The onnx frontend should import the model correctly.

Actual behavior

For the following model, it can be executed by onnxruntime.

the results of onnxruntime are as follows:

ONNXRuntime :
[array([[[[55.49899 ，52.465206]
            [56.963356，47.7378921111，dtype=float32)]

However, this model cannot be converted by the onnx frontend in TVM, a crash occurs as follows:

Error converting operator DepthToSpace, with inputs: [R.sum(lv16, axis=None, keepdims=True)]
Traceback (most recent call last):
  File "/home/carla/Documents/test/test.py", line 36, in <module>
    main()
  File "/home/carla/Documents/test/test.py", line 31, in main
    tvm_model = from_onnx(onnx_model)
                ^^^^^^^^^^^^^^^^^^^^^
  File "/home/carla/Documents/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 3695, in from_onnx
    return g.from_onnx(graph, opset)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/carla/Documents/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 3326, in from_onnx
    self._construct_nodes(graph)
  File "/home/carla/Documents/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 3506, in _construct_nodes
    raise err
  File "/home/carla/Documents/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 3501, in _construct_nodes
    op = self._convert_operator(op_name, inputs, attr, self.opset)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/carla/Documents/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 3601, in _convert_operator
    sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/carla/Documents/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 2918, in _impl_v11
    return relax.op.reshape(x, (b, c // (block_size**2), h * block_size, w * block_size))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/carla/Documents/tvm/python/tvm/relax/op/manipulate.py", line 221, in reshape
    return _ffi_api.reshape(x, shape)  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "tvm/ffi/cython/./function.pxi", line 228, in tvm.ffi.core.Function.__call__
tvm.error.InternalError: Check failed: (expr->struct_info_.defined()) is false: The struct_info is not populated, check if you have normalized the expr
[21:24:07] /home/carla/Documents/tvm/src/relax/ir/block_builder.cc:64: Warning: BlockBuilder destroyed with remaining blocks!

Environment

OS: Ubuntu 20.04
TVM: 0.21.dev0(eca92bd)

Steps to reproduce

This bug can be reproduced by the following code with the model in the attachment. As shown in the code, the model can be executed by onnxruntime, which indicates that the model is valid.

import sys

import numpy as np
import onnx
import onnxruntime

import tvm
from tvm import relax
from tvm.relax.frontend.onnx import from_onnx

import pickle

            
def main():
    onnx_model = onnx.load("11.onnx")
    
    with open("inputs.pkl", "rb") as fp:
        inputs = pickle.load(fp)
    
    try:
        ort_session = onnxruntime.InferenceSession(
            onnx_model.SerializeToString(), providers=["CPUExecutionProvider"]
        )
        ort_output = ort_session.run([], inputs)
    except Exception as e:
        print(e)
        sys.exit(1)
        
    print("ONNXRuntime:\n", ort_output)   

    tvm_model = from_onnx(onnx_model)

    
if __name__ == "__main__":
    
    main()

testcase.zip

Triage

• needs-triage

cc @KJlaccHoeUM9l

[vacu9708]

I’ve fixed the root cause and will commit shortly.