[ARM] Fix concat

[hlu1]

Fix for https://discuss.tvm.ai/t/relay-build-target-rasp3b-something-wrong/2195
I added arm_cpu schedule for concat with no vectorization.New test case is also added to cover the special case that triggered this bug.

@FrozenGene, please review

[FrozenGene]

I still meet the same error. @hlu1

v0.0.1
%3 = fn (%p0: Tensor[(3, 256), float32], %p1: Tensor[(3, 256), float32], dict=meta[StrMap][0]) -> Tensor[(3, 512), float32] {
%0 = (%p0, %p1)
%1 = concatenate(%0, axis=1) // ty=Tensor[(3, 512), float32]
%2 = nn.batch_flatten(%1) // ty=Tensor[(3, 512), float32]
%2
}
%3

[hlu1]

The root cause of this problem is that the way concat implemented is not vectorizable. It uses tvm::if_then_else (https://github.com/dmlc/tvm/blob/master/topi/include/topi/transform.h#L338), which cannot be vectorized. Writing a separate schedule for concat doesn't completely solve the problem because concat is marked as injective op and once it's fused with other injective ops, the fused_op is scheduled with the injective schedule instead of the concat schedule. One temporary solution is to remove vectorization from the injective schedule, which is not optimal. Another solution is to fix the if_then_else problem, which isn't quite obvious to me.

[FrozenGene]

@tqchen tvm::if_then_else I remember is introduced by one commit of you, right? Could you help to see the problem of @hlu1 mentioned?

[tqchen]

I think we could use some discussion here. if_then_else is a "safe" version of select when we cannot simply read out values from both sides and there are some branches involved.

Although for certain systems, OOB read is fine and select could just be used. Ideally, we would like to have a detector that detects if vectorization can be applied, and only apply it when it finds that the compute-pattern is OK. We could move some of the discussion to a new RFC.

@hlu1 @FrozenGene please share your thoughts

[tqchen]

ping @hlu1

[tqchen]

Merge this in for now as it is a strict improvement over previous ones

[tqchen]

Thanks, @hlu1 @FrozenGene