[CI Problem] Flaky Test: tests/python/driver/tvmc/test_compiler.py::test_compile_tflite_module_with_external_codegen_cmsisnn

[junrushao]

Branch/PR Failing

Several instances of CI flakiness on the same unittest: tests/python/driver/tvmc/test_compiler.py::test_compile_tflite_module_with_external_codegen_cmsisnn

Jenkins Link

Some examples:

• https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/main/2070/pipeline
• https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/main/2069/pipeline
• https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/main/2068/pipeline
• https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/PR-9486/2/pipeline
• https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/PR-9488/2/pipeline

Flakiness

Yes

[Mousius]

Hi @junrushao1994,

Can you assign this to me? I have no idea why it would continue to flip and it might just be old branches intersecting so needs monitoring some.

[mbs-octoml]

I suspect driver_api.cc:439 is picking out the host module non-deterministically since a) can end up with two kDLCPU targets for the given and the 'default' used for shapes etc and b) map iteration order is non-det.

[junrushao]

Thank you @Mousius @mbs-octoml for acting quickly on this :-)