[NativeAOT] Individual test failures may not always be surfaced as a failure in the test run summary.

[VSadov]

This, so far, has been observed only in NativeAOT and only locally, but I am not sure if this could happen in CoreCLR or in CI.
See discussion here: #90848 (comment)

A repro:

• undo the fix done in [NativeAOT] Reverted the change in IsInstanceOfException #90867 (it is a very small fix)
• run runtime tests with NativeAOT Checked (this may take a while)

build clr+libs -rc Checked -lc Release & src\tests\build.cmd nativeaot Checked &  src\tests\run.cmd runnativeaottests Checked

You will see a test failure in the output like:

Unhandled Exception: System.Diagnostics.DebugProvider+DebugAssertException: IsInstanceOfClass called with interface M
  ethodTable
     at System.Diagnostics.DebugProvider.Fail(String, String) + 0x36
     at System.Diagnostics.Debug.Fail(String, String) + 0x32
     at System.Runtime.TypeCast.IsInstanceOfClass(MethodTable*, Object) + 0x72
     at System.Runtime.EH.FindFirstPassHandler(Object, UInt32, StackFrameIterator&, UInt32&, Byte*&) + 0xfb
     at System.Runtime.EH.DispatchEx(StackFrameIterator&, EH.ExInfo&) + 0x131
     at System.Runtime.EH.RhThrowEx(Object, EH.ExInfo&) + 0x4b
     at GenericCatchInterfaceProgram.Main() + 0x2a
     at Program.<<Main>$>g__TestExecutor23|0_24(StreamWriter tempLogSw, StreamWriter statsCsvSw, Program.<>c__DisplayCl
  ass0_0&) + 0xa7
     at Program.<Main>$(String[] args) + 0x337
     at JIT.Generics!<BaseAddress>+0x26a500

However, the test is not marked as failed in the summary of the test run.

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This, so far, has been observed only in NativeAOT and only locally, but I am not sure if this could happen in CoreCLR or in CI.
See discussion here: #90848 (comment)

A repro:

• undo the fix done in [NativeAOT] Reverted the change in IsInstanceOfException #90867 (it is a very small fix)
• run runtime tests with NativeAOT Checked (this may take a while)

build clr+libs -rc Checked -lc Release & src\tests\build.cmd nativeaot Checked &  src\tests\run.cmd runnativeaottests Checked

You will see a test failure in the output like:

Unhandled Exception: System.Diagnostics.DebugProvider+DebugAssertException: IsInstanceOfClass called with interface M
  ethodTable
     at System.Diagnostics.DebugProvider.Fail(String, String) + 0x36
     at System.Diagnostics.Debug.Fail(String, String) + 0x32
     at System.Runtime.TypeCast.IsInstanceOfClass(MethodTable*, Object) + 0x72
     at System.Runtime.EH.FindFirstPassHandler(Object, UInt32, StackFrameIterator&, UInt32&, Byte*&) + 0xfb
     at System.Runtime.EH.DispatchEx(StackFrameIterator&, EH.ExInfo&) + 0x131
     at System.Runtime.EH.RhThrowEx(Object, EH.ExInfo&) + 0x4b
     at GenericCatchInterfaceProgram.Main() + 0x2a
     at Program.<<Main>$>g__TestExecutor23|0_24(StreamWriter tempLogSw, StreamWriter statsCsvSw, Program.<>c__DisplayCl
  ass0_0&) + 0xa7
     at Program.<Main>$(String[] args) + 0x337
     at JIT.Generics!<BaseAddress>+0x26a500

However, the test is not marked as failed in the summary of the test run.

Author:	VSadov
Assignees:	-
Labels:	area-Infrastructure
Milestone:	-