Possible JIT optimization bug on .NET 5 preview

[dellamonica]

Description

On a managed .NET 5 C# project, I get intermittent errors, sometimes an AccessViolationException, other times ExecutionEngineException, and even NullReferenceException (?) with no managed code in sight on the debugger.

The code does mathematical optimization and is fully deterministic, however the exceptions are not always consistent even though they are frequent.

On the following scenarios these exceptions have never appeared so far (I tried a few times):

• Debug mode.
• Release mode with Optimize=False,
• Release mode with Optimize=True and COMPlus_JITMinOpts=1.

Unfortunately, I don't know how to create a small program to reproduce the problem, but there seemed to be a particular point where the code breaks the most:

For context, _complement is declared as:
private readonly ImmutableArray<ulong> _complement
An extra detail: in the constructor I initialize a regular ulong[] and use Unsafe.As<ulong[], ImmutableArray<ulong>> to set _complement.

Here is a misterious NullReferenceException (on a value type?!)

JIT and compiler optimizations have a lot of impact on the performance in this project so I don't think disabling these optimizations is a long term solution.

Configuration

.NET 5.0.0-preview.6.20305.6

Also tried preview 4 before and was getting similar exceptions.

category:correctness
theme:testing
skill-level:expert
cost:large

[tannergooding]

Can you provide a more complete code example so we can attempt to reproduce the issue on our end?

[dellamonica]

It is a rather complex algorithm that sometimes runs for several minutes before hitting these exceptions (and some times it doesn't). If I knew how to create a reasonably small code example, I would.

Perhaps a Live Share session or with some help I could try to get more info with WinDbg.

[BruceForstall]

Could you try a current master branch build (see https://github.com/dotnet/installer)?

Another thing to try: run with COMPlus_GCStress=F. If there's a problem with GC information, this might make the failure more deterministically reproducible. (If it's not a GC related issue, it probably won't help. It will also make the app run very slow.)

[dellamonica]

Additional info: I've tried running on .net core 3.1 with basically a single change (due to MemoryExtensions.Sort not being present in 3.1) and the exceptions did not occur. I've tested for about an 1h on several instances, while on .net 5 preview I usually get these exceptions very frequently.

@BruceForstall, yes I can test the master branch build and get back here to let you know.

I can also try COMPlus_GCStress=F.

[AndyAyersMS]

A crash dump might be useful.

[dellamonica]

@BruceForstall: I've got an ExecutionEngineException on 5.0.0-preview.8.20358.9.
I've set COMPlus_GCStress=F and it took forever to get even Visual Studio/PSCore open, if it is that much slower, then we might need hours and for that I'd have to use another PC/VM, so I'll postpone that test.

@AndyAyersMS: from VS Debug > Save Dump as... I exported a crash dump and it's 201Mb and I'm not sure if source code is present in this file, if there is a way to send the file to you privately, or if there is a way to generate a smaller dump without (C#) code present, please let me know.

[AndyAyersMS]

The dump will contain IL but not sources.

You can share it securely by opening an issue on the VS Developer Portal and then attaching it there, or you can create your own share and email me the access info (andya@microsoft.com).

[dellamonica]

The dump will contain IL but not sources.

You can share it securely by opening an issue on the VS Developer Portal and then attaching it there, or you can create your own share and email me the access info (andya@microsoft.com).

Sent to your e-mail!

[AndyAyersMS]

Thanks. I should be able to look at it later today.

[AndyAyersMS]

Working with @dellamonica offline -- initial impression is bad GC reporting by the jit. Will add this to 5.0.