[Regression] Invalid program when marhsalling delegate with both value type return and parameter

[ceztko]

Description

I'm receiving an System.InvalidProgramException with message "Common Language Runtime detected an invalid program" when trying to marshal an UnmanagedFunctionPointer delegate that has both return and parameters that are value types. This happens only on x86 build with .NET core runtime. It works with x64 build or using .NET Framework runtime.

The minimal program to reproduce is with the following on the C# side:

using System;
using System.Runtime.InteropServices;

namespace TestDelegateValueTypeReturnInterop
{
    class Program
    {
        [UnmanagedFunctionPointer(CallingConvention.Cdecl)]
        delegate ValueType ValueTypeReturnDelegate(ValueType p1);

        static ValueTypeReturnDelegate Callback;

        static Program()
        {
            Callback = callback;
        }

        static void Main(string[] args)
        {
            Foo(Callback);
        }

        static ValueType callback(ValueType p1)
        {
            return new ValueType();
        }

        [DllImport("DelegateValueTypeReturnNative", CallingConvention = CallingConvention.Cdecl)]
        static extern void Foo(ValueTypeReturnDelegate func);
    }

    [StructLayout(LayoutKind.Sequential)]
    struct ValueType
    {
        IntPtr F1;
        UIntPtr F2;
    }
}

and this on C++ side:

#include <cstdint>

using namespace std;

extern "C"
{
    struct ValueType
    {
        intptr_t F1;
        uintptr_t F2;
    };

    typedef ValueType(*ValueTypeReturnDelegate)(ValueType p1);

    __declspec(dllexport) void Foo(ValueTypeReturnDelegate func)
    {
        auto v = func({ });
    }
}

The program is also attached with a full standalone solution. Running it with x86 platform ( (Debug or Release is the same) fails with "TestDelegateValueTypeReturnInteropCore" and and works "TestDelegateValueTypeReturnInteropNet452" projects. On x64 platform the program works on both runtimes.

TestDelegateValueTypeReturnInterop.zip

Configuration

• .NET Core 3.1.404
• x86
• Windows 10 2004 29041.630

Regression?

The problem is a regression, the same program works on .NET Framework both on x86 and x64. Testing with 4.8 (release 528372).

[jkotas]

cc @jkoritzinsky

[AaronRobinsonMSFT]

This doesn't crash on .NET 5, but does exhibit some unusual behavior. Once we fully root cause this we can make a servicing decision.

[AaronRobinsonMSFT]

Actually this is fine in .NET 5 - GA. In our main branch there is an issue due to some refactoring, but will be fixed by @jkoritzinsky's PR #46238.

I have been able to reproduce this locally on .NET Core 3.1 and I think (thanks @jkoritzinsky for helping root cause) this may be fixed by #35026. I will verify this locally.

[AaronRobinsonMSFT]

Thanks @jkoritzinsky for the stellar memory! This was found and fixed in .NET 5, but never back ported.

[AaronRobinsonMSFT]

@ceztko Thank you for reporting this. Given this is for 3.1 we need a strong customer case. Are you completely blocked by this issue? Is it possible to upgrade to and run on .NET 5.0?

[ceztko]

Thank you very much for taking care of this issue! Fortunately for me, I have no imminent use case. I stumbled in this issue in a big testing advanced interop strategies for strings and thought it was important to report. Feel free to stand by with the back port until an user with relevant use case replies here (if ever).

[AaronRobinsonMSFT]

@ceztko Thanks for the reply. Since this isn't blocking anyone at the moment I will close this issue. The issue will be searchable and since we now know the fix will be easy to apply if it becomes a blocking high priority concern.