ComInterfaceGenerator emits method attributed with [UnmanagedCallersOnly] and in out or ref parameters.

[jtschuster]

ComInterfaceGenerator generates code that produces error CS8977: Cannot use 'ref', 'in', or 'out' in the signature of a method attributed with 'UnmanagedCallersOnly'.

User Code:

[assembly:DisableRuntimeMarshalling]

[global::System.Runtime.InteropServices.Marshalling.GeneratedComInterfaceAttribute]
partial interface INativeAPI : IUnmanagedInterfaceType
{
    
    void Method(out S value);
}
// Try using the generated native interface
sealed class NativeAPI : IUnmanagedVirtualMethodTableProvider, INativeAPI.Native
{
    public VirtualMethodTableInfo GetVirtualMethodTableInfoForKey(System.Type type) => throw null;
}
partial interface INativeAPI
{
        static unsafe void* IUnmanagedInterfaceType.VirtualMethodTableManagedImplementation => null;
}
[NativeMarshalling(typeof(Marshaller))]
public struct S
{
#pragma warning disable CS0649 // Field is never assigned to, and will always have its default value
    public bool b;
#pragma warning restore CS0649
}

[CustomMarshaller(typeof(S), MarshalMode.ManagedToUnmanagedOut, typeof(Marshaller))]
[CustomMarshaller(typeof(S), MarshalMode.UnmanagedToManagedIn, typeof(Marshaller))]
public static class Marshaller
{
    public struct Native { }

    public static S ConvertToManaged(Native n) => default;
}

NativeToManagedStubs.g.cs

// <auto-generated/>
unsafe partial interface INativeAPI
{
    internal unsafe partial interface Native
    {
        // error CS8977: Cannot use 'ref', 'in', or 'out' in the signature of a method attributed with 'UnmanagedCallersOnly'.
        [System.Runtime.InteropServices.UnmanagedCallersOnlyAttribute]
        internal static void ABI_Method(void* __this_native, out global::S value)
        {
            global::INativeAPI @this = default;
            try
            {
                // Unmarshal - Convert native data to managed data.
                @this = (global::INativeAPI)System.Runtime.InteropServices.Marshalling.UnmanagedObjectUnwrapper.GetObjectForUnmanagedWrapper<System.Runtime.InteropServices.Marshalling.ComWrappersUnwrapper>(__this_native);
                @this.Method(out value);
            }
            catch (System.Exception __exception)
            {
                System.Runtime.InteropServices.Marshalling.SwallowExceptionMarshaller.ConvertToUnmanaged(__exception);
            }
        }
    }
}

[ghost]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

Issue Details

---

ComInterfaceGenerator generates code that produces error CS8977: Cannot use 'ref', 'in', or 'out' in the signature of a method attributed with 'UnmanagedCallersOnly'.

User Code:

[assembly:DisableRuntimeMarshalling]

[global::System.Runtime.InteropServices.Marshalling.GeneratedComInterfaceAttribute]
partial interface INativeAPI : IUnmanagedInterfaceType
{
    
    void Method(out S value);
}
// Try using the generated native interface
sealed class NativeAPI : IUnmanagedVirtualMethodTableProvider, INativeAPI.Native
{
    public VirtualMethodTableInfo GetVirtualMethodTableInfoForKey(System.Type type) => throw null;
}
partial interface INativeAPI
{
        static unsafe void* IUnmanagedInterfaceType.VirtualMethodTableManagedImplementation => null;
}
[NativeMarshalling(typeof(Marshaller))]
public struct S
{
#pragma warning disable CS0649 // Field is never assigned to, and will always have its default value
    public bool b;
#pragma warning restore CS0649
}

[CustomMarshaller(typeof(S), MarshalMode.ManagedToUnmanagedOut, typeof(Marshaller))]
[CustomMarshaller(typeof(S), MarshalMode.UnmanagedToManagedIn, typeof(Marshaller))]
public static class Marshaller
{
    public struct Native { }

    public static S ConvertToManaged(Native n) => default;
}

NativeToManagedStubs.g.cs

// <auto-generated/>
unsafe partial interface INativeAPI
{
    internal unsafe partial interface Native
    {
        // error CS8977: Cannot use 'ref', 'in', or 'out' in the signature of a method attributed with 'UnmanagedCallersOnly'.
        [System.Runtime.InteropServices.UnmanagedCallersOnlyAttribute]
        internal static void ABI_Method(void* __this_native, out global::S value)
        {
            global::INativeAPI @this = default;
            try
            {
                // Unmarshal - Convert native data to managed data.
                @this = (global::INativeAPI)System.Runtime.InteropServices.Marshalling.UnmanagedObjectUnwrapper.GetObjectForUnmanagedWrapper<System.Runtime.InteropServices.Marshalling.ComWrappersUnwrapper>(__this_native);
                @this.Method(out value);
            }
            catch (System.Exception __exception)
            {
                System.Runtime.InteropServices.Marshalling.SwallowExceptionMarshaller.ConvertToUnmanaged(__exception);
            }
        }
    }
}

Author:	jtschuster
Assignees:	-
Labels:	area-System.Runtime.InteropServices, untriaged
Milestone:	-

[jkoritzinsky]

Looks like we need a better fallback mechanism in this case when a marshaller doesn’t support the right marshal mode.

[jtschuster]

@jkoritzinsky So ref parameters aren't supported for methods in interfaces with [GeneratedComInterfaceAttribute]?

[jkoritzinsky]

The problem here is that the marshaller for S supports out parameters for "Managed to Unmanaged" scenarios, and in parameters for "Unmanaged to Managed" scenarios, whereas the case in the generated code is an out parameter in an "Unmanaged to Managed" scenario.

As a result, we emit an error from the generator and fall back to the forwarder logic (or at least we should). However, the forwarder logic generates code that isn't legal, so we also get a compiler error. The test is probably checking that we don't have any compiler errors before it checks if the generator emitted any errors. I don't know if we want to care about fixing the code to not generate compiler errors if we're already generating errors from the generator, but it's probably worth generating something that, while wrong, still compiles.

[jtschuster]

I see, in that case if we don't look for compiler errors when there are generator diagnostics, all the Compiles.cs tests pass.

[jkoritzinsky]

In the LibraryImportGenerator tests, we have a way for tests to specify how many compiler errors they expect. Might be something worth looking into