[API Proposal]: Marshaller type to support SafeHandle in source-generated marshalling without hard-coding in the generator

[jkoritzinsky]

Background and motivation

Today, we support marshalling System.Runtime.InteropServices.SafeHandle-derived types intrinsically in the interop source generators. This is one of the only types that we do this for since we moved string marshalling out of the generator during the .NET 7 development cycle. Additionally, SafeHandle marshalling is extremely complex, much more than any other built-in type.

As a result, we'd like to introduce a marshaller type so we can move the interop source generators to use this type instead of emitting the code directly.

This change will make the interop code more serviceable, since we can't service source-generated code.

API Proposal

namespace System.Runtime.InteropServices.Marshalling;

[CustomMarshaller(typeof(CustomMarshallerAttribute.GenericPlaceholder), MarshalMode.ManagedToUnmanagedIn, typeof(ManagedToUnmanagedIn))]
[CustomMarshaller(typeof(CustomMarshallerAttribute.GenericPlaceholder), MarshalMode.ManagedToUnmanagedRef, typeof(ManagedToUnmanagedRef))]
[CustomMarshaller(typeof(CustomMarshallerAttribute.GenericPlaceholder), MarshalMode.ManagedToUnmanagedOut, typeof(ManagedToUnmanagedOut))]
public static class SafeHandleMarshaller<T>
    where T : System.Runtime.InteropServices.SafeHandle
{
    public struct ManagedToUnmanagedIn
    {
        public void FromManaged(T handle);

        public IntPtr ToUnmanaged();

        public void Free();
    }

    public struct ManagedToUnmanagedRef
    {
        public ManagedToUnmanagedRef();

        public void FromManaged(T handle);

        public IntPtr ToUnmanaged();

        public void FromUnmanaged(IntPtr value);

        public T ToManagedGuaranteed();

        public void OnInvoked();

        public void Free();
    }

    public struct ManagedToUnmanagedOut
    {
        public ManagedToUnmanagedOut();

        public void FromUnmanaged(IntPtr value);

        public T ToManaged();

        public void OnInvoked();

        public void Free();
    }
}

API Usage

// Marshaller will be implicitly used
[LibraryImport("foo")]
public void Method(Microsoft.Win32.SafeFileHandle handle);
[LibraryImport("foo")]
public void Method2([MarshalUsing(typeof(SafeHandleMarshaller<Microsoft.Win32.SafeFileHandle>))] Microsoft.Win32.SafeFileHandle handle);

Alternative Designs

We considered adding a new() constraint to the generic parameter to enforce our requirements around constructing new instances of the SafeHandle-derived type for the ref and out marshallers; however, this constraint would cause breaking changes for APIs that take non-concrete SafeHandle types as by-value or by-ref in parameters.

Risks

No response

[ghost]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

Today, we support marshalling System.Runtime.InteropServices.SafeHandle-derived types intrinsically in the interop source generators. This is one of the only types that we do this for since we moved string marshalling out of the generator during the .NET 7 development cycle. Additionally, SafeHandle marshalling is extremely complex, much more than any other built-in type.

As a result, we'd like to introduce a marshaller type so we can move the interop source generators to use this type instead of emitting the code directly.

This change will make the interop code more serviceable, since we can't service source-generated code.

API Proposal

namespace System.Runtime.InteropServices.Marshalling;

[CustomMarshaller(typeof(CustomMarshallerAttribute.GenericPlaceholder), MarshalMode.ManagedToUnmanagedIn, typeof(ManagedToUnmanagedIn))]
[CustomMarshaller(typeof(CustomMarshallerAttribute.GenericPlaceholder), MarshalMode.ManagedToUnmanagedRef, typeof(ManagedToUnmanagedRef))]
[CustomMarshaller(typeof(CustomMarshallerAttribute.GenericPlaceholder), MarshalMode.ManagedToUnmanagedOut, typeof(ManagedToUnmanagedOut))]
public static class SafeHandleMarshaller<T>
    where T : System.Runtime.InteropServices.SafeHandle
{
    public struct ManagedToUnmanagedIn
    {
        public void FromManaged(T handle);

        public IntPtr ToUnmanaged();

        public void Free();
    }

    public struct ManagedToUnmanagedRef
    {
        public ManagedToUnmanagedRef();

        public void FromManaged(T handle);

        public IntPtr ToUnmanaged();

        public void FromUnmanaged(IntPtr value);

        public T ToManagedGuaranteed();

        public void OnInvoked();

        public void Free();
    }

    public struct ManagedToUnmanagedOut
    {
        public ManagedToUnmanagedOut();

        public void FromUnmanaged(IntPtr value);

        public T ToManaged();

        public void OnInvoked();

        public void Free();
    }
}

API Usage

// Marshaller will be implicitly used
[LibraryImport("foo")]
public void Method(Microsoft.Win32.SafeFileHandle handle);
[LibraryImport("foo")]
public void Method2([MarshalUsing(typeof(SafeHandleMarshaller<Microsoft.Win32.SafeFileHandle>))] Microsoft.Win32.SafeFileHandle handle);

Alternative Designs

We considered adding a new() constraint to the generic parameter to enforce our requirements around constructing new instances of the SafeHandle-derived type for the ref and out marshallers; however, this constraint would cause breaking changes for APIs that take non-concrete SafeHandle types as by-value or by-ref in parameters.

Risks

No response

Author:	jkoritzinsky
Assignees:	-
Labels:	api-suggestion, area-System.Runtime.InteropServices
Milestone:	8.0.0

[bartonjs]

Video

• Looks good as proposed.

namespace System.Runtime.InteropServices.Marshalling;

[CustomMarshaller(typeof(CustomMarshallerAttribute.GenericPlaceholder), MarshalMode.ManagedToUnmanagedIn, typeof(ManagedToUnmanagedIn))]
[CustomMarshaller(typeof(CustomMarshallerAttribute.GenericPlaceholder), MarshalMode.ManagedToUnmanagedRef, typeof(ManagedToUnmanagedRef))]
[CustomMarshaller(typeof(CustomMarshallerAttribute.GenericPlaceholder), MarshalMode.ManagedToUnmanagedOut, typeof(ManagedToUnmanagedOut))]
public static class SafeHandleMarshaller<T>
    where T : System.Runtime.InteropServices.SafeHandle
{
    public struct ManagedToUnmanagedIn
    {
        public void FromManaged(T handle);

        public IntPtr ToUnmanaged();

        public void Free();
    }

    public struct ManagedToUnmanagedRef
    {
        public ManagedToUnmanagedRef();

        public void FromManaged(T handle);

        public IntPtr ToUnmanaged();

        public void FromUnmanaged(IntPtr value);

        public T ToManagedGuaranteed();

        public void OnInvoked();

        public void Free();
    }

    public struct ManagedToUnmanagedOut
    {
        public ManagedToUnmanagedOut();

        public void FromUnmanaged(IntPtr value);

        public T ToManaged();

        public void OnInvoked();

        public void Free();
    }
}