[NativeAOT] Specific PInvoke not supported

[ThomasFOG]

Hello there,

This is a follow up to #240, but not related to the same missing feature.

While implementing another third party SDK different from #240, we stumbled upon an unsupported type of PInvoke.

An example goes as follow:

[DllImport("somelibrary.dll", EntryPoint = "SomeEntryPoint")]
public static extern IntPtr SomeEntryPoint(
    [MarshalAs(UnmanagedType.CustomMarshaler, MarshalType = "SomeICustomMarshaler")]
    string arg1,
    HandleRef arg2
);

I'm assuming that it is related to custom marshalers.

Cheers!

[MichalStrehovsky]

This is #160 - I think we should be able to add support for this, but it's not very high up the priority list at this point.

You could work around by doing what the runtime would do manually - I think this should be equivalent:

public static IntPtr SomeEntryPoint(
    string arg1,
    HandleRef arg2
)
{
    ICustomMarshaler marshaler = SomeICustomMarshaler.GetInstance(null);
    IntPtr val = marshaler.MarshalManagedToNative(arg1);
    try
    {
        SomeEntryPoint(val, arg2);
    }
    finally
    {
        marshaler.CleanUpNativeData(val);
    }
}

[DllImport("somelibrary.dll", EntryPoint = "SomeEntryPoint")]
public static extern IntPtr SomeEntryPoint(
    IntPtr arg1,
    HandleRef arg2
);

[ThomasFOG]

Ah! I should have checked the opened issues. Since it's a duplicate, I'm closing this one.

Thanks for pointing out that it's about the attribute, I'll be notifying the vendor.