How can I declare an interop COM interface as "agile"?

[smourier]

Is there a way to tell the CLR/.NET that a given COM interface doesn't need any marshaling? I may be wrong but I think this is sometimes referred as "agile" in more recent .NET/COM litterature.

For example this simple .NET 6 console app here:

using System;
using System.Runtime.InteropServices;
using System.Threading.Tasks;

namespace ConsoleApp1
{
    internal class Program
    {
        static void Main()
        {
            D3D11CreateDevice(IntPtr.Zero, D3D_DRIVER_TYPE_HARDWARE, IntPtr.Zero, 0, IntPtr.Zero, 0, D3D11_SDK_VERSION, out var ptr, out _, out _);

            var device = (ID3D11Device)Marshal.GetObjectForIUnknown(ptr);
            Task.Run(() =>
            {
                var deviceInThread = (ID3D11Device)Marshal.GetObjectForIUnknown(ptr);
                Console.WriteLine("ok");
            });
            Console.ReadLine();
        }

        const int D3D_DRIVER_TYPE_HARDWARE = 1;
        const int D3D11_SDK_VERSION = 7;

        [DllImport("d3d11")]
        public static extern int D3D11CreateDevice(IntPtr pAdapter, int DriverType, IntPtr Software, int Flags, IntPtr pFeatureLevels, int FeatureLevels, int SDKVersion, out IntPtr ppDevice, out int pFeatureLevel, out IntPtr ppImmediateContext);

        [ComImport, Guid("db6f6ddb-ac77-4e88-8253-819df9bbf140"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
        public interface ID3D11Device
        {
            // undef yet, not needed here
        }
    }
}

Works fine. But if I run Main in an STA, like this:

        [STAThread]
        static void Main()
        {
            // same code as above
	}

It will hang in the second Marshal.GetObjectForIUnknown call.
I know it hangs because I don't have a message pump, but that's not my point here. As I understand, today, .NET will consider ID3D11Device as being MTA-bound because this interface registers nothing in the registry (which is how it's behave it's not a "full" COM interface), so will try to marshal its reference from STA to MTA, and will fall into the message pump trap.

I would just be able to tell .NET "manually" that this interface doesn't need marshaling and the pointer can be passed as-is, something like:

	[Agile] // don't marshal this
        [ComImport, Guid("db6f6ddb-ac77-4e88-8253-819df9bbf140"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
        public interface ID3D11Device
        {
        }

I've remarked the Win32Metadata project does mark some interfaces with a similar attribute (but only of its own), and I guess this is the same idea. Could there be an equivalent for the .NET runtime that changes its behavior? Or maybe there's already something I've missed?

Thanks

[rickbrew]

AFAIK this isn't possible with .NET's built-in COM Interop. I wrestled with this enormously back when I was working on Windows Performance Analyzer, about 10 years ago. .NET's COM Interop seems married to the idea that COM objects belong in apartments. We resorted to creating our COM objects on the thread pool so that .NET wouldn't force method calls to marshal to the UI thread (sounds like you have the opposite situation).

My recommendation is to use @tannergooding 's excellent TerraFX.Interop.Windows package for working with things like Direct3D. It's basically #include <windows.h>. If you need the GC to handle cleanup for your COM objects then you'll want to wrap the objects in a ComRef<T> : SafeHandle class, but that's pretty easy to accomplish.

(I'm using TerraFX.Interop.Windows for all of my interop in Paint.NET nowadays -- I can't recommend it enough!)

[smourier]

Hi @rickbrew.

Thanks for this answer. I've written a similar thing https://github.com/smourier/DirectN. I believe TerraFX didn't exist back then. I didn't want unsafe code, and I also wanted deriving interfaces (ID3D11Device1 derives from ID3D11Device) to ease programming and be able to code in a way very similar to C/C++ with very natural C#, and now I have existing code base. I was also looking for .NET Framework and .NET core support.

One other solution is to maximize the use opaque IntPtr everywhere, but it also makes C# code more complex.

I actually can understand why .NET behaves like it does (consider an object w/o any registration info to marked as Apartment) because that's how COM behaves somehow. But COM has evolved since https://docs.microsoft.com/en-us/windows/win32/api/objidl/nn-objidl-iagileobject on its own.

It seems it should be quite a simple feature to add to .NET and the existing COM infrastructure do support all these.

[rickbrew]

TerraFX does support inheritance, although it's a little clumsy.

First, the COM interface structs are defined with a managed interface, and the managed interfaces have inheritance:

public struct IUnknown : IUnknown.Interface
{
    public interface Interface
    {
        HRESULT QueryInterface(...);
        uint AddRef();
        uint Release();
    }
    ...
}

public struct ID3D11Device : ID3D11Device.Interface
{
    public interface Interface : IUnknown.Interface { ... }
    ...
}

public struct ID3D11Device1 : ID3D11Device1.Interface
{
    public interface Interface : ID3D11Device.Interface { ... }
    ...
}

You can then write generic methods that have a where T : unmanaged, IUnknown.Interface to accept any COM pointer, or to restrict based on another interface.

In addition, I've published a package, PointerToolkit.TerraFX.Interop.Windows to make pointer casting easier. Otherwise C# doesn't know that you can implicitly cast e.g. ID3D11Device1* to ID3D11Device*, requiring you to put in an explicit cast. This is not only clumsy but error prone, with terrible consequences if you mismatch things. With this package you use __cast(pDevice) (__cast is a method returning a special CastPtr<...> type) and then implicit casting to all the "base" pointer types is enabled. The JIT should be smart enough to inline this and wash away the overhead.

I'm not sure you'll see any additions to .NET's original COM Interop system, it's pretty clear that the ecosystem is moving and has moved away from it. I can't speak with authority on this matter though.

[AaronRobinsonMSFT]

This isn't really safe nor something we are likely to try and support. There are two sides here. The first is all .NET implemented COM servers are Free Threaded so can be considered "agile". All native COM servers must indicate how they are implemented as the implementation (i.e., coclass) not the interface dictates apartment requirements. COM servers can do this in one of two ways. Post-Windows 8, they can implement the IAgileObject interface - very simple. Alternatively, the approach is to aggregate with the Free Threaded Marshaller and the runtime will respect that. If aggregation isn't possible the coclass can instead return the Free Threaded Marshaller when the runtime QIs for IMarshal.

[smourier]

Hi @AaronRobinsonMSFT

Thanks for your answer.

I have no problem with .NET COM servers here, and I myself know how to write COM object that can behave.

The question is more about (thousands of) existing COM interfaces, like DirectX and friends ones (that are not coclasses strictly speaking). I don't think asking for DirectX team to add IAgileObject to DirectX objects implementation is more realistic then asking DotNet team to support this :-)

I'm not sure how being able to declaratively indicate that a given interop-oriented interface is agile is this less safe than forcing people to use IntPtr or unsafe code all around, though.

[AaronRobinsonMSFT]

interop-oriented interface is agile

This is the confusion from my side. It can't be on the interface; it would have to be on the implementation.

We would need some mechanism to indicate the following call should create an RCW that doesn't capture the thread context and just assume the COM instance is agile.

Marshal.GetObjectForIUnknown(ptr);

Honestly, this seems like something we need to fold into our COM source generator, potentially for the .NET 8 timeframe. See #66674. One of the design goals there is to provide mechanism for people to indicate this on the RCWs that are generated.

I'm inclined to close this as we just aren't in a position to support this for the built-in support - that is "done" as far as the Interop team is concern. Innovation and new feature requests are relegated to our source generator solutions.

/cc @jkoritzinsky @elinor-fung

[smourier]

We would need some mechanism to indicate the following call should create an RCW that doesn't capture the thread context and just assume the COM instance is agile.

Is exactly my point. 👍

[AaronRobinsonMSFT]

Then this is where we make the hard decision and state the built-in system has no affordance to indicate this since RCWs are an implementation detail of the runtime.

The only avenue to having this expressiveness is through the upcoming source generator solution where RCWs will be configurable and by default behave in a manner that is reliant on the IUnknown ABI as opposed to fully supporting the COM threading model out of the box. Apartment aware support is a goal of the second phase.

The current recommendation then is to leverage something like the TerraFX library, which is very good and performant, or use the IntPtr and C# function pointers directly. The COM source generator will be adding the desired support in the Checkpoint 1 timeframe.

I'm not sure you'll see any additions to .NET's original COM Interop system, it's pretty clear that the ecosystem is moving and has moved away from it. I can't speak with authority on this matter though.

Yep. Regressions, high impact bugs and security fixes are the only additions to the built-in interop system, this includes P/Invokes too.