Support for TypeLibs needed for assembly consumption as VBA references

[robert-io]

Steps to reproduce

Create a .Net Core Class Library and set the Target Framework to .NET Core 3.0 and add the following code (setting you own GUIDs) in the PersonRegistration class:

using System;
using System.Runtime.InteropServices;

namespace CoreComTest {
    internal class PersonRegistration {
        public const string IID_IPerson = "########-####-####-####-############";
        public const string IID_IBlagger = "########-####-####-####-############";
        public const string IIE_IPerson = "########-####-####-####-############";
        public const string CLSID_Person = "########-####-####-####-############";
        public const string PROGID_Person = "CoreComTest.Person";
    }

    [ComVisible(true)]
    [Guid(PersonRegistration.IID_IPerson)]
    [InterfaceType(ComInterfaceType.InterfaceIsDual)]
    public interface IPerson {
        [DispId(100)]
        string Name { get; set; }

        [DispId(200)]
        DateTime BornOn { get; set; }

        [DispId(300)]
        int GetAge();

        [DispId(400)]
        IPerson GetMate();
    }

    [ComVisible(true)]
    [Guid(PersonRegistration.IIE_IPerson)]
    [InterfaceType(ComInterfaceType.InterfaceIsDual)]
    public interface IPersonEvents {
        [DispId(100)]
        void NameChanged(string original_name, string new_name);
    }

    public delegate void NameChangedDelegate(string original_name, string new_name);

    [ComVisible(true)]
    [Guid(PersonRegistration.IID_IBlagger)]
    [InterfaceType(ComInterfaceType.InterfaceIsDual)]
    public interface IBlagger {
        [DispId(100)]
        string Blag();
    }

    [ComVisible(true)]
    [ClassInterface(ClassInterfaceType.None)]
    [ComSourceInterfaces(typeof(IPersonEvents))]
    [Guid(PersonRegistration.CLSID_Person)]
    [ProgId(PersonRegistration.PROGID_Person)]
    public class Person : IPerson, IBlagger {
        private string _name = string.Empty;

        public string Name {
            get => _name;
            set {
                if (_name != value) {
                    var old_value = _name;
                    _name = value;
                    OnNameChanged(old_value, _name);
                }
            }
        }
        public DateTime BornOn { get; set; } = DateTime.MinValue;

        public int GetAge() {
            return (int)((DateTime.Now - BornOn).TotalDays / 365);
        }

        public void OnNameChanged(string original_name, string new_name) {
            var callback = NameChanged;
            try {
                callback?.Invoke(original_name, new_name);
            }
            catch {
                /* Todo: Log */
            }
        }

        public string Blag() {
            return $"{Name} is the best";
        }

        public IPerson GetMate() {
            var rnd = new Random(42);
            var years = rnd.Next(20, 50);
            var days = rnd.Next(1, 356);

            return new Person() {
                Name = "John Smith",
                BornOn = DateTime.UtcNow.AddYears(-years).AddDays(-days)
            };
        }

        public event NameChangedDelegate NameChanged;
    }
}

Desired behavior

Using VBA or VBScript create and use the object. For example the following VBS should create the object and show the persons age in a message box:

Dim sam 'As CoreComTest.Person
Set sam = CreateObject("CoreComTest.Person")

rob.Name = "Sam Carter"
rob.BornOn = dotnet/core-setup#29 December 1968#

Call MsgBox(rob.GetAge())

Actual behavior

A Windows Script Host error box with the following error is displayed:
~ ActiveX component can't create object

Environment data

Windows 10
VS 2019 with the .NET Core 3.0 Preview 8 SDK

Additional

I can see from the warnings in my code that ComInterfaceType.InterfaceIsDual & ComSourceInterfaces are marked obsolete.

~ Will Microsoft be adding support for ComInterfaceType.InterfaceIsDual & ComSourceInterfaces?
~ Or is there another way to implement a scriptable (IDispatch) COM object?

[vitek-karas]

/cc @AaronRobinsonMSFT

[AaronRobinsonMSFT]

@robert-io I will try to reproduce this issue locally, but I think the issue here is the missing ProgID support we recently added - dotnet/core-setup#7573. I will verify the above scenario works though.

[robert-io]

@AaronRobinsonMSFT hi, have you been able to reproduce this and is there any update regarding addressing the issue?

[AaronRobinsonMSFT]

@robert-io Sorry for the late reply. I verified the ProgIDs are being respected and I am able to activate them as desired with the official .NET Core 3.0 release. Have you been able to try the latest as well?

[robert-io]

@AaronRobinsonMSFT i have tested it with 3.1 Preview 2 and it works in VB Script but i can't add a reference to the DLL in VBA.

I used Excel (32bit) VBA with and compiled the dotnet Core Assembly with the platform target set to x86.

When i use the Tools -> References dialog to add a reference its not listed and when i browse to the DLL i get the following error:
~ "Can't add a reference to the specified file."

[vitek-karas]

@AaronRobinsonMSFT could you please take a second look?
Trying to figure out if we need to do something here for 3.1+ releases.

[AaronRobinsonMSFT]

@vitek-karas I am doubtful we can do anything in the 3.1 time frame for this issue. The underlying issue here is that VBA wants a TypeLib generated for the .NET types and since CoreCLR no longer generates TypeLibs, the VBA editor fails to add the reference. This is unfortunate, but unless we bring back TypeLib generation I don't see how we can make work automatically.

@robert-io In this case, the only solution here will be to create a TypeLib for the types and register that. I will need to think more about this because I don't fully know the best way to address the scenario. The issue as stated above is that the COM host DLL is being loaded via LoadTypeLibEx and since there isn't an embedded TypeLib nor is one available to be generated the reference fails.

[robert-io]

@AaronRobinsonMSFT i have tested your recommendation of creating a TypeLib and registering it and it works.

To generate the tlb file (not ideal but functional)

• i added net47 to the TargetFrameworks and build a .Net Assembly
• i used tlbexp to export the tlb file
• manually created the reg file to register it

I'm going to be thinking about how i can make this process simpler for me. i would still be interested in the CoreCLR generating the TypeLibs but understand that there is not enough time to get that into the 3.1 release.

[vitek-karas]

Thanks @AaronRobinsonMSFT - moved it to 5 for now.

[AaronRobinsonMSFT]

@robert-io Thank you for the confirmation.

I'm going to be thinking about how i can make this process simpler for me. i would still be interested in the CoreCLR generating the TypeLibs but understand that there is not enough time to get that into the 3.1 release.

Absolutely agree. I will admit ignorance and didn't realize the TypeLib generation was a VBA requirement, this is unfortunate. Adding transparent TypeLib support in .NET Core that is similar to .NET Framework is probably going to be a hard sell for various reasons. Some options exists though and I encourage you and the rest of the community to help us define what would address the needs in these scenarios.

A possible solution here would be to "port" the TlbExp tool to .NET Core and integrate that into the build system? It wouldn't really be a port though, but rather a complete rewrite since TlbExp relies on being deeply integrated with .the NET Framework CLR.

Please keep filing suggestions and/or proposals for addressing this as it is how we prioritize work and demonstrate community need.

/cc @jkotas @jeffschwMSFT @richlander

[agocke]

Unfortunately I don't think we're going to get to this for 5.0 either. Moving to 6.0.

[bclothier]

Would it be easier if you were given a IDL file and generate .NET types derived from the IDL which can be then implemented as interfaces? I'd rather be able to directly consume an IDL file which also would help guarantee ABI compatibility without getting lost in the marshalling quagmire.

[AaronRobinsonMSFT]

@bclothier Going the IDL -> TLB -> .NET interfaces is already supported, albeit with the .NET Framework tool - TlbImp. The .NET assembly generated by TlbImp will/should work with .NET Core without issue. If you discover a scenario that doesn't work please let us know. At present we are relying on the .NET Framework version of TlbImp since porting it over isn't a simple task and at present appears to generate a compatible assembly.

[BickelLukas]

@AaronRobinsonMSFT Is there any way to circumvent the missing typelib generation in .Net 5 right now? We are trying to migrate a huge legacy VB application to dotnet and will have the need for COM Server interop for at least a few years. As the API surface on the Dotnet side grows, strong typing is going to be absolutely crucial.

Right now we are solving it with late binding but the lack of intellisense and type checking makes this not really a viable option for the future.

[AaronRobinsonMSFT]

Is there any way to circumvent the missing typelib generation in .Net 5 right now?

@BickelLukas The solution for the missing TypeLib generation is to define all interfaces in IDL and then produce a TLB using the MIDL compiler. This TLB can then be registered on the machine instead of being provided in an ad-hoc manner which was the previous manner for .NET Framework. The workaround was described above in #3740 (comment).

However, it seems that I could be mistaken as to what you are asking. It sounds like you are attempting to convert a VB application to .NET - yay by the way. So is the issue not being able to project the VB types in .NET? This could be achieved using TlbImp from a .NET Framework install. The workaround above is for going the other way - consuming a .NET type in VB.

[BickelLukas]

@AaronRobinsonMSFT yes you did understand correctly. The question was about consumung .Net types in VB. This is needed since the core of the app is (and probably will stay for a while) in VB but new functionality is developed in .Net. If you are interested in hearing more about the application and our migration approach you can email me at lukas.bickel@consolidate.at

I did stumble upon the manual IDL approach before. However I was struggling with writing the IDL files by hand. If you could point me at some resources where a plain old C# dev can aquire the needed skills it would be greatly appreciated. Maybe there is even an automatic conversation tool?

[AaronRobinsonMSFT]

I did stumble upon the manual IDL approach before. However I was struggling with writing the IDL files by hand. If you could point me at some resources where a plain old C# dev can aquire the needed skills it would be greatly appreciated. Maybe there is even an automatic conversation tool?

@BickelLukas Yep, IDL is a rather esoteric language and exposes a lot of RPC concepts that clutter what most .NET developers want from it. In order to get at the IDL I would go about this as follows.

1. In a .NET Framework project, define all interfaces that are expected to be exposed to VB.
2. Compile the project into a .dll.
3. Take the .dll from the previous step and use the TlbExp.exe tool to create a TLB. TlbExp.exe is available in the .NET Framework SDK - should be on the path in a Developer Command Prompt.
4. Take the resulting TLB and pass that to oleview.exe, also available from the Developer Command Prompt.
5. From within oleview.exe if you examine the TLB, the equivalent IDL code will be displayed.

That will give you a good starting point. I would compile it locally using the MIDL compiler and then iterate as you define the contract you would like to expose from .NET 5. The MIDL language reference is helpful here.

[bclothier]

Just to follow up on the request. One problem with the manual IDL generation workaround is that there is no good way to import the definition, or at least I couldn't work it out.

1. A .NET core project does not seem to be able to reference a type library directly.
2. We can use tlbimp tool but that's going back to the .NET Framework, rather than bootstrapping from the .NET Core
3. Making a C++ project and then referencing it seems to work but then the definition is wrong or incomplete. Furthermore, it doesn't seem possible to implement a CoClass that's defined externally because the C++ project will already have defined a FooClass and decorate the interface IFoo with CoClass(typeof(FooClass)) when in fact I want my Foo class in the .NET Core assembly to be the coclass implementation.

The point here is that this requires 2 parallel processes -- we continue to decorate our classes with various interop attributes but we also have to maintain the IDL file externally. Both must be kept in the sync. That's going to be a lot of work and inherently error-prone.

I also am not sure I understand the difficulty of enabling type lib generation. If we can use an abstract type lib created out of thin air from a MIDL compiler and have it work well with .NET core objects, what's stopping the .NET Core from automatically generating a type library?

[bclothier]

For those interested, there's now a project to implement type library generation using pure .NET Core 3.1 to not have to depend on non-dotnet build tools.

[AaronRobinsonMSFT]

I also am not sure I understand the difficulty of enabling type lib generation. If we can use an abstract type lib created out of thin air from a MIDL compiler and have it work well with .NET core objects, what's stopping the .NET Core from automatically generating a type library?

There are two parts to this problem unfortunately. The first part is that the original TlbExp implementation was written in C++ and baked into the runtime itself. It leveraged many internal data structures and relied upon algorithms (e.g. stable GUID generation) that are are not exposed at the managed layer. Also, when .NET Core was released all Windows specific code (e.g. COM related logic) was completely removed. By the time .NET Core 3.0 came along a fair bit of internal code have drifted which would make just bringing it back difficult not to mention the limited desire to resurrect a rather large and poorly understood mess of code. That is basically why it is no longer in the product.

Regarding "not sure I understand the difficulty of enabling type lib generation" and "what's stopping the .NET Core from automatically generating a type library?". I see the project mentioned in #3740 (comment) and we wanted to do this but it will become difficult rather quickly - at least if one wants to make the results close or identical to the existing TlbExp tool. To that end, there is nothing stopping us other than it is a time consuming task that is difficult to fund at present. Hitting the level of support the community desires is beyond what we can fund relative to other interop needs.

All that aside, your and the community efforts here are much appreciated and something we would be more than willing to help encourage. Please keeps us posted on the project's progress and if help is needed here please let us know.

/cc @jeffschwMSFT @jkoritzinsky @elinor-fung

[bclothier]

Thank you for the background. I did notice that the existing implementation of the TypeLibConverter class calls into a magic function not exposed via the reference sources.

It's good to know that it may be challenging to get identical results as the TlbExp tool - at the moment I'm far more concerned by the warnings I get from the referenced project that various things are obsolete and may not be supported (a key example being InterfaceType.InterfaceIsDual/InterfaceIsIDispatch). If those do go the way of dodo, then the project will be for nothing. As long those will continue to be supported in .NET 5 & beyond, we can at least accommodate differences between the behaviors from TlbExp library and a pure .NET Core implementation.

[AaronRobinsonMSFT]

If those do go the way of dodo, then the project will be for nothing. As long those will continue to be supported in .NET 5 & beyond

They are marked as such merely because of the original .NET Core release where these COM types were really obsolete. Filing an issue about these types can help us to identify the ones that really are obsolete and those that will remain as long as .NET Core/.NET 5+ supports the Windows platform - so forever. I think I provided feedback in #13264 about filing issues in dotnet/corefx, those issues should now be filed in dotnet/runtime.

[BickelLukas]

@AaronRobinsonMSFT
We did get the typelib generation working by writing our own idl file.
Now we need to register the tlb file. As I found out this was previously done using regtlib.exe, however this is not part of windows 10 anymore. Also I think a seperate tlb file would cause problems with registry free com (which we're ultimately trying to get to).

Is there any option to directly embed the tlb as a win32 resource inside the comhost dll?

[AaronRobinsonMSFT]

Is there any option to directly embed the tlb as a win32 resource inside the comhost dll?

@BickelLukas The answer is no, but that is a good suggestion. I think the SxS manifest can reference non-embedded TLBs right? That might be an option for now. The idea of enabling this though is appealing and would help to further champion our RegFree approach. Can you file an issue with this suggestion? The relevant code links are below.

SDK that requests ComHost creation: CreateComHost.cs
Runtime code that composes a tailored ComHost instance: ComHost.cs

[elinor-fung]

The current path remains creating an IDL and generating a TLB. Embedding a type library is supported as of .NET 6 and the out-of-process COM server sample shows how it can be used.

We don't have plans to bring tlbexp to .NET, but are certainly interested in community efforts like #3740 (comment) or https://github.com/dspace-group/dscom. I am going to close this, as it sounds like folks are unblocked.

For C#, we will be investing in source generation for COM. One of the future goals of that is to enable a tlbexp replacement.