[Question] How to get DLLExport work correctly, even when debugging

[batzen]

Hi there,

i am not sure if i am doing something wrong or if it's an issue with dnlib.
I don't know much about IL code and this is my first try to do IL code manipulation through dnlib.
I also tried to understand the code in PatchSdataBytesBlob from ManagedExportsWriter but i don't even understand whats done there :-(.

Using the way to write dllexports from the readme (+ some more code from me) causes the following sdata to be written:

.data D_00004000 = bytearray (
                 04 00 00 06 00 00 00 00) 

This causes the MDA (managed debugging assistant) to trap access violations while trying to call the exported method(s).
Changing the sdata to (for an x64 image)

.data D_00004000 = int64(0)

or to (for an x86 image)

.data D_00004000 = int32(0)

fixes the AC issues.

Am i thinking this wrong, or does dnlib something wrong here?

A project showcasing/reproducing what i am doing can be found here: https://github.com/batzen/NativeToManagedTest
To repro just clone the project, build and run the WPFTestApp.
If you run it from within visual studio and click the button "Call managed native" while having the MDA activated (from the exception settings) you should see a "FatalExecutionEngineError" being thrown.

If you need any more details from me feel free to ask. I am also on gitter (https://gitter.im/batzen), if that helps.

[0xd4d]

Related issue #172

This could be a VS debugger issue. I couldn't repro it with dnSpy. I couldn't repro it if I started the debug build from explorer.exe.

Workaround for VS debugger: debug Release builds. Or try attaching to a debug build (didn't work for me, VS crashed).

[0xd4d]

Using the way to write dllexports from the readme (+ some more code from me) causes the following sdata to be written:

.data D_00004000 = bytearray (
                 04 00 00 06 00 00 00 00) 

This causes the MDA (managed debugging assistant) to trap access violations while trying to call the exported method(s).
Changing the sdata to (for an x64 image)

.data D_00004000 = int64(0)

or to (for an x86 image)

.data D_00004000 = int32(0)

fixes the AC issues.

Those are the tokens and the CLR updates them with the address of the method/stub. I don't know what you did to get this working but if you change dnlib to write 0s you'll get an exception.

========

You should use GetTypes() since Types only returns all non-nested types.

            var methods = module.Types.SelectMany(x => x.Methods)
                .Where(x => x.CustomAttributes.Any(IsDllExportAttribute));

[batzen]

Thanks for pointing out GetTypes. I will change that.

Those are the tokens and the CLR updates them with the address of the method/stub. I don't know what you did to get this working but if you change dnlib to write 0s you'll get an exception.

I don't want to get dnlib to write 0s but int64(0) or int32(0) instead. As the bytearray part seems to cause fatalexecution exceptions. Why does dnlib write a bytearray here? Maybe i would understand it better if i'd knew the reason.

All samples i could find use int instead of a bytearray.
Samples i found:

• Allow C# to export functions and constants to native code dotnet/csharplang#308
• https://books.google.de/books?id=g8gJKBedykUC&pg=PA52&lpg=PA52&dq=vtfixup+%22.data%22&source=bl&ots=0sYYuVWZV8&sig=ACfU3U0Q47w3n2XUnABUD0t_Zxn7mutODA&hl=en&sa=X&ved=2ahUKEwjkyMv0qebhAhWIZVAKHdGpCAYQ6AEwA3oECAcQAQ#v=onepage&q=%22.data%22&f=false
• https://books.google.de/books?id=oAcCRKd6EZgC&pg=PA384&lpg=PA384&dq=vtfixup+".data"&source=bl&ots=KCUf4ErU7s&sig=ACfU3U2mMH_GBM0E8kJAfBpFARjgvkV_gQ&hl=en&sa=X&ved=2ahUKEwjkyMv0qebhAhWIZVAKHdGpCAYQ6AEwAXoECAgQAQ#v=onepage&q=vtfixup%20".data"&f=false

[0xd4d]

It writes 4 or 8 bytes, which ILDASM shows as a byte array. You can't write 0s, it must be the token of the method.

[batzen]

Ok, now i finally understood what you mean and that int32(0) etc. gets read by ilasm and is converted to a bytearray too.

This led me to try one more thing:

• Modify assembly with dnlib (using the resulting assembly causes the MDA to trigger because of AV)
• run ildasm on that assembly
• run ilasm on that assembly (using the resulting assembly does not cause MDA to trigger)

Options i used for ilasm are ilasm ManagedWithDllExport.net462.x64.dll.il /output=ManagedWithDllExport.net462.x64.dll /dll /x64 /DEBUG /RESOURCE=ManagedWithDllExport.net462.x64.dll.res.

The resulting binaries have the exact same size of 9216 bytes on disk but behave differently.
The binary data is different though, but i was yet unable to find out whats really different and how i can avoid it.

Any idea what i or dnlib might be doing wrong here?

[0xd4d]

VS' debugger crashes if there's a DebuggableAttribute attribute and if the first ctor arg is 0x107. The workaround is to clear the EnableEditAndContinue bit:

            var ca = module.Assembly.CustomAttributes.Find("System.Diagnostics.DebuggableAttribute");
            if (ca != null && ca.ConstructorArguments.Count == 1) {
                var arg = ca.ConstructorArguments[0];
                // VS' debugger crashes if value == 0x107, so clear EnC bit
                if (arg.Type.FullName == "System.Diagnostics.DebuggableAttribute/DebuggingModes" && arg.Value is int value && value == 0x107) {
                    arg.Value = value & ~(int)DebuggableAttribute.DebuggingModes.EnableEditAndContinue;
                    ca.ConstructorArguments[0] = arg;
                }
            }

Other stuff:

You should add an assembly resolver. If this will be an MSBuild task you should pass in all references to the task and your custom asm resolver should only resolve to one of those assemblies (example). The following code adds a context which has a default asm resolver and type resolver.

            var creationOptions = new ModuleCreationOptions
                                  {
                                      // If you open more modules, they should share the same context (and asm resolver)
                                      Context = ModuleDef.CreateModuleContext(),
                                      TryToLoadPdbFromDisk = true
                                  };

EDIT: the assembly resolver needs to be disposed too after you don't need it.

[batzen]

Thanks a lot for your tips and teaching me a bit more about IL code! And thanks for this great project!

The EnC tip fixed the debug issue and the ModuleContext lead me to solve the issue of having a forced reference to System.Runtime.CompilerServices.VisualC in netcoreapp3.0 by adding an AssemblyRefUser.

I don't plan to release this as an MSBuild task but as a standalone tool. Essentially i only need it to add managed exports to libraries in Snoop (https://github.com/cplotts/snoopwpf/) to get rid of the managed c++ assemblies currently being used.
Now i only have to wait for some fixes in .net core 3.0 because it's currently not possible to call exported methods there if the assembly is not already loaded...

[batzen]

Maybe you should add the bit about EnC to the readme about dllexport. Might help others to not run into the same issues i did.

[0xd4d]

Yes, I added that to the README 2 days ago.