NativeAOT: support for building static libraries for Debug CRT on Windows

[lambdageek]

I would like to use NativeAOT to create a static library that I then link into a larger C/C++ project on Windows.

The C/C++ project has a debug configuration and a release configuration. In the debug configuration, the project links against the debug version of the C runtime. In the release configuration, it uses the release C runtime.

The static library that NativeAOT produces links against ucrt.lib:

runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Windows.targets

Lines 103 to 107 in f4f9bd5

	<ItemGroup Condition="!Exists('$(IlcSdkPath)debugucrt.txt')">
	<!-- Force ucrt to be dynamically linked for release runtime -->
	<LinkerArg Include="/NODEFAULTLIB:libucrt.lib" />
	<LinkerArg Include="/DEFAULTLIB:ucrt.lib" />
	</ItemGroup>

this is somewhat possible to change by doing some custom targets that mess with the LinkerArg item group after the SetupOSSpecificProps target.

Unfortunately, that's not enough. The reason is because the assorted helper libraries such as bootstrapperdll.obj, eventpipe-disabled.lib, Runtime.WorkstationGC.lib etc are all compiled with the release CRT, so you get linker errors when including them into a native app in Release mode.

It would be nice if the NativeAOT build integration provided versions of these libraries linked against the debug CRT.

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

I would like to use NativeAOT to create a static library that I then link into a larger C/C++ project on Windows.

The C/C++ project has a debug configuration and a release configuration. In the debug configuration, the project links against the debug version of the C runtime. In the release configuration, it uses the release C runtime.

The static library that NativeAOT produces links against ucrt.lib:

runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Windows.targets

Lines 103 to 107 in f4f9bd5

	<ItemGroup Condition="!Exists('$(IlcSdkPath)debugucrt.txt')">
	<!-- Force ucrt to be dynamically linked for release runtime -->
	<LinkerArg Include="/NODEFAULTLIB:libucrt.lib" />
	<LinkerArg Include="/DEFAULTLIB:ucrt.lib" />
	</ItemGroup>

this is somewhat possible to change by doing some custom targets that mess with the LinkerArg item group after the SetupOSSpecificProps target.

Unfortunately, that's not enough. The reason is because the assorted helper libraries such as bootstrapperdll.obj, eventpipe-disabled.lib, Runtime.WorkstationGC.lib etc are all compiled with the release CRT, so you get linker errors when including them into a native app in Release mode.

It would be nice if the NativeAOT build integration provided versions of these libraries linked against the debug CRT.

Author:	lambdageek
Assignees:	-
Labels:	untriaged, area-NativeAOT-coreclr
Milestone:	-

[lambdageek]

Sample of what I'm doing: https://github.com/lambdageek/naothello

Note: I don't think we want the support libraries to be built in the Debug configuration - we release builds of the libraries, as usual, we just want them to link against the debug CRT

[lambdageek]

No, I was wrong. it's not working.

LINK : warning LNK4098: defaultlib 'LIBCMTD' conflicts with use of other libs; use /NODEFAULTLIB:library [E:\na
othello\app\out\helloLib.vcxproj

[jkotas]

warning LNK4098: defaultlib 'LIBCMTD' conflicts with use of other libs; use /NODEFAULTLIB:library

What happens if you ignore this warning?

[lambdageek]

warning LNK4098: defaultlib 'LIBCMTD' conflicts with use of other libs; use /NODEFAULTLIB:library

What happens if you ignore this warning?

The app runs. But I'm not familiar enough with Windows development to tell if that's just an accident because my native code is a toy (so it doesn't touch enough of the CRT to get into trouble) or if everything is ok.

[jkotas]

Everything should be ok. NativeAOT does not use C++ STL where mixing debug and release would get you into trouble. You should be able to use NODEFAULTLIB to suppress the warning.

[lambdageek]

@jkotas just to double check. The warning is coming from the point where I'm linking the NativeAOT artifacts together with my C code. if I use /NODEFAULTLIB:LIBCMTD won't that effectively just always cause my C code to link against the release CRT that was already linked into the nativeaot libs? Is that what I want?

On the other hand adding /NODEFAULTLIB:ucrt.lib to my build doesn't have any effect: I think the fact that Runtime.WorkstationGC.lib (or some of the other libs - I'm not sure) were already linked against ucrt.lib means that it's too late to ask for a different CRT

---

this is the link step (without any /NODEFAULTLIB changeS)

  C:\Program Files\Microsoft Visual Studio\2022\Enterprise Preview\VC\Tools\MSVC\14.39.33519\bin\HostX64\x64\link
  .exe /ERRORREPORT:QUEUE /OUT:"E:\naothello\app\out\Debug\helloLib.dll" /INCREMENTAL /ILK:"helloLib.dir\Debug\he
  lloLib.ilk" /NOLOGO /LIBPATH:"C:/.tools/.nuget/packages/runtime.win-x64.microsoft.dotnet.ilcompiler/8.0.0/frame
  work" /LIBPATH:"C:/.tools/.nuget/packages/runtime.win-x64.microsoft.dotnet.ilcompiler/8.0.0/framework/Debug" /L
  IBPATH:"C:/.tools/.nuget/packages/runtime.win-x64.microsoft.dotnet.ilcompiler/8.0.0/sdk" /LIBPATH:"C:/.tools/.n
  uget/packages/runtime.win-x64.microsoft.dotnet.ilcompiler/8.0.0/sdk/Debug" "E:\naothello\libnaothello\bin\Debug
  \net8.0\win-x64\publish\libnaothello.lib" "C:\.tools\.nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompile
  r\8.0.0\sdk\bootstrapperdll.obj" "C:\.tools\.nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler\8.0.0\s
  dk\Runtime.WorkstationGC.lib" "C:\.tools\.nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler\8.0.0\sdk\
  eventpipe-disabled.lib" "C:\.tools\.nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler\8.0.0\sdk\Runtim
  e.VxsortEnabled.lib" "C:\.tools\.nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler\8.0.0\sdk\System.Gl
  obalization.Native.Aot.lib" "C:\.tools\.nuget\packages\runtime.win-x64.microsoft.dotnet.ilcompiler\8.0.0\sdk\Sy
  stem.IO.Compression.Native.Aot.lib" BCrypt.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32
  .lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTUAC:"level='asIn
  voker' uiAccess='false'" /manifest:embed /DEBUG /PDB:"E:/naothello/app/out/Debug/helloLib.pdb" /SUBSYSTEM:CONSO
  LE /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:"E:/naothello/app/out/Debug/helloLib.lib" /MACHINE:X64  /machine:x64
   /DLL helloLib.dir\Debug\hello.obj
     Creating library E:/naothello/app/out/Debug/helloLib.lib and object E:/naothello/app/out/Debug/helloLib.exp
LINK : warning LNK4098: defaultlib 'LIBCMTD' conflicts with use of other libs; use /NODEFAULTLIB:library [E:\naot
hello\app\out\helloLib.vcxproj]

[jkotas]

Release Runtime.WorkstationGC.lib has /DEFAULTLIB:"libcpmt" /DEFAULTLIB:"LIBCMT", so the suppress it you would need /NODEFAULTLIB:libcpmt /NODEFAULTLIB:LIBCMT`.

The easiest way to find out what is going on is to open *.lib file in an editor and look for DEFAULTLIB. You will find a string that looks like a part of the linker command line. This command line is merged with the actual linker command line, with actual linker command line taking a precedence.

[jkotas]

This is built upon #pragma comment(linker, ... https://learn.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp?view=msvc-170#linker

[lambdageek]

Thanks!

I ended up using dumpbin /directives on Runtime.WorkstationGC.lib to get the same info.

Adding your suggested /nodefaultlib directives removed the warning and the toy app still runs. I'm unblocked now.

[lambdageek]

Nope, still blocked.

If I change my native hosting app to C++ things break due to the #pragma detect_mismatch directives in Runtime.WorkstationGC.lib. dumpbin /directives output:

 Linker Directives
   -----------------
   /FAILIFMISMATCH:_CRT_STDIO_ISO_WIDE_SPECIFIERS=0
   /FAILIFMISMATCH:_MSC_VER=1900
   /FAILIFMISMATCH:_ITERATOR_DEBUG_LEVEL=0
   /FAILIFMISMATCH:RuntimeLibrary=MT_StaticRelease

The app is in the cplusplus branch https://github.com/lambdageek/naothello/tree/cplusplus

The app's own code gets these directives:

   Linker Directives
   -----------------
   /FAILIFMISMATCH:_CRT_STDIO_ISO_WIDE_SPECIFIERS=0
   /FAILIFMISMATCH:_MSC_VER=1900
   /FAILIFMISMATCH:_ITERATOR_DEBUG_LEVEL=2
   /FAILIFMISMATCH:RuntimeLibrary=MTd_StaticDebug

And the linker complains (one example of about 120):

Runtime.WorkstationGC.lib(gccommon.cpp.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value
 '0' doesn't match value '2' in hello.obj [E:\naothello\app\out\helloLib.vcxproj]
Runtime.WorkstationGC.lib(gccommon.cpp.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MT_St
aticRelease' doesn't match value 'MTd_StaticDebug' in hello.obj [E:\naothello\app\out\helloLib.vcxproj]

I see no way to tell the linker to ignore /FAILIFMISMATCH directives

[jkotas]

Hmm, I guess the only way to solve these sorts of scenarios in scalable way would be to ship the runtime sources in the package and allow the user to compile them with any options they want. We do that for ICU and OpenSSL shims today: https://github.com/dotnet/runtime/blob/main/src/coreclr/nativeaot/docs/compiling.md#using-statically-linked-icu

[MichalStrehovsky]

We track the build from source scenario here: #83611

It's not just the debug/release mismatch. There could also be ABI differences, we were just lucky that MSVC hasn't done a breaking change in ABI recently. Building static libraries out of native AOT is not recommended. I think this issue should just track adding "we really don't recommend you to do this" to the dotnet/samples sample. I never really liked we soft-documented this by adding the static library case to the dotnet/samples sample.