Refactor out TargetFramework concept from Microsoft.Interop.SourceGeneration

[jkoritzinsky]

Refactor the concept of TargetFramework out of Micrsoft.Interop.SourceGeneration. Replace it with a CodeEmitOptions struct for us to introduce emit switches based on TFM features (in particular, the ability to skip initialization of locals was only introduced in .NET 5, so we need to have a switch for it). As all other use cases of the TFM are downlevel-support-specific in LibraryImportGenerator, move the TFM types into LibraryImportGenerator.

[ghost]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Refactor the concept of TargetFramework out of Micrsoft.Interop.SourceGeneration. Replace it with a CodeEmitOptions struct for us to introduce emit switches based on TFM features (in particular, the ability to skip initialization of locals was only introduced in .NET 5, so we need to have a switch for it). As all other use cases of the TFM are downlevel-support-specific in LibraryImportGenerator, move the TFM types into LibraryImportGenerator.

Author:	jkoritzinsky
Assignees:	-
Labels:	area-System.Runtime.InteropServices
Milestone:	9.0.0

[filipnavara]

I started getting a wall of errors on dotnet/runtime builds today:

CSC : error CS8784: Generator 'VtableIndexStubGenerator' failed to initialize. It will not contribute to the output and compilation errors may occur as a result. Exception was of type 'TypeLoadException' with message 'Could not load type 'Microsoft.Interop.TargetFramework' from assembly 'Microsoft.Interop.SourceGeneration, Version=42.42.42.42, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.' [/Users/filipnavara/Projects/runtime/src/tasks/AppleAppBuilder/AppleAppBuilder.csproj]
CSC : error CS8784: Generator 'JSExportGenerator' failed to initialize. It will not contribute to the output and compilation errors may occur as a result. Exception was of type 'TypeLoadException' with message 'Could not load type 'Microsoft.Interop.TargetFramework' from assembly 'Microsoft.Interop.SourceGeneration, Version=42.42.42.42, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.' [/Users/filipnavara/Projects/runtime/src/tasks/AppleAppBuilder/AppleAppBuilder.csproj]
CSC : error CS8784: Generator 'JSImportGenerator' failed to initialize. It will not contribute to the output and compilation errors may occur as a result. Exception was of type 'TypeLoadException' with message 'Could not load type 'Microsoft.Interop.TargetFramework' from assembly 'Microsoft.Interop.SourceGeneration, Version=42.42.42.42, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.' [/Users/filipnavara/Projects/runtime/src/tasks/AppleAppBuilder/AppleAppBuilder.csproj]
CSC : error CS8784: Generator 'LibraryImportGenerator' failed to initialize. It will not contribute to the output and compilation errors may occur as a result. Exception was of type 'TypeLoadException' with message 'Could not load type 'Microsoft.Interop.TargetFramework' from assembly 'Microsoft.Interop.SourceGeneration, Version=42.42.42.42, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.' [/Users/filipnavara/Projects/runtime/src/tasks/AppleAppBuilder/AppleAppBuilder.csproj]
CSC : error AD0001: Analyzer 'Microsoft.Interop.Analyzers.ConvertToLibraryImportAnalyzer' threw an exception of type 'System.TypeLoadException' with message 'Could not load type 'Microsoft.Interop.TargetFrameworkSettings' from assembly 'Microsoft.Interop.SourceGeneration, Version=42.42.42.42, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.'. [/Users/filipnavara/Projects/runtime/src/tasks/AppleAppBuilder/AppleAppBuilder.csproj]
CSC : error CS8784: Generator 'VtableIndexStubGenerator' failed to initialize. It will not contribute to the output and compilation errors may occur as a result. Exception was of type 'TypeLoadException' with message 'Could not load type 'Microsoft.Interop.TargetFramework' from assembly 'Microsoft.Interop.SourceGeneration, Version=42.42.42.42, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.' [/Users/filipnavara/Projects/runtime/src/tasks/AotCompilerTask/MonoAOTCompiler.csproj::TargetFramework=net8.0]
CSC : error CS8784: Generator 'JSExportGenerator' failed to initialize. It will not contribute to the output and compilation errors may occur as a result. Exception was of type 'TypeLoadException' with message 'Could not load type 'Microsoft.Interop.TargetFramework' from assembly 'Microsoft.Interop.SourceGeneration, Version=42.42.42.42, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.' [/Users/filipnavara/Projects/runtime/src/tasks/AotCompilerTask/MonoAOTCompiler.csproj::TargetFramework=net8.0]
CSC : error CS8784: Generator 'JSImportGenerator' failed to initialize. It will not contribute to the output and compilation errors may occur as a result. Exception was of type 'TypeLoadException' with message 'Could not load type 'Microsoft.Interop.TargetFramework' from assembly 'Microsoft.Interop.SourceGeneration, Version=42.42.42.42, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.' [/Users/filipnavara/Projects/runtime/src/tasks/AotCompilerTask/MonoAOTCompiler.csproj::TargetFramework=net8.0]
...

Could it be related?

[filipnavara]

Apparently the errors went away after the next arcade version bump commit... I just happened to land in the window in-between. Sorry for the noise.

[jkoritzinsky]

Any chance you can send me the commit sha that failed? And did this fail in VS or on the command line? We shouldn't be hitting failures like that ever.

[filipnavara]

Any chance you can send me the commit sha that failed?

It was 24ead90 but I cannot reproduce it anymore.

I simply woke up and tried to run ./build.sh on something that worked the day before. The only reasonable explanation is that something got restored from NuGet that caused the breakage.

[filipnavara]

Not sure what is going on but it started happening again... commit bbe5c66, macOS, running

./build.sh
./dotnet.sh test src/libraries/System.Formats.Cbor/tests -f net9.0

The build passed, but the second command did not:

CSC : error CS8784: Generator 'VtableIndexStubGenerator' failed to initialize. It will not contribute to the output and compilation errors may occur as a result. Exception was of type 'TypeLoadException' with message 'Could not load type 'Microsoft.Interop.TargetFramework' from assembly 'Microsoft.Interop.SourceGeneration, Version=42.42.42.42, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.' [/Users/filipnavara/Projects/runtime/src/libraries/Common/tests/TestUtilities/TestUtilities.csproj::TargetFramework=net8.0]
CSC : error CS8784: Generator 'JSExportGenerator' failed to initialize. It will not contribute to the output and compilation errors may occur as a result. Exception was of type 'TypeLoadException' with message 'Could not load type 'Microsoft.Interop.TargetFramework' from assembly 'Microsoft.Interop.SourceGeneration, Version=42.42.42.42, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.' [/Users/filipnavara/Projects/runtime/src/libraries/Common/tests/TestUtilities/TestUtilities.csproj::TargetFramework=net8.0]
CSC : error CS8784: Generator 'JSImportGenerator' failed to initialize. It will not contribute to the output and compilation errors may occur as a result. Exception was of type 'TypeLoadException' with message 'Could not load type 'Microsoft.Interop.TargetFramework' from assembly 'Microsoft.Interop.SourceGeneration, Version=42.42.42.42, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.' [/Users/filipnavara/Projects/runtime/src/libraries/Common/tests/TestUtilities/TestUtilities.csproj::TargetFramework=net8.0]

...and now any subsequent attempt to build (even just ./build.sh) fails.

Yesterday I deleted the full NuGet cache and dotnet installation. If you have any tips how to analyze this I would certainly appreciate it.

I had a theory why is it happening (semi-)randomly... I tend to have the root of the repository opened in VS Code, and I have the C# and C# Dev Kit extensions installed there. I though it may trigger some kind of restore that brings the repository into uncompilable state. However, closing VS Code and cleaning the repo and NuGet cache didn't help.

[filipnavara]

...and the problem is gone again after machine restart. Could it be MSBuild node reuse (or Roslyn process sharing?) kicking in and hitting a shared process that has a wrong version of the library already loaded?

[jkoritzinsky]

I think node reuse might be the problem, but the builds should be isolated. If it happens again, can you email me a binlog at the email in my profile?

[filipnavara]

I think node reuse might be the problem, but the builds should be isolated. If it happens again, can you email me a binlog at the email in my profile?

Sure. I tried to disable MSBuild node reuse to no effect, so I am leaning towards putting the blame on Roslyn Compiler Server. Running killall dotnet was enough to bring everything back to normal, so I use it to unblock my flow.

[ericstj]

This appears to be happening because other analyzers build using the SDK's ref-pack:

runtime/src/libraries/Directory.Build.targets

Lines 58 to 59 in 97a51cc

	<!-- Inbox analyzers shouldn't use the live targeting / runtime pack. They better depend on an LKG to avoid layering concerns. -->
	<UseLocalTargetingRuntimePack Condition="'$(IsNETCoreAppAnalyzer)' == 'true'">false</UseLocalTargetingRuntimePack>

But they aren't doing anything to disable the generators from that. So the roslyn process gets a mix of the LKG generators and live built generator assemblies in process.

I think the fix here is to disable all the generators from the ref-pack.