Ngen DLLs generates "Failed to load type token" and "Failed to load method token"

[tohling]

I am trying to generate native assemblies using Ngen on a root assembly that has dependency on Microsoft.CodeAnalysis and Microsoft.CodeAnalysis.CSharp. I am seeing some errors below and have reached out the Ngen team. While the native assemblies Microsoft.CodeAnalysis.ni.dll and Microsoft.CodeAnalysis.CSharp.ni.dll are generated, Ngen team has indicated that having these errors will impact performance. We are generating native binaries to improve performance, so resolving these errors is critical for us. Your help will be greatly appreciated.

Version Used: Microsoft.CodeAnalysis, Version=1.3.1.0 and Microsoft.CodeAnalysis.CSharp, Version=1.3.1.0

Steps to Reproduce:

1. Run ngen on any root assembly that has dependency on Microsoft.CodeAnalysis and Microsoft.CodeAnalysis.CSharp.

Expected Behavior:
Native assemblies for Microsoft.CodeAnalysis.ni.dll and Microsoft.CodeAnalysis.CSharp.ni.dll should be generated without error messages.

Actual Behavior:
Lots of messages for many object types are generated for these 2 DLLs, e.g.:

1>    Compiling assembly Microsoft.CodeAnalysis, Version=1.3.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35 (CLR v4.0.30319) ...
1>Failed to load type token 680003c9 for the profile data in [D:\Program Files (x86)\SiteExtensions\Functions\1.0.10690\bin\Microsoft.CodeAnalysis.dll] The type specified by the IBC data is no longer present in the assembly's metadata.  The type was [System.Linq]System.Linq.Enumerable+Iterator`1.
1>Failed to load type token 680004c1 for the profile data in [D:\Program Files (x86)\SiteExtensions\Functions\1.0.10690\bin\Microsoft.CodeAnalysis.dll] The type specified by the IBC data is no longer present in the assembly's metadata.  The type was [System.Collections.Concurrent]System.Collections.Concurrent.IProducerConsumerCollection`1.
1>Failed to load type token 680004c2 for the profile data in [D:\Program Files (x86)\SiteExtensions\Functions\1.0.10690\bin\Microsoft.CodeAnalysis.dll] The type specified by the IBC data is no longer present in the assembly's metadata.  The type was [System.Collections.Concurrent]System.Collections.Concurrent.ConcurrentQueue`1.
1>Failed to load type token 680004fe for the profile data in [D:\Program Files (x86)\SiteExtensions\Functions\1.0.10690\bin\Microsoft.CodeAnalysis.dll] The type specified by the IBC data is no longer present in the assembly's metadata.  The type was [System.Linq]System.Linq.Enumerable+d__1`2.
1>Failed to load type token 68000500 for the profile data in [D:\Program Files (x86)\SiteExtensions\Functions\1.0.10690\bin\Microsoft.CodeAnalysis.dll] The type specified by the IBC data is no longer present in the assembly's metadata.  The type was [System.Linq]System.Linq.Buffer`1.
1>Failed to load type token 68000501 for the profile data in [D:\Program Files (x86)\SiteExtensions\Functions\1.0.10690\bin\Microsoft.CodeAnalysis.dll] The type specified by the IBC data is no longer present in the assembly's metadata.  The type was [System.Linq]System.ArrayT`1.
[long list of similar errors...]

1>    Compiling assembly Microsoft.CodeAnalysis.CSharp, Version=1.3.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35 (CLR v4.0.30319) ...
1>Failed to load type token 68000440 for the profile data in [D:\Program Files (x86)\SiteExtensions\Functions\1.0.10690\bin\Microsoft.CodeAnalysis.CSharp.dll] The type specified by the IBC data is no longer present in the assembly's metadata.  The type was [System.Linq]System.Linq.Enumerable+d__1`1.
1>Failed to load type token 68000451 for the profile data in [D:\Program Files (x86)\SiteExtensions\Functions\1.0.10690\bin\Microsoft.CodeAnalysis.CSharp.dll] The type specified by the IBC data is no longer present in the assembly's metadata.  The type was [Microsoft.CodeAnalysis]Microsoft.CodeAnalysis.ArrayBuilder`1+<>c__DisplayClass39_0.
1>Failed to load type token 68000472 for the profile data in [D:\Program Files (x86)\SiteExtensions\Functions\1.0.10690\bin\Microsoft.CodeAnalysis.CSharp.dll] The type specified by the IBC data is no longer present in the assembly's metadata.  The type was [Microsoft.CodeAnalysis]Microsoft.CodeAnalysis.ArrayBuilder`1+<>c__DisplayClass39_0.
1>Failed to load type token 6800047c for the profile data in [D:\Program Files (x86)\SiteExtensions\Functions\1.0.10690\bin\Microsoft.CodeAnalysis.CSharp.dll] The type specified by the IBC data is no longer present in the assembly's metadata.  The type was [Microsoft.CodeAnalysis]Microsoft.CodeAnalysis.ArrayBuilder`1+<>c__DisplayClass39_0.
1>Failed to load type token 680004a7 for the profile data in [D:\Program Files (x86)\SiteExtensions\Functions\1.0.10690\bin\Microsoft.CodeAnalysis.CSharp.dll] The type specified by the IBC data is no longer present in the assembly's metadata.  The type was [Microsoft.CodeAnalysis]Microsoft.CodeAnalysis.CommonReferenceManager`2+ReferencedAssembly.
1>Failed to load type token 680004d1 for the profile data in [D:\Program Files (x86)\SiteExtensions\Functions\1.0.10690\bin\Microsoft.CodeAnalysis.CSharp.dll] The type specified by the IBC data is no longer present in the assembly's metadata.  The type was [System.Linq]System.Linq.EmptyEnumerable`1.
[long list of similar errors...]

Suggested fix:
I engaged the help from Ngen team and their response is as follows:

"If your team owns Microsoft.Codeanalysis.CSharp.dll, please talk to the people responsible for doing the build, and ask them to re-run IBC data generation procedure. This is usually a post-build optimization step, which runs some typical apps that use this DLL, and collect profiling data. Each team does this procedure a little differently, but hopefully the build team in your group knows how to do it. If this assembly came from another team, you’ll need to talk to its owner."

Kindly see if you can fix this soon.

[jasonmalinowski]

@KevinH-MS Does this make any sense?

We do embed IBC data, but the errors you're showing doesn't imply to me that we have bad data there, but rather than ngen isn't able to find our dependencies properly in your setup.

[KevinH-MS]

It sounds like the ngen logs are from a machine that has a different/newer version of the .NET Framework than we trained on back when 1.3 was released.

We don't plan to retrain/re-release those packages, and we only embed IBC data that matches the VS configuration that the binaries are intended to target.

What sort of scenario is this in service of? I assume it has nothing to do with VS, because if you did have VS installed, these binaries should already be ngen'ed. If this is not a product scenario where you might have to exist SxS with a VS installation, then one option to avoid JIT'ing might be to just strip the IBC data from the assembly and "fully" ngen everything on the target machine.

[tohling]

@KevinH-MS, thanks for the prompt response. Based on your comment "It sounds like the ngen logs are from a machine that has a different/newer version of the .NET Framework than we trained on back when 1.3 was released.", does this mean that if I updated 1.3 to 2.0, the error will go away?

You also mentioned that another option would be to strip the IBC data from the assembly. How would I go about doing this?

[KevinH-MS]

2.0 may indeed work better for you.

[tohling]

@KevinH-MS, 2.0 is still a preview release. Is there an ETA on a stable release?

[tohling]

OK, I will try the 2.0 approach and see if that resolves the issue. I will close this issue for now an re-open if necessary.