FCS solution tests fail on Mono

[auduchinok]

It looks related to #8721. cc @KevinRansom @cartermp

Tests.Service.ProjectAnalysisTests.Test project28 all symbols in signature

System.TypeInitializationException : The type initializer for '<StartupCode$FSharp-Compiler-Service-Tests>.$Tests.Service.ProjectAnalysisTests' threw ...

System.TypeInitializationException : The type initializer for '<StartupCode$FSharp-Compiler-Service-Tests>.$Tests.Service.ProjectAnalysisTests' threw an exception.
  ----> System.TypeLoadException : Could not load type of field 'Microsoft.Interactive.DependencyManager.AssemblyResolveHandlerCoreclr:handler' (1) due to: Could not resolve type with token 01000013 from typeref (expected class 'System.Runtime.Loader.AssemblyLoadContext' in assembly 'System.Runtime.Loader, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a') assembly:System.Runtime.Loader, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a type:System.Runtime.Loader.AssemblyLoadContext member:(null)
  at (wrapper managed-to-native) System.Object.__icall_wrapper_mono_generic_class_init(intptr)
  at Tests.Service.ProjectAnalysisTests.Test project28 all symbols in signature () [0x00000] in /Users/eugene/Developer/fsharp/tests/service/ProjectAnalysisTests.fs:3944 
  at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
  at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-10/external/bockbuild/builds/mono-x64/mcs/class/corlib/System.Reflection/RuntimeMethodInfo.cs:395 
--TypeLoadException
  at Microsoft.Interactive.DependencyManager.DependencyProvider..ctor (Microsoft.Interactive.DependencyManager.AssemblyResolutionProbe assemblyProbingPaths, Microsoft.Interactive.DependencyManager.NativeResolutionProbe nativeProbingRoots) [0x00014] in /Users/eugene/Developer/fsharp/src/fsharp/Interactive.DependencyManager/DependencyProvider.fs:239 
  at Microsoft.Interactive.DependencyManager.DependencyProvider..ctor (Microsoft.Interactive.DependencyManager.NativeResolutionProbe nativeProbingRoots) [0x00000] in /Users/eugene/Developer/fsharp/src/fsharp/Interactive.DependencyManager/DependencyProvider.fs:298 
  at FSharp.Compiler.CompileOps+TcConfigBuilder.CreateNew (FSharp.Compiler.ReferenceResolver+Resolver legacyReferenceResolver, System.String defaultFSharpBinariesDir, FSharp.Compiler.AbstractIL.ILBinaryReader+ReduceMemoryFlag reduceMemoryUsage, System.String implicitIncludeDir, System.Boolean isInteractive, System.Boolean isInvalidationSupported, FSharp.Compiler.CompileOps+CopyFSharpCoreFlag defaultCopyFSharpCore, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] tryGetMetadataSnapshot) [0x00399] in /Users/eugene/Developer/fsharp/src/fsharp/CompileOps.fs:2431 
  at FSharp.Compiler.CompileOps+ScriptPreprocessClosure.CreateScriptTextTcConfig[a] (FSharp.Compiler.ReferenceResolver+Resolver legacyReferenceResolver, System.String defaultFSharpBinariesDir, System.String filename, FSharp.Compiler.CompileOps+CodeContext codeContext, System.Boolean useSimpleResolution, System.Boolean useFsiAuxLib, Microsoft.FSharp.Core.FSharpOption`1[T] basicReferences, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] applyCommandLineArgs, System.Boolean assumeDotNetFramework, System.Boolean useSdkRefs, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] tryGetMetadataSnapshot, FSharp.Compiler.AbstractIL.ILBinaryReader+ReduceMemoryFlag reduceMemoryUsage) [0x00036] in /Users/eugene/Developer/fsharp/src/fsharp/CompileOps.fs:5205 
  at FSharp.Compiler.CompileOps+ScriptPreprocessClosure.GetFullClosureOfScriptText (FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, FSharp.Compiler.ReferenceResolver+Resolver legacyReferenceResolver, System.String defaultFSharpBinariesDir, System.String filename, FSharp.Compiler.Text.ISourceText sourceText, FSharp.Compiler.CompileOps+CodeContext codeContext, System.Boolean useSimpleResolution, System.Boolean useFsiAuxLib, System.Boolean useSdkRefs, FSharp.Compiler.Lexhelp+LexResourceManager lexResourceManager, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] applyCommandLineArgs, System.Boolean assumeDotNetFramework, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] tryGetMetadataSnapshot, FSharp.Compiler.AbstractIL.ILBinaryReader+ReduceMemoryFlag reduceMemoryUsage) [0x00000] in /Users/eugene/Developer/fsharp/src/fsharp/CompileOps.fs:5459 
  at FSharp.Compiler.CompileOps+LoadClosure.ComputeClosureOfScriptText (FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, FSharp.Compiler.ReferenceResolver+Resolver legacyReferenceResolver, System.String defaultFSharpBinariesDir, System.String filename, FSharp.Compiler.Text.ISourceText sourceText, FSharp.Compiler.CompileOps+CodeContext implicitDefines, System.Boolean useSimpleResolution, System.Boolean useFsiAuxLib, System.Boolean useSdkRefs, FSharp.Compiler.Lexhelp+LexResourceManager lexResourceManager, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] applyCompilerOptions, System.Boolean assumeDotNetFramework, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] tryGetMetadataSnapshot, FSharp.Compiler.AbstractIL.ILBinaryReader+ReduceMemoryFlag reduceMemoryUsage) [0x0000b] in /Users/eugene/Developer/fsharp/src/fsharp/CompileOps.fs:5500 
  at <StartupCode$FSharp-Compiler-Service>.$Service+GetProjectOptionsFromScript@775-2.Invoke (FSharp.Compiler.SourceCodeServices.ErrorScope _arg27) [0x000ac] in /Users/eugene/Developer/fsharp/src/fsharp/service/service.fs:799 
  at FSharp.Compiler.AbstractIL.Internal.Library+CancellableBuilder.Using[c,d] (c resource, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] e) [0x00000] in /Users/eugene/Developer/fsharp/src/absil/illib.fs:811 
  at FSharp.Compiler.AbstractIL.Internal.Library+CancellableModule+delay@746[T].Invoke (System.Threading.CancellationToken ct) [0x00000] in /Users/eugene/Developer/fsharp/src/absil/illib.fs:746 
  at <StartupCode$FSharp-Compiler-Service>.$Reactor+EnqueueAndAwaitOpAsync@181-2[T].Invoke (FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok) [0x00000] in /Users/eugene/Developer/fsharp/src/fsharp/service/Reactor.fs:184 
--- End of stack trace from previous location where exception was thrown ---

  at Microsoft.FSharp.Control.AsyncResult`1[T].Commit () [0x0002c] in <5c64a986dff9fae1a745038386a9645c>:0 
  at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a] (System.Threading.CancellationToken cancellationToken, Microsoft.FSharp.Control.FSharpAsync`1[T] computation) [0x00028] in <5c64a986dff9fae1a745038386a9645c>:0 
  at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T] (System.Threading.CancellationToken cancellationToken, Microsoft.FSharp.Control.FSharpAsync`1[T] computation, Microsoft.FSharp.Core.FSharpOption`1[T] timeout) [0x00013] in <5c64a986dff9fae1a745038386a9645c>:0 
  at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T] (Microsoft.FSharp.Control.FSharpAsync`1[T] computation, Microsoft.FSharp.Core.FSharpOption`1[T] timeout, Microsoft.FSharp.Core.FSharpOption`1[T] cancellationToken) [0x0006e] in <5c64a986dff9fae1a745038386a9645c>:0 
  at <StartupCode$FSharp-Compiler-Service-Tests>.$Tests.Service.ProjectAnalysisTests..cctor () [0x01a93] in /Users/eugene/Developer/fsharp/tests/service/ProjectAnalysisTests.fs:4511 



-----

One or more child tests had errors
  Exception doesn't have a stacktrace



System.TypeInitializationException : The type initializer for '<StartupCode$FSharp-Compiler-Service-Tests>.$Tests.Service.ProjectAnalysisTests' threw an exception.
  ----> System.TypeLoadException : Could not load type of field 'Microsoft.Interactive.DependencyManager.AssemblyResolveHandlerCoreclr:handler' (1) due to: Could not resolve type with token 01000013 from typeref (expected class 'System.Runtime.Loader.AssemblyLoadContext' in assembly 'System.Runtime.Loader, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a') assembly:System.Runtime.Loader, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a type:System.Runtime.Loader.AssemblyLoadContext member:(null)  at (wrapper managed-to-native) System.Object.__icall_wrapper_mono_generic_class_init(intptr)
  at Tests.Service.ProjectAnalysisTests.Test project28 all symbols in signature () [0x00000] in /Users/eugene/Developer/fsharp/tests/service/ProjectAnalysisTests.fs:3944 
  at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
  at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-10/external/bockbuild/builds/mono-x64/mcs/class/corlib/System.Reflection/RuntimeMethodInfo.cs:395 
--TypeLoadException
  at Microsoft.Interactive.DependencyManager.DependencyProvider..ctor (Microsoft.Interactive.DependencyManager.AssemblyResolutionProbe assemblyProbingPaths, Microsoft.Interactive.DependencyManager.NativeResolutionProbe nativeProbingRoots) [0x00014] in /Users/eugene/Developer/fsharp/src/fsharp/Interactive.DependencyManager/DependencyProvider.fs:239 
  at Microsoft.Interactive.DependencyManager.DependencyProvider..ctor (Microsoft.Interactive.DependencyManager.NativeResolutionProbe nativeProbingRoots) [0x00000] in /Users/eugene/Developer/fsharp/src/fsharp/Interactive.DependencyManager/DependencyProvider.fs:298 
  at FSharp.Compiler.CompileOps+TcConfigBuilder.CreateNew (FSharp.Compiler.ReferenceResolver+Resolver legacyReferenceResolver, System.String defaultFSharpBinariesDir, FSharp.Compiler.AbstractIL.ILBinaryReader+ReduceMemoryFlag reduceMemoryUsage, System.String implicitIncludeDir, System.Boolean isInteractive, System.Boolean isInvalidationSupported, FSharp.Compiler.CompileOps+CopyFSharpCoreFlag defaultCopyFSharpCore, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] tryGetMetadataSnapshot) [0x00399] in /Users/eugene/Developer/fsharp/src/fsharp/CompileOps.fs:2431 
  at FSharp.Compiler.CompileOps+ScriptPreprocessClosure.CreateScriptTextTcConfig[a] (FSharp.Compiler.ReferenceResolver+Resolver legacyReferenceResolver, System.String defaultFSharpBinariesDir, System.String filename, FSharp.Compiler.CompileOps+CodeContext codeContext, System.Boolean useSimpleResolution, System.Boolean useFsiAuxLib, Microsoft.FSharp.Core.FSharpOption`1[T] basicReferences, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] applyCommandLineArgs, System.Boolean assumeDotNetFramework, System.Boolean useSdkRefs, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] tryGetMetadataSnapshot, FSharp.Compiler.AbstractIL.ILBinaryReader+ReduceMemoryFlag reduceMemoryUsage) [0x00036] in /Users/eugene/Developer/fsharp/src/fsharp/CompileOps.fs:5205 
  at FSharp.Compiler.CompileOps+ScriptPreprocessClosure.GetFullClosureOfScriptText (FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, FSharp.Compiler.ReferenceResolver+Resolver legacyReferenceResolver, System.String defaultFSharpBinariesDir, System.String filename, FSharp.Compiler.Text.ISourceText sourceText, FSharp.Compiler.CompileOps+CodeContext codeContext, System.Boolean useSimpleResolution, System.Boolean useFsiAuxLib, System.Boolean useSdkRefs, FSharp.Compiler.Lexhelp+LexResourceManager lexResourceManager, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] applyCommandLineArgs, System.Boolean assumeDotNetFramework, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] tryGetMetadataSnapshot, FSharp.Compiler.AbstractIL.ILBinaryReader+ReduceMemoryFlag reduceMemoryUsage) [0x00000] in /Users/eugene/Developer/fsharp/src/fsharp/CompileOps.fs:5459 
  at FSharp.Compiler.CompileOps+LoadClosure.ComputeClosureOfScriptText (FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, FSharp.Compiler.ReferenceResolver+Resolver legacyReferenceResolver, System.String defaultFSharpBinariesDir, System.String filename, FSharp.Compiler.Text.ISourceText sourceText, FSharp.Compiler.CompileOps+CodeContext implicitDefines, System.Boolean useSimpleResolution, System.Boolean useFsiAuxLib, System.Boolean useSdkRefs, FSharp.Compiler.Lexhelp+LexResourceManager lexResourceManager, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] applyCompilerOptions, System.Boolean assumeDotNetFramework, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] tryGetMetadataSnapshot, FSharp.Compiler.AbstractIL.ILBinaryReader+ReduceMemoryFlag reduceMemoryUsage) [0x0000b] in /Users/eugene/Developer/fsharp/src/fsharp/CompileOps.fs:5500 
  at <StartupCode$FSharp-Compiler-Service>.$Service+GetProjectOptionsFromScript@775-2.Invoke (FSharp.Compiler.SourceCodeServices.ErrorScope _arg27) [0x000ac] in /Users/eugene/Developer/fsharp/src/fsharp/service/service.fs:799 
  at FSharp.Compiler.AbstractIL.Internal.Library+CancellableBuilder.Using[c,d] (c resource, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] e) [0x00000] in /Users/eugene/Developer/fsharp/src/absil/illib.fs:811 
  at FSharp.Compiler.AbstractIL.Internal.Library+CancellableModule+delay@746[T].Invoke (System.Threading.CancellationToken ct) [0x00000] in /Users/eugene/Developer/fsharp/src/absil/illib.fs:746 
  at <StartupCode$FSharp-Compiler-Service>.$Reactor+EnqueueAndAwaitOpAsync@181-2[T].Invoke (FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok) [0x00000] in /Users/eugene/Developer/fsharp/src/fsharp/service/Reactor.fs:184 
--- End of stack trace from previous location where exception was thrown ---

[cartermp]

Ugh, yeah this looks like an inconsistency in .NET Standard implementations.

[KevinRansom]

Hmmm!!!!
Okay, I think I see a way forward. We need to handle, a netstandard 2.0 implementation that can't load AssemblyLoadContext for some reason.

Sigh !!!!! the real world is so much more complicated than slide decks.

Thanks for the heads up @auduchinok

Kevin

[cartermp]

@auduchinok now that #8722 is in, does this resolve the issue you're seeing?

[cartermp]

oh, derp. I can just run the test

[cartermp]

So a huge amount of tests now fail with this on my machine:

  Message: 
    System.TypeInitializationException : The type initializer for '<StartupCode$FSharp-Compiler-Service-Tests>.$Tests.Service.ProjectAnalysisTests' threw an exception.
      ----> System.ArgumentException : The index was outside the range of elements in the list. (Parameter 'n')
  Stack Trace: 
    Project1.get_options()
    ProjectAnalysisTests.Test project1 all uses of all symbols() line 414
    --ArgumentException
    PrivateListHelpers.nth[a](FSharpList`1 l, Int32 n)
    ProjectAnalysisTests.cctor() line 4569

[cartermp]

@KevinRansom methinks something is off

[cartermp]

Noticed when on this branch: #9389

[auduchinok]

@auduchinok now that #8722 is in, does this resolve the issue you're seeing?

Sorry, missed it somehow.

So a huge amount of tests now fail with this on my machine:

Yes, this is one of exceptions I was seeing too.

[KevinRansom]

@cartermp , What do i need to run to see these test failures?

[cartermp]

Just run the tests in teh FCS solution in visual studio and you'll see it.

[KevinRansom]

I tried: fcs\build.cmd TestAndNuget

It passed:

---------------------------------------------------------------------
Build Time Report
---------------------------------------------------------------------
Target         Duration
------         --------
Start          00:00:00.0000247
Restore        00:00:14.9996283
Build          00:01:58.4907343
NuGet          00:01:02.3201396
Test           00:05:41.5850583
TestAndNuGet   00:00:00.0000985
Total:         00:08:57.4458058
Status:        Ok
---------------------------------------------------------------------
Hint: Could not find a version in your paket.dependencies file, consider adding 'version 5.226.0' at the top of your dependencies file (c:\kevinransom\fsharp\fcs\paket.dependencies).
Read https://github.com/fsharp/FAKE/issues/2193 for details.
Hint: The fake-runner has not been updated for at least 6 months. Please consider upgrading to get latest bugfixes, improved suggestions and F# features.
Performance:
 - Cli parsing: 278 milliseconds
 - Packages: 57 milliseconds
   - Creating Runtime Graph: 43 milliseconds
   - Retrieve Assembly List: 1 second
 - Script compiling: 3 seconds
 - Script analyzing: 558 milliseconds
 - Script running: 9 minutes, 49 seconds
 - Script cleanup: 6 milliseconds
 - Runtime: 9 minutes, 55 seconds

[cartermp]

Right, but what about when you run the tests in VS?