Skip to content
This repository has been archived by the owner on Dec 14, 2018. It is now read-only.

Re-enable functional tests on full .NET Framework #5873

Closed
dougbu opened this issue Mar 1, 2017 · 6 comments
Closed

Re-enable functional tests on full .NET Framework #5873

dougbu opened this issue Mar 1, 2017 · 6 comments
Assignees
@dougbu
Labels
3 - Done task
Milestone
2.0.0-preview1

Comments

@dougbu
Copy link
Member

dougbu commented Mar 1, 2017

These tests fail on AppVeyor and some (but not all) other Windows machines. See https://ci.appveyor.com/project/aspnetci/mvc/build/1.0.8116 for example.

Innermost stack trace is always some variant of

Error Message:
 System.MissingMethodException : Method not found: 'System.Collections.Immutable.ImmutableArray`1<Byte> System.Reflection.Metadata.MetadataReader.GetBlobContent(System.Reflection.Metadata.BlobHandle)'.
Stack Trace:
   at Microsoft.CodeAnalysis.MetadataReaderExtensions.CreateAssemblyIdentityOrThrow(MetadataReader reader, Version version, AssemblyFlags flags, BlobHandle publicKey, StringHandle name, StringHandle culture, Boolean isReference)
   at Microsoft.CodeAnalysis.MetadataReaderExtensions.ReadAssemblyIdentityOrThrow(MetadataReader reader)
   at Microsoft.CodeAnalysis.PEAssembly..ctor(AssemblyMetadata owner, ImmutableArray`1 modules)
   at Microsoft.CodeAnalysis.AssemblyMetadata.GetOrCreateData()
   at Microsoft.CodeAnalysis.AssemblyMetadata.GetModules()
   at Microsoft.CodeAnalysis.AssemblyMetadata.IsValidAssembly()
   at Microsoft.CodeAnalysis.CommonReferenceManager`2.GetMetadata(PortableExecutableReference peReference, CommonMessageProvider messageProvider, Location location, DiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CommonReferenceManager`2.ResolveMetadataReferences(TCompilation compilation, Dictionary`2 assemblyReferencesBySimpleName, ImmutableArray`1& references, IDictionary`2& boundReferenceDirectiveMap, ImmutableArray`1& boundReferenceDirectives, ImmutableArray`1& assemblies, ImmutableArray`1& modules, DiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.ReferenceManager.CreateAndSetSourceAssemblyFullBind(CSharpCompilation compilation)
   at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.ReferenceManager.CreateSourceAssemblyForCompilation(CSharpCompilation compilation)
   at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.CommonGetTypeByMetadataName(String metadataName)
   at Microsoft.CodeAnalysis.Razor.TagHelperTypeVisitor.Create(Compilation compilation, List`1 results)
   at Microsoft.CodeAnalysis.Razor.TagHelpers.VisitTagHelpers(Compilation compilation, List`1 results, ErrorSink errors)
   at Microsoft.CodeAnalysis.Razor.TagHelpers.GetTagHelpers(Compilation compilation)
   at Microsoft.CodeAnalysis.Razor.DefaultTagHelperFeature.InnerResolver.Resolve(IList`1 errors)
   at Microsoft.AspNetCore.Razor.Evolution.TagHelperBinderSyntaxTreePass.Execute(RazorCodeDocument codeDocument, RazorSyntaxTree syntaxTree)
   at Microsoft.AspNetCore.Razor.Evolution.DefaultRazorSyntaxTreePhase.ExecuteCore(RazorCodeDocument codeDocument)
   at Microsoft.AspNetCore.Razor.Evolution.RazorEnginePhaseBase.Execute(RazorCodeDocument codeDocument)
   at Microsoft.AspNetCore.Razor.Evolution.DefaultRazorEngine.Process(RazorCodeDocument document)
   at Microsoft.AspNetCore.Mvc.Razor.Internal.RazorCompilationService.ProcessCodeDocument(RazorCodeDocument codeDocument)
   at Microsoft.AspNetCore.Mvc.Razor.Internal.RazorCompilationService.Compile(RelativeFileInfo file)
   at Microsoft.AspNetCore.Mvc.Razor.Internal.CompilerCache.CreateCacheEntry(String relativePath, String normalizedPath, Func`2 compile)
@dougbu
Copy link
Member Author

dougbu commented Mar 1, 2017

This might be a misdirection but…

All machines where tests fail have System.Collections.Immutable in the GAC; it isn't in the GAC where the tests succeed. For example, everything worked on my laptop 'til I did a couple of VS installations today. Even then, tests worked before I rebuilt the Mvc repo.

My registry now contains the following. From the installation logs, VS 2017 added this. Not sure why the GAC addition didn't happen in earlier installations of that product.

Key Name:          HKEY_CLASSES_ROOT\Installer\Assemblies\Global
Class Name:        <NO CLASS>
Last Write Time:   2/28/2017 - 10:52
...
Value 218
  Name:            System.Collections.Immutable,fileVersion="4.6.23123.0",version="1.1.37.0000",culture="neutral",publicKeyToken="B03F5F7F11D50A3A",processorArchitecture="MSIL"
  Type:            REG_MULTI_SZ
  Data:            OV)&OvMsP=.+xLp`A*BdBuildTools_MSBuild_Core_amd64>OFYkSvP,(Dv0'}^46=SQ
                   4V_dJd)v_9O7wvTD8L&6BuildTools_MSBuild_Core_x86>vMYcV@gS(E{xW_SwPofL
...

dougbu pushed a commit that referenced this issue Mar 1, 2017
@NTaylorMullen @dougbu
Migrate to MSBuild
acfad83 
- thanx to @NTaylorMullen for initial conversion
  - e.g. AssemblyInfo.cs files were already minimized or removed :)
- allow `>=` RC3 CLI's to build and run MVC
- work around several dotnet migration issues; see #5482
- disable full .NET Framework runs of functional tests; see #5873
- remove `Microsoft.DotNet.InternalAbstractions` and `System.Xml.XmlDocument` dependencies
- remove project.json (!!), *.xproj, .notest, and web.config files

Redo earlier changes:
- apply test migration to .NET 4.5.2 in *.csproj world
  - see 63507c8 for previous, project.json work
- apply dependency version downgrade from 0097e40 in *.csproj world

Make other test-related changes:
- make Microsoft.AspNetCore.Mvc.TestDiagnosticListener a regular class library
- add support for `/p:GenerateBaselines=true` for functional and Razor.Host tests
- separate `GetCSharpTypeName_ReturnsCorrectTypeNames_ForOutParameter()` test
  - work around inability to deserialize a odd `ref` type
  - xUnit and vstest now serialize / deserialze test data more often
- skip poor test mentioned in #5768
- work around microsoft/vstest#392
  - rename tests to avoid duplicates
- work around microsoft/vstest#419
  - set up created `AppDomain`s with current `ApplicationBase`
@dougbu
Copy link
Member Author

dougbu commented Mar 1, 2017

Doing some initial work on this since I've been debugging it already. Will turn issue over if I run out of ideas or reach the end of the day w/o getting a repro to share with potential culprits.

@dougbu dougbu self-assigned this Mar 1, 2017
dougbu added a commit that referenced this issue Mar 2, 2017
@dougbu
Re-enable functional tests on full .NET Framework
9b3b3e9 
- #5873
- creating an EXE for the test project seems to work around #5873
  - also avoids dotnet/sdk#926 when building in Visual Studio

nit: clean up duplicate test data
@dougbu dougbu mentioned this issue Mar 2, 2017
Merged
@dougbu
Copy link
Member Author

dougbu commented Mar 2, 2017

Worked around with 9b3b3e9. Will close this issue after I've filed a bug wherever the problem appears to be (likely in the SDK) and an MVC issue tracking that to revert 9b3b3e9.

@danroth27 danroth27 added the task label Mar 2, 2017
@danroth27 danroth27 added this to the 2.0.0-preview1 milestone Mar 2, 2017
@dougbu
Copy link
Member Author

dougbu commented Mar 4, 2017

When testing a more limited repro that I can run completely within VS, the test gets further and instead fails with the following. Note the test hits the MissingMethodException with the stack trace captured above when run from the command line.

Message: System.IO.FileLoadException : Could not load file or assembly 'System.Reflection.Metadata, Version=1.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
Result StackTrace:
at Microsoft.AspNetCore.Mvc.Razor.Internal.DefaultRoslynCompilationService.Compile(RelativeFileInfo fileInfo, String compilationContent)
   at Microsoft.AspNetCore.Mvc.Razor.Internal.RazorCompilationService.Compile(RelativeFileInfo file)
   at Microsoft.AspNetCore.Mvc.Razor.Internal.CompilerCache.CreateCacheEntry(String relativePath, String normalizedPath, Func`2 compile)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Mvc.Razor.Internal.CompilerCache.GetOrAdd(String relativePath, Func`2 compile)
   at Microsoft.AspNetCore.Mvc.Razor.Internal.DefaultRazorPageFactoryProvider.CreateFactory(String relativePath)
   at Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.CreateCacheResult(HashSet`1 expirationTokens, String relativePath, Boolean isMainPage)
   at Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.OnCacheMiss(ViewLocationExpanderContext expanderContext, ViewLocationCacheKey cacheKey)
   at Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.LocatePageFromViewLocations(ActionContext actionContext, String pageName, Boolean isMainPage)
   at Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.FindView(ActionContext context, String viewName, Boolean isMainPage)
   at Microsoft.AspNetCore.Mvc.ViewEngines.CompositeViewEngine.FindView(ActionContext context, String viewName, Boolean isMainPage)
   at Microsoft.AspNetCore.Mvc.ViewFeatures.Internal.ViewResultExecutor.FindView(ActionContext actionContext, ViewResult viewResult)
   at Microsoft.AspNetCore.Mvc.ViewResult.<ExecuteResultAsync>d__26.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.<InvokeResultAsync>d__30.MoveNext()

Fusion log from Visual Studio Exception:

=== Pre-bind state information ===
LOG: DisplayName = System.Reflection.Metadata, Version=1.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
 (Fully-specified)
LOG: Appbase = file:///C:/dd/Projects/LoadingWebSiteInTest/Test/bin/Debug/net452/win7-x86
LOG: Initial PrivatePath = NULL
Calling assembly : Microsoft.CodeAnalysis.CSharp, Version=1.3.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35.
===
LOG: This bind starts in default load context.
LOG: Using application configuration file: C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2017\INT\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\TESTWINDOW\vstest.executionengine.x86.exe.Config
LOG: Using host configuration file:
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config.
LOG: Post-policy reference: System.Reflection.Metadata, Version=1.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
LOG: Attempting download of new URL file:///C:/dd/Projects/LoadingWebSiteInTest/Test/bin/Debug/net452/win7-x86/System.Reflection.Metadata.DLL.
WRN: Comparing the assembly name resulted in the mismatch: Minor Version
ERR: Failed to complete setup of assembly (hr = 0x80131040). Probing terminated.

The mentioned copy of System.Reflection.Metadata.dll has [assembly: AssemblyVersion("1.4.1.0")].
Microsoft.CodeAnalysis.CSharp.dll is in the same folder, has version 1.3.0.0, and does reference version
1.2.0.0 of System.Reflection.Metadata.dll.

The Test assembly references WebSite.exe. WebSite.exe.config contains

<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
  <dependentAssembly>
    <assemblyIdentity name="System.Reflection.Metadata" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
    <bindingRedirect oldVersion="0.0.0.0-1.4.1.0" newVersion="1.4.1.0" />
  </dependentAssembly>
</assemblyBinding>

vstest.executionengine.x86.exe.Config contains nothing similar (of course). Bottom line, the WebSite's
configuration isn't being used.

@dougbu
Copy link
Member Author

dougbu commented Mar 5, 2017

This problem is a variant of microsoft/vstest#428. Main oddity is binding redirects are not needed unless System.Collections.Immutable.dll is in the GAC.

Smaller repo

https://github.com/dougbu/XunitTestImmutable

Includes a test that hits a MissingMethodException earlier than either of the above stack traces. But, it's all the same problem. And, problem goes away with

<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
<GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>

(Note tests in that repro succeed. Need to back up one commit to observe the issue because the above lines are in the project file.)

Details

The copy of Microsoft.CodeAnalysis.CSharp.dll (v1.3.0) in the NuGetPackages cache depends on

<dependency id="System.Collections.Immutable" version="1.1.37" />
<dependency id="System.Reflection.Metadata" version="1.2.0" />

Everything is fine as long as nothing depends on a newer copy of either assembly because the GAC contains System.Collections.Immutable.dll v1.1.37. As soon as something depends on a newer copy e.g. v1.3.1 (also in the cache), the test needs binding redirects and those aren't added automatically. Failures also start when something depends on a newer copy of System.Reflection.Metadata.dll e.g. v1.4.1 because that depends on System.Collections.Immutable.dll v1.3.0.

dougbu added a commit that referenced this issue Mar 5, 2017
@dougbu
Use an alternate workaround for #5873
4c23d94 
- use specific workaround for microsoft/vstest#428
  - generating an executable for a test project is a bit too weird
- also work around dotnet/sdk#926
  - generating an executable covered this as well

nit: add and update comments about other workarounds in the functional tests project
@NTaylorMullen NTaylorMullen mentioned this issue Mar 5, 2017
Closed
55 tasks
dougbu added a commit that referenced this issue Mar 6, 2017
@dougbu
Use an alternate workaround for #5873
5c96a7c 
- use specific workaround for microsoft/vstest#428
  - generating an executable for a test project is a bit too weird
- also work around dotnet/sdk#926
  - generating an executable covered this as well

nit: add and update comments about other workarounds in the functional tests project
@dougbu
Copy link
Member Author

dougbu commented Mar 6, 2017

Improved workaround with 5c96a7c. Will track removing this workaround (once microsoft/vstest#428 is fixed or worked around elsewhere) with other bullets in #5482.

@dougbu dougbu closed this as completed Mar 6, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@dougbu dougbu

Labels
3 - Done task
Projects
None yet
Milestone
2.0.0-preview1
Development

No branches or pull requests
2 participants
@danroth27 @dougbu