Selenium web tests: in-process hosting of ASP.NET Core web application

[drauch]

For our Selenium web test suite we are trying to host our ASP.NET Core 1.1 web application (targetting net462) using in-process hosting with the TestServer. Both projects reside in the same solution.

We encounter two problems when doing so, and I have the feeling those are related.
In our NUnit3 setup fixture we try to host the page using the following code:

_server = new TestServer(
    new WebHostBuilder()
        .UseKestrel()
        .UseContentRoot(@"C:\Development\MyProject\MyProject.WebClient")
        .UseIISIntegration() // also tried it without this line
        .UseEnvironment("WebTests")
       .UseStartup<Startup>());

1. server.BaseAddress is http://localhost -> when I surf to that location it does not show the web application, instead it shows the IIS default page. Which is somewhat "right", because I have a local full-fledged IIS server running on port 80 ... don't know how this is support to work ...

2. Interestingly enough, server.CreateRequest("/").GetAsync().Result does not return the IIS default page but instead results in an exception:

   OneTimeSetUp: System.AggregateException : One or more errors occurred.
   ----> Microsoft.AspNetCore.Mvc.Razor.Compilation.CompilationFailedException : One or more compilation failures occurred:
   [...]

Which looks a lot like Razor isn't able to compile the view due to missing assemblies (the web test project references the web app project using a project reference, which looks like it isn't enough to pull the required assemblies to compile razor views?!

What am I doing wrong? How to use in-process hosting to run my web tests? I tried to find something in the docs @ https://docs.microsoft.com/en-us/aspnet/core/testing/integration-testing but no hit...

[Tratcher]

TestServer is an in memory server only, it does not open any ports accessible from a web browser. It's designed for use in unit tests.

There are a few other issues about geting Razor to work with TestServer.

For Selenium testing why not just drop the TestServer and start Kestrel normally?

[drauch]

Okay, I tried to run the web server with Kestrel now:

_host = new WebHostBuilder()
      .UseKestrel()
      .UseContentRoot(@"C:\Development\MyProject\MyProject.WebClient")
      .UseEnvironment("WebTests")
      .UseStartup<Startup>()
      .Build();
_host.Start();
_baseAddress = _host.ServerFeatures.Get<IServerAddressesFeature>().Addresses.Single();

However, on _host.Run() it throws a DllNotFoundException (Unable to load libuv...). What am I doing wrong now? :-)

Edit: indeed the libuv.dll resides in the output of MyProject.WebClient but not in the output of MyProject.WebClient.WebTests - do I have to reference some special NuGet package for libuv?

[drauch]

Second problem: even when manually copying the libuv.dll to the output of the WebTests project, the test starts to run, but the Kestrel server never answers to any request. Is it important to run it on a different thread?

[drauch]

Here is a repro solution:
https://www.dropbox.com/s/3ps6o98qtijanu8/InProcKestrel.zip?dl=0

• Change directory in UseContentRoot() (MyTestFixture.cs:21) to point to the web application's main directory
• Recompile everything and copy libuv.dll from web app's output to webtests' output (why is this necessary?!)
• Execute and observe that the unit test (MyTest) receives an empty page instead of the web application's body. Observe also that while the server is open (e.g. breakpoint in MyTest), you are not able to request something via any browser.

Info 1: note, that although using the new project format, I have to target net47, otherwise we wouldn't even be able to use Selenium)

Info 2: I set the platform to x86, seems to be the default for the web app and wanted to make sure that the webtests project targets the same platform.

[Tratcher]

@moozzyk @halter73 for the libuv issue.

[khellang]

I imagine the libuv issue is the same as all the others. Just do a GitHub search for org:aspnet libuv and see all the existing issues. I think there are 4-5 of them scattered around the different repos.

[drauch]

@khellang : I found the following workaround:

<RuntimeIdentifier>win7-x86</RuntimeIdentifier>
<BaseNuGetRuntimeIdentifier>win7-x86</BaseNuGetRuntimeIdentifier>

Why is this necessary? Is this going to get fixed? What are those two settings doing to my project?

[drauch]

I have the libuv workarounded now + I am using the developer error page. My problem now is that Kestrel can't compile the Razor views from the other project:

/Views/Home/Index.cshtml
One or more compilation references are missing. Possible causes include a missing 
'preserveCompilationContext' property under 'buildOptions' in the application's project.json.

The type or namespace name 'System' could not be found (are you missing a using directive or an assembly reference?)
[...]

I don't even have a project.json. How to overcome this problem?

Edit: I found dotnet/sdk#958, however, it doesn't help. I switched my test project to reference the Sdk="Microsoft.NET.Sdk.Web" and <OutputType>Library</OutputType> and then run my tests again -> same error as above. Even explicitly setting <PreserveCompilationContext>true</PreserveCompilationContext> doesn't help.

[Tratcher]

See #954 for view discovery.

[Tratcher]

What _baseAddress do you get back?

[moozzyk]

@drauch - libuv is native code. As such you have to use the version that matches the processor architecture your application runs on.

[drauch]

@Tratcher : http://localhost:5000/

Finally got it working, I had to use #959 (comment) which uses the code from https://github.com/aspnet/Mvc/blob/66deaf0b3fb935027730cf470cf6a331e69f2cee/test/Microsoft.AspNetCore.Mvc.FunctionalTests/Microsoft.AspNetCore.Mvc.FunctionalTests.csproj#L79-L85 to fix the Razor issue. It seems better than to manually try to load some assemblies.

If anybody else wants to wait for a "real" fix, follow the aspnet/Mvc#6233 issue.

Thank you everybody for your time!

[giantryansaul]

@drauch the workaround you linked to + the instantiation code from above was able to get my instance working with Selenium as well. Hopefully others can find this page, the Kestral error was not immediately obvious to me in VS2017. Thanks!

[CurlyFire]

@drauch I was able to make this work with a core 2.0 test library (Microsoft.NET.Sdk) targeting net70, however when I try the exact same thing with a full framework 4.7 old style csproj, I am getting these errors:

Microsoft.AspNetCore.Mvc.Razor.Compilation.CompilationFailedException: One or more compilation failures occurred:
5quktzhs.dfw(4,62): error CS0012: The type 'Attribute' is defined in an assembly that is not referenced. You must add a reference to assembly 'netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.
5quktzhs.dfw(4,81): error CS0518: Predefined type 'System.String' is not defined or imported
5quktzhs.dfw(4,112): error CS0518: Predefined type 'System.Type' is not defined or imported
5quktzhs.dfw(4,11): error CS0518: Predefined type 'System.Void' is not defined or imported
5quktzhs.dfw(8,11): error CS0246: The type or namespace name 'System' could not be found (are you missing a using directive or an assembly reference?)
5quktzhs.dfw(9,11): error CS0246: The type or namespace name 'System' could not be found (are you missing a using directive or an assembly reference?)
5quktzhs.dfw(10,11): error CS0246: The type or namespace name 'System' could not be found (are you missing a using directive or an assembly reference?)
...

I am unsure if the fix for the razor issue you posted is compatible with the old style csproj format. Any ideas ?

[drauch]

It certainly looks like either your deps file or your refs folder is not in the build output. Are you sure you have added the copy-deps-file task? Are you sure you have shadow copying disabled?

[CurlyFire]

I copied the same fixes in the old style csproj. I ran the test from the test explorer, and from the console with nunit3-console (by default, it does not shadow copy), and I still have the same problem.

I copied your InProcKestrel solution, fixed it with the fixes in this post and made it work. I then added the OldProjectFormat as a full framework 4.7 project, made the same fixes, and I still have the same problem, even in your solution. I uploaded it to github at https://github.com/CurlyFire/InProcKestrel

Can you have a peak ?

Thanks

[CurlyFire]

I think I figured out what was missing. I compared the bin folders of your test project dans my old format project, and the refs folder was missing from the old format bin folder. I started it up and everything worked. However, I don't know why it's not copied...

[drauch]

I have opened your solution with VS2017 and ReSharper installed. I could execute your test in the InProcKestrel.WebTests project without any problems. I even debugged it to look if the page renders correctly, no problem at all. The old project didn't even compile.

Note: In my solution above I'm not using the old project format! I'm using the new project format, I just switched the TargetFramework manually from .NET Core to .NET 4.7!

I don't know anything about how to run it using the old project format, sorry!

[CurlyFire]

Allright thanks. You probably don't have the full framework 4.7 installed...

[CurlyFire]

I isolated what fix copies the refs folder to the output folder of the testproject. It seems it's true

I'm not sure why it's not working for the old style csproj project... Gonna keep digging

[CurlyFire]

Was explained in aspnet/Razor#2146