1187 add support for xplat vstest console in translationlayer

[rouke-broersma]

Description

With this PR it will be possible to use the translationlayer with vstest.console.dll instead of only with vstest.console.exe. I have also removed all test logic from the unit tests that determined vstest.console.dll was not supported. I have successfully used the translationlayer dll resulting from this change to start unit tests against a dotnet core project using vstest.console.dll from the portable nuget package.

However, I have not been able to get all unit tests working yet. The vstest.console.dll used in the acceptance tests seems to be invalid (compared to the one in the portable nuget package). I get this error message:

PS U:\> dotnet C:\git\vstest\src\package\package\bin\Debug\netcoreapp2.0\vstest.console.dll /parentprocessid:28400 /port
:63362 /diag:"C:\Users\roukeb\AppData\Local\Temp\VSTestConsoleWrapperLogs\tmp1C9F.tmp";tracelevel=Verbose
A fatal error was encountered. The library 'hostpolicy.dll' required to execute the application was not found in 'C:\git\vstest\src\package\package\bin\Debug\netcoreapp2.0\'.
Failed to run as a self-contained app. If this should be a framework-dependent app, add the C:\git\vstest\src\package\package\bin\Debug\netcoreapp2.0\vstest.console.runtimeconfig.json file specifying the appropriate framework.
tracelevel=Verbose : The term 'tracelevel=Verbose' is not recognized as the name of a cmdlet, function, script file, or
 operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try a
gain.
At line:1 char:200
+ ... a\Local\Temp\VSTestConsoleWrapperLogs\tmp1C9F.tmp";tracelevel=Verbose
+                                                        ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (tracelevel=Verbose:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

I would appreciate assistance with fixing this before this PR can be merged.

Related issue

Fixes #1887

[msftclas]

All CLA requirements met.

[mayankbansal018]

A bunch of tests are failing because the number of processes(dotnet) being spawned is changing, can you please give me access to your branch. I'll help you fix these tests.

[rouke-broersma]

A bunch of tests are failing because the number of processes(dotnet) being spawned is changing, can you please give me access to your branch. I'll help you fix these tests.

I believe I created this PR with change permissions for reviewers enabled, so you should have access to my branch.

[singhsarab]

    /// <summary>

Fix the docs as well.

---

Refers to: test/Microsoft.TestPlatform.TestUtilities/IntegrationTestBase.cs:365 in cda8bf9. [](commit_id = cda8bf9, deletion_comment = False)

[singhsarab]

Would have loved to see some unit tests for this class.

[rouke-broersma]

Is there anything more I should do for this PR to be merged?

[mayankbansal018]

@Mobrockers , I'll push the PR, once my colleague has gone through.

[rouke-broersma]

@mayankbansal018 I am seeing some weird results using a nuget package created from this branch. vstest.console.dll fails to start (I see the dotnet.exe process for vstest.console.dll start up then immediately shut down) when connecting from the nuget package (uploaded to my own myget feed). When I was testing using the dll's itself referenced in my test project it worked as expected.

I have the packages Microsoft.TestPlatform.TranslationLayer and Microsoft.TestPlatform.ObjectModel which contain the changes from this branch uploaded here: https://www.myget.org/feed/Packages/stryker-mutator-integrationtest

They do not work in my POC. When the dll's on my local system are referenced as assemblies through visual studio it does work. Any idea why?

[mayankbansal018]

@Mobrockers can you make sure you have dotnet core 2.0 installed when you are trying? Also I could be access that feed.

[rouke-broersma]

@mayankbansal018 I have dotnet core 2.0 installed. I believe you should be able to access that feed, as it is public.
The v3 feed url is: https://www.myget.org/F/stryker-mutator-integrationtest/api/v3/index.json
The v2 feed url is: https://www.myget.org/F/stryker-mutator-integrationtest/api/v2

[mayankbansal018]

@Mobrockers can you also share instructions on how can try our your scenario?

[rouke-broersma]

Checkout: https://github.com/stryker-mutator/stryker-net/tree/183-vstest-integration
Open: https://github.com/stryker-mutator/stryker-net/blob/183-vstest-integration/src/Stryker.CLI/Stryker.CLI.sln
Restore project using: https://www.myget.org/F/stryker-mutator-integrationtest/api/v3/index.json (this contains the nuget package from this pull request)
Add this launchSettings.json in visual studio:

{
  "profiles": {
    "vstest": {
      "commandName": "Project",
      "commandLineArgs": "-p \"ExampleProject.csproj\" -r \"['All']\" -f -tr vstest",
      "workingDirectory": "<REPO_LOCATION>stryker-net\\integrationtests\\IntegrationTests\\ExampleProject.XUnit"
    }
  }
}

Now if you start a debug run, my project should start using vstest.console.dll and it will not work.

If I then do the following, it does work for me
Checkout: https://github.com/Mobrockers/vstest/tree/1187_add_support_for_xpat_vstest_console_in_translationlayer
Build vstest from above branch^
Remove nuget references from Stryker.Core.csproj:

<PackageReference Include="Microsoft.TestPlatform.ObjectModel" Version="16.0.0-dev" />
<PackageReference Include="Microsoft.TestPlatform.TranslationLayer" Version="16.0.0-dev" />

Add assembly references to Microsoft.TestPlatform.ObjectModel.dll and Microsoft.TestPlatform.TranslationLayer.dll built locally from this PR.

Now it vstest.console.dll successfully starts up and runs tests on my machine.

[mayankbansal018]

@Mobrockers can you please drop a mail to maban at microsoft dot com. I'll set up a meeting with you to diagnose this issue.
Please do let me know what time zone you work in, & what will be your preferred work timings. I work in India Time Zone

[rouke-broersma]

@mayankbansal018 Is it possible to give an indication of when a new (preview) nuget package can be release with these changes?

[mayankbansal018]

@Mobrockers we recently release a package last week itself, so most likely our next release will be 2 weeks from now.

[rouke-broersma]

@mayankbansal018 Thank you for your help :)