UWP Test Runner

[kristiandupont]

In order to run our tests on UWP platforms, we need to create a test runner similar to the ones we have for Android and iOS.

TODO:

• Fully functional test runner: Changes required for Realm runner nirinchev/nunit.xamarin#1
• iOS runner: Use the nunit3 test runner for iOS #1338
• Android runner: Use the nunit3 test runner for Android #1355
• UWP runner: Add UWP test runner #1358
• NuGetReleaseTests for iOS: Add Manual Nuget runners #1370
• NuGetReleaseTests for Android: Add Manual Nuget runners #1370
• NuGetReleaseTests for UWP: Add Manual Nuget runners #1370
• NuGetReleaseTests for Win32: Add Manual Nuget runners #1370

[AndyDentFree]

Maybe we can just use integrated testing as noted here with workarounds for bait-and-switch?

That doesn't give us an on-phone runner but maybe good enough and would also be worth doing as a separate article for people who want to run their pure PCL unit tests.

[kristiandupont]

But that would not exercise the UWP build of the library? Unless with "workarounds" you have mean some trick that I don't know of?

[AndyDentFree]

Unless needed before Mar 7 am pushing this back on the stack so I can update to VS2017. I have had private feedback from MS that the latest VS 2017 RC fixes the clash which wipes out VS2015 Xamarin but don't want to go through installing it then the release one a few days later.

Note that you cannot currently build Win32 with VS2015.

[AndyDentFree]

Task waiting to merge with .NET Standard library layout.

Note that for future Windows Phone 8.1 testing, if ever done, will need to use VS2015 to build projects due to lack of support in VS2017.

Two sample test runners uploaded see VS2017 readme

[AndyDentFree]

Status of this effort, sadly incomplete.

A runner using NUnit3 has been added with two additional projects, one targeting UWP and a parallel Tests.XamarinIOSn3 to allow continued use of the existing iOS tester whilst debugging the new runner.

The latter was added to be able to separate NUnit runner issues from the issues in running under UWP. It works, with a notable difference in that a few more tests fail than normal.

The intent was to enhance the existing Nunit runner, from the nunit.xamarin project with test browsing and other features to make it a good replacement for the current iOS and Android test runner. To enable this, rather than using a nuget to pull the runner in, it is currently included directly in the realm.sln.

Problems with UWP

Currently the runner crashes early with UWP builds both on desktop and mobile ARM device.

These problems are due to pulling in Realm - a simple test without Realm was fine.