NUnit v2 Test Adapter to support .runsettings file

[Erezinho]

Hi
While working with a 3rd party tool which relies on NUnit 2, and running night builds with TFS vNext 2017, we need a way to pass on some parameters to NUnit.
As I read, with NUnit 3 adapter it is easy since it supports .runsettings.

Any possibilty to add this capability in the (very) near future?
Thanks!!

[OsirisTerje]

Hi! We will be adding some of the parameters from the runsettings to this adapter, but had really hoped people could migrate to NUnit3.
I would like to ask you some questions:

1. What is the reason the 3rd party tool can't go to NUnit3? Can you elaborate a bit on how you use it, and what prevents it? Also, what 3rd party tool is this?
2. What parameters do you need to send to NUnit? Note that NUnit2 doesnt support the same number of parameters that NUnit3 does.

[rprouse]

Would it not be more appropriate to ask the 3rdParty tool to start supporting NUnit 3 which has been out for 4 years now? Just asking 😄

[CharliePoole]

@Erezinho Same question as @rprouse plus "How do you run your tests now?" 😃

[Erezinho]

Hi All,

1. The 3rd party tool is Difido-reports and a short discussion about shifting to NUnit3 has already been taken.
   Please see here: https://stackoverflow.com/questions/38749270/how-to-use-itesteventlistener-in-nunit-3
   Difido listens to NUnit’s events and log some data internally (elastic) to be displayed for end users.

2. We run our tests mostly from TFS vNext Build system and sometimes (during R&D) manually and directly from command line.

3. We need the .runsettings capability with NUnit2 for TFS vNext Builds.
   The parameters that should be passed are:
   a) /runlist
   b) /results
   c) /output
   d) /err
   e) /work

BTW,
I wish Difido would have already supported NUnit3 as there are several capabilities we really want to use but it is unclear when it will happen.
With no real solution we might eventually do the migration on our local copy of the source
Nevertheless, we are currently a bit blocked.

Thank you!

[OsirisTerje]

3. Sorry for confusing you, but what we need to consider this is which runsettings parameters you need. We can't implement them all, and the NUnit2 doesn't even support all possibilities. If unsure, please see the template here: https://marketplace.visualstudio.com/items?itemName=OsirisTerje.Runsettings-19151 . (The settings we currently support is those that are in the registry settings described here under section for NUnit2: https://github.com/nunit/docs/wiki/Tips-And-Tricks)

[Erezinho]

I read the Tips and Tricks and some more and trying to understand:
The adapter which basically allows execution of NUnit from VS may use .runsettings to load different configurations.
The .runsettings holds, among others, some configurations used to be stored in registry but not anymore .
Now, can the .runsettings have some NUnit command line parameters like ‘runlist’ (under for <!—Nunit -->) which the adapter can read and pass on when executing NUnit ?

I hope I’ve not got this all wrong….

[CharliePoole]

The .runsettings file is used by the NUnit 3 adapter, where we basically took some of the command-line arguments from nunit3-console and defined settings in the file. Many settings are not there, however, because they don't make sense in the VS environment. You can see a sample .runsettings file that lists the supported values for that adapter here: https://github.com/nunit/nunit3-vs-adapter/blob/master/demo/demo.runsettings

For the V2 adapter, we would need to do the same thing: (1) examine nunit-console options and determine which ones make sense to emulate under VS, (2) implement code to get the settings from the file and (3) implement code to instruct the V2 core to use those settings.

Step 1 is crucial. Some console settings, like --process, --domain, --result, etc. don't make sense for the adapter. However, you could create a "wish-list" here and one of us can react to it as a next step forward.

[Erezinho]

Thanks for the clarifications.

The console settings that would be very useful to have with the adapter are:

1. --result (/result) . I saw you wrote it does not make sense but I'll try to explain the motivation: Since the adapter is used with TFS build system, using this parameter in the runsettings will allow you to get the results directly (nunit's raw data) and not to relay on VS to show it or TFS to process it.
2. --testlist (/runlist) - will give much more flexibility on which tests to run with TFS (instead of complex category phrase, on demand tests by QA)

Thanks.

[CharliePoole]

I indicated --result doesn't make sense because it tells us where to save the XML output, which is not generated by the adapter. It would make sense if we implemented the XML output file for the adapter. The reason we never did that was that I assumed folks using VS would use the VS output.