Define console / engine runtime platform support

[CharliePoole]

Platform support from Microsoft changes rapidly, so we need to look at this periodically. I think it's time to do it again, since .NET 7.0 went out of support this month and 6.0 has six months left.

It might be helpful to have a statement that describes our support, to include

• What runtime are needed to run the console
• What runtimes we support for execution of tests.
• What agents we supply - this is different from the preceding point since, for example, a .NET 8.0 agent can run .NET 5.0 tests.
• If possible, how we will change the above in the future as Microsoft drops support and adds new runtimes.

NOTE: The above may differ between versions 3 and 4.

I'll start working on a draft document to include in the root

@OsirisTerje @manfred-brands @anyone Feel free to comment right away and/or after there is a doc to review

[manfred-brands]

I know you just did work to fix it, but are there still users of X86?
Any small project more likely use ARM processors and Linux.

We need to support as a minimum the Microsoft supported .NET versions.
At the moment that would be .NET Framework 4.6.2 and later and NET6.0, NET8.0
If this can be supported by only having an .NET8.0 runner then we only ever need the latest framework.
NUnit itself is .NET6.0, but I have a NET8.0 version to see if I can support TimeoutAttribute there.

For existing releases (3x), there is no need to remove what is supported.
For new releases (4x), drop things if we want to use new functionality that is not supported in older releases.
Although as we still need to support .NET Framework that will be the common denominator unless we have different versions/code bases.

You told me once that you are in favour of keeping support for old versions.
If it is not a security risk and doesn't take effort then OK.
One the other hand it will blow up package sizes with multiple versions.

[CharliePoole]

@manfred-brands Thanks for the comments!

Re X86: Well, I fixed it due to a bug report. There are a number of other bug reports, some of which may be fixed by what I've done. Microsoft continues to support it with 8.0. I believe most folks who need X86 are on 64 bit processors but have dependencies on packages that require X86. Microsoft continues to deliver X86 builds of the runtime with 8.0.

MS Supported Versions: I agree. I think that support in this case should mean full support, i.e. delivering an agent that runs tests under those versions. In the case of Framework, of course, the tests will actually run on whatever level of 4.x is installed on the machine and that is usually going to be 4.8. As far as running tests goes, one .NET Core version should run everything, maybe even back as far as .NET Core 1.1/ OTOH, for some fields (e.g. aero and medical) critical software may need to be tested under the exact runtime to be used in order to be certified.

3X vs 4X: Even in 3X, we might want to remove any code of ours which is marked as insecure, rather than repressing the error. What I have found is that new releases of VS start giving warnings or errors for things that could be built on old versions. We can suppress out of support warnings but security notifications may be another matter except possibly for things like mock-assembly, which we use as test data rather than part of the application.

After I wrote this issue, I realized that it somewhat interacts with the question of pluggable agents - #909. That is, if we allow agents to be installed as extensions, that gives an easy way to support old versions even after we archive the code for that agent. Of course, the pluggable agent feature, requires a greater degree of isolation between engine code and agent code than we now have, but that might be a very good thing anyway. I'll try to deal with that as part of my writeup, i.e., v4 with vs v4 without pluggable agents.

As for my own view of maintaining backward compatibility, that was part of the original project vision, circa 2000. We disliked Microsoft's aggressive end-of-life practices and wanted to do better for working developers, who were often unable to upgrade as often as MS would like. The degree and manner in which backward compatibility is provided by NUnit is really up to the present-day project members, but I think I can help by drafting something.

[OsirisTerje]

About a Lifecycle policy, we had this discussion on the framework: nunit/nunit#4424

The current suggestion is to keep supporting LTS 6 months longer than MS.
and

only do the minimal wrt to STS to make tests run

The reasoning

is that companies and developers that aren't keeping their projects up to date aren't updating their NuGet packages either.

I believe more companies are now understanding the need to keep their products maintained and upgrading accordingly than they did earlier.

We are also doing [some] backporting of issues to the 3.X version series. which do support older frameworks.

[CharliePoole]

Thanks @OsirisTerje , this is very helpful. I'll try a similar approach for the console / engine itself.

[CharliePoole]

Blocking work on #1427 and #1428