High-Level Goals of the V4 Release

[CharliePoole]

All the information about how V4 was planned in the past is available in GitHub issues, but it's hard to find and even harder to evaluate. Things we discussed and finally decided not to do are mixed with important features we implemented. As a start towards re-booting V4 development, I'm posting this list of what I believe to be the important high-level goals. Clearly, their implementation will involve many features, enhancements and fixes, probably closer to 60 or 80 than the six items listed here.

The first three of the following high-level goals for V4 were initially created by Chris Maddock in 2020. I’m quoting his words from issue #752. I’ve added three others, which are slightly lower in level, but important enough to be considered goals of the release.

1. Test Anywhere

Since NUnit 3.0 was released, the .NET scene has evolved from being largely Windows focused, to running across a whole bunch of different platforms. Currently users often need to set up large amounts of boilerplate code to test across these platforms - the next generation engine should provide that functionality directly to the user. The engine should also be suitably extensible, to allow users to run tests on less-common platforms that the project itself isn't resourced to support.

2. Improving the Developer Experience

NUnit has a long, proud history and a big user-base, utilizing a range of technologies from the last 20 years of .NET development. This is not without it's cost - we have a large codebase still restricted to building for legacy platforms, and a drawn-out and manual release process, which impacts our ability to move quickly.

For v4, I'd like to see us improve the experience for developers of the engine and console. This would cover things like moving as much code as possible to modern platforms, reducing our reliance on manual testing and release processes, and considering code complexity as a factor against feature value and development.

3. Clarity of supported interfaces, and protection for the future

A big cost of on-going development, particularly in the framework, is handling code which has been exposed to end users, but was never intended to be as supported interface. The team are often in a position of 'guessing' how user's may be using a class to decide on an appropriate change - which delays and confuses external contributions, and often unintentionally breaks end-users' setups.
For v4, I'd like for us to consider locking everything down as far as possible, and only exposing guaranteed interfaces, intended for ongoing support. Docs are great too - but locking things down prevents people making the mistake right from the start.

4. Simplify the Console Runner User Interface

Users should not need to control all the internal details of how tests are run. To an extent, this simplification has occurred already in version 3. if the user is able to take the default settings, things generally just work. However, we encounter many issues in which the user, faced by something that isn’t working correctly, tries a random collection of optional settings, often making the problem worse.

Moving to an agent-based model, where the agent is automatically selected based on the target runtime framework, will solve most of these problems. Manual agent selection through a command-line option will provide for exceptional cases. Agents will themselves make the necessary choices about how the test is run and the user will not need to specify additional command-line options.

5. Independence of Agents from the Engine

The code for the engine should be entirely separate from the code for agents, just as the runner code is separate from the engine. Ideally, agents should be plugins, so that users can continue to use agents for older runtimes even if we are no longer producing new releases of such agents.

Facilities intended for use by multiple runners should be published separately so we can eliminate duplication that arises by implementing them separately.

6. Increased Availability of Extensibility

The extension points available within the engine is expected to grow and the console runner itself will become extensible. As a side effect, it will become possible for other runners, such as NUnitLite and the VS Adapter, to support extensibility themselves.

---

Hopefully, this helps to provide everyone with the "Big Picture" of V4. Questions, comments, suggestions are welcome.