Code coverage

[KtorZ]

What is your idea? Provide a use case.

Leverage the trace system of the Plutus Virtual Machine to capture which expressions are being evaluated by the machine. On large programs, this can possibly be quite expensive so there's a fair amount of uncertainty and feasibility study on this.

A code coverage summary should be shown in a human-readable and machine-readable ways, so that it can also be used in tools such as CI workflows, while also remaining useful during development.

Depending on how heavy is the process, we could even imagine integrating some of the results directly into the LSP protocol to provide a more direct feedback during development.

A good solution should:

• Provide an accurate coverage report of what expressions are evaluated across all tests executions.
• A mechanism to automate the failure of a CI workflow in case of low coverage.
• A way to run coverage in isolation (for a set of test, piggypacking on the existing test filter system).

Why is it a good idea?

We currently provides integrated testing in the form of unit tests and property tests. These are great, but heavily relies on one's ability to identify gaps in the testing coverage. Code coverage analysis is something pretty common as a way to identify those gaps.

What is the current alternative and why is it not good enough?

There's no alternative at the moment, other than building this mental model yourself. Which is perfectly inconvenient because it comes with bias and extra cognitive load.