end-to-end test generated coverage profile

[qweeah]

What is the version of your ORAS CLI

1.0.0

What would you like to be added?

Run end-to-end tests based on instrumented application and generate coverage report accordingly.

The released application will still not be instrumented.

Why is this needed for ORAS?

Currently oras' code coverage statistics are generated by unit testing, which is not accurate since:

1. By design, unit tests coverage report will only include files within the package. E.g if package a uses package b and a have unit test specs in a_test.go, unit test coverage report will not include package b.
2. If a package is not included, Codecov report will skip the line counting, which means some uncovered code may be silently ignored. E.g. oras.land/oras/cmd/oras/... are not covered and not calculated in oras 0.16. There are approximately 2K lines of golang code in oras but Codecov only show coverage for 550+ lines.
3. Code coverage of end-to-end tests are not counted.

Are you willing to submit PRs to contribute to this feature?

• Yes, I am willing to implement it.