Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable doctest in coverage #1372

Open
ghost opened this issue Sep 14, 2016 · 3 comments
Open

Disable doctest in coverage #1372

ghost opened this issue Sep 14, 2016 · 3 comments

Comments

@ghost
Copy link

ghost commented Sep 14, 2016

Whilst testing the sbt-scoverage plugin for scala.js on the cats code, I noticed that there was greater test coverage in JVM. Turns out that this is because doctest is currently jvm only and there is some code that is not tested in the real tests, but happen to be called in doctest where the methods are in a comment. E.g. Bifunctor.leftWiden and all of Choice. About 1% of tests are like this.

So doctest should be disabled in coverage, so that the proper test coverage is measured. It could be that the doctest really tests the code, but more likely is just an example of usage.

OTOH.. @andyscott recently had a PR #1352 with two lines missed

Turns out this a bug in scoverage-scalac-plugin as it currently "detects" macro implementations by looking at methods that return an Expr, and here a Tree is returned. So actually, the whole macro method should be excluded, but currently is included.

@non
Copy link
Contributor

non commented Sep 14, 2016

👍

@ghost
Copy link
Author

ghost commented Sep 14, 2016

FYI, fix for macros coverage is scoverage/scalac-scoverage-plugin#178

@ceedubs
Copy link
Contributor

ceedubs commented Sep 18, 2016

I don't feel too strongly either way about this, but I'm inclined to think that doctest examples should count toward test coverage. Usually when we use them we are checking that the result matches the expected value (by doing something like res0: Int = 3. Personally I tend to take advantage of this when I realize that we have some syntax that just trivially delegates through to another method (which is tested) but we don't have any tests for the syntax itself. Throwing together a doctest example is a good way to document the syntax while also making sure that we can use it as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants