Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test coverage results don't seem to be right #715

Closed
heisen-li opened this issue Nov 19, 2021 · 7 comments
Closed

Test coverage results don't seem to be right #715

heisen-li opened this issue Nov 19, 2021 · 7 comments

Comments

@heisen-li
Copy link

heisen-li commented Nov 19, 2021

I tested the randproject and found that it had zero branch coverage.

Is this normal?
image

@marco-c
Copy link
Collaborator

marco-c commented Nov 24, 2021

Are you using source-based coverage or gcov-based coverage?

@heisen-li
Copy link
Author

This is based on the coverage of the source.Will the results of the two be different?

@heisen-li
Copy link
Author

Thanks, I tested it. The results of the two are still quite different.

image

@heisen-li
Copy link
Author

The reason I reopened is that I want to know what is the difference between these two test methods, and why is there such a big gap in the results of branch coverage?

@heisen-li heisen-li reopened this Nov 25, 2021
@victor-aembit
Copy link

+1.

I tried to run it against my small project (with code based coverage) and I can see quite a lot of unexpected things:

  • Zero branch coverage info (even though I used "--branch" flag) (This is my biggest concern)
  • Some of macros for tests marked as hit. Some tests function are included (green) and some excluded (white)
  • Function coverage percentage is strangely off. As an example, there are two functions (marked as green). However, it shows 66.67% (which implies that there is a third function, but nothing is marked as red)

@heisen-li
Copy link
Author

  • Zero branch coverage info (even though I used "--branch" flag) (This is my biggest concern)

This is because the source-based approach does not currently support branch coverage.
refer to:rust-lang/rust#79649

I agree to other situations. In addition, you can continue the discussion here.rust-lang/rust#91661

@marco-c
Copy link
Collaborator

marco-c commented Dec 17, 2021

The reason I reopened is that I want to know what is the difference between these two test methods, and why is there such a big gap in the results of branch coverage?

gcov-based coverage is using DebugInfo.
Source-based coverage instead is using the AST directly.

There is some information on https://clang.llvm.org/docs/SourceBasedCodeCoverage.html. It's about Clang, but should apply similarly to Rust.

@marco-c marco-c closed this as completed Dec 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants