Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Research and implement tarpaulin instead of grcov #252

Closed
greg-szabo opened this issue May 4, 2020 · 6 comments
Closed

Research and implement tarpaulin instead of grcov #252

greg-szabo opened this issue May 4, 2020 · 6 comments
Labels
ci Related to continuous integration structure High level repo-wide structural concerns

Comments

@greg-szabo
Copy link
Member

Not a done deal, but according to Tony, tarpaulin has less false negatives.
https://github.com/xd009642/tarpaulin

@liamsi
Copy link
Member

liamsi commented May 4, 2020

related: #250

@thanethomson
Copy link
Contributor

I'm assuming this will be superseded by #811?

@thanethomson thanethomson added the ci Related to continuous integration label Mar 23, 2021
@thanethomson
Copy link
Contributor

Since our code coverage seems broken (and seems like it has been broken for some time), we should consider fixing it or integrating tarpaulin-based coverage calculation sooner rather than later.

@romac are you planning on taking #811 further? What are the pros/cons at this point of going with cargo-llvm-cov? I see it's still labelled as being "experimental" right now?

@romac
Copy link
Member

romac commented Mar 23, 2021

are you planning on taking #811 further?

I wish I could but I don't have the bandwidth at the moment.

What are the pros/cons at this point of going with cargo-llvm-cov?

Pros:

  • the coverage should be much more precise
    Cons:
  • at the moment I was not able to get codecov.io to understand it's output on anything other than toy crates
  • -Zinstrument-coverage is still pretty new so there might be issues

I see it's still labelled as being "experimental" right now?

The wrapper itself is marked as experimental, but it basically only wraps cargo build (adding nightly-only options), llvm-profdata, and llvm-cov.

@romac
Copy link
Member

romac commented Mar 24, 2021

By the way, we use tarpaulin on ibc-rs but have seen no real improvements (eg. the coverage still jumps around by 30% sometimes, etc) hence why I was looking into source-based code coverage.

@thanethomson
Copy link
Contributor

Currently superseded by #840.

We can reopen this issue again in future if we want to reconsider using tarpaulin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci Related to continuous integration structure High level repo-wide structural concerns
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants