Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coverage: Verify that LCOV reporter is working #1969

Closed
Tracked by #1961
onbjerg opened this issue Jun 14, 2022 · 1 comment
Closed
Tracked by #1961

Coverage: Verify that LCOV reporter is working #1969

onbjerg opened this issue Jun 14, 2022 · 1 comment
Assignees
Labels
C-forge Command: forge Cmd-forge-coverage Command: forge coverage P-normal Priority: normal

Comments

@onbjerg
Copy link
Member

onbjerg commented Jun 14, 2022

We need to verify that the LCOV reporter does not output a malformed report - genhtml did not work for me, but I'm unsure if that is expected since it was made for C LCOV reports, so I am looking for other ways to test.

@onbjerg onbjerg mentioned this issue Jun 14, 2022
8 tasks
@onbjerg onbjerg added C-forge Command: forge Cmd-forge-coverage Command: forge coverage P-normal Priority: normal labels Jun 14, 2022
@onbjerg
Copy link
Member Author

onbjerg commented Jun 23, 2022

I just verified this, although it did highlight some odd behavior with forge coverage I am tracking down/finding explanations for.

To reproduce:

forge coverage --report lcov
genhtml -o report lcov.info

(note: this spams your project folder with a bunch of HTML and CSS so probably not the wisest to do)

See below for an example genhtml run on an lcov.info generated by Forge.

Some observations:

  • I have not created a line item for function definitions, so this info is not surfaced in the LCOV report (i.e. you can't see how many times a function was called in the source view, but it does work in the "functions" view, see below)
  • branchRequire looks like it should given I test it 2 times, once for true/false.
  • Same for branchAssert
  • Same for branchIf
  • For oddEven:
    • The first two lines look OK, given they are only executed once - but it is unclear if we should report how many times the loop condition is run instead of how many times you "enter" the loop?
    • I am not entirely sure why the if statement in the for loop shows 12 executions instead of 10
    • Similarly, I am not sure why the two statements in the for loop show up as executed 6 and 7 times instead of 5 each.

I will go over them in a bit more detail to figure out what's going on and open up relevant issues.

Example report

Source view

image

Functions view

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-forge Command: forge Cmd-forge-coverage Command: forge coverage P-normal Priority: normal
Projects
Archived in project
Development

No branches or pull requests

1 participant