In clover.xml, count all method definitions once #115
Conversation
[The `clover.xml` specification] does not answer what a `count` means for a line with `type="method"`. In the case of PHP's CodeCoverage library, it is set to [the maximum count] for any statement within the method. Therefore, even if a file is parsed and run (so the method is defined), if the method is never called then it will have a count of `0`. This differs from how other PHP tools display `"method"` lines, which instead treat them as run. So, treat all methods as if they have a count of `1`. [The `clover.xml` specification]: https://bitbucket.org/atlassian/clover/src/master/etc/schema/clover.xsd [the maximum count]: https://github.com/sebastianbergmann/php-code-coverage/blob/main/src/Report/Clover.php#L89-L104
Pull Request Test Coverage Report for Build 8443662829Details
💛 - Coveralls |
There was a problem hiding this comment.
@mike-burns please see my comment on Line 60 in clover_parser.cr. I'm mostly seeking clarity---not sure if I've assessed the code properly.
| else | ||
| hits = line_node.attributes["count"].content.to_u64 | ||
| coverage[line_number] = hits | ||
| end | ||
| end |
There was a problem hiding this comment.
@mike-burns question:
Will this solution place the correct number of hits for the method on the line declaring the method (in the Coveralls source file report)? See the example I refer to below, pls.
I see that this code will solve the problem of lines declaring functions displaying as (relevant but) uncovered. It appears this will always at least make sure they are shown as covered (green) in Coveralls reports.
However, given your discovery that the number of hits for a method is taken to be equivalent to the highest number of hits for any of its (statement) lines, won't this miss the opportunity to show the correct number of hits for the method, and always just show (1)?
I can understand using this code as a failsafe to make sure lines declaring a method appear as covered, but it seems to me that the correct number of hits should be what we get from a correct XML report, which appears to be that highest number for the method's lines.
Check out the example pertaining to the AbstractArrayDeclarationSniff.php file in my comment on the OP's public issue:
To be perfectly honest, I kind of forget the point of my comment there, but now I'm stuck on the fact that recent source file reports (like this one, as opposed to this older version from Aug 2023) seem to show the correct hit count for the method. in the case of Line 178, 52.
Isn't that what a correct clover.xml report should show as the hit count for the method? And isn't that what we want to show in our reports?
Forgive me for being rusty on this one, I may have missed something.
There was a problem hiding this comment.
won't this miss the opportunity to show the correct number of hits for the method, and always just show (
1)?
This really comes down to: what number do we want to show for line 178 in that screenshot? What is "correct", in this case?
52, the highest number of any line within the method (so0for an untested method).max(52, 1), the highest number of any line within the method (or1for an untested method, meaning that line 178 is OK).1, simply indicating that the method was defined.
As a base case, what numbers do we want for this method definition:
public function someNoOpMethod()
{
}Do we want to say that the public function someNoOpMethod() line was never run? No amount of test coverage can change that -- calling the method executes zero statements, so the method definition line is still considered 0.
I'm happy to go with your decision -- you might have more context than I do around what these numbers are used for.
|
Hi @Thompson1985. Thanks for reviewing the changes. We had left this open because we're not sure about the proper representation of coverage for method declarations in It's been a while since I spent time with this, but I left last time feeling that the expectation is that method declarations should not be shown as Unfortunately, I also seem to remember a lack of canonical documentation on the matter. Are you a user of TIA. |
The
clover.xmlspecification does not answer what acountmeans for a line withtype="method".In the case of PHP's CodeCoverage library, it is set to the maximum count for any statement within the method.
Therefore, even if a file is parsed and run (so the method is defined), if the method is never called then it will have a count of
0. This differs from how other PHP tools display"method"lines, which instead treat them as run.So, treat all methods as if they have a count of
1.☑️ Checklist