Tracking: Coverage

[onbjerg]

There's still some missing features and improvements for coverage:

Data/functionality

• Coverage: More robust test/dependency contract checks #1962
• Coverage: Select anchors for constructors using creation source maps #1963
• Coverage: Fuzz test support #1964
• Coverage: Better branch handling (require/assert) #1967
• Coverage: Abstract contract coverage #1965

User facing

• Coverage: HTML reporter #1966

Misc

• Coverage: Verify that LCOV reporter is working #1969
• Coverage: Unhandled AST nodes #1968

[aahna-ashina]

@onbjerg When I run forge coverage at https://github.com/nation3/court, I get this error:

could not find anchor: no matching instruction in range CollateralAgreementFramework:3834:2:7

forge coverage 
warning: Unknown section [default] found in foundry.toml. This notation for profiles has been deprecated and may result in the profile not being registered in future versions. Please use [profile.default] instead or run `forge config --fix`.
warning: Unknown section [default] found in lib/solmate/foundry.toml. This notation for profiles has been deprecated and may result in the profile not being registered in future versions. Please use [profile.default] instead or run `forge config --fix`.
warning: Unknown section [intense] found in lib/solmate/foundry.toml. This notation for profiles has been deprecated and may result in the profile not being registered in future versions. Please use [profile.default] instead or run `forge config --fix`.
[⠒] Compiling...
[⠒] Compiling 11 files with 0.8.14
[⠑] Solc 0.8.14 finished in 992.33ms
Compiler run successful
Analysing contracts...
Error: 
could not find anchor: no matching instruction in range CollateralAgreementFramework:3834:2:7

Does could not find anchor mean that we are missing some code coverage configuration in our project?

forge --version
forge 0.2.0 (37e4376 2022-07-16T00:05:51.461078Z)

[onbjerg]

Please check the other issues on coverage - this is not on your side, but coverage is not stable yet, so there are edge cases that are not covered. This is one edge case where it could not find an opcode that matches the source range, so we fail and return instead of showing a potentially incorrect coverage report

[aahna-ashina]

@onbjerg Is there a way to exclude *.sol files in the test/ folder when running forge coverage?

[elenadimitrova]

@aahna-ashina I had the same question and what I ended up doing is exclude them in codecov.yml instead with:

ignore:
  - "./tests/"

[PaulRBerg]

Thanks @elenadimitrova, great idea to ignore the tests in Codecov directly.

However, it should be possible to do this with Forge itself - ideally, there would be a --ignore flag in the forge coverage command.

I created an issue to track this as a feature request: #4006.

Update: it looks like it is not possible to ignore globs on all coverage tracking platforms - for instance, Coveralls doesn't offer this feature. Thus, it would be really helpful for Foundry to have an --ignore flag.

[PaulRBerg]

I have recently discovered several bugs in the coverage command. Posting here for cross-reference:

• feat(forge): disable coverage for assertions #4294
• bug(forge): coverage does not work with return values assigned to library calls #4305
• bug(forge): incorrect branch coverage when passing array elements to custom errors #4309
• bug(forge): accessing array elements breaks coverage for if statements #4310
• bug(forge): no-effect statements on the stack are not ignored in the coverage report #4311
• bug(forge): empty if/else statements are not covered #4314
• bug(forge): coverage does not work for enum assignments #4315
• bug(forge): coverage does not work when the test contract inherits the target contract #4316

[zerosnacks]

Closing in favor of #4442 as updated metatracking ticket, includes the bugs stated in #1961 (comment)