Observe test case distribution

[bitemyapp]

https://hackage.haskell.org/package/QuickCheck-2.10.1/docs/Test-QuickCheck-Property.html#v:classify

Example: http://hackage.haskell.org/package/checkers-0.4.9.5/docs/src/Test-QuickCheck-Utils.html#isCommutable

isTotalOrder :: (Arbitrary a,Show a,Ord a) => a -> a -> Property
isTotalOrder x y =
    classify (x > y)  "less than" $
    classify (x == y) "equals" $
    classify (x < y)  "greater than" $
    x < y || x == y || x > y

[bitemyapp]

https://hackage.haskell.org/package/QuickCheck-2.10.1/docs/Test-QuickCheck-Property.html#v:cover

[mbrock]

It seems hard to feel confident in property tests without something like this, no? Especially when input domains are large and there's some class of rare cases that you still want to make sure are represented.

[thumphries]

Not really; if you have rare cases with unusual significance you can write a separate generator and properties exercising them. Coverage tooling would be a great addition to Hedgehog; none of us really use it in QuickCheck so it has been a low priority. PRs welcome

…

On Mar 25, 2018, at 6:25 AM, Mikael Brockman ***@***.***> wrote: It seems hard to feel confident in property tests without something like this, no? Especially when input domains are large and there's some class of rare cases that you still want to make sure are represented. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[mankyKitty]

I have a basic implementation of such a thing: https://github.com/qfpl/tasty-hedgehog-coverage

Happy to try to make this into a more useful "built-in" part of Hedgehog, if there's interest.

EDIT: This is now on hackage https://hackage.haskell.org/package/tasty-hedgehog-coverage

[ruhatch]

@thumphries is the approach taken in tasty-hedgehog-coverage good enough to make it into hedgehog proper? I'd be happy to submit a PR to try and combine them

[mankyKitty]

@ruhatch Happy to help if required, feel free to throw issues/PRs at tasty-hedgehog-coverage too.

[moodmosaic]

Resolved via #253, #262, #288, and #289.