Show test.check's generated input for failing tests

[sebastianpoeplau]

CIDER nicely displays the expected and actual outcome for tests in the clojure.test framework. However, when the tests are based on test.check the output lacks information about the generated inputs.

For example, consider the following property:

(defspec whatever
  (prop/for-all [x gen/int]
    (= 0 x)))

Checking it in CIDER yields this output:

Fail in whatever
expected: result
  actual: false

It would be nice to also learn which x caused the test to fail. To this end, I would suggest to capture that very piece of information from the test.check results and display it in the result buffer whenever applicable.

Technically, the information about the failing input has to be propagated to cider-nrepl.middleware.test/report, where it could be stored and passed to the Emacs plugin for display. Unfortunately, clojure.test.check.clojure-test.defspec doesn't do so yet, throwing away everything except the success status, but the extension should be relatively simple. The utility library test.chuck already demonstrates a way to go: it calls the report function, in our case cider-nrepl.middleware.test/report, with an event of :type ::shrunk when it has worked out the smallest failing inputs. As a first step, one might extend cider-nrepl.middleware.test/report to handle events of that type and have CIDER display the information in the result buffer. Then the same concept could be applied to test.check's defspec. I'll play around with it...

[jeffvalk]

Definitely. This was always on my radar for cider-test, but clearly I hadn't gotten to that blip. :)

[sebastianpoeplau]

I've created a ticket for test.check to forward shrunk inputs as well: http://dev.clojure.org/jira/browse/TCHECK-118.