Gradually grow test size

[zwizwa]

Haskell quickcheck allows gradual growing of test via the size parameter. In Lua quickcheck it appears this parameter is fixed to the sample size. Was this difference intentional? If not are there plans to allow for gradual growing test sizes? The bias this introduces towards the smaller corner cases (zero, one, ...) seems to be a better default in practice.

[zwizwa]

After sleeping on this, it does seem that my bias argument is not a good one: it is possible to make generators that bias towards smaller "test size" also in the current implementation. The argument made in docs of Haskell quickcheck to ramp up the size of the tests is that this is more efficient.

I'm thinking of trying out to the following solution: to leave the current sample size as a parameter to the pick method as is, but to also add a sequence number as a second argument. That would be backwards compatible. What do you think about that?

( And thanks for writing and publishing this library! )

[luc-tielen]

Hi there!

It's been a few years since I created this library. I think I mainly didn't do it to avoid additional complexity (the codebase is already complicated due to dynamic typing). I don't use Lua anymore nowadays (except for configuring neovim), so no plans as of this moment.
Also nowadays I'm too busy with my freelance work and working on my datalog compiler, don't really have much time for anything else. 😅

Feel free to fork it, but I probably won't have the time to review your changes (unless they're really small).

[zwizwa]

Thanks for taking the time to answer. I'm going to play with it for a while as is and when I decide to fork it I will post about it here. My client uses Lua as test scripting language so I figured I give this a try first before subjecting them to the wonderful world of Haskell :-)

[luc-tielen]

@zwizwa sure, go ahead! Glad my library could help.

[zwizwa]

Inspired by half-understanding your and Haskell QC source and some blog posts I ended up re-implementing a minimal version to learn and find nails for some hammers I had. Gives me a bit more control, and the minimal version is good enough for what I need now.

Obscure summary: Using HOAS for the type specs. Those type specs can then evaluate directly to a generator and a to a shrinker (final tagless style) which are both just functions. Generators compose like a state monad threading the seed. The shrinker's output list is represented as a visitor (inverted control, applies a function to elements in the shrink list), which makes them compose easily as well by nesting the iterations and it keeps the representation lazy. Shrinkers by default iterate over all shrinks, so stopping when counterexample is found is implemented by raising an exception (basic idea: Oleg's fold to generator inversion). All this is quite compact with very little bookkeeping but diverges from your implementation. I'll publish as part of my uc_tools library.

EDIT: https://github.com/zwizwa/uc_tools/blob/exo/lua/qcrun.lua (test runner), which depends on qctools.lua (generators, shrinkers) and qcprop.lua (example property file)

[luc-tielen]

Cool. Yeah, this approach sounds better, my library isn't lazy at all.

I built it years ago when I was still a junior and getting to know Haskell. 🙂

Good to see alternatives popping up!