You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The shrinker is already really great, and it's producing some excellent counterexamples to populate our regression test suite (thanks @Rewbert !). But it could be even better:
There are a lot of skips in the generated code that can should be easily pruned out.
For a lot of the test cases, the values don't actually matter---can we try to replace more complicated expressions with constants where possible? And can we try to replace constants with smaller constants (so it's not just a syntactically smaller test case)?
Can we try to simplify the types? For instance, if a test case fails with an Int32, does it still fail with a UInt32? And what about a Bool?
I guess the overall theme of this PR is about being more flexible with what constitutes a "smaller" test case---I believe implementing the above may lead to even more intuitive test cases (right now, the test cases are small enough to understand but still require some manual doctoring to be self-explanatory).
The text was updated successfully, but these errors were encountered:
Changing the types of things is very difficult, as the program might not (most likely not) be type correct anymore. And furthermore, if the shrinker can change an Int32 to an Int64, it can probably also do the opposite, and thus the shrinker will never terminate. When we are shrinking a program we are not maintaining any state, so there's no way to know that we've not already tried to give a variable a certain type.
Concerning the constants, the shrinker is already trying to do a syntactic reduction of expressions. If only a single constant remain after this step it could probably be shrunk, yes, but I don't personally think it's worth the effort. We're just trading one constant for another. I did prepare the expression shrinker to do this though, so you'd just need to write a function SSMExp -> [SSMExp] where the resulting list contains mutations of the expression under scrutiny. Then that function can be plugged into the rest of the machinery. Empty list = unsuccessful shrinking.
The skips can be pruned out, but keeping them won't change any semantics or anything. It should be easy to add this as a last shrinking step though, if we think it's worth the time.
Ah gotcha. In terms of type errors, @yc2454 's type checker should come in handy, but I do understand your concern.
I think when I have some spare cycles, I'll write these shrinkers myself to get my hands dirty with the shrinker, and see if it's actually useful. For now, I'll punt on this issue.
The shrinker is already really great, and it's producing some excellent counterexamples to populate our regression test suite (thanks @Rewbert !). But it could be even better:
I guess the overall theme of this PR is about being more flexible with what constitutes a "smaller" test case---I believe implementing the above may lead to even more intuitive test cases (right now, the test cases are small enough to understand but still require some manual doctoring to be self-explanatory).
The text was updated successfully, but these errors were encountered: