Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deterministically vary the number of seed slides in Test.check #911

Merged

Conversation

leviramsey
Copy link
Contributor

(re-attempt of #841, after that was inadvertently turned into an empty PR in the process of moving to a feature branch in my fork)

A Prop.forAll of multiple parameters will use the initial seed to generate the first parameter and slide the seed once for each of the subsequent parameters. When combined with Test.check which slides the seed before evaluating the property, this means that for a property with m > 1 parameters, if the i th evaluation of the property had seeds (seed1 ... seedm), then the i+1 th evaluation will have seeds (seed2 ... seedm, seedm.slide). If the same generator is used for multiple parameters, then a generated value in one evaluation will be reused in a different position in a specific later evaluation.

This addresses this by varying the number of slides done in Test.check based on the number of times a given property has been evaluated. The number of slides remains deterministic, just less amenable to pattern recognition.

The seed sliding behavior between checks is configurable:

  • If maxRNGSpins is 1, 0, or negative: 1 spin is performed
  • Otherwise at most maxRNGSpins will be performed; the number of spins is deterministic and as the number of checks of a given property increases, the total number of spins performed between checks converges to N * (maxRNGSpins + 1)/2

@leviramsey
Copy link
Contributor Author

Addresses #840 and scalatest/scalatest#2054

@SethTisue
Copy link
Member

Just one more thing before we merge — for the history, does this make sense as 6 commits?

@leviramsey leviramsey force-pushed the spin-again-like-we-did-last-summer branch from 9b20458 to 58a8620 Compare August 11, 2022 21:06
@SethTisue SethTisue merged commit 7bb1a47 into typelevel:main Aug 15, 2022
@leviramsey leviramsey deleted the spin-again-like-we-did-last-summer branch August 15, 2022 19:07
@SethTisue SethTisue mentioned this pull request Aug 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants