Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explore interaction of model-based and property-based testing #822

Open
2 of 6 tasks
andrey-kuprianov opened this issue Feb 25, 2021 · 0 comments
Open
2 of 6 tasks
Assignees
Labels
enhancement New feature or request tests

Comments

@andrey-kuprianov
Copy link
Contributor

andrey-kuprianov commented Feb 25, 2021

Following the efforts to apply PBT in #709 (see backward.rs), #792, #815, as well as discussion within the MBT team, we need to understand how to properly integrate MBT and PBT.

The high-level idea is not to reinvent the wheel by writing our own fuzzers, and instead to leverage the PBT infrastructure as done by proptest. In that, the test cases that span over several computation steps will be generated by MBT, and then mutated by PB strategies tendermint-rs datastructures. The main complication here is how to correctly express what changes should happen to the expected test outcome when the intermediate computation steps are mutated.

As the first step for the exploration we have decided to replace one of the manual fuzzing steps as done in model_based.rs with the proptest-based one, in particular to replace fuzzing of Time.

For successful completion of this issue:

  • write a PB-test that accepts the MB-test case, and fuzzes time fields in various test case blocks
  • fuzzing should be done using combinators over the proptest strategies developed in Property based tests of Time #815
  • the PB-test should explore different options wrt. how the fuzzed time relates to other times in the test case:
    • to the trusting period
    • to the time of neighbouring blocks
      Depending on that the expected verdict should be modified accordingly.
  • (optional) explore possible new macros to concisely express such MBT-specific PB-tests/strategies.
@andrey-kuprianov andrey-kuprianov added enhancement New feature or request tests labels Feb 25, 2021
shonfeder added a commit that referenced this issue Mar 17, 2021
Closes #828

This allows us to use the timestamp and DateTime generators in any other
crate in our workspace. It should help advance #822 and #821.
thanethomson added a commit that referenced this issue Mar 22, 2021
* Move time proptest generators into their own crate

Closes #828

This allows us to use the timestamp and DateTime generators in any other
crate in our workspace. It should help advance #822 and #821.

* Fix clippy warning

* Correct documentation example

* Update pbt-gen/src/time.rs

Co-authored-by: Thane Thomson <thane@informal.systems>

* Update pbt-gen/src/time.rs

Co-authored-by: Thane Thomson <thane@informal.systems>

* Fix typos in pbt-gen/Cargo.toml

* Rename pbt-gen crate and guard modules with feature

* Correct wording

Co-authored-by: Thane Thomson <thane@informal.systems>
Shivani912 added a commit that referenced this issue Mar 25, 2021
Shivani912 added a commit that referenced this issue Mar 29, 2021
thanethomson added a commit that referenced this issue Mar 30, 2021
* Move time proptest generators into their own crate

Closes #828

This allows us to use the timestamp and DateTime generators in any other
crate in our workspace. It should help advance #822 and #821.

* Fix clippy warning

* Correct documentation example

* Update pbt-gen/src/time.rs

Co-authored-by: Thane Thomson <thane@informal.systems>

* Update pbt-gen/src/time.rs

Co-authored-by: Thane Thomson <thane@informal.systems>

* Fix typos in pbt-gen/Cargo.toml

* Rename pbt-gen crate and guard modules with feature

* Correct wording

Co-authored-by: Thane Thomson <thane@informal.systems>
Shivani912 added a commit that referenced this issue Apr 1, 2021
Shivani912 added a commit that referenced this issue Apr 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request tests
Projects
None yet
Development

No branches or pull requests

2 participants