Fuzz testing #1255
Fuzz testing #1255
Conversation
packages/token/src/tests/erc721/test_fuzz_erc721_enumerable.cairo
Outdated
Show resolved
Hide resolved
.github/workflows/test.yml
Outdated
| @@ -48,7 +48,7 @@ jobs: | |||
| run: scarb fmt --check --workspace | |||
|
|
|||
| - name: Run tests and generate coverage report | |||
| run: snforge test --workspace --coverage | |||
| run: snforge test --workspace --coverage --features fuzzing --fuzzer-runs 10000 | |||
There was a problem hiding this comment.
I have the feeling that this will soon make tests run timeout in the action.
There was a problem hiding this comment.
Tests are not running on the PR btw, when that's fixed I'm curious to see how much time will this take with this few examples.
There was a problem hiding this comment.
I reduced the number of fuzzer runs to 500 which seems a reasonable value to me, at least for now
With 10k runs the tests took 1h 55m to complete which is way too much, here are the run results: https://github.com/OpenZeppelin/cairo-contracts/actions/runs/12374506824/job/34537193923
With 1k runs it took 17m 27s to run all the tests. 10 minutes more than our usual test runs.
Results: https://github.com/OpenZeppelin/cairo-contracts/actions/runs/12379685661/job/34554383936
Although it's an acceptable duration, it can increase much when we add more fuzz test cases. So now with 500 runs the tests job takes 12 minutes (5 minutes more than it was without fuzz tests)
| fn test__spend_allowance(supply: u256, spend_amount: u256) { | ||
| if supply == Bounded::MAX { | ||
| return; | ||
| } |
There was a problem hiding this comment.
Just to be as comprehensive as we can, what do you think about moving the if statement past the _spend_allowance call and asserting the values don't change if supply == u256 max?
| @@ -50,7 +50,7 @@ jobs: | |||
| run: scarb fmt --check --workspace | |||
|
|
|||
| - name: Run tests | |||
| run: snforge test --workspace | |||
| run: snforge test --workspace --features fuzzing --fuzzer-runs 500 | |||
PR Checklist