Fix for RTQ EQC prop_main and prop_parallel #422
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
During statem test execution, riak_repl2_rtq_eqc reads from the RTQ queue using its pull/2 function. This function defines an anonymous DeliverFun to be called by the RTQ, which uses erlang messaging to send a pulled object back to the client (test) process. Messaging was found to be occasionally timing out. This cased the RTQ, when seeing the DeliverFun to fail, not to mark the pulled object as completed, and thus, when the test client was re-registered, the same object would be delivered. The test model would then identify this duplicate and error out.
A similar problem was affecting the prop_parallel test, except that the timeouts were occurring in the test, not in the DeliverFun ack clause. This was causing the test to believe that the queue was empty (return -> none).
Bumping this up to 500 ms appears to fix the problem on a late model MPB, and not slow down the test considerably. We may want to test this on various platforms and find a better value, but this fixes it for dev for now.