Ignore calls in other threads when counting ruby calls for a test #114
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #113
Problem
As stated in #113 we have flaky test failures in a test that is counting ruby calls using a tracepoint.
I had trouble reproducing the test failure locally, so I commented out the test using TracePoint and started a tracepoint in test_helper.rb, which was able to capture calls on the Thread module we were seeing. After a bit of debugging, I found the source of the problem, that can be demonstrated using this in test/test_helper.rb
which outputs
So these calls are coming from this line in minitest: https://github.com/seattlerb/minitest/blob/v5.14.2/lib/minitest/parallel.rb#L29
Solution
Filter calls to another thread in the test that was asserting on the number of ruby calls made.