-
Notifications
You must be signed in to change notification settings - Fork 313
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make result order deterministic #126
Conversation
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #126 +/- ##
==========================================
- Coverage 99.31% 98.94% -0.38%
==========================================
Files 12 12
Lines 441 474 +33
==========================================
+ Hits 438 469 +31
- Misses 3 4 +1
- Partials 0 1 +1 ☔ View full report in Codecov by Sentry. |
type resultAggregator[T any] struct { | ||
mu sync.Mutex | ||
len int | ||
results []T | ||
errored []int | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The changes to resultAggregator
are central to this PR.
Summary:
- Add a
nextIndex
method that makes it possible to reserve a slot in the result slice - Change the
add
method tosave
, which also takes an the index fromnextIndex
and whether the result errored - Add a
collect
method that respects theWithCollectErrored
option by filtering out errored results.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, LGTM - this does add an extra set of lock/unlock on the mutex, though I imagine it doesn't make much difference in practice
Yes, it does. It would be possible to use an atomic integer for reserving a slot, but it adds more complexity of the implementation and I wouldn't expect these to be a high-contention since in most cases, the task being done in a goroutine is large. We can do some benchmarking and improve perf later if the difference turns out to be significant. |
Any plans to tag a new release with this change? |
This PR makes the order of results in a
Result.*Pool
deterministic so that the order of the result slice corresponds with the order of tasks submitted. As an example of why this would be useful, it makes it easy to rewriteiter.Map
in terms ofResultPool
. Additionally, it's a generally nice and intuitive property to be able to match the index of the result slice with the index of the input slice.Comments inline.