-
Notifications
You must be signed in to change notification settings - Fork 326
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: concurrency with 1.23 iterator #137
Comments
The part I find more troublesome when implementing is that letting the user decide when to stop the loop means developer have to have to implement the complex mechanism to close the channel at receiver side. I think we also need to consider how to use it with the |
Hello @amikai! I've been toying with ideas in this direction on my end as well, so I'm glad to see others think this is an exciting possibility. Thank you for writing up this proposal.
This one is a quite tricky since it requires One way around it is just to leave cancellation to the caller. I personally like this because although it is a little harder to use, it's more predictable IMO and leaves control fully in the hands of the user.
I think we can design this so that the consumer doesn't have to worry about it. Because of the design of the iter package, control will be yielded back to |
This is a good idea, but I want to ask if it might cause a goroutine leak if the user does not call cancel and the for loop has already been broken. Or in another situation, the context is already canceled but there is no break in for loop. Perhaps we can pass the context by changing the method to This will make it easier for users because they won’t need to worry about forgetting to cancel it. But I think this will cause problems for developers, including increased implementation difficulty and how to handle panic situations. // function declaration
func SeqMapCtx[In, Out any](context.Context, iter.Seq[In], func(context.Context, In) Out ) iter.Seq[Out] {...}
ids := slices.Values([]int{1,2,3,4,5,6})
func fetchSquare(ctx context.Context, id int) int {
...
}
ctx := context.Background()
// SeqMapCtx will derived the context with cancel
for v := range iter.SeqMapCtx(ctx, fetchSquare}) {
if v == 25 {
break // when yield return false, cancel the context and clean up goroutines
}
fmt.Println(v)
} Summary of my opinionThis is my opinion. I hope it helps.
NOTE: I haven't PoC yet. I'm not sure if it can be implemented. |
Proposal
I propose the four functions
SeqMap
under conciter
:SeqMap2
under conciter
: similar toiter.SeqMap
but returns two values, allowing it to be used for error handling.SeqMap
under concstream
: similar toiter.SeqMap
, but in order way.SeqMap2
under concconc
: similar tostream.SeqMap
but returns two values, allowing it to be used for error handling.Some thought
I believe the best aspects of a concurrent iterator are its ability to calculate lazily and its resource-saving capability. Functions like
iter.Map
anditer.ForEach
will exhaust all elements, but with an iterator, when the output iterator stops, the input iterator should also cease retrieving elements and stop all goroutines.The text was updated successfully, but these errors were encountered: