You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A pattern in structured concurrency is to pass the lifetime object into the concurrent tasks so that they can launch additional tasks with the same lifetime. This way, the originator can wait on – and receive errors from – all work at a single point, but subtasks can run concurrently when they don't have dependencies between each other. The "Go statement considered harmful" article describes this pattern and its advantages in more detail.
However, conc/pool prevents this pattern. Once a call to Wait occurs, new tasks cannot be submitted to the pool, because the task submission channel becomes closed:
As a concrete example, I wanted to use a ContextPool to process SQS messages in batches roughly as follows:
Decode the message body. If this fails, return immediately; the message must remain in the queue so that it eventually becomes a dead letter.
Delete the decoded message from the queue so that it isn't retried anymore.
Process the message according to business logic.
While steps 2 and 3 can each fail, there's no dependency between them, so they can in principle run currently. The approach I attempted was to p.Go a task to run step 1, then p.Go a subtask for step 2, and then finish step 3. This approach causes a send on closed channel panic.
Tested on both v0.3.0 and v0.3.1-0.20240108182409-4afefce20f9b (current main).
The text was updated successfully, but these errors were encountered:
A pattern in structured concurrency is to pass the lifetime object into the concurrent tasks so that they can launch additional tasks with the same lifetime. This way, the originator can wait on – and receive errors from – all work at a single point, but subtasks can run concurrently when they don't have dependencies between each other. The "Go statement considered harmful" article describes this pattern and its advantages in more detail.
However, conc/pool prevents this pattern. Once a call to Wait occurs, new tasks cannot be submitted to the pool, because the task submission channel becomes closed:
conc/pool/pool.go
Lines 72 to 75 in 4afefce
As a concrete example, I wanted to use a ContextPool to process SQS messages in batches roughly as follows:
While steps 2 and 3 can each fail, there's no dependency between them, so they can in principle run currently. The approach I attempted was to
p.Go
a task to run step 1, thenp.Go
a subtask for step 2, and then finish step 3. This approach causes a send on closed channel panic.Tested on both v0.3.0 and v0.3.1-0.20240108182409-4afefce20f9b (current main).
The text was updated successfully, but these errors were encountered: