Parallel should not drop data if paused. #331
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Yet another
parallel
bug.Simplest case:
Given 3 streams
s1
,s2
, ands3
being consumed in this order, when s1 ends, ifs2
has already ended, andthen anything that
s3
has already emitted gets dropped on the floor.For those who care, the problem with the proof sketch in #302 (comment) was the assertion
This is not true if the
n
th ended before one of the preceeding streams. Since it wasn't the top stream when it ended, it won't flush the buffer for then+1
th stream.The fix is obviously make any stream that ends flush as much as it can (not just the next stream's data). Incidentally, this allows us to drop the flush block at the start of the generator, since it's fairly easy to show that there is never anything to flush at that point.