Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel should not drop data if paused. #331

Merged
merged 1 commit into from
Jun 30, 2015

Conversation

vqvu
Copy link
Collaborator

@vqvu vqvu commented Jun 30, 2015

Yet another parallel bug.

Simplest case:

Given 3 streams s1, s2, and s3 being consumed in this order, when s1 ends, if

  • s2 has already ended, and
  • the parallel stream is paused (caused by the downstream consumer)

then anything that s3 has already emitted gets dropped on the floor.

For those who care, the problem with the proof sketch in #302 (comment) was the assertion

Otherwise, when the nth stream ends, and it immediately flushes the buffer for the n+1th stream.

This is not true if the nth ended before one of the preceeding streams. Since it wasn't the top stream when it ended, it won't flush the buffer for the n+1th stream.

The fix is obviously make any stream that ends flush as much as it can (not just the next stream's data). Incidentally, this allows us to drop the flush block at the start of the generator, since it's fairly easy to show that there is never anything to flush at that point.

vqvu added a commit that referenced this pull request Jun 30, 2015
Parallel should not drop data if paused (fixes #328).
@vqvu vqvu merged commit 5db4fc1 into caolan:master Jun 30, 2015
@vqvu vqvu added this to the v2.6.0 milestone Nov 18, 2015
@vqvu vqvu deleted the parallel-issue-328 branch November 18, 2015 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant