bpo-32751: Wait for task cancellation in asyncio.wait_for() #7216
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently, asyncio.wait_for(fut), upon reaching the timeout deadline,
cancels the future and returns immediately. This is problematic for
when *fut* is a Task, because it will be left running for an arbitrary
amount of time. This behavior is iself surprising and may lead to
related bugs such as the one described in bpo-33638:
condition = asyncio.Condition()
async with condition:
await asyncio.wait_for(condition.wait(), timeout=0.5)
Currently, instead of raising a TimeoutError, the above code will fail
with `RuntimeError: cannot wait on un-acquired lock`, because
`__aexit__` is reached _before_ `condition.wait()` finishes its
cancellation and re-acquires the condition lock.
To resolve this, make `wait_for` await for the task cancellation.
The tradeoff here is that the `timeout` promise may be broken if the
task decides to handle its cancellation in a slow way. This represents
a behavior change and should probably not be back-patched to 3.6 and
earlier.
1st1
reviewed
May 29, 2018
Lib/asyncio/tasks.py
Outdated
| # We cannot wait on *fut* directly to make | ||
| # sure _cancel_and_wait itself is reliably cancellable. | ||
| await waiter | ||
| except futures.CancelledError: |
There was a problem hiding this comment.
No, it's an oversight from an earlier code iteration.
1st1
reviewed
May 29, 2018
Lib/asyncio/tasks.py
Outdated
| waiter = loop.create_future() | ||
| cb = functools.partial(_release_waiter, waiter) | ||
| fut.add_done_callback(cb) | ||
| fut.cancel() |
There was a problem hiding this comment.
I'd move 'fut.cancel()' into the 'try' block
1st1
approved these changes
May 29, 2018
asvetlov
approved these changes
May 29, 2018
|
|
||
| try: | ||
| fut.cancel() | ||
| # We cannot wait on *fut* directly to make |
Yeah. Broken timeouts is a small price to have the rest of asyncio code working correctly. A related bug where wait_for caused an incorrect |
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this pull request
May 29, 2018
…-7216) Currently, asyncio.wait_for(fut), upon reaching the timeout deadline, cancels the future and returns immediately. This is problematic for when *fut* is a Task, because it will be left running for an arbitrary amount of time. This behavior is iself surprising and may lead to related bugs such as the one described in bpo-33638: condition = asyncio.Condition() async with condition: await asyncio.wait_for(condition.wait(), timeout=0.5) Currently, instead of raising a TimeoutError, the above code will fail with `RuntimeError: cannot wait on un-acquired lock`, because `__aexit__` is reached _before_ `condition.wait()` finishes its cancellation and re-acquires the condition lock. To resolve this, make `wait_for` await for the task cancellation. The tradeoff here is that the `timeout` promise may be broken if the task decides to handle its cancellation in a slow way. This represents a behavior change and should probably not be back-patched to 3.6 and earlier. (cherry picked from commit e2b340a) Co-authored-by: Elvis Pranskevichus <elvis@magic.io>
|
GH-7223 is a backport of this pull request to the 3.7 branch. |
miss-islington
added a commit
that referenced
this pull request
May 29, 2018
Currently, asyncio.wait_for(fut), upon reaching the timeout deadline, cancels the future and returns immediately. This is problematic for when *fut* is a Task, because it will be left running for an arbitrary amount of time. This behavior is iself surprising and may lead to related bugs such as the one described in bpo-33638: condition = asyncio.Condition() async with condition: await asyncio.wait_for(condition.wait(), timeout=0.5) Currently, instead of raising a TimeoutError, the above code will fail with `RuntimeError: cannot wait on un-acquired lock`, because `__aexit__` is reached _before_ `condition.wait()` finishes its cancellation and re-acquires the condition lock. To resolve this, make `wait_for` await for the task cancellation. The tradeoff here is that the `timeout` promise may be broken if the task decides to handle its cancellation in a slow way. This represents a behavior change and should probably not be back-patched to 3.6 and earlier. (cherry picked from commit e2b340a) Co-authored-by: Elvis Pranskevichus <elvis@magic.io>
elprans
added a commit
to elprans/cpython
that referenced
this pull request
Aug 15, 2020
When I was fixing bpo-32751 back in pythonGH-7216 I missed the case when *timeout* is zero or negative. This takes care of that. Props to @aaliddell for noticing the inconsistency.
1st1
pushed a commit
that referenced
this pull request
Aug 26, 2020
… 0 (#21895) When I was fixing bpo-32751 back in GH-7216 I missed the case when *timeout* is zero or negative. This takes care of that. Props to @aaliddell for noticing the inconsistency.
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this pull request
Aug 26, 2020
… 0 (pythonGH-21895) When I was fixing bpo-32751 back in pythonGH-7216 I missed the case when *timeout* is zero or negative. This takes care of that. Props to @aaliddell for noticing the inconsistency. (cherry picked from commit c517fc7) Co-authored-by: Elvis Pranskevichus <elvis@magic.io>
elprans
added a commit
to elprans/cpython
that referenced
this pull request
Aug 26, 2020
…out <= 0 (pythonGH-21895) When I was fixing bpo-32751 back in pythonGH-7216 I missed the case when *timeout* is zero or negative. This takes care of that. Props to @aaliddell for noticing the inconsistency.. (cherry picked from commit c517fc7) Co-authored-by: Elvis Pranskevichus <elvis@magic.io>
elprans
added a commit
to elprans/cpython
that referenced
this pull request
Aug 26, 2020
…out <= 0 (pythonGH-21895) When I was fixing bpo-32751 back in pythonGH-7216 I missed the case when *timeout* is zero or negative. This takes care of that. Props to @aaliddell for noticing the inconsistency.. (cherry picked from commit c517fc7)
xzy3
pushed a commit
to xzy3/cpython
that referenced
this pull request
Oct 18, 2020
… 0 (python#21895) When I was fixing bpo-32751 back in pythonGH-7216 I missed the case when *timeout* is zero or negative. This takes care of that. Props to @aaliddell for noticing the inconsistency.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently,
asyncio.wait_for(fut), upon reaching the timeout deadline,cancels the future and returns immediately. This is problematic when
fut is a Task, because it will be left running for an arbitrary
amount of time. This behavior is iself surprising and may lead to
related bugs such as the one described in bpo-33638:
Currently, instead of raising a TimeoutError, the above code will fail
with
RuntimeError: cannot wait on un-acquired lock, because__aexit__is reached beforecondition.wait()finishes itscancellation and re-acquires the condition lock.
To resolve this, make
wait_forawait for the task cancellation.The tradeoff here is that the
timeoutpromise may be broken if thetask decides to handle its cancellation in a slow way. This represents
a behavior change and should probably not be back-patched to 3.6 and
earlier.
https://bugs.python.org/issue32751