Allow Usage without Async Generators #56
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a new option that runs the code that generates responses in the same TaskGroup that is used for streaming the response. Doing so avoids the need for an async generator or some other form of persistent iterator that exists across requests.
The motivation for doing that is that async generators have less well defined error and cleanup behavior especially when they are interrupted see here.
In this new implementation, the coroutine that is generating outputs can be straightforwardly cancelled just like all the others in response to disconnects or server shutdowns and will be able to run its cleanup code right then and there which makes it easier to understand what is going to happen in all cases and importantly to propagate all exceptions where they should go.
This is conceptually similar to what is being done in the stream generator examples except it works within a single endpoint and doesn't require any shared state.