Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an API configuration for request timeout #1452

Open
deliahu opened this issue Oct 15, 2020 · 0 comments
Open

Add an API configuration for request timeout #1452

deliahu opened this issue Oct 15, 2020 · 0 comments
Labels
enhancement New feature or request

Comments

@deliahu
Copy link
Member

deliahu commented Oct 15, 2020

Description

If configured, when a request is taken off of the queue for processing, if the time spent on the queue is greater than the configured timeout, the request would not be processed and an error would be returned (a 503 error is probably most appropriate).

This timeout could also be applied if the request is still waiting in the queue, or if the request is already being processed. This could be separated into separate issue(s).

For timing out requests that are currently in progress, it might be possible with async (e.g. via this). Question: assuming we figure out how to cap the thread pool when using async (which should be possible), will there be other unintended consequences of using async? Here is FastAPI's discussion.

Relevant question on gitter: https://gitter.im/cortexlabs/cortex?at=5fe1eea0dbb17f28c59329b0

May be related to #1453

@deliahu deliahu added the enhancement New feature or request label Oct 15, 2020
@RobertLucian RobertLucian added this to the v0.33 milestone Apr 2, 2021
@deliahu deliahu removed this from the v0.33 milestone Apr 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants