Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update in-flight request counter, switch to FastAPI + Uvicorn #838

Merged
merged 64 commits into from
Mar 4, 2020
Merged

Update in-flight request counter, switch to FastAPI + Uvicorn #838

merged 64 commits into from
Mar 4, 2020

Conversation

vishalbollu
Copy link
Contributor

@vishalbollu vishalbollu commented Mar 2, 2020 •
edited by deliahu
Loading

closes #590, closes #646

checklist:

  • run make test and make lint
  • test manually (i.e. build/push all images, restart operator, and re-deploy APIs)
  • update examples
  • update docs and add any new files to summary.md (view in gitbook after merging)

deliahu and others added 30 commits February 23, 2020 22:00
@deliahu
FastAPI demo
aa6cee5
@vishalbollu
Switch gunicorn to uvicorn
72abaed
@vishalbollu
flask to fast-api rough
c0aaaf9
@deliahu
Merge branch 'master' of github.com:cortexlabs/cortex into fastapi
b79b7d1
@vishalbollu
Cleanup gunicorn code
d3df3af
@deliahu
Add max_queue_length
0f2aacd
@deliahu
Update run.sh
7de8652
@deliahu
Set somaxconn
0c56233
@vishalbollu
Use regular function instead of async for predict endpoint
1c005dd
@deliahu
Update log formatter
da08f18
@deliahu
Remove api is live log line
4363113
@deliahu
Update tf.requirements.txt
bbf4304
@deliahu
Add pyyaml
37dd3ec
@deliahu
Use configuration in APIs
263061f
@vishalbollu
Add healthchecks and liveness probes
75359d8
@deliahu
Add comment for uvicorn config
789498e
@deliahu
Add license
1a8fc9c
@deliahu
Update avg request header
7efa210
@vishalbollu
Use background tasks and dedicated threadpool for requests
23fb6d6
@vishalbollu
Merge branch 'fastapi' of github.com:cortexlabs/cortex into fastapi
241ec7a
@deliahu
Remove extra log
14f09e5
@vishalbollu
Put latency metrics post request and use json-tricks if default json …
2fcd86d 
…serialization failes
@vishalbollu
Merge branch 'fastapi' of github.com:cortexlabs/cortex into fastapi
e8115d6
@deliahu
Add timestamp readiness probe
d70e4bb
@deliahu
Merge branch 'fastapi' of github.com:cortexlabs/cortex into fastapi
74f0799
@deliahu
Write current time to api readiness file
5130d9b
@deliahu
Update liveness and readiness checks
e410f5f
@deliahu
Update OOM calculation
e1c9453
@deliahu
Move request tracking file deletion to background
2294d08
@deliahu
Use request ID for request tracker files
c1a0b51
@deliahu deliahu changed the title Fastapi Update in-flight request counter, switch to FastAPI + Uvicorn Mar 4, 2020
@deliahu deliahu merged commit ae25f18 into master Mar 4, 2020
@deliahu deliahu deleted the fastapi branch March 4, 2020 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@deliahu deliahu deliahu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Allow users to configure request queue length Support API parallelism
2 participants
@vishalbollu @deliahu