doc: Update vllm.rst #4824
Merged
doc: Update vllm.rst #4824
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sherlock113
added
the
documentation
frostming
added a commit
that referenced
this pull request
Jul 18, 2024
* feat: support adding response background tasks via bentoml.Context (#4754) * feat: add secret management commands (#4764) * fix: add secrets management command * chore: add file type secret * chore: update command help text * chore: add deployment option secret * fix: add delete command * chore: add secret Apply command * chore: update mountAs * chore: update import * chore: update secret api route * feat: impl client async endpoints and results (#4768) * feat: support adding response background tasks via bentoml.Context Signed-off-by: Frost Ming <me@frostming.com> * fix: remove init Signed-off-by: Frost Ming <me@frostming.com> * fix: reorganize Signed-off-by: Frost Ming <me@frostming.com> * fix: cancel the timeout check at the first send Signed-off-by: Frost Ming <me@frostming.com> * feat: async task and result Signed-off-by: Frost Ming <me@frostming.com> * feat: new route and openapi schema Signed-off-by: Frost Ming <me@frostming.com> * feat: retry endpoint Signed-off-by: Frost Ming <me@frostming.com> * feat: impl client async endpoints and results Signed-off-by: Frost Ming <me@frostming.com> * fix: test client Signed-off-by: Frost Ming <me@frostming.com> * fix: rename is_async to is_task Signed-off-by: Frost Ming <me@frostming.com> * fix: add timestamp Signed-off-by: Frost Ming <me@frostming.com> --------- Signed-off-by: Frost Ming <me@frostming.com> * feat: external_queue default to 1 when using tasks Signed-off-by: Frost Ming <me@frostming.com> * feat: get_service api for selected frameworks (#4782) * feat: support adding response background tasks via bentoml.Context (#4754) * feat: add secret management commands (#4764) * fix: add secrets management command * chore: add file type secret * chore: update command help text * chore: add deployment option secret * fix: add delete command * chore: add secret Apply command * chore: update mountAs * chore: update import * chore: update secret api route * feat: get_service api for selected frameworks Signed-off-by: Frost Ming <me@frostming.com> * fix: framework importer Signed-off-by: Frost Ming <me@frostming.com> * fix: typing import Signed-off-by: Frost Ming <me@frostming.com> * fix: deduplicate finder Signed-off-by: Frost Ming <me@frostming.com> * doc: update docs Signed-off-by: Frost Ming <me@frostming.com> * fix tests Signed-off-by: Frost Ming <me@frostming.com> * fix tests Signed-off-by: Frost Ming <me@frostming.com> * fix tests Signed-off-by: Frost Ming <me@frostming.com> * fix: tests for get_service Signed-off-by: Frost Ming <me@frostming.com> * fix: install dependencies Signed-off-by: Frost Ming <me@frostming.com> * fix: show dependency tree Signed-off-by: Frost Ming <me@frostming.com> * fix: upgrade tensorflow Signed-off-by: Frost Ming <me@frostming.com> * fix: dependencies Signed-off-by: Frost Ming <me@frostming.com> * fix: catboost test Signed-off-by: Frost Ming <me@frostming.com> * ci: auto fixes from pre-commit.ci For more information, see https://pre-commit.ci --------- Signed-off-by: Frost Ming <me@frostming.com> Co-authored-by: xianxian.zhang <141706136+xianml@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * feat: cli auto login (#4817) * chore: support cli auto login * feat: add auth server * ci: auto fixes from pre-commit.ci For more information, see https://pre-commit.ci * ci: auto fixes from pre-commit.ci For more information, see https://pre-commit.ci * chore: fix by comments * chore: encode uri * ci: auto fixes from pre-commit.ci For more information, see https://pre-commit.ci * chore: add cors * ci: auto fixes from pre-commit.ci For more information, see https://pre-commit.ci * chore: clean unused code * ci: auto fixes from pre-commit.ci For more information, see https://pre-commit.ci * ci: auto fixes from pre-commit.ci For more information, see https://pre-commit.ci * chore: clean unused code --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * feat: use rich.print to replace click.echo (#4827) * chore: update print and fix cors error (#4829) * feat: login to current context instead of default context if --context is not passed (#4828) * feat: switch to uv as the resolver and installer (#4822) * feat: switch to uv Signed-off-by: Frost Ming <me@frostming.com> * feat: Officially support Python 3.12 (#4830) * Docs: Add auto cli login docs (#4844) Add auto cli login docs Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> * doc: Fix typo and format (#4821) Fix typo and format Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> * fix(cli): hanging issue when deployment ready (#4825) * fix(cli): hanging issue when deployment ready Signed-off-by: Frost Ming <me@frostming.com> * fix(deployment-client): add build process and extend url (#4790) * fix(deployement-client): add build process and extend url * chore: mute if not using cli * ci: auto fixes from pre-commit.ci For more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Frost Ming <me@frostming.com> Signed-off-by: Frost Ming <me@frostming.com> * doc: Update vllm.rst (#4824) Update vllm.rst * docs: Add custom models use case doc (#4819) * Add custom models use case doc Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> * ci: auto fixes from pre-commit.ci For more information, see https://pre-commit.ci * Update examples/xgboost/save_model.py Co-authored-by: Frost Ming <mianghong@gmail.com> * Update examples/mlflow/save_model.py --------- Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Frost Ming <mianghong@gmail.com> Signed-off-by: Frost Ming <me@frostming.com> * docs: Reorg blip doc to multimodality section (#4826) Reorg blip doc to multimodality section Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> * fix: inject prometheus multiproc_dir into env var when worker starts (#4820) * fix: inject prometheus multiproc_dir into env var when worker starts Signed-off-by: Frost Ming <me@frostming.com> * fix: remove unneeded tests Signed-off-by: Frost Ming <me@frostming.com> * fix: tests Signed-off-by: Frost Ming <me@frostming.com> * fix: replace bentoml.metrics with prometheus_client Signed-off-by: Frost Ming <me@frostming.com> --------- Signed-off-by: Frost Ming <me@frostming.com> * docs: Add logging and tracing docs (#4725) * Add logging and tracing docs Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> * Fix format * ci: auto fixes from pre-commit.ci For more information, see https://pre-commit.ci --------- Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * feat: --label option for bentoml build (#4832) * feat: --label option for bentoml build Signed-off-by: Frost Ming <me@frostming.com> * docs: Update the concurrency doc (#4834) * Update the concurrency doc Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> * Update descriptions Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> --------- Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> * docs: Update services.rst (#4837) Update services.rst In the Python classes I changed the reference to the class to `self` from `this`. `this` is not a keyword in Python * docs: Update concurrency links (#4840) Update concurrency links Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> * docs: Fix broken link (#4841) Fix broken link Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> * ci: pre-commit autoupdate [skip ci] (#4838) updates: - [github.com/astral-sh/ruff-pre-commit: v0.4.7 → v0.5.0](astral-sh/ruff-pre-commit@v0.4.7...v0.5.0) - [github.com/pdm-project/pdm: 2.15.4 → 2.16.1](pdm-project/pdm@2.15.4...2.16.1) - [github.com/bufbuild/buf: v1.32.2 → v1.34.0](bufbuild/buf@v1.32.2...v1.34.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * doc: Fix typo and format (#4821) Fix typo and format Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> Signed-off-by: Frost Ming <me@frostming.com> * fix: stop spinner before starting to stream logs (#4846) Signed-off-by: Frost Ming <me@frostming.com> * docs: Add azure byoc setup doc (#4847) * Add azure byoc step doc Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> * Fix space Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> * ci: auto fixes from pre-commit.ci For more information, see https://pre-commit.ci --------- Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update azure.rst * docs: Update the BYOC architecture image (#4848) Update the BYOC architecture image Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> * update lock file Signed-off-by: Frost Ming <me@frostming.com> * feat: retry and cancel task (#4852) * fix: options are not required (#4861) * fix: options are not required * ci: auto fixes from pre-commit.ci For more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * fix: don't fail when image pod is not found (#4862) * fix: async task response deserialization crash (#4865) * fix: raise deploy timeout to 1hr Signed-off-by: Frost Ming <me@frostming.com> * fix: remove references to yatai from cli help Signed-off-by: Frost Ming <me@frostming.com> * feat: change spinner text on status change and show spinner after image build done (#4867) Signed-off-by: Frost Ming <me@frostming.com> * fix: do not raise deployment pods not found error (#4868) * fix: console print Signed-off-by: Frost Ming <me@frostming.com> --------- Signed-off-by: Frost Ming <me@frostming.com> Signed-off-by: Sherlock113 <sherlockxu07@gmail.com> Co-authored-by: xianxian.zhang <141706136+xianml@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sherlock Xu <65327072+Sherlock113@users.noreply.github.com> Co-authored-by: Steve Guo <44340365+Haivilo@users.noreply.github.com> Co-authored-by: Vikramjit Sidhu <vikram.sidhu.007@gmail.com> Co-authored-by: yetone <yetoneful@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR address?
Fixes #(issue)
Before submitting:
guide on how to create a pull request.
pre-commit run -ascript has passed (instructions)?those accordingly? Here are documentation guidelines and tips on writting docs.