feat(frontend): python frontend alternative to dynamo-run #1887
Conversation
grahamking
requested review from
biswapanda,
hhzhang16,
hutm,
ishandhanani,
julienmancuso,
mohammedabdulwahhab and
nnshah1
as code owners
WalkthroughA new CLI command, Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant CLI (Typer)
participant Frontend Module
participant DistributedRuntime
participant Engine
participant HTTP Server
User->>CLI (Typer): dynamo frontend [options]
CLI (Typer)->>Frontend Module: Invoke frontend()
Frontend Module->>Frontend Module: Parse options (port, cache size)
Frontend Module->>DistributedRuntime: Initialize runtime
Frontend Module->>Engine: make_engine (with options)
Engine->>Engine: Initialize engine (auto-discovery, etc.)
Frontend Module->>HTTP Server: run_input()
HTTP Server-->>User: Serve HTTP requests (OpenAI-compatible)
Poem
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
deploy/sdk/src/dynamo/sdk/cli/cli.py(2 hunks)deploy/sdk/src/dynamo/sdk/cli/frontend.py(1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
deploy/sdk/src/dynamo/sdk/cli/cli.py (1)
deploy/sdk/src/dynamo/sdk/cli/frontend.py (2)
cli(31-45)frontend(22-27)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Build and Test - vllm
🔇 Additional comments (5)
deploy/sdk/src/dynamo/sdk/cli/cli.py (2)
27-27: LGTM: Import statement is correctly placed.The import follows the existing alphabetical ordering pattern and correctly imports the
frontendfunction from the new module.
79-82: LGTM: Command registration follows established patterns.The registration uses the same context settings as the
runcommand, which is appropriate for a command that needs to handle extra arguments and unknown options. The disabled help option is consistent with the pattern used for similar commands.deploy/sdk/src/dynamo/sdk/cli/frontend.py (3)
10-18: LGTM: Imports are well-organized and appropriate.The imports are logically grouped and all appear to be used correctly in the implementation.
22-27: LGTM: Typer command implementation is correct.The function signature and implementation follow Typer best practices with appropriate defaults and type hints.
67-68: LGTM: Standard Python entry point pattern.The
if __name__ == "__main__"guard is correctly implemented.
grahamking
force-pushed
the
gk-dy-frontend
branch
from
4b8c890 to
91a36e1
Compare
grahamking
requested review from
a team,
GuanLuo,
PeaBrane,
alec-flowers,
jthomson04,
kkranen,
paulhendricks,
piotrm-nvidia,
ptarasiewiczNV,
rmccorm4,
ryanolson,
tanmayv25,
tedzhouhk and
tmonty12
as code owners
grahamking
changed the title
dynamo frontend alternative to dynamo-run
grahamking
force-pushed
the
gk-dy-frontend
branch
from
91a36e1 to
be2b390
Compare
grahamking
force-pushed
the
gk-dy-frontend
branch
from
be2b390 to
bd71f35
Compare
grahamking
force-pushed
the
gk-dy-frontend
branch
from
9ef7592 to
c7c0007
Compare
grahamking
changed the title
No need to build or install the `dynamo-run` Rust binary. This will have the same performance as `dynamo-run`, it uses the Rust library. In time we could split http server, pre-processor and router into separate bindings, and call them here, allow easy customization.
grahamking
force-pushed
the
gk-dy-frontend
branch
from
c7c0007 to
9b8d177
Compare
grahamking
force-pushed
the
gk-dy-frontend
branch
from
9b8d177 to
9767893
Compare
|
Will resubmit now that I understand what's needed. |
pip install -e .(from the root) anddynamo-frontendis now the Python equivalent ofdynamo-run in=http out=dyn.No need to build or install the
dynamo-runRust binary. This will have the same performance asdynamo-run, it uses the Rust library.In time we could split http server, pre-processor and router into separate bindings, and call them here, allow easy customization.
Temporarily the directory is
components/ingressbecausefrom components.frontend import ..conflicts with some examples.