Skip to content

Commit

Permalink
update model path in readme
Browse files Browse the repository at this point in the history
  • Loading branch information
tybalex committed Jun 28, 2024
1 parent d49ffbc commit 8a38ff2
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ npm install jsonrepair

2. Start the Server with a Rubra function calling model:
```
python -m vllm.entrypoints.openai.api_server --model sanjay920/Phi-3-mini-128k-instruct-function-calling-alpha-v1 --dtype auto --api-key token-abc123 --max-model-len 8000 --gpu-memory-utilization 0.96 --enforce-eager
python -m vllm.entrypoints.openai.api_server --model rubra-ai/Phi-3-mini-128k-instruct-function-calling-alpha-v1 --dtype auto --api-key token-abc123 --max-model-len 8000 --gpu-memory-utilization 0.96 --enforce-eager
```
The model will get downloaded automatically from huggingface.

Expand All @@ -39,7 +39,7 @@ curl localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer token-abc123" \
-d '{
"model": "sanjay920/Phi-3-mini-128k-instruct-function-calling-alpha-v1",
"model": "rubra-ai/Phi-3-mini-128k-instruct-function-calling-alpha-v1",
"messages": [
{
"role": "system",
Expand Down Expand Up @@ -81,7 +81,7 @@ tools = [
]
messages = [{"role": "user", "content": "What's the weather like in Boston today?"}]
completion = client.chat.completions.create(
model="sanjay920/Phi-3-mini-128k-instruct-function-calling-alpha-v1",
model="rubra-ai/Phi-3-mini-128k-instruct-function-calling-alpha-v1",
messages=messages,
tools=tools,
tool_choice="auto"
Expand Down

0 comments on commit 8a38ff2

Please sign in to comment.