Skip to content

Commit 2698a94

Browse files
committed
small fixes for pre deployment profiling
1 parent b753391 commit 2698a94

File tree

2 files changed

+15
-38
lines changed

2 files changed

+15
-38
lines changed

benchmarks/profiler/deploy/profile_sla_job.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ spec:
2929
command: ["python", "-m", "benchmarks.profiler.profile_sla"]
3030
args:
3131
- --config
32-
- /data/configs/disagg.yaml
32+
- ${DGD_CONFIG_FILE}
3333
- --output-dir
3434
- /data/profiling_results
3535
- --namespace

docs/benchmarks/pre_deployment_profiling.md

Lines changed: 14 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -100,31 +100,6 @@ pip install -r deploy/utils/requirements.txt
100100

101101
Use the injector utility to place your DGD manifest into the PVC. The profiling job will read the path you specify.
102102

103-
```bash
104-
# Inject your disagg manifest
105-
python3 -m deploy.utils.inject_manifest \
106-
--namespace $NAMESPACE \
107-
--src components/backends/vllm/deploy/disagg.yaml \
108-
--dest /data/configs/disagg.yaml
109-
110-
# Set the docker image for the profiling job; any docker image that contains your script.
111-
export DOCKER_IMAGE=nvcr.io/nvidia/dynamo:latest-vllm
112-
```
113-
114-
### Configure container image (optional)
115-
116-
You have two options for configuring your profiling setup:
117-
118-
**Option A: Use pre-built image with custom config injection (recommended)**
119-
120-
Use the default pre-built image and inject custom configurations via PVC:
121-
122-
1. **Set the container image:**
123-
```bash
124-
export DOCKER_IMAGE=nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.4.1 # or any existing image tag
125-
```
126-
127-
2. **Inject your custom disagg configuration:**
128103
```bash
129104
# Use default disagg.yaml config
130105
python3 -m deploy.utils.inject_manifest --namespace $NAMESPACE --src components/backends/vllm/deploy/disagg.yaml --dest /data/configs/disagg.yaml
@@ -138,16 +113,6 @@ Use the default pre-built image and inject custom configurations via PVC:
138113

139114
> **Note**: All paths must start with `/data/` for security reasons. If you forget this prefix, the script will show a helpful error message with the correct path.
140115
141-
3. **Set the config path for the profiling job:**
142-
```bash
143-
export DGD_CONFIG_FILE=/workspace/profiling_results/disagg.yaml # or your custom path
144-
```
145-
146-
This approach allows you to:
147-
- Customize DGD configurations without rebuilding container images
148-
- Test different model configurations easily
149-
- Version control your DGD configs alongside your code
150-
151116
> **Important**: For profiling, disagg configs should be run with Grove disabled by adding the annotation `nvidia.com/enable-grove: "false"` to avoid alpha Grove status issues.
152117
153118
**Step 2: Set SLA target**
@@ -173,13 +138,25 @@ spec:
173138
- <vllm/sglang>
174139
```
175140
176-
**Step 3: Run profiling (required)**
141+
**Step 3: Define the container image and config path**
142+
143+
1. **Set the container image:**
144+
```bash
145+
export DOCKER_IMAGE=nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.4.1 # or any existing image tag (TODO: update to 0.5.0 upon release as profiling with 0.4.1 is broken)
146+
```
147+
148+
3. **Set the config path for the profiling job:**
149+
```bash
150+
export DGD_CONFIG_FILE=/data/configs/disagg.yaml # should be the same path you set for --dest in Step 1
151+
```
152+
153+
**Step 4: Run profiling (required)**
177154

178155
```bash
179156
envsubst < benchmarks/profiler/deploy/profile_sla_job.yaml | kubectl apply -f -
180157
```
181158

182-
**Step 4: Wait for profiling to complete**
159+
**Step 5: Wait for profiling to complete**
183160
```bash
184161
kubectl get jobs -n $NAMESPACE
185162
kubectl logs job/profile-sla -n $NAMESPACE

0 commit comments

Comments
 (0)