Skip to content

Commit 37d6dc1

Browse files
committed
update docs on frontend image not existing for next release
1 parent b5db4e7 commit 37d6dc1

File tree

2 files changed

+6
-3
lines changed

2 files changed

+6
-3
lines changed

deploy/inference-gateway/README.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,7 @@ cd deploy/inference-gateway
111111

112112
# Export the Dynamo image you have used when deploying your model in Step 3.
113113
export DYNAMO_IMAGE=<the-dynamo-image-you-have-used-when-deploying-the-model>
114-
# Export the FrontEnd image tag provided by Dynamo (recommended) or build the Dynamo EPP image by following the commands later in this README.
114+
# ⚠️ Note: The frontend Dynamo image will not be included in the 0.7.0 release. You must build your own Dynamo EPP image by following the instructions in the "Build the custom EPP image" section below.
115115
export EPP_IMAGE=<the-epp-image-you-built>
116116
```
117117

@@ -146,11 +146,12 @@ You can configure the plugin by setting environment vars in your [values-dynamo-
146146

147147
Dynamo provides a custom routing plugin `pkg/epp/scheduling/plugins/dynamo_kv_scorer/plugin.go` to perform efficient kv routing.
148148
The Dynamo router is built as a static library, the EPP router will call to provide fast inference.
149-
You can either use the special FrontEnd image for the EPP_IMAGE in the Helm deployment command and proceed to the step 2 or you can build the image yourself following the steps below.
149+
150+
**⚠️ Important:** The frontend Dynamo image (`nvcr.io/nvidia/ai-dynamo/frontend:<my-tag>`) will not be included in the 0.7.0 release. You must build your own Dynamo EPP image following the steps below.
150151

151152
##### 1. Build the custom EPP image #####
152153

153-
If you choose to build your own image use the steps below.
154+
Build your own Dynamo EPP image using the steps below.
154155

155156
##### 1.1 Clone the official GAIE repo in a separate folder #####
156157

recipes/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -186,6 +186,8 @@ First, deploy the Dynamo Graph per instructions above.
186186

187187
Then follow [Deploy Inference Gateway Section 2](../deploy/inference-gateway/README.md#2-deploy-inference-gateway) to install GAIE.
188188

189+
**⚠️ Warning:** The frontend Dynamo image (`nvcr.io/nvidia/ai-dynamo/frontend:<my-tag>`) will not be included in the 0.7.0 release. You must build your own Dynamo EPP image following the instructions in [Build the custom EPP image](../deploy/inference-gateway/README.md#1-build-the-custom-epp-image) section of the Inference Gateway README.
190+
189191
Update the containers.epp.image in the deployment file, i.e. llama-3-70b/vllm/agg/gaie/k8s-manifests/epp/deployment.yaml. It should match the release tag and be in the format `nvcr.io/nvidia/ai-dynamo/frontend:<my-tag>` i.e. `nvcr.io/nvstaging/ai-dynamo/dynamo-frontend:0.7.0rc2-amd64`
190192

191193
```bash

0 commit comments

Comments
 (0)