You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#Export the FrontEnd image tag provided by Dynamo (recommended) or build the Dynamo EPP image by following the commands later in this README.
114
+
#⚠️ Note: The frontend Dynamo image will not be included in the 0.7.0 release. You must build your own Dynamo EPP image by following the instructions in the "Build the custom EPP image" section below.
115
115
export EPP_IMAGE=<the-epp-image-you-built>
116
116
```
117
117
@@ -146,11 +146,12 @@ You can configure the plugin by setting environment vars in your [values-dynamo-
146
146
147
147
Dynamo provides a custom routing plugin `pkg/epp/scheduling/plugins/dynamo_kv_scorer/plugin.go` to perform efficient kv routing.
148
148
The Dynamo router is built as a static library, the EPP router will call to provide fast inference.
149
-
You can either use the special FrontEnd image for the EPP_IMAGE in the Helm deployment command and proceed to the step 2 or you can build the image yourself following the steps below.
149
+
150
+
**⚠️ Important:** The frontend Dynamo image (`nvcr.io/nvidia/ai-dynamo/frontend:<my-tag>`) will not be included in the 0.7.0 release. You must build your own Dynamo EPP image following the steps below.
150
151
151
152
##### 1. Build the custom EPP image #####
152
153
153
-
If you choose to build your own image use the steps below.
154
+
Build your own Dynamo EPP image using the steps below.
154
155
155
156
##### 1.1 Clone the official GAIE repo in a separate folder #####
Copy file name to clipboardExpand all lines: recipes/README.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -186,6 +186,8 @@ First, deploy the Dynamo Graph per instructions above.
186
186
187
187
Then follow [Deploy Inference Gateway Section 2](../deploy/inference-gateway/README.md#2-deploy-inference-gateway) to install GAIE.
188
188
189
+
**⚠️ Warning:** The frontend Dynamo image (`nvcr.io/nvidia/ai-dynamo/frontend:<my-tag>`) will not be included in the 0.7.0 release. You must build your own Dynamo EPP image following the instructions in [Build the custom EPP image](../deploy/inference-gateway/README.md#1-build-the-custom-epp-image) section of the Inference Gateway README.
190
+
189
191
Update the containers.epp.image in the deployment file, i.e. llama-3-70b/vllm/agg/gaie/k8s-manifests/epp/deployment.yaml. It should match the release tag and be in the format `nvcr.io/nvidia/ai-dynamo/frontend:<my-tag>` i.e. `nvcr.io/nvstaging/ai-dynamo/dynamo-frontend:0.7.0rc2-amd64`
0 commit comments