|
1 | 1 | # Dynamo on AKS |
2 | 2 |
|
3 | 3 |
|
4 | | -This document covers the process of deploying Dynamo Cloud and running inference in a vLLM distributed runtime within a Azure Kubernetes environment. The Dynamo Cloud Platform provides a managed deployment experience: |
5 | | - |
6 | | -- Contains the infrastructure components required for the Dynamo cloud platform |
7 | | -- Used when deploying with the `dynamo deploy` CLI commands |
8 | | -- Provides a managed deployment experience |
9 | | - |
10 | | -This overview covers the setup process on a Azure Kubernetes Cluster, all the way from setup to testing inference. |
| 4 | +This document covers the process of deploying Dynamo Cloud and running inference in a vLLM distributed runtime within a Azure Kubernetes environment, covering the setup process on a Azure Kubernetes Cluster, all the way from setup to testing inference. |
11 | 5 |
|
12 | 6 |
|
13 | 7 | ### Task 1. Infrastructure Deployment |
@@ -130,7 +124,7 @@ dynamo-platform-nats-0 2/2 Runnin |
130 | 124 | dynamo-platform-nats-box-5dbf45c748-kln82 1/1 Running 0 2m51s |
131 | 125 | ``` |
132 | 126 |
|
133 | | -There are other ways to install Dynamo, you can find them [here] (https://github.com/ai-dynamo/dynamo/blob/main/docs/guides/dynamo_deploy/quickstart.md) |
| 127 | +There are other ways to install Dynamo, you can find them [here](https://github.com/ai-dynamo/dynamo/blob/main/docs/guides/dynamo_deploy/quickstart.md) |
134 | 128 |
|
135 | 129 | ### Task 4. Deploy a model |
136 | 130 |
|
@@ -160,7 +154,7 @@ kubectl get pods -n ${NAMESPACE} -o wide |
160 | 154 | kubectl logs <pod-name> -n ${NAMESPACE} |
161 | 155 | kubectl exec -it <pod-name> -n ${NAMESPACE} -- nvidia-smi |
162 | 156 |
|
163 | | -# Enable Port forwarding |
| 157 | +# Enable Port forwarding to be able to hit a curl request |
164 | 158 | kubectl get svc -n ${NAMESPACE} |
165 | 159 |
|
166 | 160 | #Look for one that ends in -frontend and use it for port forward. |
|
0 commit comments