Skip to content

Commit eebc741

Browse files
authored
docs: Adjust the path to examples (#2056)
1 parent ffb5409 commit eebc741

File tree

6 files changed

+18
-32
lines changed

6 files changed

+18
-32
lines changed

README.md

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -68,12 +68,13 @@ pip install "ai-dynamo[all]"
6868
6969
### Building the Dynamo Base Image
7070

71-
Although not needed for local development, deploying your Dynamo pipelines to Kubernetes will require you to build and push a Dynamo base image to your container registry. You can use any container registry of your choice, such as:
71+
Although not needed for local development, deploying your Dynamo pipelines to Kubernetes will require you to use a Dynamo base image to your container registry. You can use any container registry of your choice, such as:
7272
- Docker Hub (docker.io)
7373
- NVIDIA NGC Container Registry (nvcr.io)
7474
- Any private registry
7575

76-
Here's how to build it:
76+
We publish our images in [nvcr.io](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime) and you can use them.
77+
Alternatively you could build and push an image from source:
7778

7879
```bash
7980
./container/build.sh
@@ -83,8 +84,10 @@ docker push <your-registry>/dynamo-base:latest-vllm
8384
```
8485

8586
Notes about builds for specific frameworks:
86-
- For specific details on the `--framework vllm` build, see [here](examples/vllm/README.md).
87-
- For specific details on the `--framework tensorrtllm` build, see [here](examples/tensorrt_llm/README.md).
87+
- For specific details on the `--framework vllm` build [read about the VLLM backend](components/backends/vllm/README.md)
88+
.
89+
- For specific details on the `--framework tensorrtllm` build, see [Read about the TensorRT-LLM backend](components/backends/trtllm/README.md)
90+
.
8891

8992
Note about AWS environments:
9093
- If deploying Dynamo in AWS, make sure to build the container with EFA support using the `--make-efa` flag.
@@ -197,8 +200,6 @@ pip install .
197200
cd ../../../
198201
pip install ".[all]"
199202

200-
# To test
201-
docker compose -f deploy/metrics/docker-compose.yml up -d
202-
cd examples/llm
203-
dynamo serve graphs.agg:Frontend -f configs/agg.yaml
203+
Follow the [Quickstart Guide](docs/guides/dynamo_deploy/quickstart.md)
204+
204205
```

deploy/helm/README.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,10 @@ This approach allows you to install Dynamo directly using a DynamoGraphDeploymen
2929

3030
### Basic Installation
3131

32+
Here is how you would install a VLLM inference backend example.
33+
3234
```bash
33-
helm upgrade --install dynamo-graph ./deploy/helm/chart -n dynamo-cloud -f ./examples/vllm/deploy/agg.yaml
35+
helm upgrade --install dynamo-graph ./deploy/helm/chart -n dynamo-cloud -f ./components/backends/vllm/deploy/agg.yaml
3436
```
3537

3638
### Customizable Properties
@@ -39,7 +41,7 @@ You can override the default configuration by setting the following properties:
3941

4042
```bash
4143
helm upgrade --install dynamo-graph ./deploy/helm/chart -n dynamo-cloud \
42-
-f ./examples/vllm/deploy/agg.yaml \
44+
-f ./components/backends/vllm/deploy/agg.yaml \
4345
--set "imagePullSecrets[0].name=docker-secret-1" \
4446
--set etcdAddr="my-etcd-service:2379" \
4547
--set natsAddr="nats://my-nats-service:4222"

deploy/inference-gateway/example/README.md

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -16,11 +16,7 @@ This guide provides instructions for setting up the Inference Gateway with Dynam
1616
[See Quickstart Guide](../../../docs/guides/dynamo_deploy/quickstart.md) to install Dynamo Cloud.
1717

1818

19-
2. **Launch Dynamo Deployments**
20-
21-
[See VLLM Example](../../../examples/vllm/README.md)
22-
23-
3. **Deploy Inference Gateway**
19+
2. **Deploy Inference Gateway**
2420

2521
First, deploy an inference gateway service. In this example, we'll install `kgateway` based gateway implementation.
2622

@@ -54,7 +50,7 @@ kubectl get gateway inference-gateway
5450
# inference-gateway kgateway True 1m
5551
```
5652

57-
4. **Apply Dynamo-specific manifests**
53+
3. **Apply Dynamo-specific manifests**
5854

5955
The Inference Gateway is configured through the `inference-gateway-resources.yaml` file.
6056

docs/examples/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
## Serving examples locally
44

5-
TODO: Follow individual examples to serve models locally.
5+
Follow individual examples under components/backends/ to serve models locally.
66

77

88
## Deploying Examples to Kubernetes
@@ -38,7 +38,7 @@ export NAMESPACE=<your-namespace> # the namespace you used to deploy Dynamo clou
3838
Deploying an example consists of the simple `kubectl apply -f ... -n ${NAMESPACE}` command. For example:
3939

4040
```bash
41-
kubectl apply -f examples/vllm/deploy/agg.yaml -n ${NAMESPACE}
41+
kubectl apply -f components/backends/vllm/deploy/agg.yaml -n ${NAMESPACE}
4242
```
4343

4444
You can use `kubectl get dynamoGraphDeployment -n ${NAMESPACE}` to view your deployment.

docs/get_started.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -167,7 +167,7 @@ docker compose -f deploy/docker-compose.yml up -d
167167

168168
### Start Dynamo LLM Serving Components
169169

170-
[Explore the VLLM Example](../examples/vllm/README.md)
170+
[Explore the VLLM Example](../components/backends/vllm/README.md)
171171

172172

173173
## Local Development

docs/guides/dynamo_deploy/quickstart.md

Lines changed: 0 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -187,19 +187,6 @@ We provide a script to uninstall CRDs should you need a clean start.
187187

188188
## Explore Examples
189189

190-
Pick your deployment destination.
191-
192-
If local
193-
194-
```bash
195-
export DYNAMO_CLOUD=http://localhost:8080
196-
```
197-
198-
If kubernetes
199-
```bash
200-
export DYNAMO_CLOUD=https://dynamo-cloud.nvidia.com
201-
```
202-
203190
If deploying to Kubernetes, create a Kubernetes secret containing your sensitive values if needed:
204191

205192
```bash

0 commit comments

Comments
 (0)