Skip to content

Commit 93b078e

Browse files
rmccorm4rmccorm4
authored
docs: Simplify sphinx build and table of contents on webpage (#2519) (#2703)
1 parent fd8b52f commit 93b078e

File tree

29 files changed

+303
-697
lines changed

29 files changed

+303
-697
lines changed

‎components/backends/sglang/deploy/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,7 @@ All templates use **DeepSeek-R1-Distill-Llama-8B** as the default model. But you
145145
## Further Reading
146146

147147
- **Deployment Guide**: [Creating Kubernetes Deployments](../../../../docs/guides/dynamo_deploy/create_deployment.md)
148-
- **Quickstart**: [Deployment Quickstart](../../../../docs/guides/dynamo_deploy/quickstart.md)
148+
- **Quickstart**: [Deployment Quickstart](../../../../docs/guides/dynamo_deploy/README.md)
149149
- **Platform Setup**: [Dynamo Cloud Installation](../../../../docs/guides/dynamo_deploy/dynamo_cloud.md)
150150
- **Examples**: [Deployment Examples](../../../../docs/examples/README.md)
151151
- **Kubernetes CRDs**: [Custom Resources Documentation](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
@@ -159,4 +159,4 @@ Common issues and solutions:
159159
3. **Health check failures**: Review model loading logs and increase `initialDelaySeconds`
160160
4. **Out of memory**: Increase memory limits or reduce model batch size
161161

162-
For additional support, refer to the [deployment guide](../../../../docs/guides/dynamo_deploy/quickstart.md).
162+
For additional support, refer to the [deployment guide](../../../../docs/guides/dynamo_deploy/README.md).

‎components/backends/trtllm/deploy/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ extraPodSpec:
8181

8282
Before using these templates, ensure you have:
8383

84-
1. **Dynamo Cloud Platform installed** - See [Quickstart Guide](../../../../docs/guides/dynamo_deploy/quickstart.md)
84+
1. **Dynamo Cloud Platform installed** - See [Quickstart Guide](../../../../docs/guides/dynamo_deploy/README.md)
8585
2. **Kubernetes cluster with GPU support**
8686
3. **Container registry access** for TensorRT-LLM runtime images
8787
4. **HuggingFace token secret** (referenced as `envFromSecret: hf-token-secret`)
@@ -257,7 +257,7 @@ Configure the `model` name and `host` based on your deployment.
257257
## Further Reading
258258

259259
- **Deployment Guide**: [Creating Kubernetes Deployments](../../../../docs/guides/dynamo_deploy/create_deployment.md)
260-
- **Quickstart**: [Deployment Quickstart](../../../../docs/guides/dynamo_deploy/quickstart.md)
260+
- **Quickstart**: [Deployment Quickstart](../../../../docs/guides/dynamo_deploy/README.md)
261261
- **Platform Setup**: [Dynamo Cloud Installation](../../../../docs/guides/dynamo_deploy/dynamo_cloud.md)
262262
- **Examples**: [Deployment Examples](../../../../docs/examples/README.md)
263263
- **Architecture Docs**: [Disaggregated Serving](../../../../docs/architecture/disagg_serving.md), [KV-Aware Routing](../../../../docs/architecture/kv_cache_routing.md)
@@ -277,4 +277,4 @@ Common issues and solutions:
277277
6. **Git LFS issues**: Ensure git-lfs is installed before building containers
278278
7. **ARM deployment**: Use `--platform linux/arm64` when building on ARM machines
279279

280-
For additional support, refer to the [deployment troubleshooting guide](../../../../docs/guides/dynamo_deploy/quickstart.md#troubleshooting).
280+
For additional support, refer to the [deployment troubleshooting guide](../../../../docs/guides/dynamo_deploy/README.md).

‎components/backends/vllm/deploy/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ extraPodSpec:
8282

8383
Before using these templates, ensure you have:
8484

85-
1. **Dynamo Cloud Platform installed** - See [Quickstart Guide](../../../../docs/guides/dynamo_deploy/quickstart.md)
85+
1. **Dynamo Cloud Platform installed** - See [Quickstart Guide](../../../../docs/guides/dynamo_deploy/README.md)
8686
2. **Kubernetes cluster with GPU support**
8787
3. **Container registry access** for vLLM runtime images
8888
4. **HuggingFace token secret** (referenced as `envFromSecret: hf-token-secret`)
@@ -236,7 +236,7 @@ args:
236236
## Further Reading
237237
238238
- **Deployment Guide**: [Creating Kubernetes Deployments](../../../../docs/guides/dynamo_deploy/create_deployment.md)
239-
- **Quickstart**: [Deployment Quickstart](../../../../docs/guides/dynamo_deploy/quickstart.md)
239+
- **Quickstart**: [Deployment Quickstart](../../../../docs/guides/dynamo_deploy/README.md)
240240
- **Platform Setup**: [Dynamo Cloud Installation](../../../../docs/guides/dynamo_deploy/dynamo_cloud.md)
241241
- **SLA Planner**: [SLA Planner Deployment Guide](../../../../docs/guides/dynamo_deploy/sla_planner_deployment.md)
242242
- **Examples**: [Deployment Examples](../../../../docs/examples/README.md)
@@ -252,4 +252,4 @@ Common issues and solutions:
252252
4. **Out of memory**: Increase memory limits or reduce model batch size
253253
5. **Port forwarding issues**: Ensure correct pod UUID in port-forward command
254254

255-
For additional support, refer to the [deployment troubleshooting guide](../../../../docs/guides/dynamo_deploy/quickstart.md#troubleshooting).
255+
For additional support, refer to the [deployment troubleshooting guide](../../../../docs/guides/dynamo_deploy/README.md).

‎deploy/inference-gateway/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ Currently, these setups are only supported with the kGateway based Inference Gat
2020

2121
1. **Install Dynamo Platform**
2222

23-
[See Quickstart Guide](../../docs/guides/dynamo_deploy/quickstart.md) to install Dynamo Cloud.
23+
[See Quickstart Guide](../../docs/guides/dynamo_deploy/README.md) to install Dynamo Cloud.
2424

2525

2626
2. **Deploy Inference Gateway**

‎docs/_includes/dive_in_examples.rst‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
The examples below assume you build the latest image yourself from source. If using a prebuilt image follow the examples from the corresponding branch.
2+
3+
.. grid:: 1 2 2 2
4+
:gutter: 3
5+
:margin: 0
6+
:padding: 3 4 0 0
7+
8+
.. grid-item-card:: :doc:`Hello World <../examples/runtime/hello_world/README>`
9+
:link: ../examples/runtime/hello_world/README
10+
:link-type: doc
11+
12+
Demonstrates the basic concepts of Dynamo by creating a simple GPU-unaware graph
13+
14+
.. grid-item-card:: :doc:`vLLM <../components/backends/vllm/README>`
15+
:link: ../components/backends/vllm/README
16+
:link-type: doc
17+
18+
Presents examples and reference implementations for deploying Large Language Models (LLMs) in various configurations with VLLM.
19+
20+
.. grid-item-card:: :doc:`SGLang <../components/backends/sglang/README>`
21+
:link: ../components/backends/sglang/README
22+
:link-type: doc
23+
24+
Presents examples and reference implementations for deploying Large Language Models (LLMs) in various configurations with SGLang.
25+
26+
.. grid-item-card:: :doc:`TensorRT-LLM <../components/backends/trtllm/README>`
27+
:link: ../components/backends/trtllm/README
28+
:link-type: doc
29+
30+
Presents examples and reference implementations for deploying Large Language Models (LLMs) in various configurations with TensorRT-LLM.
31+
32+

‎docs/_includes/install.rst‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
Pip (PyPI)
2+
----------
3+
4+
Install a pre-built wheel from PyPI.
5+
6+
.. code-block:: bash
7+
8+
# Create a virtual environment and activate it
9+
uv venv venv
10+
source venv/bin/activate
11+
12+
# Install Dynamo from PyPI (choose one backend extra)
13+
uv pip install "ai-dynamo[sglang]==0.4.1" # or [vllm], [trtllm]
14+
15+
16+
Pip from source
17+
---------------
18+
19+
Install directly from a local checkout for development.
20+
21+
.. code-block:: bash
22+
23+
# Clone the repository
24+
git clone https://github.com/ai-dynamo/dynamo.git
25+
cd dynamo
26+
27+
# Create a virtual environment and activate it
28+
uv venv venv
29+
source venv/bin/activate
30+
uv pip install ".[sglang]" # or [vllm], [trtllm]
31+
32+
33+
Docker
34+
------
35+
36+
Pull and run prebuilt images from NVIDIA NGC (`nvcr.io`).
37+
38+
.. code-block:: bash
39+
40+
# Run a container (mount your workspace if needed)
41+
docker run --rm -it \
42+
--gpus all \
43+
--network host \
44+
nvcr.io/nvidia/ai-dynamo/sglang-runtime:0.4.1 # or vllm, tensorrtllm

‎docs/_includes/quick_start_local.rst‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
Get started with Dynamo locally in just a few commands:
2+
3+
**1. Install Dynamo**
4+
5+
.. code-block:: bash
6+
7+
# Install uv (recommended Python package manager)
8+
curl -LsSf https://astral.sh/uv/install.sh | sh
9+
10+
# Create virtual environment and install Dynamo
11+
uv venv venv
12+
source venv/bin/activate
13+
uv pip install "ai-dynamo[sglang]==0.4.1" # or [vllm], [trtllm]
14+
15+
**2. Start etcd/NATS**
16+
17+
.. code-block:: bash
18+
19+
# Fetch and start etcd and NATS using Docker Compose
20+
curl -fsSL -o docker-compose.yml https://raw.githubusercontent.com/ai-dynamo/dynamo/release/0.4.1/deploy/docker-compose.yml
21+
docker compose -f docker-compose.yml up -d
22+
23+
**3. Run Dynamo**
24+
25+
.. code-block:: bash
26+
27+
# Start the OpenAI compatible frontend (default port is 8080)
28+
python -m dynamo.frontend
29+
30+
# In another terminal, start an SGLang worker
31+
python -m dynamo.sglang --model-path Qwen/Qwen3-0.6B
32+
33+
**4. Test your deployment**
34+
35+
.. code-block:: bash
36+
37+
curl localhost:8080/v1/chat/completions \
38+
-H "Content-Type: application/json" \
39+
-d '{"model": "Qwen/Qwen3-0.6B",
40+
"messages": [{"role": "user", "content": "Hello!"}],
41+
"max_tokens": 50}'
42+
43+

‎docs/_sections/architecture.rst‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
Overview
2+
============
3+
4+
.. include:: ../architecture/architecture.md
5+
:parser: myst_parser.sphinx_
6+
7+
.. toctree::
8+
:hidden:
9+
10+
Overview <self>
11+
Disaggregated Serving <../architecture/disagg_serving>

‎docs/_sections/backends.rst‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
..
2+
SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
SPDX-License-Identifier: Apache-2.0
4+
5+
Licensed under the Apache License, Version 2.0 (the "License");
6+
you may not use this file except in compliance with the License.
7+
You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
17+
Backends
18+
========
19+
20+
NVIDIA Dynamo supports multiple inference backends to provide flexibility and performance optimization for different use cases and model architectures. Backends are the underlying engines that execute AI model inference, each optimized for specific scenarios, hardware configurations, and performance requirements.
21+
22+
Overview
23+
--------
24+
25+
Dynamo's multi-backend architecture allows you to:
26+
27+
* **Choose the optimal engine** for your specific workload and hardware
28+
* **Switch between backends** without changing your application code
29+
* **Leverage specialized optimizations** from each backend
30+
* **Scale flexibly** across different deployment scenarios
31+
32+
Supported Backends
33+
------------------
34+
35+
Dynamo currently supports the following high-performance inference backends:
36+
37+
.. toctree::
38+
:maxdepth: 1
39+
40+
vLLM <../components/backends/vllm/README>
41+
SGLang <../components/backends/sglang/README>
42+
TensorRT-LLM <../components/backends/trtllm/README>

‎docs/_sections/examples.rst‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
..
2+
Quickstart Page (left sidebar target)
3+
..
4+
5+
Examples
6+
========
7+
8+
.. include:: ../_includes/dive_in_examples.rst

0 commit comments

Comments
 (0)