Skip to content

Commit 29dcb6d

Browse files
committed
[Doc] Elaborated basic pipeline parallelism tutorial example.
Signed-off-by: insukim1994 <insu.kim@moreh.io>
1 parent 6d11ed5 commit 29dcb6d

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

tutorials/15-basic-pipeline-parallel.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ This tutorial provides a step-by-step guide for configuring and deploying the vL
4444
- **`headNode`**: Specifies the resource requirements for the Kuberay head node and must be defined accordingly:
4545
- **`requestCPU`**: The amount of CPU resources requested for Kuberay head pod.
4646
- **`requestMemory`**: Memory allocation for Kuberay head pod. Sufficient memory is required to load the model.
47-
- **`requestGPU`**: Specifies the number of GPUs to allocate for Kuberay head pod.
47+
- **`requestGPU`**: Defines the number of GPUs to allocate for the KubeRay head pod. Currently, the Ray head node must also participate in both tensor parallelism and pipeline parallelism. This requirement exists because the `vllm serve ...` command is executed on the Ray head node, and vLLM mandates that the pod where this command is run must have at least one visible GPU.
4848
- **`name`**: The unique identifier for your model deployment.
4949
- **`repository`**: The Docker repository containing the model's serving engine image.
5050
- **`tag`**: Specifies the version of the model image to use.
@@ -69,6 +69,8 @@ This tutorial provides a step-by-step guide for configuring and deploying the vL
6969

7070
### Example Snippet
7171

72+
In the following example, we configure a total of two Ray nodes each equipped with two GPUs (one head node and one worker node) to serve a distilgpt2 model. We set the tensor parallelism size to 2, as each node contains two GPUs, and the pipeline parallelism size to 2, corresponding to the two Ray nodes being utilized.
73+
7274
```yaml
7375
servingEngineSpec:
7476
runtimeClassName: ""

0 commit comments

Comments
 (0)