[train] Update vicuna release test example to use V2 #57767
Conversation
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This PR updates the cluster configuration for the Vicuna example to use 16 worker GPUs for training, which is a valid change. However, this introduces an inconsistency with the documentation in the accompanying Jupyter Notebook, which has not been updated. I've left a specific comment on the configuration file change. Please update the notebook documentation to match the new cluster setup to avoid user confusion.
| instance_type: m5.4xlarge | ||
|
|
||
| worker_node_types: | ||
| - name: worker_node | ||
| instance_type: g5.4xlarge | ||
| min_workers: 15 | ||
| max_workers: 15 | ||
| min_workers: 16 | ||
| max_workers: 16 |
There was a problem hiding this comment.
These changes correctly adjust the cluster to use 16 worker GPUs for training. However, this makes the documentation in the corresponding notebook (vicuna_13b_lightning_deepspeed_finetune.ipynb) outdated and misleading.
The notebook's 'Cluster Setting' section needs to be updated to reflect:
- The head node is now
m5.4xlarge(a CPU instance). - There are now 16 worker nodes.
- The tip about using a GPU head node for inference is no longer accurate and should be revised, as inference will now run on a worker node.
Please update the notebook to ensure the example remains consistent and clear for users.
| "name": "stderr", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "2025-10-15 15:50:45,333\tINFO worker.py:1833 -- Connecting to existing Ray cluster at address: 10.0.171.127:6379...\n", |
There was a problem hiding this comment.
We should delete these output cells here and elsewhere so they don't show up in the docs.
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
ray-gardener
bot
added
docs
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
Signed-off-by: Justin Yu <justinvyu@anyscale.com>
Updates the vicuna lightning deepspeed example to run w/ Train V2. --------- Signed-off-by: Justin Yu <justinvyu@anyscale.com>
Updates the vicuna lightning deepspeed example to run w/ Train V2. --------- Signed-off-by: Justin Yu <justinvyu@anyscale.com> Signed-off-by: xgui <xgui@anyscale.com>
Updates the vicuna lightning deepspeed example to run w/ Train V2. --------- Signed-off-by: Justin Yu <justinvyu@anyscale.com>
Updates the vicuna lightning deepspeed example to run w/ Train V2. --------- Signed-off-by: Justin Yu <justinvyu@anyscale.com> Signed-off-by: Aydin Abiar <aydin@anyscale.com>
2 participants
Description
Updates the vicuna lightning deepspeed example to run w/ Train V2.
Related issues
Types of change
Checklist
Does this PR introduce breaking changes?
Testing:
Code Quality:
git commit -s)Documentation:
doc/source/(if applicable)Additional context