docs: Adding document for running Dynamo on Azure Kubernetes Services #2080
Conversation
|
👋 Hi saurabh-nvidia! Thank you for contributing to ai-dynamo/dynamo. Just a reminder: The 🚀 |
github-actions
bot
added
the
external-contribution
saurabh-nvidia
changed the title
WalkthroughA new markdown documentation file has been added that provides a detailed, step-by-step guide for deploying Dynamo Cloud with vLLM distributed runtime on Azure Kubernetes Service (AKS). The guide covers infrastructure setup, GPU configuration, model deployment, testing, and resource cleanup procedures. Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Poem
Note ⚡️ Unit Test Generation - BetaCodeRabbit's unit test generation is now available in Beta! Automatically generate comprehensive unit tests for your code changes, ensuring better test coverage and catching edge cases you might miss. Our AI analyzes your code structure and creates tests that follow best practices and your project's testing patterns. Learn more here, or just try it under ✨ Finishing Touches. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
examples/deployments/AKS-deployment.md (2)
10-15: Add language identifiers to fenced code blocks (MD040)All fenced blocks lack a language tag, triggering MD040 warnings and disabling syntax highlighting. Prefix them with an appropriate language (mostly
bash):-``` +```bash az login @@ -``` +```bash ssh-keygen -t rsa -b 4096 -C "<email@id.com>" @@ -``` +```bash export REGION=<region> @@ -``` +```bash az aks nodepool add … @@ -``` +```bash helm repo add nvidia … @@ -``` +```bash helm install … @@ -``` +```bash kubectl get pods -A -o wide @@ -``` +```bash helm fetch https://helm.ngc.nvidia.com/… @@ -``` +```bash curl localhost:8000/v1/chat/completions …This minor edit greatly improves legibility.
Also applies to: 18-20, 23-31, 46-48, 55-57, 60-62, 65-67, 71-77, 93-106, 167-183
17-20: Parameterise the SSH public-key path to avoid hard-coded/home/userReaders who run the guide from a different account will hit a file not found error. Export the key path once and reuse it:
-export REGION=<region> +export REGION=<region> +export SSH_KEY=$HOME/.ssh/id_rsa.pub # or your custom path @@ -az aks create -g $RESOURCE_GROUP … --ssh-key-value /home/user/.ssh/id_rsa.pub +az aks create -g $RESOURCE_GROUP … --ssh-key-value $SSH_KEYSmall but eliminates a common stumbling block.
Also applies to: 30-31
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
examples/deployments/AKS-deployment.md(1 hunks)
🧰 Additional context used
🪛 LanguageTool
examples/deployments/AKS-deployment.md
[grammar] ~9-~9: Ensure spelling is correct
Context: ...ent 1. Open Azure Cloud Shell or a ternimal on an Azure VM and install pre-reqs: ``...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
[style] ~189-~189: Consider a more concise word here.
Context: ...: "stop"}]} ``` ## Clean Up Resources In order to clean up any Dynamo related resources, ...
(IN_ORDER_TO_PREMIUM)
🪛 markdownlint-cli2 (0.17.2)
examples/deployments/AKS-deployment.md
7-7: Heading levels should only increment by one level at a time
Expected: h2; Actual: h3
(MD001, heading-increment)
10-10: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
18-18: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
23-23: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
46-46: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
55-55: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
60-60: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
65-65: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
71-71: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
93-93: Bare URL used
(MD034, no-bare-urls)
167-167: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🪛 GitHub Actions: Pre Merge Validation of (ai-dynamo/dynamo/refs/pull/2080/merge) by saurabh-nvidia.
examples/deployments/AKS-deployment.md
[error] 26-190: Pre-commit hook 'trailing-whitespace' failed and fixed trailing whitespace issues in this file. Please run 'pre-commit run --all-files' locally to apply fixes.
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Build and Test - vllm
🔇 Additional comments (2)
examples/deployments/AKS-deployment.md (2)
9-9: Correct typos for a professional document
- “ternimal” → “terminal”
- “neccessary” → “necessary”
- “number os replicas” → “number of replicas”
-1. Open **Azure Cloud Shell** or a ternimal on an Azure VM … +1. Open **Azure Cloud Shell** or a terminal on an Azure VM … -#The above linked document says to authenticate using NGC_API_KEY, not neccessary, since … +#The above linked document says to authenticate using NGC_API_KEY, not necessary, since … -# You can edit the number os replicas of encoder/ decoder independently … +# You can edit the number of replicas of encoder/decoder independently …[ suggest_nitpick ]
Also applies to: 98-99, 145-146
193-198: Trailing whitespace fixed by pre-commit—re-run hooks before pushingCI auto-stripped whitespace (see pipeline failure). To keep history clean, execute:
pre-commit run --all-files git add examples/deployments/AKS-deployment.md git commit --amend --no-edit git push --force-with-leaseEnsures the file passes future hooks.
|
Tagging 🥇 @mc-nv who recently set up our dev/ci dynamo cluster on AKS ! |
ishandhanani
left a comment
ishandhanani
left a comment
There was a problem hiding this comment.
Have not tested this myself but looks good overall. We should somehow figure out how to get this into CI
5 participants
Document showing how to create an AKS cluster with GPUs, install GPU operator, install Dynamo (using the public image on NGC) and run a Phi 3.5 Vision Instruct model.
I have created the folder directory as recommended by Anish.
Summary by CodeRabbit