[Core] Support custom image on Azure #3362
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Michaelvll
reviewed
Mar 25, 2024
There was a problem hiding this comment.
Thanks for adding the support for Azure custom image @cblmemo! This is awesome. Left several comments.
| @@ -194,6 +194,10 @@ Available fields: | |||
| # Or machine image: https://cloud.google.com/compute/docs/machine-images | |||
| # image_id: projects/my-project/global/machineImages/my-machine-image | |||
| # | |||
| # Azure | |||
| # To find Azure images: https://docs.microsoft.com/en-us/azure/virtual-machines/linux/cli-ps-findimage | |||
| # image_id: microsoft-dsvm:ubuntu-2004:2004:21.11.04 | |||
There was a problem hiding this comment.
Wondering if we should use / for separation or :
There was a problem hiding this comment.
this is the standard format on Azure (look at the fifth column):
az vm image list --publisher microsoft-dsvm --all --output table | grep ubuntu-hpc
x64 ubuntu-hpc microsoft-dsvm 1804 microsoft-dsvm:ubuntu-hpc:1804:18.04.2021051701 18.04.2021051701
x64 ubuntu-hpc microsoft-dsvm 1804 microsoft-dsvm:ubuntu-hpc:1804:18.04.2021110101 18.04.2021110101
x64 ubuntu-hpc microsoft-dsvm 1804 microsoft-dsvm:ubuntu-hpc:1804:18.04.2021120101 18.04.2021120101
x64 ubuntu-hpc microsoft-dsvm 1804 microsoft-dsvm:ubuntu-hpc:1804:18.04.2022061601 18.04.2022061601
x64 ubuntu-hpc microsoft-dsvm 1804 microsoft-dsvm:ubuntu-hpc:1804:18.04.2022121201 18.04.2022121201
x64 ubuntu-hpc microsoft-dsvm 1804 microsoft-dsvm:ubuntu-hpc:1804:18.04.2023031501 18.04.2023031501
x64 ubuntu-hpc microsoft-dsvm 1804 microsoft-dsvm:ubuntu-hpc:1804:18.04.2023063001 18.04.2023063001
x64 ubuntu-hpc microsoft-dsvm 2004 microsoft-dsvm:ubuntu-hpc:2004:20.04.2021051401 20.04.2021051401
x64 ubuntu-hpc microsoft-dsvm 2004 microsoft-dsvm:ubuntu-hpc:2004:20.04.2021110101 20.04.2021110101
x64 ubuntu-hpc microsoft-dsvm 2004 microsoft-dsvm:ubuntu-hpc:2004:20.04.2021120101 20.04.2021120101
x64 ubuntu-hpc microsoft-dsvm 2004 microsoft-dsvm:ubuntu-hpc:2004:20.04.2022121201 20.04.2022121201
x64 ubuntu-hpc microsoft-dsvm 2004 microsoft-dsvm:ubuntu-hpc:2004:20.04.2023011101 20.04.2023011101
x64 ubuntu-hpc microsoft-dsvm 2004 microsoft-dsvm:ubuntu-hpc:2004:20.04.2023031501 20.04.2023031501
x64 ubuntu-hpc microsoft-dsvm 2004 microsoft-dsvm:ubuntu-hpc:2004:20.04.2023063001 20.04.2023063001
x64 ubuntu-hpc microsoft-dsvm 2004 microsoft-dsvm:ubuntu-hpc:2004:20.04.2023080201 20.04.2023080201
x64 ubuntu-hpc microsoft-dsvm 2004 microsoft-dsvm:ubuntu-hpc:2004:20.04.2023111801 20.04.2023111801
x64 ubuntu-hpc microsoft-dsvm 2004-preview-ndv5 microsoft-dsvm:ubuntu-hpc:2004-preview-ndv5:20.04.2023041401 20.04.2023041401
x64 ubuntu-hpc microsoft-dsvm 2004-preview-ndv5 microsoft-dsvm:ubuntu-hpc:2004-preview-ndv5:20.04.2023041701 20.04.2023041701
x64 ubuntu-hpc microsoft-dsvm 2004-preview-ndv5 microsoft-dsvm:ubuntu-hpc:2004-preview-ndv5:20.04.2023042401 20.04.2023042401
x64 ubuntu-hpc microsoft-dsvm 2004-preview-ndv5 microsoft-dsvm:ubuntu-hpc:2004-preview-ndv5:20.04.2023051001 20.04.2023051001
x64 ubuntu-hpc microsoft-dsvm 2004-preview-ndv5 microsoft-dsvm:ubuntu-hpc:2004-preview-ndv5:20.04.2023063001 20.04.2023063001
x64 ubuntu-hpc microsoft-dsvm 2004-preview-ndv5 microsoft-dsvm:ubuntu-hpc:2004-preview-ndv5:20.04.2023080201 20.04.2023080201
x64 ubuntu-hpc microsoft-dsvm 2204 microsoft-dsvm:ubuntu-hpc:2204:22.04.2023041401 22.04.2023041401
x64 ubuntu-hpc microsoft-dsvm 2204 microsoft-dsvm:ubuntu-hpc:2204:22.04.2023041701 22.04.2023041701
x64 ubuntu-hpc microsoft-dsvm 2204 microsoft-dsvm:ubuntu-hpc:2204:22.04.2023042401 22.04.2023042401
x64 ubuntu-hpc microsoft-dsvm 2204 microsoft-dsvm:ubuntu-hpc:2204:22.04.2023051001 22.04.2023051001
x64 ubuntu-hpc microsoft-dsvm 2204 microsoft-dsvm:ubuntu-hpc:2204:22.04.2023061401 22.04.2023061401
x64 ubuntu-hpc microsoft-dsvm 2204 microsoft-dsvm:ubuntu-hpc:2204:22.04.2023063001 22.04.2023063001
x64 ubuntu-hpc microsoft-dsvm 2204 microsoft-dsvm:ubuntu-hpc:2204:22.04.2023080201 22.04.2023080201
x64 ubuntu-hpc microsoft-dsvm 2204 microsoft-dsvm:ubuntu-hpc:2204:22.04.2023111801 22.04.2023111801
x64 ubuntu-hpc microsoft-dsvm 2204-preview-ndv5 microsoft-dsvm:ubuntu-hpc:2204-preview-ndv5:22.04.2023041401 22.04.2023041401
x64 ubuntu-hpc microsoft-dsvm 2204-preview-ndv5 microsoft-dsvm:ubuntu-hpc:2204-preview-ndv5:22.04.2023041701 22.04.2023041701
x64 ubuntu-hpc microsoft-dsvm 2204-preview-ndv5 microsoft-dsvm:ubuntu-hpc:2204-preview-ndv5:22.04.2023042401 22.04.2023042401
x64 ubuntu-hpc microsoft-dsvm 2204-preview-ndv5 microsoft-dsvm:ubuntu-hpc:2204-preview-ndv5:22.04.2023051001 22.04.2023051001
x64 ubuntu-hpc microsoft-dsvm 2204-preview-ndv5 microsoft-dsvm:ubuntu-hpc:2204-preview-ndv5:22.04.2023061401 22.04.2023061401
x64 ubuntu-hpc microsoft-dsvm 2204-preview-ndv5 microsoft-dsvm:ubuntu-hpc:2204-preview-ndv5:22.04.2023070601 22.04.2023070601
x64 ubuntu-hpc microsoft-dsvm 2204-preview-ndv5 microsoft-dsvm:ubuntu-hpc:2204-preview-ndv5:22.04.2023080201 22.04.2023080201There was a problem hiding this comment.
Good point! I : sounds good to me then.
Michaelvll
approved these changes
Mar 29, 2024
There was a problem hiding this comment.
Thanks for adding the support for Azure custom image @cblmemo! LGTM. Please feel free to merge it once the tests pass.
6 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds support for custom images on Azure. Partially fixes #2910.
Related: skypilot-org/skypilot-catalog#66
Tested (run the relevant ones):
bash format.shsky launch --cloud azure --image-id microsoft-dsvm:ubuntu-2004:2004:21.11.04and check the image on the cloud consolepytest tests/test_smoke.pypytest tests/test_smoke.py::test_fill_in_the_namebash tests/backward_comaptibility_tests.sh