Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Packaging] Bump embedded Python version to 3.12.7 #29887

Merged
merged 5 commits into from
Oct 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .azure-pipelines/breaking-change-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ jobs:
fetchTags: true
persistCredentials: true
- task: UsePythonVersion@0
displayName: 'Use Python 3.11'
displayName: 'Use Python 3.12'
inputs:
versionSpec: 3.11
versionSpec: 3.12
- template: ${{ variables.Pipeline.Workspace }}/.azure-pipelines/templates/azdev_setup.yml
- bash: |
set -ev
Expand Down
45 changes: 2 additions & 43 deletions azure-pipelines-full-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,6 @@ jobs:
matrix:
Python39:
python.version: '3.9'
Python311:
python.version: '3.11'
Python312:
python.version: '3.12'
steps:
Expand All @@ -46,8 +44,6 @@ jobs:
matrix:
Python39:
python.version: '3.9'
Python311:
python.version: '3.11'
Python312:
python.version: '3.12'
steps:
Expand All @@ -67,8 +63,6 @@ jobs:
matrix:
Python39:
python.version: '3.9'
Python311:
python.version: '3.11'
Python312:
python.version: '3.12'
steps:
Expand Down Expand Up @@ -113,40 +107,6 @@ jobs:
fullTest: true
jobName: 'FullTest'

- job: AutomationFullTestPython311ProfileLatest
displayName: Automation Full Test Python311 Profile Latest
timeoutInMinutes: 9999
strategy:
maxParallel: 8
matrix:
instance1:
Instance_idx: 1
instance2:
Instance_idx: 2
instance3:
Instance_idx: 3
instance4:
Instance_idx: 4
instance5:
Instance_idx: 5
instance6:
Instance_idx: 6
instance7:
Instance_idx: 7
instance8:
Instance_idx: 8
pool:
name: ${{ variables.ubuntu_pool }}
steps:
- template: .azure-pipelines/templates/automation_test.yml
parameters:
pythonVersion: '3.11'
profile: 'latest'
instance_cnt: '8'
instance_idx: '$(Instance_idx)'
fullTest: true
jobName: 'FullTest'

- job: AutomationFullTestPython312ProfileLatest
displayName: Automation Full Test Python312 Profile Latest
timeoutInMinutes: 9999
Expand Down Expand Up @@ -187,16 +147,15 @@ jobs:
- AutomationTest20190301
- AutomationTest20180301
- AutomationFullTestPython39ProfileLatest
- AutomationFullTestPython311ProfileLatest
condition: and(failed(), in(variables['Build.Reason'], 'BatchedCI'))
displayName: Notify CI Errors
pool:
name: ${{ variables.ubuntu_pool }}
steps:
- task: UsePythonVersion@0
displayName: 'Use Python 3.11'
displayName: 'Use Python 3.12'
inputs:
versionSpec: 3.11
versionSpec: 3.12
- task: AzureCLI@2
inputs:
azureSubscription: 'Azure CLI'
Expand Down
107 changes: 50 additions & 57 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -129,9 +129,9 @@ jobs:

steps:
- task: UsePythonVersion@0
displayName: 'Use Python 3.11'
displayName: 'Use Python 3.12'
inputs:
versionSpec: 3.11
versionSpec: 3.12

- bash: ./scripts/ci/dependency_check.sh
displayName: 'Verify src/azure-cli/requirements.py3.Linux.txt'
Expand All @@ -144,9 +144,9 @@ jobs:

steps:
- task: UsePythonVersion@0
displayName: 'Use Python 3.11'
displayName: 'Use Python 3.12'
inputs:
versionSpec: 3.11
versionSpec: 3.12

- bash: ./scripts/ci/dependency_check.sh
displayName: 'Verify src/azure-cli/requirements.py3.Darwin.txt'
Expand All @@ -159,9 +159,9 @@ jobs:

steps:
- task: UsePythonVersion@0
displayName: 'Use Python 3.11'
displayName: 'Use Python 3.12'
inputs:
versionSpec: 3.11
versionSpec: 3.12

- task: BatchScript@1
inputs:
Expand All @@ -176,9 +176,9 @@ jobs:
name: ${{ variables.ubuntu_pool }}
steps:
- task: UsePythonVersion@0
displayName: 'Use Python 3.11'
displayName: 'Use Python 3.12'
inputs:
versionSpec: 3.11
versionSpec: 3.12
- template: .azure-pipelines/templates/azdev_setup.yml
- bash: |
set -ev
Expand Down Expand Up @@ -437,9 +437,9 @@ jobs:
name: ${{ variables.ubuntu_pool }}
steps:
- task: UsePythonVersion@0
displayName: 'Use Python 3.11'
displayName: 'Use Python 3.12'
inputs:
versionSpec: 3.11
versionSpec: 3.12

- task: PipAuthenticate@1
condition: eq(variables['Build.SourceBranch'], 'refs/heads/release')
Expand Down Expand Up @@ -469,7 +469,12 @@ jobs:

- job: TestPythonWheel
displayName: Test Python Wheels

strategy:
matrix:
Python39:
python.version: '3.9'
Python312:
python.version: '3.12'
dependsOn: BuildPythonWheel
condition: succeeded()
pool:
Expand All @@ -480,35 +485,23 @@ jobs:
inputs:
TargetPath: '$(Build.ArtifactStagingDirectory)/metadata'
artifactName: metadata


- task: DownloadPipelineArtifact@1
displayName: 'Download PyPI Packages'
inputs:
TargetPath: '$(Build.ArtifactStagingDirectory)/pypi'
artifactName: pypi


- task: UsePythonVersion@0
displayName: 'Use Python $(python.version)'
inputs:
versionSpec: '$(python.version)'
- bash: |
#!/usr/bin/env bash

# Verify the pip wheels

set -ex

CLI_VERSION=`cat $BUILD_ARTIFACTSTAGINGDIRECTORY/metadata/version`
PYPI_FILES=$(cd $BUILD_ARTIFACTSTAGINGDIRECTORY/pypi; pwd)

echo "== Testing pip install on Python 3.9 =="
docker run \
--rm -v $PYPI_FILES:/mnt/pypi mcr.microsoft.com/mirror/docker/library/python:3.9-slim \
/bin/bash -c "cd /mnt/pypi && ls && pip install --find-links ./ azure_cli-$CLI_VERSION*whl && az self-test && az --version && sleep 5"

echo "== Testing pip install on Python 3.11 =="
docker run \
--rm -v $PYPI_FILES:/mnt/pypi mcr.microsoft.com/mirror/docker/library/python:3.11-slim \
Copy link
Contributor Author

@bebound bebound Sep 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAR does not provide newer Python docker image anymore, use local Python instead.

Ref: https://mcr.microsoft.com/en-us/artifact/mar/mirror/docker/library/python/tags

/bin/bash -c "cd /mnt/pypi && ls && pip install --find-links ./ azure_cli-$CLI_VERSION*whl && az self-test && az --version && sleep 5"

echo "== Testing pip install on $PYTHON_VERSION =="
cd $BUILD_ARTIFACTSTAGINGDIRECTORY/pypi
pip install --find-links ./ azure_cli-$CLI_VERSION*whl && az self-test && az --version && sleep 5
displayName: 'Test pip Install'

- job: TestCore
Expand All @@ -520,8 +513,8 @@ jobs:
matrix:
Python39:
python.version: '3.9'
Python311:
python.version: '3.11'
Python312:
python.version: '3.12'
steps:
- template: .azure-pipelines/templates/automation_test.yml
parameters:
Expand All @@ -537,8 +530,8 @@ jobs:
matrix:
Python39:
python.version: '3.9'
Python311:
python.version: '3.11'
Python312:
python.version: '3.12'
steps:
- template: .azure-pipelines/templates/automation_test.yml
parameters:
Expand All @@ -557,14 +550,14 @@ jobs:
matrix:
Python39:
python.version: '3.9'
Python311:
python.version: '3.11'
Python312:
python.version: '3.12'
steps:
- task: UsePythonVersion@0
displayName: 'Use Python $(python.version)'
inputs:
versionSpec: '$(python.version)'
- bash: pip install --upgrade pip wheel
- bash: pip install --upgrade pip wheel setuptools
displayName: 'Install pip and wheel'
- bash: ./scripts/ci/test_profile_integration.sh
displayName: 'Run Integration Test against Profiles'
Expand All @@ -578,15 +571,15 @@ jobs:
name: ${{ variables.ubuntu_pool }}
strategy:
matrix:
Python311:
python.version: '3.11'
Python312:
python.version: '3.12'
steps:
- task: UsePythonVersion@0
displayName: 'Use Python $(python.version)'
inputs:
versionSpec: '$(python.version)'
- bash: pip install --upgrade pip wheel
displayName: 'Install pip and wheel'
- bash: pip install --upgrade pip wheel setuptools
displayName: 'Install pip and wheel setuptools'
- bash: ./scripts/ci/test_extensions.sh
displayName: 'Load extensions'

Expand Down Expand Up @@ -623,7 +616,7 @@ jobs:
-e CLI_VERSION=$CLI_VERSION \
-e HOMEBREW_UPSTREAM_URL=$HOMEBREW_UPSTREAM_URL \
--name azurecli \
mcr.microsoft.com/mirror/docker/library/python:3.11-slim \
mcr.microsoft.com/azurelinux/base/python:3 \
Copy link
Contributor Author

@bebound bebound Sep 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Formula generation is independent of the Python version.
Use Azure Linux's 3.12 to avoid updates for the next four years.

docker source: https://mcr.microsoft.com/en-us/artifact/mar/azurelinux/base/python/tags

/mnt/scripts/run.sh

# clean up
Expand Down Expand Up @@ -670,7 +663,7 @@ jobs:
set -ev
# Force relink python@3.xx in Homebrew to resolve the conflict with pre-installed python 3.xx on macOS-12 image
# See: https://github.com/Azure/azure-cli/issues/29054
python_version=3.11
python_version=3.12
brew unlink python@$python_version && brew link --overwrite python@$python_version

echo == Remove pre-installed azure-cli ==
Expand Down Expand Up @@ -1050,9 +1043,9 @@ jobs:
name: ${{ variables.ubuntu_pool }}
steps:
- task: UsePythonVersion@0
displayName: 'Use Python 3.11'
displayName: 'Use Python 3.12'
inputs:
versionSpec: 3.11
versionSpec: 3.12
- template: .azure-pipelines/templates/azdev_setup.yml
- bash: |
set -ev
Expand All @@ -1066,9 +1059,9 @@ jobs:
name: ${{ variables.ubuntu_pool }}
steps:
- task: UsePythonVersion@0
displayName: 'Use Python 3.11'
displayName: 'Use Python 3.12'
inputs:
versionSpec: 3.11
versionSpec: 3.12
- template: .azure-pipelines/templates/azdev_setup.yml
- bash: |
set -ev
Expand All @@ -1083,8 +1076,8 @@ jobs:
matrix:
Python39:
python.version: '3.9'
Python311:
python.version: '3.11'
Python312:
python.version: '3.12'
pool:
name: ${{ variables.ubuntu_pool }}
steps:
Expand Down Expand Up @@ -1112,9 +1105,9 @@ jobs:
name: ${{ variables.ubuntu_pool }}
steps:
- task: UsePythonVersion@0
displayName: 'Use Python 3.11'
displayName: 'Use Python 3.12'
inputs:
versionSpec: 3.11
versionSpec: 3.12
- template: .azure-pipelines/templates/azdev_setup.yml
- bash: |
set -ev
Expand All @@ -1135,9 +1128,9 @@ jobs:
name: ${{ variables.ubuntu_pool }}
steps:
- task: UsePythonVersion@0
displayName: 'Use Python 3.11'
displayName: 'Use Python 3.12'
inputs:
versionSpec: 3.11
versionSpec: 3.12
- template: .azure-pipelines/templates/azdev_setup.yml
- bash: |
set -ev
Expand All @@ -1163,9 +1156,9 @@ jobs:
name: ${{ variables.ubuntu_pool }}
steps:
- task: UsePythonVersion@0
displayName: 'Use Python 3.11'
displayName: 'Use Python 3.12'
inputs:
versionSpec: 3.11
versionSpec: 3.12
- template: .azure-pipelines/templates/azdev_setup.yml
- bash: |
set -ev
Expand Down Expand Up @@ -1253,9 +1246,9 @@ jobs:
name: ${{ variables.ubuntu_pool }}
steps:
- task: UsePythonVersion@0
displayName: 'Use Python 3.11'
displayName: 'Use Python 3.12'
inputs:
versionSpec: 3.11
versionSpec: 3.12
- task: AzureCLI@2
inputs:
azureSubscription: 'Azure CLI'
Expand Down
Loading