Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{C} Use Azure Artifacts feeds in release branch #29866

Merged
merged 17 commits into from
Oct 16, 2024
Merged

{C} Use Azure Artifacts feeds in release branch #29866

merged 17 commits into from
Oct 16, 2024

Conversation

bebound
Copy link
Contributor

@bebound bebound commented Sep 9, 2024

Use Azure Artifacts feeds in release branch build process: PyPI, MSI, ZIP, DEB, and RPM

The public pipeline still uses PyPI as it's hard to keep the CFS feed token secret.

Ref:

Copy link

azure-client-tools-bot-prd bot commented Sep 9, 2024

️✔️AzureCLI-FullTest
️✔️acr
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️acs
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️ams
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️apim
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️aro
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️backup
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️batch
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️billing
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️compute_recommender
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️config
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️configure
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️container
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️containerapp
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️core
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️databoxedge
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️dla
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️dls
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️dms
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️find
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️identity
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️iot
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️keyvault
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️kusto
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️lab
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️maps
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️mysql
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️network
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️profile
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️redis
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️relay
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️resource
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️role
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️search
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️security
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️sql
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️storage
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️telemetry
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️util
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9
️✔️vm
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.12
️✔️3.9

Copy link

azure-client-tools-bot-prd bot commented Sep 9, 2024

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@yonzhan
Copy link
Collaborator

yonzhan commented Sep 9, 2024

Packaging

yonzhan
yonzhan previously approved these changes Sep 9, 2024
@bebound bebound changed the title {Packaging} Use CFS during build process {Packaging} Use Azure Artifacts feeds during build process Sep 9, 2024
@@ -16,6 +16,7 @@ docker build \
--target build-env \
--build-arg cli_version=${CLI_VERSION} \
--build-arg image=${IMAGE} \
--secret id=PIP_INDEX_URL \
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--build-arg also works, but I prefer --secret as the index url contains credential and secret does not record its value in image.

Ref: https://docs.docker.com/build/building/secrets/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is interesting that env can be omitted:

https://docs.docker.com/build/building/secrets/#sources

When you use secrets from environment variables, you can omit the env parameter to bind the secret to a file with the same name as the variable.

@bebound bebound marked this pull request as ready for review September 10, 2024 03:23
wangzelin007
wangzelin007 previously approved these changes Sep 11, 2024
- task: PipAuthenticate@1
displayName: 'Pip Authenticate'
inputs:
artifactFeeds: 'public/azure-cli-feed'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wangzelin007
Copy link
Member

/azp where

Copy link

Azure DevOps orgs getting events for this repository:

@bebound bebound changed the title {Packaging} Use Azure Artifacts feeds during build process {Packaging} Use Azure Artifacts feeds during release Oct 11, 2024
wangzelin007
wangzelin007 previously approved these changes Oct 12, 2024
@@ -6,6 +6,8 @@ trigger:
branches:
include:
- '*'
exclude:
- 'release'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The public pipeline does not have the permission to run PipAuthenticate@1 task, so skip it. This also saves some resource during CLI release.

@@ -11,11 +11,15 @@ set -exv

CLI_VERSION=`cat src/azure-cli/azure/cli/__main__.py | grep __version__ | sed s/' '//g | sed s/'__version__='// | sed s/\"//g`

# PIP_INDEX_URL env must exist in `docker build --secret`, use an empty string if it doesn't exist.
export PIP_INDEX_URL=${PIP_INDEX_URL}
Copy link
Contributor Author

@bebound bebound Oct 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Error message without empty env during docker build: ERROR: failed to stat PIP_INDEX_URL: stat PIP_INDEX_URL: no such file or directory

wangzelin007
wangzelin007 previously approved these changes Oct 15, 2024
condition: eq(variables['Build.SourceBranch'], 'refs/heads/release')
displayName: 'Pip Authenticate'
inputs:
artifactFeeds: ${{ variables.AZURE_ARTIFACTS_FEEDS }}
Copy link
Member

@jiasli jiasli Oct 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other tasks use macro syntax as $(variables.AZURE_ARTIFACTS_FEEDS). Does template expression syntax work here?

My understanding is that ${{ variables.AZURE_ARTIFACTS_FEEDS }} only works for templates: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops&pivots=templates-includes#variable-reuse

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, the variable defined in ADO UI is empty when use ${{ variables.AZURE_ARTIFACTS_FEEDS }}. I've changed to $(AZURE_ARTIFACTS_FEEDS)

@bebound bebound changed the title {Packaging} Use Azure Artifacts feeds during release {C} Use Azure Artifacts feeds in release branch Oct 16, 2024
@bebound bebound merged commit 9fd6b4f into Azure:dev Oct 16, 2024
61 checks passed
@bebound bebound deleted the cfs branch October 16, 2024 07:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Auto-Assign Auto assign by bot Packaging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants