Update prepare-pipelines.yml to use the new jobs-based template #32999
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
konrad-jamrozik
requested review from
hallipr,
weshaggard and
benbp
as code owners
konrad-jamrozik
added
the
Central-EngSys
ghost
pushed a commit
to Azure/azure-sdk-tools
that referenced
this pull request
Dec 12, 2022
… to use Ubuntu 20.04 (#4930) This PR, in tandem with PRs #4915 and #4916, addresses #4888. This PR subsumes the abandoned PR of #4911. Specifically, this PR ensures that `prepare-pipelines.yml` is configured to use specific `pool`, set to use Ubuntu 20.04: ``` pool: name: azsdk-pool-mms-ubuntu-2004-general vmImage: MMSUbuntu20.04 ``` Without the [pool](https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/pool?view=azure-pipelines) definition pointing to Ubuntu 20.04, the steps within the file used an implicit default, which resolves to `ubuntu-latest`. This caused breakage as explained in #4888. Setting the pool to Ubuntu 20.04 will allow us to: - Immediately unblock the language repos `prepare-pipelines.yml` pipelines [1], as Ubuntu 20.04 has the .NET Core version used by it [2]; - Allow us to migrate our tooling to .NET 6 as both Ubuntu 20.04 and `ubuntu-latest`, which is Ubuntu 22.04 [3], have .NET on them [4]. This step is accomplished by PRs #4916 and #4915; - Allow us to migrate to Ubuntu 22.04 once we migrate to .NET 6. ## Technical considerations of the changes - The current `prepare-pipelines.yml` has only [steps](https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/steps?view=azure-pipelines) definition, which does not support `pool` definition. One needs to wrap the steps in [job](https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/jobs-job?view=azure-pipelines) definition to support `pool` [5]. - Because the `.yml` file is located in `eng/common/pipelines/templates/steps/` directory (note the `steps`), we cannot wrap the steps in job and leave it there; the introduction of `job` necessitates introducing the changes in `eng/common/pipelines/templates/jobs/` directory instead (note the `jobs`), which is done in this PR. - Because this PR introduces a new file, once this PR is merged, the language-specific `prepare-pipelines.yml` language pipelines need to be rewired to point to the newly introduced file. This approach was proposed in #4911 (comment). - Language repos PRs doing the rewiring: Azure/azure-sdk-for-net#32999 ... more to come! ## Testing done I have confirmed the proposed modification to the `.yml` file will work [by modifying the existing `prepare-pipelines.yml` directly](Azure/azure-sdk-for-java@f8cef4c) to have exactly the same contents as the `/jobs/prepare-pipelines.yml` introduced in this PR, and observing the [relevant build succeeds on azure-sdk-for-java](https://dev.azure.com/azure-sdk/internal/_build/results?buildId=2044120&view=logs&j=7f42699e-eeb0-56c8-40c2-c88ae4093e4f&t=a02502cb-238e-5603-14ee-8bb7bd07f0c6):  while it failed before this change:  # Footnotes [1] for example, [internal / java / prepare-pipelines](https://dev.azure.com/azure-sdk/internal/_build?definitionId=2158&_a=summary) pipeline, or [internal / net / prepare-pipelines](https://dev.azure.com/azure-sdk/internal/_build?definitionId=2179) pipeline. [2] as evidenced by the [Ubuntu 20.04 image software page](https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md#net-core-sdk), it has .NET Core 3.1. The pool definition value points to a 1ES hosted image which has the software listed on that page, as explained by this comment: #4888 (comment). [3] See this comment: #4888 (comment). [4] as evidenced by the [Ubuntu 22.04 image software page](https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2204-Readme.md#net-core-sdk). [5] Observe that on [pool definition](https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/pool?view=azure-pipelines) YAML reference page, we can read: > Properties that use this definition: pipeline.pool, stages.stage.pool, jobs.job.pool, jobs.deployment.pool. i.e. `steps.pool` is not listed.
konrad-jamrozik
removed
the
EngSys
weshaggard
approved these changes
Dec 13, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
This is an Azure SDK Engineering System Team PR, made to address failing prepare-pipelines pipeline, as explained in Azure/azure-sdk-tools#4888. The issue is currently mitigated with a manual configuration override of the machine pool used. This PR is a proper fix.
PR merging prerequisite
This PR MUST NOT be merged until this PR is merged:
Azure/azure-sdk-tools#4930
and the changes made by it reflected in this language repository.
Changes made
This PR rewires
prepare-pipelines.ymlto use the new template introduced by Azure/azure-sdk-tools#4930. This template pins, viapooldefinition, the Ubuntu image used by theprepare-pipelinespipeline, thus properly addressing the issue. See that PR description for more context.Testing done
See section on testing in Azure/azure-sdk-tools#4930