Add job matrix generation scripts #16450
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds scripts, docs and samples supporting dynamic, cross-product matrix generation for azure pipeline jobs.
It aims to replicate the cross-product matrix functionality in github actions,
but also adds some additional features like sparse matrix generation, cross-product includes and excludes, parameter grouping and matrix filters.
These changes are live for the .NET tests and coming soon for Java, so this PR is just a port of the code that already exists in the those repositories. The intent is to standardize it across the language repos, and then migrate all the common code into azure-sdk-tools. I will make a follow-on PR with the actual configuration changes to use this functionality in the pipelines, because I want to keep the changes separate to make reviewing/reverting easier.
This functionality is made possible by the ability for the azure pipelines yaml to take a dynamic variable as an input
for a job matrix definition (see the code sample at the bottom of the linked section).
See the README.md file for more details on the config file syntax and usage, as well as implementation details.
The tests (
test-matrix-functions.tests.ps1
) contain a lot of detail on expected data structures at various processing stages. The-
test-matrix-functions.ps1` file could perhaps be split up or use some more organization, so let me know if it's hard to navigate.Example (from a Java scenario...you get the idea):
Sparse matrix job generation in a pipeline: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=705622&view=results
Related discussion: microsoft/azure-pipelines-yaml#20