Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using Dynamic Matrix Config #31542

Closed
wants to merge 16 commits into from

Conversation

scbedd
Copy link
Member

@scbedd scbedd commented Oct 28, 2024

I've been thinking a lot about how we can abstract away both the package distribution as well as when specific packages have restrictions on what matrix they run.

This PR updates generate-job-matrix.yml with a possible solution. (A lot of the powershell is not fully filled out as this is just a proto)

We will:

  • Gather all custom matrix configs
  • Group the packages we need to test (discovered from package properties) by what matrix they are associated with
  • For each discovered matrix config (starting with a default provided)
    • Create the matrix
      • For the packages that belong to the matrix, batch them up and distribute them to the matrix

This way we will be able to generate multiple platform matrixes while also allowing the custom selections from the users.

@scbedd
Copy link
Member Author

scbedd commented Oct 28, 2024

/azp run js - pullrequest

@scbedd
Copy link
Member Author

scbedd commented Oct 28, 2024

/azp run js - pullrequest

@scbedd
Copy link
Member Author

scbedd commented Oct 28, 2024

/azp run js - pullrequest

@scbedd
Copy link
Member Author

scbedd commented Oct 28, 2024

/azp run js - pullrequest

… and no matter what I do AZDO is refusing to queue the build
@scbedd
Copy link
Member Author

scbedd commented Oct 28, 2024

/azp run js - matrix-pullrequest

@scbedd scbedd marked this pull request as ready for review October 28, 2024 02:42
@scbedd
Copy link
Member Author

scbedd commented Oct 28, 2024

/azp run js - matrix-pullrequest

1 similar comment
@scbedd
Copy link
Member Author

scbedd commented Oct 28, 2024

/azp run js - matrix-pullrequest

}

# if we know this is the matrix for our file, we should now see if there is a custom matrix config for the package
$matrixConfigList = GetValueSafelyFrom-Yaml $content @("extends", "parameters", "MatrixConfigs")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this also handle values behind devops template conditionals? e.g. https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/identity/tests.yml#L33

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not currently.

@benbp
Copy link
Member

benbp commented Oct 28, 2024

As I mentioned I was wondering if the generation complexity would be high, but the changes necessary are so small that I think it's very reasonable. I think if you add more examples on inline comments of an inputted matrix set to an outputted matrix set, with the actual package name additions then it will be very approachable to maintain/update going forwards.

@scbedd
Copy link
Member Author

scbedd commented Oct 28, 2024

As I mentioned I was wondering if the generation complexity would be high, but the changes necessary are so small that I think it's very reasonable. I think if you add more examples on inline comments of an inputted matrix set to an outputted matrix set, with the actual package name additions then it will be very approachable to maintain/update going forwards.

Executing!

@scbedd
Copy link
Member Author

scbedd commented Oct 29, 2024

/azp run js - matrix-pullrequest

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@scbedd
Copy link
Member Author

scbedd commented Oct 31, 2024

All the conversations in this PR ended up being commited in Azure/azure-sdk-tools#9281. Closing this PR in favor of #31420 in this repo.

@scbedd scbedd closed this Oct 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants