Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Start moving to parse the configuration of the build in the tests pipelines #21009

Merged
merged 11 commits into from
Sep 6, 2024

Conversation

mandel-macaque
Copy link
Member

@mandel-macaque mandel-macaque commented Aug 9, 2024

At the moment we are recaculating the configuration that was used in the build to decide which tests to run, that is not needed since the configuration was uploaded to the artifacts.

This change will allow to do the following:

  • Load the default variables on the build pipeline this will allow us to set the name of the tests to match those of the build for easy parsing.
  • Load the default variables to set the property comment in the PR.
  • Do not recalculate the built platforms on the tests matrix.
  • Do not calculate the API scan matrix, it is not needed for the tests.

@vs-mobiletools-engineering-service2

This comment has been minimized.

tools/devops/automation/scripts/VSTS.psm1 Outdated Show resolved Hide resolved
Comment on lines +59 to +76
- bash: ./xamarin-macios/tools/devops/automation/scripts/bash/configure-platforms.sh
name: configure_platforms
displayName: 'Configure platforms'

- pwsh: |
Import-Module $Env:SYSTEM_DEFAULTWORKINGDIRECTORY/xamarin-macios/tools/devops/automation/scripts/MaciosCI.psd1
$jsonPath = Join-Path -Path "$(Build.ArtifactStagingDirectory)" -ChildPath "configuration.json"
Write-Host "##vso[task.setvariable variable=CONFIG_PATH]$jsonPath"
New-BuildConfiguration -ConfigFile $jsonPath
env:
GITHUB_TOKEN: $(GitHub.Token)
ACCESSTOKEN: $(AzDoBuildAccess.Token)
name: labels
displayName: 'Configure build'

- bash: ./xamarin-macios/tools/devops/automation/scripts/bash/configure-decisions.sh
name: decisions
displayName: 'Make decisions'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a copy of configure.yml, and seems to create (and later upload) configuration.xml like configure.yml does. Isn't this file supposed to load the already created configuration.xml instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a copy, the idea is that I'll be making changes one by one to keep the compatibility and remove those steps that are not needed. ATM we are loading the test matrix from the config, yet we are generating the config. That will be the next step.

@vs-mobiletools-engineering-service2

This comment has been minimized.

Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2
Copy link
Collaborator

📚 [CI Build] Artifacts 📚

Packages generated

View packages

Pipeline on Agent
Hash: [PR build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

💻 [CI Build] Tests on macOS M1 - Mac Ventura (13) passed 💻

All tests on macOS M1 - Mac Ventura (13) passed.

Pipeline on Agent
Hash: bf46ecedfe800e538379128cd38656c8f38768b7 [PR build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

✅ API diff for current PR / commit

NET (empty diffs)
  • iOS: (empty diff detected)
  • tvOS: (empty diff detected)
  • MacCatalyst: (empty diff detected)
  • macOS: (empty diff detected)

✅ API diff vs stable

.NET (No breaking changes)

ℹ️ Generator diff

Generator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes)

Pipeline on Agent
Hash: bf46ecedfe800e538379128cd38656c8f38768b7 [PR build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

💻 [CI Build] Tests on macOS M1 - Mac Big Sur (11) passed 💻

All tests on macOS M1 - Mac Big Sur (11) passed.

Pipeline on Agent
Hash: bf46ecedfe800e538379128cd38656c8f38768b7 [PR build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

💻 [CI Build] Tests on macOS M1 - Mac Monterey (12) passed 💻

All tests on macOS M1 - Mac Monterey (12) passed.

Pipeline on Agent
Hash: bf46ecedfe800e538379128cd38656c8f38768b7 [PR build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

💻 [CI Build] Tests on macOS X64 - Mac Sonoma (14) passed 💻

All tests on macOS X64 - Mac Sonoma (14) passed.

Pipeline on Agent
Hash: bf46ecedfe800e538379128cd38656c8f38768b7 [PR build]

@vs-mobiletools-engineering-service2

This comment has been minimized.

@vs-mobiletools-engineering-service2
Copy link
Collaborator

❌ [CI Build] Windows Integration Tests failed ❌

❌ Failed ❌

Pipeline on Agent
Hash: bf46ecedfe800e538379128cd38656c8f38768b7 [PR build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

🔥 [CI Build] Test results 🔥

Test results

❌ Tests failed on VSTS: test results

0 tests crashed, 1 tests failed, 97 tests passed.

Failures

❌ dotnettests tests (MacCatalyst) [attempt 2]

1 tests failed, 0 tests passed.
  • DotNet tests: Failed (Execution failed with exit code 1)

Html Report (VSDrops) Download

Successes

✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (iOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (macOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (Multiple platforms): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (tvOS): All 1 tests passed. Html Report (VSDrops) Download
✅ framework: All 2 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 4 tests passed. Html Report (VSDrops) Download
✅ generator: All 1 tests passed. Html Report (VSDrops) Download
✅ install-source: All 1 tests passed. Html Report (VSDrops) Download
✅ interdependent-binding-projects: All 4 tests passed. Html Report (VSDrops) Download
✅ introspection: All 4 tests passed. Html Report (VSDrops) Download
✅ linker: All 40 tests passed. Html Report (VSDrops) Download
⚠️ mac-binding-project: No tests selected. Html Report (VSDrops) Download
⚠️ mmp: No tests selected. Html Report (VSDrops) Download
⚠️ mononative: No tests selected. Html Report (VSDrops) Download
✅ monotouch (iOS): All 7 tests passed. Html Report (VSDrops) Download
✅ monotouch (MacCatalyst): All 7 tests passed. Html Report (VSDrops) Download
✅ monotouch (macOS): All 8 tests passed. Html Report (VSDrops) Download
✅ monotouch (tvOS): All 7 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
⚠️ mtouch: No tests selected. Html Report (VSDrops) Download
⚠️ xammac: No tests selected. Html Report (VSDrops) Download
✅ xcframework: All 4 tests passed. Html Report (VSDrops) Download
✅ xtro: All 1 tests passed. Html Report (VSDrops) Download

Pipeline on Agent
Hash: bf46ecedfe800e538379128cd38656c8f38768b7 [PR build]

@vs-mobiletools-engineering-service2
Copy link
Collaborator

❌ [CI Build] Windows Integration Tests failed ❌

❌ Failed ❌

Pipeline on Agent
Hash: bf46ecedfe800e538379128cd38656c8f38768b7 [PR build]

@mandel-macaque mandel-macaque merged commit 52ee455 into main Sep 6, 2024
38 checks passed
@mandel-macaque mandel-macaque deleted the parse-config branch September 6, 2024 11:15
mandel-macaque added a commit that referenced this pull request Sep 6, 2024
The pwsh used on windows is based on dotnet framework. Dotnet framework
has a limit in the size of the env that can be passed to a child
process. We are very close to reach the limit, but after #21009 we can
ignore that env variable since it is parsed from the args.

This is just needed for the ci pipeline because the issues are going to
have are related with the vsdrops task used to upload nuget results
etc..
mandel-macaque added a commit that referenced this pull request Sep 10, 2024
The pwsh used on windows is based on dotnet framework. Dotnet framework
has a limit in the size of the env that can be passed to a child
process. We are very close to reach the limit, but after #21009 we can
ignore that env variable since it is parsed from the args.

This is just needed for the ci pipeline because the issues are going to
have are related with the vsdrops task used to upload nuget results
etc..
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants