[dotnet] Create and ship a WorkloadDependencies.json file. Fixes #21768. #21779

rolfbjarne · 2024-12-09T18:33:00Z

rolfbjarne · 2024-12-09T18:36:45Z

Checking the produced content:

$ cd _build/nupkgs
$ for i in *NET*; do echo "### $i"; echo ""; echo '```json'; unzip -p $i data/WorkloadDependencies.json; echo '```'; echo ""; done

Microsoft.NET.Sdk.MacCatalyst.Manifest-9.0.100.18.1.9209-ci.workloaddependencies-json.nupkg

{
  "microsoft.net.sdk.maccatalyst": {
    "workload": {
      "alias": [ "maccatalyst" ],
      "version": "18.1.9209-ci.workloaddependencies-json"
    },
    "xcode": {
      "version": "[16.1,)",
      "recommendedVersion": "16.1"
    },
    "sdk": {
      "version": "18.1"
    }
  }
}

Microsoft.NET.Sdk.iOS.Manifest-9.0.100.18.1.9209-ci.workloaddependencies-json.nupkg

{
  "microsoft.net.sdk.ios": {
    "workload": {
      "alias": [ "ios" ],
      "version": "18.1.9209-ci.workloaddependencies-json"
    },
    "xcode": {
      "version": "[16.1,)",
      "recommendedVersion": "16.1"
    },
    "sdk": {
      "version": "18.1"
    }
  }
}

Microsoft.NET.Sdk.macOS.Manifest-9.0.100.15.1.9209-ci.workloaddependencies-json.nupkg

{
  "microsoft.net.sdk.macos": {
    "workload": {
      "alias": [ "macos" ],
      "version": "15.1.9209-ci.workloaddependencies-json"
    },
    "xcode": {
      "version": "[16.1,)",
      "recommendedVersion": "16.1"
    },
    "sdk": {
      "version": "15.1"
    }
  }
}

Microsoft.NET.Sdk.tvOS.Manifest-9.0.100.18.1.9209-ci.workloaddependencies-json.nupkg

{
  "microsoft.net.sdk.tvos": {
    "workload": {
      "alias": [ "tvos" ],
      "version": "18.1.9209-ci.workloaddependencies-json"
    },
    "xcode": {
      "version": "[16.1,)",
      "recommendedVersion": "16.1"
    },
    "sdk": {
      "version": "18.1"
    }
  }
}

jonathanpeppers · 2024-12-09T18:58:29Z

scripts/generate-workloaddependencies-json/generate-workloaddependencies-json.cs

+
+using (var writer = new StreamWriter (outputPath)) {
+	writer.WriteLine ($"{{");
+	writer.WriteLine ($"  \"microsoft.net.sdk.{platformLowerCase}\": {{");


Is microsoft.net.sdk.[platform] the right key? Or should it be the workload name?

I think we want a mapping "required Xcode" -> iOS workload, and not a relationship between a "required Xcode" -> "workload manifest"?

Mainly thinking if we need .NET SDK team to review.

Yeah, you could have multiple workloads within one manifest, that has different requirements per workload?

For example, we might have a android-nativeaot workload one day that requires the Android NDK and the plain android workload doesn't.

I think we do want the actual workload identifier here as is in the PR. The aliases were meant to capture the name we might use in the CLI, but for looking up this type of info, for example, C:\Program Files\dotnet\sdk-manifests\9.0.100\microsoft.net.sdk.ios\[version]\WorkloadDependencies.json is where i'd expect to find the info, so I think it makes sense for the key in this file to match the id here.

How would we define one workload as having different requirements as the other?

For example, here in .NET 9 Preview 6:

https://github.com/dotnet/android/blob/release/9.0.1xx-preview6/src/Xamarin.Android.Build.Tasks/Microsoft.NET.Sdk.Android/WorkloadManifest.in.json

We would require API-34 for the android workload and API-35 for the preview android-35 workload.

I think we've mostly sorted this out for now. Android is listing these as optional, and we'll eventually add a way to query the context for a given project state (probably invoking a specific msbuild target on a project) at which point having a net9.0-android35.0 TFM for example could cause the returned info to require the sdk package for android 35 instead of the default 34 for net9.0 as per the file in the workload.

In the case of macios, I don't think this will be a problem as the workload doesn't differentiate anywhere at this point that different xcode versions would be needed for different target platform versions for example.

vs-mobiletools-engineering-service2 · 2024-12-11T18:09:34Z

✅ API diff for current PR / commit

.NET (No breaking changes)

iOS: vsdrops gist (No breaking changes)
tvOS: vsdrops gist (No breaking changes)
MacCatalyst: vsdrops gist (No breaking changes)
macOS: vsdrops gist (No breaking changes)

✅ API diff vs stable

.NET (No breaking changes)

iOS: vsdrops gist (No breaking changes)
tvOS: vsdrops gist (No breaking changes)
MacCatalyst: vsdrops gist (No breaking changes)
macOS: vsdrops gist (No breaking changes)

ℹ️ Generator diff

Generator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes)

Pipeline on Agent
Hash: 673524070d17a95d667a2a1774c6eb6275220d94 [PR build]

Context: [`Releases.json` loop][0] Context: xamarin/xamarin-macios#21779 (comment) There is a desire to have the .NET Workloads have a machine readable description of what dependencies they require in order to run, in order to facilitate tooling that would check for these dependencies. Add `tools/workload-dependencies`, a new tool which parses a "Xamarin Manifest" to generate `workload-dependencies.json`. The "canonical" location for the "Xamarin Manifest" is within `external/android-platform-support/Feeds/AndroidManifestFeed_d17.12.xml`; failing that, <https://aka.ms/AndroidManifestFeed/d17-12> can be used. Output of the tool is a JSON document specifying ther required JDK and Android SDK packages which the .NET for Android workload requires: { "microsoft.net.sdk.android": { "workload": { "alias": [ "android" ], "version": "35.0.100" }, "jdk": { "version": "[17.0,18.0)", "recommendedVersion": "17.0.12" }, "androidsdk": { "packages": [ { "desc": "Android SDK Build-Tools 35", "sdkPackage": { "id": "build-tools;*", "version": "[30.0.2,30.0.3,31.0.0,32.0.0,33.0.0,33.0.1,33.0.2,33.0.3,34.0.0,35.0.0]", "recommendedId": "build-tools;35.0.0", "recommendedVersion": "35.0.0" }, "optional": "false" }, { "desc": "Android SDK Command-line Tools", "sdkPackage": { "id": "cmdline-tools;*", "version": "[5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0]", "recommendedId": "cmdline-tools;13.0", "recommendedVersion": "13.0" }, "optional": "false" }, { "desc": "Android SDK Platform 28", "sdkPackage": { "id": "platforms;android-*", "version": "[1,1,1,2,2,2,2,3,3,3,3,3,3,5,6]", "recommendedId": "platforms;android-28", "recommendedVersion": "6" }, "optional": "false" }, { "desc": "Android SDK Platform-Tools", "sdkPackage": { "id": "platform-tools", "version": "[33.0.2,33.0.3,34.0.1,34.0.3,34.0.4,34.0.5,35.0.1,35.0.2]", "recommendedId": "platform-tools", "recommendedVersion": "35.0.2" }, "optional": "false" }, … ] } } } [0]: https://loop.cloud.microsoft/p/eyJ1IjoiaHR0cHM6Ly9taWNyb3NvZnQuc2hhcmVwb2ludC1kZi5jb20vc2l0ZXMvYzIyZmVjMDMtN2I4OS00OTJhLTgzNzQtZmZjMTI4YjMwMWRhP25hdj1jejBsTWtaemFYUmxjeVV5Um1NeU1tWmxZekF6TFRkaU9Ea3RORGt5WVMwNE16YzBMV1ptWXpFeU9HSXpNREZrWVNaa1BXSWxNakZXTUhSeU9XY3dRbk5WYlhVdFJWUjNRVEZNY0dOSmQwdG1VVEZUZFVFeFRuRk5XbVZ3TUhVd1dUaEhkVVpKVlRSUGIxWnlVMWxoZFRaT2RFODRTamhISm1ZOU1ERlhSelkwU0RNMU56TlZRbEpITWs1TU1rSkdTemRZV1ZCWFJqSlNTRVJQVENaalBTVXlSaVpoUFV4dmIzQkJjSEFtY0QwbE5EQm1iSFZwWkhnbE1rWnNiMjl3TFhCaFoyVXRZMjl1ZEdGcGJtVnlKbmc5SlRkQ0pUSXlkeVV5TWlVelFTVXlNbFF3VWxSVlNIaDBZVmRPZVdJelRuWmFibEYxWXpKb2FHTnRWbmRpTW14MVpFTXhhMXBwTldwaU1qRTRXV2xHVjAxSVVubFBWMk4zVVc1T1ZtSllWWFJTVmxJelVWUkdUV05IVGtwa01IUnRWVlJHVkdSVlJYaFVia1pPVjIxV2QwMUlWWGRYVkdoSVpGVmFTbFpVVWxCaU1WcDVWVEZzYUdSVVdrOWtSVGcwVTJwb1NHWkVRWGhXTUdNeVRrVm5lazU2VGtwVU1WRXdWMnQwUmxGNldrdFRSbXhaVlRCYVVWUlZhRlJXUlVaRFYyeEZKVE5FSlRJeUpUSkRKVEl5YVNVeU1pVXpRU1V5TWprNE5HUXhZMlpoTFRnMVpXUXROR1kyWXkxaU9EWmlMVFJtTXpZMU1EQXlaak5tTnlVeU1pVTNSQT09In0%3D?ct=1728330820992&

Context: [`Releases.json` loop][0] Context: xamarin/xamarin-macios#21779 (comment) There is a desire to have the .NET Workloads have a machine readable description of what dependencies they require in order to run, in order to facilitate tooling that would check for these dependencies. Add `tools/workload-dependencies`, a new tool which parses a "Xamarin Manifest" to generate `workload-dependencies.json`. The "canonical" location for the "Xamarin Manifest" is within `external/android-platform-support/Feeds/AndroidManifestFeed_d17.12.xml`; failing that, <https://aka.ms/AndroidManifestFeed/d17-12> can be used. Output of the tool is a JSON document specifying ther required JDK and Android SDK packages which the .NET for Android workload requires: { "microsoft.net.sdk.android": { "workload": { "alias": [ "android" ], "version": "35.0.100" }, "jdk": { "version": "[17.0,18.0)", "recommendedVersion": "17.0.12" }, "androidsdk": { "packages": [ { "desc": "Android SDK Build-Tools 35", "sdkPackage": { "id": "build-tools;*", "version": "[30.0.2,30.0.3,31.0.0,32.0.0,33.0.0,33.0.1,33.0.2,33.0.3,34.0.0,35.0.0]", "recommendedId": "build-tools;35.0.0", "recommendedVersion": "35.0.0" }, "optional": "false" }, { "desc": "Android SDK Command-line Tools", "sdkPackage": { "id": "cmdline-tools;*", "version": "[5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0]", "recommendedId": "cmdline-tools;13.0", "recommendedVersion": "13.0" }, "optional": "false" }, { "desc": "Android SDK Platform 35", "sdkPackage": { "id": "platforms;android-*", "version": "[2,2,3,2,3,2,3,6,5,3,1,1,3,3,1]", "recommendedId": "platforms;android-35", "recommendedVersion": "1" }, "optional": "false" }, { "desc": "Android SDK Platform-Tools", "sdkPackage": { "id": "platform-tools", "version": "[33.0.2,33.0.3,34.0.1,34.0.3,34.0.4,34.0.5,35.0.1,35.0.2]", "recommendedId": "platform-tools", "recommendedVersion": "35.0.2" }, "optional": "false" }, … ] } } } [0]: https://loop.cloud.microsoft/p/eyJ1IjoiaHR0cHM6Ly9taWNyb3NvZnQuc2hhcmVwb2ludC1kZi5jb20vc2l0ZXMvYzIyZmVjMDMtN2I4OS00OTJhLTgzNzQtZmZjMTI4YjMwMWRhP25hdj1jejBsTWtaemFYUmxjeVV5Um1NeU1tWmxZekF6TFRkaU9Ea3RORGt5WVMwNE16YzBMV1ptWXpFeU9HSXpNREZrWVNaa1BXSWxNakZXTUhSeU9XY3dRbk5WYlhVdFJWUjNRVEZNY0dOSmQwdG1VVEZUZFVFeFRuRk5XbVZ3TUhVd1dUaEhkVVpKVlRSUGIxWnlVMWxoZFRaT2RFODRTamhISm1ZOU1ERlhSelkwU0RNMU56TlZRbEpITWs1TU1rSkdTemRZV1ZCWFJqSlNTRVJQVENaalBTVXlSaVpoUFV4dmIzQkJjSEFtY0QwbE5EQm1iSFZwWkhnbE1rWnNiMjl3TFhCaFoyVXRZMjl1ZEdGcGJtVnlKbmc5SlRkQ0pUSXlkeVV5TWlVelFTVXlNbFF3VWxSVlNIaDBZVmRPZVdJelRuWmFibEYxWXpKb2FHTnRWbmRpTW14MVpFTXhhMXBwTldwaU1qRTRXV2xHVjAxSVVubFBWMk4zVVc1T1ZtSllWWFJTVmxJelVWUkdUV05IVGtwa01IUnRWVlJHVkdSVlJYaFVia1pPVjIxV2QwMUlWWGRYVkdoSVpGVmFTbFpVVWxCaU1WcDVWVEZzYUdSVVdrOWtSVGcwVTJwb1NHWkVRWGhXTUdNeVRrVm5lazU2VGtwVU1WRXdWMnQwUmxGNldrdFRSbXhaVlRCYVVWUlZhRlJXUlVaRFYyeEZKVE5FSlRJeUpUSkRKVEl5YVNVeU1pVXpRU1V5TWprNE5HUXhZMlpoTFRnMVpXUXROR1kyWXkxaU9EWmlMVFJtTXpZMU1EQXlaak5tTnlVeU1pVTNSQT09In0%3D?ct=1728330820992&

…ndencies-json

vs-mobiletools-engineering-service2 · 2024-12-19T22:11:24Z

🔥 [CI Build] Build failed (Detect API changes) 🔥

Build failed for the job 'Detect API changes' (with job status 'Failed')

Pipeline on Agent
Hash: 03475b1f8b3819dc950676d64a55cdaae36254eb [PR build]

rolfbjarne · 2024-12-20T07:06:55Z

/sudo backport release/9.0.1xx

vs-mobiletools-engineering-service2 · 2024-12-20T07:08:04Z

Backport Job to branch release/9.0.1xx Created! The magic is happening here

vs-mobiletools-engineering-service2 · 2024-12-20T07:11:03Z

Oh no! Backport failed! Please see https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=10739822 for more details.

rolfbjarne · 2024-12-20T08:16:17Z

/azp run xamarin-macios-pr,xamarin-macios-apidiff

azure-pipelines · 2024-12-20T08:16:36Z

Azure Pipelines successfully started running 1 pipeline(s).

vs-mobiletools-engineering-service2 · 2024-12-20T09:03:34Z

✅ [CI Build] Build passed (Build packages) ✅

Pipeline on Agent
Hash: 7fc9591e1a42c15aad57b1f037132a7ee9cb07b8 [PR build]

vs-mobiletools-engineering-service2 · 2024-12-20T09:26:27Z

✅ [CI Build] Build passed (Build macOS tests) ✅

Pipeline on Agent
Hash: 7fc9591e1a42c15aad57b1f037132a7ee9cb07b8 [PR build]

vs-mobiletools-engineering-service2 · 2024-12-20T09:34:57Z

💻 [CI Build] Windows Integration Tests passed 💻

✅ All Windows Integration Tests passed.

Pipeline on Agent
Hash: 7fc9591e1a42c15aad57b1f037132a7ee9cb07b8 [PR build]

vs-mobiletools-engineering-service2 · 2024-12-20T09:39:55Z

💻 [CI Build] Tests on macOS M1 - Mac Monterey (12) passed 💻

✅ All tests on macOS M1 - Mac Monterey (12) passed.

Pipeline on Agent
Hash: 7fc9591e1a42c15aad57b1f037132a7ee9cb07b8 [PR build]

vs-mobiletools-engineering-service2 · 2024-12-20T09:40:53Z

💻 [CI Build] Tests on macOS arm64 - Mac Sequoia (15) passed 💻

✅ All tests on macOS arm64 - Mac Sequoia (15) passed.

Pipeline on Agent
Hash: 7fc9591e1a42c15aad57b1f037132a7ee9cb07b8 [PR build]

vs-mobiletools-engineering-service2 · 2024-12-20T09:41:34Z

💻 [CI Build] Tests on macOS X64 - Mac Sonoma (14) passed 💻

✅ All tests on macOS X64 - Mac Sonoma (14) passed.

Pipeline on Agent
Hash: 7fc9591e1a42c15aad57b1f037132a7ee9cb07b8 [PR build]

vs-mobiletools-engineering-service2 · 2024-12-20T09:41:56Z

💻 [CI Build] Tests on macOS M1 - Mac Ventura (13) passed 💻

✅ All tests on macOS M1 - Mac Ventura (13) passed.

Pipeline on Agent
Hash: 7fc9591e1a42c15aad57b1f037132a7ee9cb07b8 [PR build]

vs-mobiletools-engineering-service2 · 2024-12-20T15:05:07Z

🚀 [CI Build] Test results 🚀

Test results

✅ All tests passed on VSTS: test results.

🎉 All 107 tests passed 🎉

Tests counts

✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (iOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (MacCatalyst): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (macOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (Multiple platforms): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (tvOS): All 1 tests passed. Html Report (VSDrops) Download
✅ framework: All 2 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 4 tests passed. Html Report (VSDrops) Download
✅ generator: All 5 tests passed. Html Report (VSDrops) Download
✅ interdependent-binding-projects: All 4 tests passed. Html Report (VSDrops) Download
✅ introspection: All 4 tests passed. Html Report (VSDrops) Download
✅ linker: All 40 tests passed. Html Report (VSDrops) Download
✅ monotouch (iOS): All 7 tests passed. Html Report (VSDrops) Download
✅ monotouch (MacCatalyst): All 10 tests passed. Html Report (VSDrops) Download
✅ monotouch (macOS): All 9 tests passed. Html Report (VSDrops) Download
✅ monotouch (tvOS): All 9 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. [attempt 2] Html Report (VSDrops) Download
✅ xcframework: All 4 tests passed. Html Report (VSDrops) Download
✅ xtro: All 1 tests passed. Html Report (VSDrops) Download

Pipeline on Agent
Hash: 7fc9591e1a42c15aad57b1f037132a7ee9cb07b8 [PR build]

[dotnet] Create and ship a WorkloadDependencies.json file. Fixes #21768.

6735240

Fixes #21768.

rolfbjarne requested a review from Redth December 9, 2024 18:45

rolfbjarne mentioned this pull request Dec 9, 2024

[Workloads] Publish a WorkloadDependencies file inside each manifest pack #21768

Closed

jonathanpeppers reviewed Dec 9, 2024

View reviewed changes

dalexsoto approved these changes Dec 9, 2024

View reviewed changes

This comment has been minimized.

Sign in to view

dalexsoto approved these changes Dec 12, 2024

View reviewed changes

jonpryor mentioned this pull request Dec 12, 2024

[build] Generate workload-dependencies.json dotnet/android#9613

Merged

Merge remote-tracking branch 'origin/main' into dev/rolf/workloaddepe…

2ddbe2a

…ndencies-json

This comment has been minimized.

Sign in to view

Merge branch 'main' into dev/rolf/workloaddependencies-json

7fc9591

This comment has been minimized.

Sign in to view

rolfbjarne merged commit 9853fef into main Dec 20, 2024
12 checks passed

rolfbjarne deleted the dev/rolf/workloaddependencies-json branch December 20, 2024 15:05

[dotnet] Create and ship a WorkloadDependencies.json file. Fixes #21768. #21779

[dotnet] Create and ship a WorkloadDependencies.json file. Fixes #21768. #21779

Conversation

rolfbjarne commented Dec 9, 2024

rolfbjarne commented Dec 9, 2024

Microsoft.NET.Sdk.MacCatalyst.Manifest-9.0.100.18.1.9209-ci.workloaddependencies-json.nupkg

Microsoft.NET.Sdk.iOS.Manifest-9.0.100.18.1.9209-ci.workloaddependencies-json.nupkg

Microsoft.NET.Sdk.macOS.Manifest-9.0.100.15.1.9209-ci.workloaddependencies-json.nupkg

Microsoft.NET.Sdk.tvOS.Manifest-9.0.100.18.1.9209-ci.workloaddependencies-json.nupkg

jonathanpeppers Dec 9, 2024

Choose a reason for hiding this comment

jonathanpeppers Dec 9, 2024

Choose a reason for hiding this comment

Redth Dec 9, 2024

Choose a reason for hiding this comment

jonathanpeppers Dec 9, 2024

Choose a reason for hiding this comment

Redth Dec 19, 2024

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

vs-mobiletools-engineering-service2 commented Dec 11, 2024

✅ API diff for current PR / commit

✅ API diff vs stable

ℹ️ Generator diff

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

vs-mobiletools-engineering-service2 commented Dec 19, 2024

🔥 [CI Build] Build failed (Detect API changes) 🔥

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rolfbjarne commented Dec 20, 2024

vs-mobiletools-engineering-service2 commented Dec 20, 2024

vs-mobiletools-engineering-service2 commented Dec 20, 2024

This comment has been minimized.

This comment has been minimized.

rolfbjarne commented Dec 20, 2024

azure-pipelines bot commented Dec 20, 2024

vs-mobiletools-engineering-service2 commented Dec 20, 2024

✅ [CI Build] Build passed (Build packages) ✅

vs-mobiletools-engineering-service2 commented Dec 20, 2024

✅ [CI Build] Build passed (Build macOS tests) ✅

vs-mobiletools-engineering-service2 commented Dec 20, 2024

💻 [CI Build] Windows Integration Tests passed 💻

vs-mobiletools-engineering-service2 commented Dec 20, 2024

💻 [CI Build] Tests on macOS M1 - Mac Monterey (12) passed 💻

vs-mobiletools-engineering-service2 commented Dec 20, 2024

💻 [CI Build] Tests on macOS arm64 - Mac Sequoia (15) passed 💻

vs-mobiletools-engineering-service2 commented Dec 20, 2024

💻 [CI Build] Tests on macOS X64 - Mac Sonoma (14) passed 💻

vs-mobiletools-engineering-service2 commented Dec 20, 2024

💻 [CI Build] Tests on macOS M1 - Mac Ventura (13) passed 💻

This comment has been minimized.

vs-mobiletools-engineering-service2 commented Dec 20, 2024

🚀 [CI Build] Test results 🚀

Test results

Tests counts