Fix Additional Metadata for App Submission packages

Documentation/USAGE.md

@@ -24,6 +24,7 @@
     *   [IAP Overview](#iap-overview)
     *   [Creating Your IAP Payload](#creating-your-iap-payload)
     *   [IAP Commands](#iap-commands)
+*   [Schema Versions](#schema-versions)
 *   [Using INT vs PROD](#using-int-vs-prod)
 *   [Telemetry](#telemetry)
 *   [FAQ](#faq)
@@ -879,6 +880,104 @@ To monitor an IAP submission:
 Follow the steps in [monitoring a submission](#monitoring-a-submission), and be sure to include
 **`-IapId`** in the function parameters _instead of_ **`-AppId`**.
 
+## Schema Versions
+
+StoreBroker's packaging commands (`New-SubmissionPackage` and `New-InAppProductSubmissionPackage`)
+add properties to the generated .json file that are not part of the official Submission API JSON
+schema.  These additional properties are added in order to enable additional StoreBroker scenarios
+(like validating the `AppId` or `IapId` being used with a submission package, and determining which
+packages are redundant when using the `-UpdatePackages` switch).
+
+It is our intention to never change how these custom properties work once we've published out that
+version of StoreBroker, however it is possible that bugs or additional circumstances may simply make
+it necessary.
+
+To that end, starting with version `1.11.1` of StoreBroker, we will always add a `sbSchema` property
+to the generated json file, with a simple **integer value** that will be incremented whenever we make any
+change to those additional properties.  We'll document what those differences are below so that you'll
+be able to make corresponding changes to your own code if you depend on these properties.
+
+### App Submission Packages
+
+#### Version 1
+
+We never used the value `1` for a published schema.  Any App JSON package file that doesn't have an
+`sbSchema` property is effectively version 1.
+
+**Properties Added**
+ * `appId` - Stores the `appId` of the In-App Product that the submission package is for.
+ * `applicationPackages.minOSVersion` - An array of all Min Versions referenced in the app packages.
+ * `applicationPackages.targetFamiliesEx` - An array of all device families that the package targets.
+ * `applicationPackages.innerPackages.[architecture].minOSVersion` - An array of all Min OS Versions
+    referenced in the specific architecture app package.
+ * `applicationPackages.innerPackages.[architecture].targetFamiliesEx` - An array of all device
+    families targeted in this specific architecture app package.
+
+**Properties Removed**
+ * _None_
+
+**Properties Changed**
+ * _None_
+
+#### Version 2
+
+**Properties Added**
+ * `sbSchema` - Tracks the version number of the schema to identify what additional StoreBroker properties
+   should be expected within.
+
+**Properties Removed**
+ * `applicationPackages.minOSVersion` - This information can now be found in better context in
+   `applicationPackages.targetFamiliesEx.minOSVersion`
+ * `applicationPackages.innerPackages.[architecture].minOSVersion` - This information can now
+    be found in better context in `applicationPackages.innerPackages.[architecture].targetFamiliesEx.minOSVersion`
+
+**Properties Changed**
+ * `applicationPackages.targetDeviceFamiliesEx` - This used to be an array of strings of each
+   device family. This is now an array of dictionaries where each dictionary is a
+   `name`/`minOSVersion` pair, since a single appx can have different Min OS Versions depending
+   on which platform is being looked at.
+ * `applicationPackages.targetDeviceFamilies` - There was a bug here that if there was more than
+   one Min OS Version within a package, the string would look like this:
+   `Windows.Desktop min version System.Object[]`.  This is now fixed.
+ * `applicationPackages.innerPackages.[architecture].targetDeviceFamiliesEx` - This used to be
+   an array of strings of each device family. This is now an array of dictionaries where each dictionary
+   is a `name`/`minOSVersion` pair, since a single appx can have different Min OS Versions depending on
+   which platform is being looked at.
+ * `applicationPackages.innerPackages.[architecture].targetDeviceFamilies` - There was a bug here that
+   if there was more than one Min OS Version within a package, the string would look like this:
+   `Windows.Desktop min version System.Object[]`.  This is now fixed.
+
+### In-App Product (IAP) Submission Packages
+
+#### 1
+
+We never used the value `1` for a published schema.  Any IAP JSON package file that doesn't have an
+`sbSchema` property is effectively version 1.
+
+**Properties Added**
+ * `iapId` - Stores the `iapId` of the In-App Product that the submission package is for.
+
+**Properties Removed**
+ * _None_
+
+**Properties Changed**
+ * _None_
+
+#### 2
+
+This schema is equivalent to v1, but now with the addition of `sbSchema` tracking the schema version.
+
+**Properties Added**
+ * `sbSchema` - Tracks the version number of the schema to identify what additional StoreBroker properties
+   should be expected within.
+
+**Properties Removed**
+ * _None_
+
+**Properties Changed**
+ * _None_
+
+
 ## Using INT vs PROD
 
 > This option is only available for Microsoft employees using the official Microsoft

StoreBroker/PackageTool.ps1

@@ -33,10 +33,20 @@ $script:applicationMetadataProperties = @(
     "capabilities",
     "targetDeviceFamilies",
     "targetDeviceFamiliesEx",
-    "minOSVersion",
     "innerPackages"
 )
 
+# The API formats version numbers as "[device family] min version [min version]"
+$script:minVersionFormatString = "{0} min version {1}"
+
+# The current version of the StoreBroker schema that PackageTool is authoring for app and IAP submissions.
+# The StoreBroker schema may include metadata that isn't a core part of the official Submission API
+# JSON schema (like the appId or iapId, package metadata, etc...). These values should be updated any time
+# we alter what additional metadata is added to the schema for that submission type.
+$script:appSchemaVersion = 2
+$script:iapSchemaVersion = 2
+$script:schemaPropertyName = 'sbSchema'
+
 function Get-StoreBrokerConfigFileContentForIapId
 {
 <#
@@ -1296,7 +1306,7 @@ function New-ApplicationMetadataTable
 
         The hashtable has keys for
          "version", "architecture", "targetPlatform", "languages",
-        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx", "minOSVersion",
+        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx"
         and "innerPackages"
 
     .OUTPUTS
@@ -1384,7 +1394,7 @@ function Read-AppxMetadata
         Reads various metadata properties about the input .appx file.
 
         The metadata read is "version", "architecture", "targetPlatform", "languages",
-        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx", "minOSVersion",
+        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx"
         and "innerPackages".  Not all of the metadata read is actually passed as
         part of the Store submission; some metadata is used as part of an app flighting
         workflow.
@@ -1437,7 +1447,6 @@ function Read-AppxMetadata
 
         $metadata.version        = $manifest.Package.Identity.Version
         $metadata.architecture   = $manifest.Package.Identity.ProcessorArchitecture
-        $metadata.minOSVersion   = $manifest.Package.Dependencies.TargetDeviceFamily.MinVersion
         $metadata.targetPlatform = Get-TargetPlatform -AppxManifestPath $appxManifest
         $metadata.name           = $manifest.Package.Identity.Name -creplace '^Microsoft\.', ''
 
@@ -1453,14 +1462,15 @@ function Read-AppxMetadata
             Sort-Object -Unique
 
         $metadata.targetDeviceFamiliesEx = @()
-        $metadata.targetDeviceFamiliesEx += $manifest.Package.Dependencies.TargetDeviceFamily.Name |
-            Where-Object { $null -ne $_ } |
-            Sort-Object -Unique
+        $metadata.targetDeviceFamiliesEx += $manifest.Package.Dependencies.TargetDeviceFamily |
+            Where-Object { $null -ne $_.Name } |
+            Sort-Object -Property Name -Unique |
+            ForEach-Object { [PSCustomObject]@{ 'name' = $_.Name; 'minOSVersion' = $_.MinVersion } }
 
         $metadata.targetDeviceFamilies = @()
         foreach ($family in $metadata.targetDeviceFamiliesEx)
         {
-            $metadata.targetDeviceFamilies += ("{0} min version {1}" -f $family, $metadata.minOSVersion)
+            $metadata.targetDeviceFamilies += ($script:minVersionFormatString -f $family.Name, $family.minOSVersion)
         }
 
         # A single .appx will never have an inner package, but we will still set this property to
@@ -1497,7 +1507,7 @@ function Read-AppxUploadMetadata
         Reads various metadata properties about the input .appxupload.
 
         The metadata read is "version", "architecture", "targetPlatform", "languages",
-        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx", "minOSVersion",
+        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx"
         and "innerPackages".  Not all of the metadata read is actually passed as
         part of the Store submission; some metadata is used as part of an app flighting
         workflow.
@@ -1596,7 +1606,7 @@ function Read-AppxBundleMetadata
         Reads various metadata properties about the input .appxbundle.
 
         The metadata read is "version", "architecture", "targetPlatform", "languages",
-        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx", "minOSVersion",
+        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx"
         and "innerPackages".  Not all of the metadata read is actually passed as
         part of the Store submission; some metadata is used as part of an app flighting
         workflow.
@@ -1677,21 +1687,20 @@ function Read-AppxBundleMetadata
             $appxFilePath = (Get-ChildItem -Recurse -Path $expandedContainerPath -Include $application).FullName
             $appxMetadata = Read-AppxMetadata -AppxPath $appxFilePath -AppxInfo $AppxInfo
 
-            # minOSVersion and targetPlatform will always be the values of the last .appx processed.
-            $metadata.minOSVersion    = $appxMetadata.minOSVersion
+            # targetPlatform will always be the values of the last .appx processed.
             $metadata.targetPlatform  = $appxMetadata.targetPlatform
 
             $capabilities            += $appxMetadata.capabilities
             $targetDeviceFamilies    += $appxMetadata.targetDeviceFamilies
             $targetDeviceFamiliesEx  += $appxMetadata.targetDeviceFamiliesEx
 
             $metadata.innerPackages.$($appxMetadata.architecture) = @{
-                version              = $appxMetadata.version;
-                targetDeviceFamilies = $appxMetadata.targetDeviceFamiliesEx;
-                languages            = $appxMetadata.languages;
-                capabilities         = $appxMetadata.capabilities;
-                minOSVersion         = $appxMetadata.minOSVersion;
-                targetPlatform       = $appxMetadata.targetPlatform;
+                version                = $appxMetadata.version;
+                targetDeviceFamiliesEx = $appxMetadata.targetDeviceFamiliesEx
+                targetDeviceFamilies   = $appxMetadata.targetDeviceFamiliesEx | ForEach-Object { $script:minVersionFormatString -f $_.name, $_.minOSVersion }
+                languages              = $appxMetadata.languages;
+                capabilities           = $appxMetadata.capabilities;
+                targetPlatform         = $appxMetadata.targetPlatform;
             }
         }
 
@@ -1701,7 +1710,11 @@ function Read-AppxBundleMetadata
         # results in $m.capabilities being a String type instead of Array type.
         $metadata.capabilities           += $capabilities | Sort-Object -Unique
         $metadata.targetDeviceFamilies   += $targetDeviceFamilies | Sort-Object -Unique
-        $metadata.targetDeviceFamiliesEx += $targetDeviceFamiliesEx | Sort-Object -Unique
+
+        # https://stackoverflow.com/questions/31343752/how-can-you-select-unique-objects-based-on-two-properties-of-an-object-in-powers
+        $metadata.targetDeviceFamiliesEx += $targetDeviceFamiliesEx |
+            Group-Object -Property name, minOSVersion |
+            ForEach-Object { $_.Group | Select-Object -Property name, minOSVersion -First 1 }
     }
     finally
     {
@@ -1741,7 +1754,7 @@ function Get-FormattedFilename
         System.String. The ManifestType_AppName_Version_Architecture string.
 
     .EXAMPLE
-        Get-FormattedFilename @{ name="Maps"; version="2.13.22002.0"; architecture="x86"; targetDeviceFamiliesEx=@("Windows.Desktop") }
+        Get-FormattedFilename @{ name="Maps"; version="2.13.22002.0"; architecture="x86"; targetDeviceFamiliesEx=@(@{ name = "Windows.Desktop"); minOSVersion="1.2.3.0" } }
 
         Would return something like "Desktop_Maps_2.13.22002.0_x86.appxupload"
 #>
@@ -1776,7 +1789,7 @@ function Get-FormattedFilename
     }
 
     # Simplify 'Windows.Universal' to 'Universal'
-    $deviceFamilyCollection = $Metadata.targetDeviceFamiliesEx | ForEach-Object { $_ -replace '^Windows\.', '' }
+    $deviceFamilyCollection = $Metadata.targetDeviceFamiliesEx.Name | ForEach-Object { $_ -replace '^Windows\.', '' }
 
     $formattedBundleTags = @($Metadata.name, $version, $architectureTag)
     if ($deviceFamilyCollection.Count -gt 0)
@@ -1798,7 +1811,7 @@ function Read-ApplicationMetadata
         Reads metadata used for submission of an .appx, .appxbundle, or appxupload.
 
         The metadata read is "version", "architecture", "targetPlatform", "languages",
-        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx", "minOSVersion",
+        "capabilities", "targetDeviceFamilies", "targetDeviceFamiliesEx",
         "innerPackages", and "name".  Not all of the metadata read is actually passed as
         part of the Store submission; some metadata is used as part of an app flighting
         workflow.
@@ -2149,6 +2162,8 @@ function Get-SubmissionRequestBody
 
     $submissionRequestBody = Remove-DeprecatedProperties -SubmissionRequestBody $submissionRequestBody
 
+    $submissionRequestBody | Add-Member -Name $script:schemaPropertyName -Value $script:appSchemaVersion -MemberType NoteProperty
+
     return $submissionRequestBody
 }
 
@@ -2261,6 +2276,8 @@ function Get-InAppProductSubmissionRequestBody
         $submissionRequestBody.listings = Convert-InAppProductListingsMetadata @listingsResources
     }
 
+    $submissionRequestBody | Add-Member -Name $script:schemaPropertyName -Value $script:iapSchemaVersion -MemberType NoteProperty
+
     return $submissionRequestBody
 }
 

StoreBroker/StoreBroker.psd1

@@ -6,7 +6,7 @@
     CompanyName = 'Microsoft Corporation'
     Copyright = 'Copyright (C) Microsoft Corporation.  All rights reserved.'
 
-    ModuleVersion = '1.11.0'
+    ModuleVersion = '1.11.1'
     Description = 'Provides command-line access to the Windows Store Submission REST API.'
 
     RootModule = 'StoreIngestionApi'