Azure VM Data Disk Max Partition Size is Different to Actual Size

[PlagueHO]

Details of the scenario you tried and the problem that is occurring

Using Disk resource to create a partition on an empty data disk using all free space reports that the size of the partition is smaller than it maximum size:

I was using a 1023 GB data disk, so it is possible that this is a behavior with specific disk sizes.

Verbose logs showing the problem

VERBOSE: Perform operation 'Invoke CimMethod' with following parameters, ''methodName' = SendConfigurationApply,'className' = MSFT_DSCLocalConfigurationManager,'namespaceName' = root/Microsoft/Windows/DesiredStateConfiguration'.
VERBOSE: An LCM method call arrived from computer bmddsctest with user sid S-1-5-21-393396716-4191445587-4237219874-500.
VERBOSE: [bmddsctest]: LCM:  [ Start  Set      ]
VERBOSE: [bmddsctest]: LCM:  [ Start  Resource ]  [[WaitForDisk]Disk2]
VERBOSE: [bmddsctest]: LCM:  [ Start  Test     ]  [[WaitForDisk]Disk2]
VERBOSE: [bmddsctest]:                            [[WaitForDisk]Disk2] Test-TargetResource: Checking for disk with Number '2'.
VERBOSE: [bmddsctest]:                            [[WaitForDisk]Disk2] Test-TargetResource: Found disk with Number '2' named 'Msft Virtual Disk'.
VERBOSE: [bmddsctest]: LCM:  [ End    Test     ]  [[WaitForDisk]Disk2]  in 2.0780 seconds.
VERBOSE: [bmddsctest]: LCM:  [ Skip   Set      ]  [[WaitForDisk]Disk2]
VERBOSE: [bmddsctest]: LCM:  [ End    Resource ]  [[WaitForDisk]Disk2]
VERBOSE: [bmddsctest]: LCM:  [ Start  Resource ]  [[Disk]EVolume]
VERBOSE: [bmddsctest]: LCM:  [ Start  Test     ]  [[Disk]EVolume]
VERBOSE: [bmddsctest]:                            [[Disk]EVolume] Test-TargetResource: Testing disk with Number '2' status for drive letter 'F'.
VERBOSE: [bmddsctest]:                            [[Disk]EVolume] Test-TargetResource: Checking if disk with Number '2' is initialized.
VERBOSE: [bmddsctest]:                            [[Disk]EVolume] Test-TargetResource: Partition assigned to drive F has size 1098301571072, which does not match expected size 1098302602752. Set AllowDestructive to $true to enable resizing of partition.
VERBOSE: [bmddsctest]:                            [[Disk]EVolume] Perform operation 'Query CimInstances' with following parameters, ''queryExpression' = SELECT BlockSize from Win32_Volume WHERE DriveLetter = 'F:','queryDialect' = WQL,'namespaceName' = root\c
imv2'.
VERBOSE: [bmddsctest]:                            [[Disk]EVolume] Operation 'Query CimInstances' complete.
VERBOSE: [bmddsctest]: LCM:  [ End    Test     ]  [[Disk]EVolume]  in 2.3230 seconds.
VERBOSE: [bmddsctest]: LCM:  [ Skip   Set      ]  [[Disk]EVolume]
VERBOSE: [bmddsctest]: LCM:  [ End    Resource ]  [[Disk]EVolume]
VERBOSE: [bmddsctest]: LCM:  [ End    Set      ]
VERBOSE: [bmddsctest]: LCM:  [ End    Set      ]    in  9.0260 seconds.
VERBOSE: Operation 'Invoke CimMethod' complete.
VERBOSE: Time taken for configuration job to complete is 9.149 seconds

The problem can be replicated outside of DSC, so is not specifically caused by the Disk resource but the behavior of Get-PartitionSupportedSize, New-Partition -UseMaximumSize and Get-Partition.

For example:

$null = Clear-Disk -Number 2 -RemoveData -Confirm:$false
$null = Initialize-Disk -Number 2 -PartitionStyle GPT
$partition = New-Partition -DiskNumber 2 -UseMaximumSize

('Actual Partition Size: {0}' -f (Get-Partition -DiskNumber 2 -PartitionNumber 2).Size)
('Supported Partition Size: {0}' -f (Get-PartitionSupportedSize -DiskNumber 2 -PartitionNumber ($partition.PartitionNumber)).SizeMax)

Suggested solution to the issue

The DSC configuration that is used to reproduce the issue (as detailed as possible)

Configuration Storage
{
    Import-DSCResource -ModuleName StorageDsc

    Node localhost
    {
        WaitForDisk Disk2
        {
             DiskId = 2
             RetryIntervalSec = 60
             RetryCount = 60
        }

        Disk EVolume
        {
             DiskId = 2
             DriveLetter = 'F'
             DependsOn = '[WaitForDisk]Disk2'
        }
    }
}

Install-Module StorageDsc
Storage -OutputPath . -Verbose
Start-DscConfiguration -Wait -Force -Path .\Storage -ComputerName Localhost -Verbose

The operating system the target node is running

OsName               : Microsoft Windows Server 2016 Datacenter
OsOperatingSystemSKU : DatacenterServerEdition
OsArchitecture       : 64-bit
WindowsBuildLabEx    : 14393.2608.amd64fre.rs1_release.181024-1742
OsLanguage           : en-US
OsMuiLanguages       : {en-US}

Version and build of PowerShell the target node is running

Name                           Value                                                                                                                                                                                                                             
----                           -----                                                                                                                                                                                                                             
PSVersion                      5.1.14393.2608                                                                                                                                                                                                                    
PSEdition                      Desktop                                                                                                                                                                                                                           
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                                                                                                                                                           
BuildVersion                   10.0.14393.2608                                                                                                                                                                                                                   
CLRVersion                     4.0.30319.42000                                                                                                                                                                                                                   
WSManStackVersion              3.0                                                                                                                                                                                                                               
PSRemotingProtocolVersion      2.3                                                                                                                                                                                                                               
SerializationVersion           1.1.0.1  

Version of the DSC module that was used ('dev' if using current dev branch)

4.3.0.0

[nyanhp]

@PlagueHO I stumbled upon this because my customer is seeing this issue. We are using -AllowDestructive, which means a neverending clear disk -> create partition loop since the size is different. It is not due to the size of the volume though - we are getting this issue with various sizes.

[PlagueHO]

Hi @nyanhp - What we probably need to check next is if this is an issue with the underlying CIM information - which I think underpins the PS cmdlets here. If it is in the CIM then we'll need to raise this issue with the Windows Server team.

E.g. the fact that when using UseMaximumSize you don't actually get a partition with the PartitionSupportedSize.SizeMax seems wrong to me, but I might be implying something that isn't the case.

$null = Clear-Disk -Number 2 -RemoveData -Confirm:$false
$null = Initialize-Disk -Number 2 -PartitionStyle GPT
$partition = New-Partition -DiskNumber 2 -UseMaximumSize

('Actual Partition Size: {0}' -f (Get-Partition -DiskNumber 2 -PartitionNumber 2).Size)
('Supported Partition Size: {0}' -f (Get-PartitionSupportedSize -DiskNumber 2 -PartitionNumber ($partition.PartitionNumber)).SizeMax)

What you could probably do though is set a static partition size when using the resource. This will prevent the UseMaximumSize and hopefully will result in this not happening. Have you tried this and did it work?

[nyanhp]

Hi @PlagueHO I went another route... I need to run a few more tests with different file system formats and so on, but in my environment, the following alteration to Test-DscResource looked good:

# Check the partition size
if ($partition -and -not ($PSBoundParameters.ContainsKey('Size')))
{
    $supportedSize = ($partition | Get-PartitionSupportedSize)

    $Size = if ($partition.Size - $supportedSize.SizeMax -lt 1MB) {$partition.Size} else {$supportedSize.SizeMax}
}

I am unsure how this will behave for larger volumes, which I cannot really test. The diff between $Partition.Size and SizeMax varied with values between 0.8 and 0.98 MiB for my tested sizes of 20, 200, 500 and 1000 GiB. The difference in my lab environment's vhdx files was always 0.98 MiB.

To be honest, it just looks like the offset is not taken into consideration when generating the supported size.

In my scenario a fixed size is not desirable. We do not want to waste space if the size is too small or run into errors because it is too big.

In the end, an issue might make more sense. There definitely is something wrong, but that goes deeper than my knowledge. The MSFT_Partition class documents the following:
Method GetSupportedSize: Min/Max size the partition can be resized to. The maximum size is determined by adding the size of any free extents immediately after the current partition
Property Size: The uint64 total partition size in byte

I have no clue why there is a difference, but I assume it has to do with "adding the size of any free extents" and the way UseMaximumSize will calculate the maximum partition size. The Resize method of the Partition class requires a valid partition size and will fail if the size is too small or too big.

[PlagueHO]

Wow! Great info @nyanhp - I reckon we might need to look into using your work around. I'll also reach out to the Windows Server team via the MVP PGI and see if they can enlighten us (or if I've been misusing/misunderstanding the behavior). Although, you might have better contacts over there 😁

But in the mean time we should look at the risks in using your work around.

[PlagueHO]

Doh! Wrong URL posted in one of the other issues! So ignore ^

[nyanhp]

@PlagueHO you would be surprised how little impact we premier field engineers have vs the MVP community, but I'll try to find out nevertheless.

In the meantime I will create a PR with the workaround.

[nyanhp]

@PlagueHO , it is the offset. I can't believe that I forgot about that. I will need to invest some time to properly calculate the result for Test-TargetResource and test the behavior. The default offset should be 1MB, the resource does not allow specifying Alignment and Offset which makes this a bit easier.

[nyanhp]

Sorry for the spam, but I experimented some more. It is the alignment (multiple of PhysicalSectorSize) of the partition that is used in the calculations, at least in some form. I have yet to find out how Offset and Alignment are related here.

$sectorsize = (Get-Disk -Number 1).PhysicalSectorSize

foreach ($off in 1..100)
{
    $null = Clear-Disk -Number 1 -RemoveData -Confirm:$false
    $null = Initialize-Disk -Number 1 -PartitionStyle GPT
    $reservedpartition = get-partition -number 1 -DiskNumber 1

    $junkpartition = New-Partition -DiskNumber 1 -Size 5gb -Offset ($reservedpartition.offset + $reservedpartition.size + $($off *1mb)) -Alignment ($sectorsize * $off)
    $partition = New-Partition -DiskNumber 1 -UseMaximumSize -Offset ($junkpartition.offset + $junkpartition.size + $($off *1mb)) -Alignment ($sectorsize * $off)

    'Offset: {0}, Alignment = {1}' -f ($junkpartition.offset + $junkpartition.size + $($off *1mb)), ($sectorsize * $off)
    'Actual Partition Size: {0}' -f $partition.Size
    'Supported Partition Size: {0}' -f ($partition | Get-PartitionSupportedSize).SizeMax
    'Diff: {0:n2}MB' -f ((($partition | Get-PartitionSupportedSize).SizeMax - $partition.Size) / 1mb)
}

I created an issue on UserVoice, let's see how that goes: https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/36967870-get-partitionsupportedsize-and-msft-partition-clas

[johnwc]

When should we expect to see this fix released?

[PlagueHO]

Hi @johnwc - it will go out in the next DSC Resource Kit release, which is due in about 5 weeks time. It is a week after the next community call (which is on 23 October IIRC). So the release is likely right at the end of Oct.

[johnwc]

@PlagueHO Thanks for the update. Do y'all publish preview releases?

[PlagueHO]

@johnwc - not yet, but it is coming soon. We're in the process of migrating these modules over to the DSCCommunity.org, and when we do we'll adopt "per resource" release cadence, allowing maintainers to do preview releases etc.