Storage: Show correct instance root disk size on API

[hamistao]

This aims to get instance root disk sizes correctly, as it currently often shows as 0 or null on the API, either though /1.0/instances?recursion=2 or /1.0/instances/<instance_name>/state, as reported by #14277.

This is still not a perfect solution but is a significant improvement as it:

• Fixes not returning the disk total size when getting the disk usage is not supported by the driver (relevant for dir and lvm)
• Returns default block size for block typed/based instances when the disk device does not have the size key set.
• Returns disk size -1 for instances that have unbounded storage and have access to the entire pool storage.

[hamistao]

There are two situations that aren't fixed by this PR:

• Since we do not yet support live resizing of block volumes, setting the disk size key for a VM or a container on a block based driver with the instance running will only take effect after the instance is restarted. But the API will show the size specified on the size key from the moment it is set, thus being inconsistent until instance restart. This will be partly mitigated by Storage: Add support for online disk growing of zfs and lvm block volumes (from Incus) #14211. Depending on how our plans for live resizing develop in the near future, we can consider developing more intricate solutions to prevent showing incorrect disk sizes in these scenarios.
• When the instance size was influenced by volume.size on the pool level. Refer to '/1.0/instances?recursion=2' Endpoint has missing information. #14277 (comment) for more information on this limitation.

[roosterfish]

There seems to be one more usage of DefaultBlockSize in the instance scriptlet https://github.com/canonical/lxd/blob/main/lxd/scriptlet/instance_placement.go#L224.

Can we do something similar to https://github.com/canonical/lxd/blob/main/lxd/project/limits/permissions.go#L1472 to load the storage driver and fetch the respective DefaultBlockSize? Not sure if it will cause an import loop though.

[hamistao]

@roosterfish I believe everything was addressed. I did something different from what is done on https://github.com/canonical/lxd/blob/main/lxd/project/limits/permissions.go#L1472 to get DefaultBlockSize on InstancePlacementRun, but I think you will like this approach.

Tests for VMs will follow on LXD CI.

[hamistao]

Going back to Draft to add support for the situations described in #14511 (comment).

[hamistao]

This includes the simpler fixes related to showing accurate disk sizes on the API. To address the use cases described in #14511 (comment), a follow up draft PR was opened to unblock these changes from getting merged: #14511 (comment).

[hamistao]

@tomponline This includes the simpler fixes for getting correct root disk sizes and don't address the scenarios described in #14511 (comment). These are addressed in a follow up PR: #14595.

The tests for these changes are included in the follow up PR, but we can check that they are passing on the follow up's test runs.

All that said, this is ready for a review :D

[tomponline]

Thanks!

[tomponline]

@edlerd please can you see whether this helps with storage reporting in the UI. Thanks

fixed