Increase the `MAX_DISKS_PER_INSTANCE` limit to support using up to all disks in a sled for local storage

[AlejandroME]

The local storage feature should be able to use (attach) all disks in a sled to a single VM instance, but we currently enforce a limit of 8 disks per instance.

Several discussions have converged on raising the limit to 12:

• 1 for the initial boot disk, which must be a crucible volume
• 1 reserved for cloud-init
• Up to 10 to use all the physical disks available in a sled

This issue will track the work required to implement the change.

References

• Tracking: local disks UI console#2968

[rmustacc]

I don't really get how we got to 12 here as I've mentioned in other places. If we're increasing the number of devices here and we're making incompatible changes, we should probably move to a full PCI buses worth of devices. To be clear, I get how 12 is 10 + 1 +1, but I don't think really see why that's the right limit here and it seems like we're basically saying folks are going to be at the maximum and assuming there will never be a case where there is more than one volume crucible volume which can easily happen beyond just boot.

[jmpesp]

We talked this over and will be doing two things:

• a short term PR to bump the maximum supported disks to 12 (which including the cloud init volume makes a baker's dozen :)). this will be compatible with all existing instances.

• a longer term change next year to flesh out different instance platform versions, which will include adding more buses to propolis (meaning a breaking change) and a plan for existing instances. this will be flexible enough for us for a long time.