vmm: Correctly suspend and resume the vmm driver.

[MegaManSec]

Previously, VMXON would be executed on a resume, contrary to proper initalization. The MSR lock may be lost on suspension, therefore must be taken again. Likewise, the VMX Enable bit may be cleared upon suspend, requiring to be re-set.

Concretely disable VMX on suspend, and re-enable it on resume.

Note: any IOMMU context will remain lost for any enabled vmm devices.

[bsdjhb]

I think it might be good to be explicit that the contents of MSR_IA32_FEATURE_CONTROL need to be restored in resume (I think that's what "MSR lock" is referring to?) in the log message.

[emaste]

Note: any IOMMU context will remain lost for any enabled vmm devices.

Even if this isn't a new issue from your change, should we have this noted somewhere?

[MegaManSec]

I think it might be good to be explicit that the contents of MSR_IA32_FEATURE_CONTROL need to be restored in resume (I think that's what "MSR lock" is referring to?) in the log message.

Yes, MSR lock refers to MSR_IA32_FEATURE_CONTROL and specifically the IA32_FEATURE_CONTROL_LOCK bit. I will change the commit message to state that the contents MSR_IA32_FEATURE_CONTROL needs to be restored.

[MegaManSec]

Note: any IOMMU context will remain lost for any enabled vmm devices.

Even if this isn't a new issue from your change, should we have this noted somewhere?

Probably (and I may have used the incorrect terminology about "IOMMU context"). FWIW, using vmm with a pci device on my system stalls the system completely upon its use after a resume (more details: https://joshua.hu/brcmfmac-bcm43602-suspension-shutdown-hanging-freeze-linux-freebsd-wifi-bug-pci-passthru) -- when I'm passthru'ing the device to a Linux VM, I have to "remove" the device inside the VM before a suspension, otherwise the system crashes (just completely freezes/stalls) after resume and the device is used again.

As far as I could tell when debugging a bit more lately, the irq (at least) of the device is lost on resume, which results in the system freezing when the device is interacted with.