Shutting down or rebooting dom0 while VMs are paused takes a very long time

[UndeadDevel]

Qubes OS release

4.1.2

Brief summary

When shutting down or rebooting QubesOS while some VMs are paused, the process hangs for a long time (first there is a stop job running for the full 3 mins and then dracut hangs because it can't umount old-root), so shutdown/reboot take >3 mins longer than usual.

If I unpause all VMs beforehand this doesn't happen. I have added autostart commands that will unpause relevant VMs automatically on shut down / reboot / logout etc and that fixes the problem, but I feel like this should be handled by the OS. The best solution would probably be to kill paused VMs shortly after shut down or reboot is triggered. The alternative is to unpause and qvm-shutdown them, but that may not be desired security-wise. It could also be that the real issue lies somewhere else and paused VMs during shutdown merely trigger it, I suppose.

Steps to reproduce

Pause some VMs and then shut down.

Expected behavior

System shuts down normally

Actual behavior

Hangs twice and takes >3 mins longer than usual to shut down

Related issues

may be related to #5987

[UndeadDevel]

Still an issue on Q4.2rc5. I solve it by adding triggers in startup menu to unpause my DVMs on restart / shutdown / logout etc.

[alimirjamali]

This is easy. QubesVM.shutdown function could check via is_paused function if the qube is paused and un-pause it prior to shutdown. It will need a small unittest. I will work on it.

p.s.: Some internal logs of shutting down a paused qube before the patch:

Sep 03 12:29:54 dom0 libvirtd[1296]: internal error: Failed to shutdown domain '10' with libxenlight
Sep 03 12:29:54 dom0 qubesd[28700]: unhandled exception while calling src=b'dom0' meth=b'admin.vm.Shutdown' dest=b'disp9564' arg=b'' len(untrusted_payload)=0
Sep 03 12:29:54 dom0 qubesd[28700]: Traceback (most recent call last):
Sep 03 12:29:54 dom0 qubesd[28700]:   File "/usr/lib/python3.11/site-packages/qubes/api/__init__.py", line 291, in respond
Sep 03 12:29:54 dom0 qubesd[28700]:     response = await self.mgmt.execute(
Sep 03 12:29:54 dom0 qubesd[28700]:                ^^^^^^^^^^^^^^^^^^^^^^^^
Sep 03 12:29:54 dom0 qubesd[28700]:   File "/usr/lib/python3.11/site-packages/qubes/api/admin.py", line 884, in vm_shutdown
Sep 03 12:29:54 dom0 qubesd[28700]:     await self.dest.shutdown(force=force, wait=wait)
Sep 03 12:29:54 dom0 qubesd[28700]:   File "/usr/lib/python3.11/site-packages/qubes/vm/qubesvm.py", line 1371, in shutdown
Sep 03 12:29:54 dom0 qubesd[28700]:     self.libvirt_domain.shutdown()
Sep 03 12:29:54 dom0 qubesd[28700]:   File "/usr/lib/python3.11/site-packages/qubes/app.py", line 103, in wrapper
Sep 03 12:29:54 dom0 qubesd[28700]:     return attr(*args, **kwargs)
Sep 03 12:29:54 dom0 qubesd[28700]:            ^^^^^^^^^^^^^^^^^^^^^
Sep 03 12:29:54 dom0 qubesd[28700]:   File "/usr/lib64/python3.11/site-packages/libvirt.py", line 2972, in shutdown
Sep 03 12:29:54 dom0 qubesd[28700]:     raise libvirtError('virDomainShutdown() failed')
Sep 03 12:29:54 dom0 qubesd[28700]: libvirt.libvirtError: internal error: Failed to shutdown domain '10' with libxenlight