Use Python API for managing AppVM state

[conorsch]

The Salt management logic (YAML files stored in dom0/*.{sls,top}) has proven rather problematic to work with. See this recent qubes-users thread (in which the SecureDrop Workstation project is named as one of the few examples of Salt usage in Qubes) for examples of community frustration. We can reasonably summarize the relevant pain points with Salt in terms of our needs as:

• High learning curve with Salt, so quite difficult for newcomers
• Unclear packaging strategy for dom0 configs
• Difficult to explicitly declare dependency chains, given parallel execution
• Errors do not percolate well (commands still exit 0)
• Additional dependencies required for Salt management but not for e.g. qvm-run.

We already use the Qubes Python API for running a small suite of config tests in dom0. Expanding the use of the API to enforce state, rather than simply validate it, would dramatically reduce complexity of the VM config.

We have several near-term changes incoming on the VM setup, so we should first push forward with the packaging logic in order to reduce the amount of Salt we're currently maintaining (by offloading the file-management logic into packages, for example). Once that work is done, we'll be left with a relatively small amount of Salt logic to port to Python.

[kushaldas]

@conorsch Finally!!! I was waiting for this ticket for some time now :) I would love to work on this.

[emkll]

This ticket was raised in the context of #324 (comment) where we have opted, for now, for a udev-rule-based approach in sys-usb and qubes-rpc grants to allow sys-usb to attach usb mass storage devices and printers to sd-export-usb

We have not opted for the Qubes Python API approach (for now), based on a discussion during the Nov 14th engineering meeting:

• Any code running in dom0 requires high degree of reliability and assurance. The udev solution in Support automatic attachment of export devices #324 simply requires RPC policies in dom0, attach logic is run in sys-usb.
• We are not yet using the Qubes Python API for tasks other than tests, and there would be a certain overhead in setting the automation and packaging in place.
• Because scripts in dom0 (e.g.: qvm-usb-attach) can override Qubes RPC policies, we would need to manage our own policies and enforce these. Since Qubes provides these policies for inter-vm communication, we have opted for this upstream-maintained authorization for now.
• Since the attachment to sd-export-usb mirror the attachment to sys-usb (the drives aren't auto mounted) , there is no incremental risk in doing this (sd-export-usb is grsecurity-hardened, disposable, and should generally last for a session at most).

Once we use Qubes Admin API for other tasks, it may be worth re-investigating the way devices are attached to sd-export-usb

[eloquence]

Discussed briefly in backlog review today with @creviera and @conorsch . For configuration management, Salt remains our best option. VM creation and preference management seem good candidate for refactoring to use the Python API in the near future.

[zenmonkeykstop]

Keeping this around, as we're reducing overall use of Salt we can take an incremental approach here.