Support external monitor

[osy]

iPad allows an external monitor to be used as a separate display.

[conath]

I have recently implemented external screen support in my app, Present Anything. I can make it happen for UTM app, but I have a few questions. The iOS external screen API (including on iOS 14) is UIKit based, you basically provide a UIWindow to the external screen to display. It is possible to change the resolution of the ext. screen (iOS defaults to the max supported by the device and screen).

1. Which View or ViewController renders the VM screen?
2. Is it possible to remove this view from the window and move it to the external UIWindow? (might need to find that out myself)
3. How is the touchscreen feature implemented? Like, is it the same view that does the VM rendering or another one? (that would be better)
4. Do you want to support using the external screen as an additional display for the VM? As a first step, I would implement that the VM runs at the native resolution of the external screen, in full screen, and I would try to get touch and cursor input from the iPad working with the render view moved to the other window. As a second step, extended screen could be considered. However note that it is not possible to move the iOS cursor to an external screen, some more work would be needed to capture the cursor (iOS 14+) or to redirect touches (iOS 13+).

[osy]

I see a couple of main steps mostly from reading https://www.spice-space.org/multiple-monitors.html which gives a high level view of how SPICE handles multiple monitors.

CSDisplayMetal is the interface between libspice-gtk and the Metal rendering. So here, we have to handle notifying SPICE server that a new display arrived with some dimension. This is also where the pixel framebuffer is copied to a Metal texture. That would have to be extended to handle multiple textures representing each display and/or a mapping of texture to displays if we support display mirroring. We also need some sort of syncing with CSInput which handles the cursor position.

UTMRenderer does the Metal rendering which renders both the display and cursor through delegate calls to CSDisplayMetal and CSInput. A MTKView uses UTMRenderer. VMDisplayMetalController hosts the MTKView. VMDisplayMetalController also does the touch inputs, keyboard handling, etc. So VMDisplayMetalController (or VMDisplayTerminalController) would be good candidates to move to UIWindow.

Without SPICE agent installed, the only thing we can do is mirror or move the display to the external display. For extending the display, the guest has to be aware of the SPICE protocol. I think in the case of tablet/pencil mode cursor, it is fine to not be able to input on the external display.

Also, for new files and features, I would prefer moving to Swift from this point on.

[conath]

I have added experimental iOS external screen support, with new files written in Swift. I need help with the external VMDisplayMetalViewController to get mirroring and (potentially) multiple displays working. See my pull request.

Additionally, currently the aspect ratio of the VM display becomes distorted when it moves back from external to internal screen.

Here'a a very short video demoing Windows XP on an external screen connected to my iPad Pro. 😄