Engine Display Server

One of the primary goals for Arcan is to be able to be used as a display server for building desktop, tablet and smart phone environments, particularly for dedicated settings that have special needs but not the resources or critical mass to take advantage of more established platforms. As such, there are some specific challenges and design decisions to address, that game engines, web browsers and other networked multimedia applications do not have to consider -- or at least not to the same degree.

This page intends to address such issues in a Q&A sort of manner. Recall that the previous role of the Window Manager, Compositor etc. are combined into the current running appl: it dictates policy, sharing, I/O routing, post-processing and so on. A non-authoritative connection or semi-trusted frameserver ('client') has a very restricted set of privileges. Some questions that are more involved get their own detailed description (accessibility, input, multi-display etc), refer to the main wiki page for links to pages that deal with those.

Q: Who is responsible for drawing decorations (border, titlebar, icons, ...)

Decorations should primarily be the responsibility of the appl, though the application can request subsegments for hints as to how border and titlebar should be drawn or define an active viewport on a segment that indicates the border area. It is the running appl that dictates what will actually be drawn and how. Even if a client would enforce borders and titlebar, it is rather trivial for the running appl to adjust texture coordinates to crop that away based on other measurable properties (edge detection, user-configuration + identification of underlying toolkit used etc).

Q: Who is responsible for mouse cursor state?

The running appl is responsible for determining which input events get forward to any connected client. As such, current cursor position can be forwarded. The client can submit events that hint that there is a native cursor for the current state and transfer that as a sub window. Alternatively, there is a cursor-hint event that takes a preset label (listed in the shmif_event.h) and let the running appl set the appropriate icon. It is however up to the appl if that cursor will actually be drawn.

Q: How is device/monitor hot-plugging managed?

Transparently to any connected client. For monitors, the only information provided to a client is as part of a DISPLAYHINT event that indicates what properties (dimensions, dpi, led orientation, ...) that are currently in use, which may not necessarily correspond to the entire truth, only what the appl wants the client to see. The appl dictates what is going to be redirected to which monitor and how.

Q: How is resizing behavior controlled?

Resizing is complicated in the sense that it can be a rather costly affair to reallocate and agree on buffer sizes etc. The principle here is that there is no direct correlation between the dimensions something is drawn and what the dimensions a current display segment has. The running appl may chose to hint to a client that it should internally use certain dimensions and would periodically do so during the worst case ('drag-resize').

Q: How are windows and subwindows controlled?

Any connected client gets a main segment to work with. A segment combines an input/output event queue, an audio buffer and a video buffer. It is possible for a client to request additional segments, but it can never be assumed that such requests will always be granted. Each subsegment also has a connected type that hints as to how the segment is intended to be used. This is primarily to cover for very specific data sources, e.g. a segment for odd fields and one for even fields in an interlaced video source, or a segment for the left eye and one for the right eye in stereoscopic applications.

Q: How is fullscreen options controlled?

Entirely on the appl side, 'fullscreen' is a privileged user/display server decision, not something that a client will be permitted to control.

Q: How is input device management controlled? (keyboard and/or mouse grabbing)

Entirely on the appl side; the appl decides which input events that gets injected into the event loop of a segment or subsegment. These can be entirely synthesized (both for testing and automation) and do not have to correlate to a physical device.

The basic flow of events is essentially to push as much 'signal processing' into the engine where the penalty for processing input samples is rather low, and increase the level of abstraction as it moves from engine -> appl -> client.

The actual origins and properties surrounding an input event are privileged information. As an extreme example, this can be used to hint eye tracking / focus information to a subsegment that that is responsible for a certain eye in a stereoscopic rendering scenario.

Q: How is data sharing controlled?

For shorter messages (e.g. selected text), the event loop will suffice and would be treated like any other input event. For larger data transfers, there are BCHUNK transfers that come with a file descriptor / handle to write to- or read from-, and related explicit state transfers that should also be used to communication and package complete serialization of client state. The running appl is always responsible for expressing the descriptor routing.

Q: How is legacy compatibility managed?

The full arcan-shmif interface is intended for Arcan dedicated applications, as it covers synchronization, audio, video, input and state- management/serialization. For exotic legacy cases where source is not available or development is explicitly tied to some graphics or input API, it is possible to use a specialized hijack library that is preloaded or injected to redirect relevant input. There is also WIP separate projects for a wayland compositor implementation, a QEmu backend, a SDL2 backend and an XOrg backend.

Q: How is power state managed?

The running appl can determine how and when a client is allowed to update in a fine-grained manner, like the rate at which input events are delivered, or switching synchronization strategies to cut down on display refreshes to conserve energy. It is furthermore an engine requirement/feature to be able to deallocate as much re-createable resources as possible, release locks on input/output device locks and suspend to an external application. This is intended for both virtual terminal- style switching, and for hibernate/suspend scenarios.

Q: Who is responsible for font management (drawing, hinting, ...)?

The client is free to use its own fonts for rasterization, though there are transfer mechanisms for providing fonts and fallback font chians, along with event hints that indicate desired hinting and desired 'readable-baseline' font size.

Q: Is the client permitted to grab or send manipulated input to other connected clients?

Not without further intervention. There are restricted means for IPC between connected clients, but none of them are direct; all of them require explicit instructions from the running appl. For specific applications such as screen grabbing etc. the running appl can decide to push an output segment to a connected client. The contents of this segment is entirely appl- defined.

Q: What about clipboards?

Clipboard is treated as two kinds of subsegments, and a client starts out with no available clipboard. It can request a subsegment of the apropriate type, and if provided, can use that to transfer short or multipart messages (selected text), longer binary state transfers or audio/video like any normal window would. For pasting operations, the running appl can inject a 'nulltarget' or a 'recordtarget' and send data to the client that way.

Q: Why have 'audio' in a 'display' server?

Because A/V are closely coupled experiences with many problem-space overlaps in terms of buffering, processing and routing, especially when it comes to normal tasks like sharing for conferencing, or synchronization in audio / video playback and gaming scenarios. The context sensitive information needed to tune gain, filtering, output devices etc. is already present. Creating a side-channel with additional IPC, management to uphold some illusion of separation, as is the current state is in many environments, seems highly irrational. Granted, there is a trade-off here and the large number of more professional use-cases that there already exist multiple great solutions for, simply means that we should provide mechanisms for handing over trusted access and control in such cases.

Q: What about post-processing transfers, e.g. color and gamma adjustments?

The default is for the running appl to decide on which gamma tables to alter or which postprocessing shaders to apply. There are provisions for a client to request LUT ramp in and/or out synchronization, so the script can explicitly launch a trusted third party application and synchronize color management with it.