Screen API refactoring

[maxkatz6]

Is your feature request related to a problem? Please describe.

Current Screens API was designed and implemented very long time ago, when Avalonia supported way less platforms.

Main problems that needs to be solved:

• API is only available on Desktop. While both Mobile and Browser provide similar capabilities.
• There is no way to compare screens. Users have to write ugly hacks comparing screen sizes. It's also a problem for XPF.
• After retrieving Screen instance, it won't reflect any changes on the actual display. Developer needs to re-retrieve this Screen to get latest information.
• No way to subscribe on screen changes.
• No screen orientation, display name, unique handle available.

Browser related nuance:
Browser returns only single active display, where web browser is located. It also doesn't include display name info by default.
If developer want to retrieve extended information, like other screens and additional info, they need to run async Window.getScreenDetails() API. At least once. In Avalonia, it can be covered by RequestScreenDetails-like API.

Describe the solution you'd like

public class WindowBase : TopLevel
{
-    public Screens Screens { get; }
+    public new Screens Screens => base.Screens!;
}
public class TopLevel
{
+    public Screens? Screens { get; }
}

public class Screen
{
+    public string? DisplayName { get; }
+    public ScreenOrientation CurrentOrientation { get; }

+    public IPlatformHandle? TryGetPlatformHandle();
}

public class Screens
{
+    public event EventHandler? Changed;
+    public Screen? ScreenFromTopLevel(TopLevel topLevel);
+    Task<bool> RequestScreenDetails();
}

+public enum ScreenOrientation
+{
+    None, // Unspecified, Unknown ?
+    Landscape,
+    Portrait,
+    LandscapeFlipped,
+    PortraitFlipped
+}

Describe alternatives you've considered

No response

Additional context

Child issues:

• a way to get native Screen handle & rotation info #15382
• Screenname #11512
• Add an event to detect the screen changes #15637

[maxkatz6]

Media query API can re-use the same new Screen API as well.

[maxkatz6]

cc @emmauss @kekekeks

[kekekeks]

How would we identify a particular screen? By connector id?

[maxkatz6]

@kekekeks does it work as an answer? #15382 (comment)

[thevortexcloud]

I think a user readable identifier would also be useful, even if it's only implemented on some platforms. EG if you can read EDID (should be possible on all desktops) you can get the manufacturer name, model number and serial number of the display. EDID however is not always the most reliable thing (EG it's very easy to spoof on Linux), but it makes it much easier to show/handle monitors with the same model number/display name. Even if the ports they are connected to changes.

But otherwise this change looks pretty good. There is probably also some other things that might be useful (EG colour depth, DPI, supported resolutions, etc), but they are fairly minor. SDL's display APIs can probably be used for some inspiration as SDL supports pretty much every platform ever created.

https://wiki.libsdl.org/wiki/index

[anose001]

@maxkatz6 , @kekekeks , @simonhaines, @thevortexcloud
I came to this thread while looking for responsive Avalonia app resources #15991 with the goal to clear up on how to properly design a UI for a range of devices using Avalonia. I found the Avalonia Materials library but no clear guidelines on how to properly approach adaptive or responsive design other than this rudimentary example, as UI/UX across different devices is not something I have done before and I am fairly new to Avalonia.

The mobile community has thought about this problem for a little while longer it looks like.
For example, the Android developer guide for different screen sizes suggests to "avoid using physical, hardware values for making layout decisions" (link) and instead use the actual portion of the screen that is covered by the app. They argue to think about this in terms of window size classes which they define as opinionated viewport breakpoints classifying the UI layout into size categories such as compact, medium, large, etc. They argue a little bit more on why a physical devices class should not be used to design your window layout (unless you develop for one specific device size). A very helpful read, overall touching a good amount pain points that developers face down the road.

Their guideline on window size classes is in agreement with the guideline from the Materials Design Foundation on how to design across window classes. Guidelines and how to's like these are something that I am missing in Avalonia's official documentation as well as the communities Avalonia Materials library and I believe should rather be part of a UI framework library.

It seems like this feature request is moving along these lines if I am not mistaken?
My questions for the architects of the library then:

• Is adaptive design with different window classes the proper way to use Avalonia capabilities to design for a variety of devices and app window sizes?
• Is there a proper approach on how to rearrange a navigation bar (or even change the type of navigation control) if the window size changes from let's say compact to large? I would rather try to avoid multiple views for something like this.
• Last but not least, will this Screen API return the window size in term of dp's, density-independent pixels, as this is the only platform-agnostic and device-agnostic measure that I have found to make the sense for designing layouts of different sizes. Or will you focus on returning a recommended window size class instead?

Since the term dp is not uniform across platforms I add this overview, as px are used for Web, pts for iOS, and dps for Android.

Also, this is how Android developers retrieve the window size class val windowSizeClass = currentWindowAdaptiveInfo().windowSizeClass as described here.

Apple has a similar approach via their UserInterfaceSizeClass, based on adaptive design principles.

• When developing for iOS and Android are these the approaches for window sizes recommended by Avalonia as well?

[thevortexcloud]

Is there a proper approach on how to rearrange a navigation bar (or even change the type of navigation control) if the window size changes from let's say compact to large? I would rather try to avoid multiple views for something like this.

You can do this already with a converter/markup extension and binding to the window width. Although you will have to define what counts as large/small yourself.

Last but not least, will this Screen API return the window size in term of dp's, density-independent pixels, as this is the only platform-agnostic and device-agnostic measure that I have found to make the sense for designing layouts of different sizes. Or will you focus on returning a recommended window size class instead?

Avalonia already has device independent pixels. EG the PixelSize struct contains many methods for calculating them.

Avalonia/src/Avalonia.Base/PixelSize.cs

Lines 154 to 184 in b1e96ad

	/// <summary>
	/// Converts the <see cref="PixelSize"/> to a device-independent <see cref="Size"/> using the
	/// specified scaling factor.
	/// </summary>
	/// <param name="scale">The scaling factor.</param>
	/// <returns>The device-independent size.</returns>
	public Size ToSize(double scale) => new Size(Width / scale, Height / scale);
	/// <summary>
	/// Converts the <see cref="PixelSize"/> to a device-independent <see cref="Size"/> using the
	/// specified scaling factor.
	/// </summary>
	/// <param name="scale">The scaling factor.</param>
	/// <returns>The device-independent size.</returns>
	public Size ToSize(Vector scale) => new Size(Width / scale.X, Height / scale.Y);
	/// <summary>
	/// Converts the <see cref="PixelSize"/> to a device-independent <see cref="Size"/> using the
	/// specified dots per inch (DPI).
	/// </summary>
	/// <param name="dpi">The dots per inch.</param>
	/// <returns>The device-independent size.</returns>
	public Size ToSizeWithDpi(double dpi) => ToSize(dpi / 96);
	/// <summary>
	/// Converts the <see cref="PixelSize"/> to a device-independent <see cref="Size"/> using the
	/// specified dots per inch (DPI).
	/// </summary>
	/// <param name="dpi">The dots per inch.</param>
	/// <returns>The device-independent size.</returns>
	public Size ToSizeWithDpi(Vector dpi) => ToSize(new Vector(dpi.X / 96, dpi.Y / 96));

Many things already use them.

[maxkatz6]

Completed in 11.2 nightly. Mobile support is coming later.