Hi DPI support

[ytsarko]

Hi!

In this thread on YCombinator 2 weeks ago you mentioned that the library supports defining coordinates both in pixels and in "fractions of window". Could you please elaborate more on the latter? You also mentioned that this functionality is handled in one function inside the library. What is the name of that function?
I would like to investigate more on the high DPI support in this library (which itself is very good, thank you!) and would like to know a starting point.

[dumblob]

Before jumping to detail, please see #74 (comment) .

[ytsarko]

After reading linked comments I was only confirmed my mind that current state of nuklear does not support automatic UI scaling for different DPIs simply because it works in pixels.
So for the time being if user wants to make her UI scalable one has to do it by hand in client(user) code.

For example:
nk_stroke_rect(..., nk_rect(0, 0, 100, 100), ...); // always draws 100x100 rectangle regardless of current DPI

If user wants scalability one must do something like:
nk_stroke_rect(..., nk_rect(0, 0, 100 * scaling_factor_x, 100 * scaling_factor_y), ...); - where scaling_factor_x(y) is some scaling factor obtained from OS/window system. For example for Windows DPI set to 120 points scaling_factor_x==1.25 and the rectangle will be 125x125 pixels on such screen.

[malv-c]

without opengl* pixels are fine

[vurtun]

@ytsarko OK could you please elaborate what you need. Because what I personally read out of your description is to have the correct scaling between normal and High DPI screen.
If so it already exists in this library (at least for all SDL2/OpenGL and GLFW/OpenGL examples out of the box and the others can be easily converted). In these cases rendering output is already getting scaled by OpenGL (glViewport) to fit the screen and it would make no sense modifying the actual library core for this case.

[ytsarko]

@vurtun > OK could you please elaborate what you need.

Basically I would like to author UI code not in hard-coded pixels but in some "units" which scale appropriately for current DPI. So the mentioned early call
nk_stroke_rect(..., nk_rect(0, 0, 100, 100), ...);
would be just fine if depending on DPI my rect will be drawn 125x125, 150x150, 200x200 and so on. This would be the best since all the internal "units"->pixels conversion is done inside the toolkit.

As to

If so it already exists in this library (at least for all SDL2/OpenGL and GLFW/OpenGL examples out of the box and the others can be easily converted). In these cases rendering output is already getting scaled by OpenGL (glViewport) to fit the screen and it would make no sense modifying the actual library core for this case.

Could you please elaborate more? I have made a quick test (based on SDL nuklear demo) and following code:

float ddpi;
float hdpi;
float vdpi;
int ret = SDL_GetDisplayDPI(0, &ddpi, &hdpi, &vdpi);
if (ret != 0)
{
SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
"failed to get DPI for display: %s\n", SDL_GetError());
return 1;
}
printf("DPI: %0.2f, %0.2f, %0.2f\n", ddpi, hdpi, vdpi);

nk_stroke_rect(canvas, nk_rect(20, 60, 100, 20),
0.0f, 0.0f, nk_rgb(255, 255, 0));

produces yellow rectangle 100x20 pixels on my current machine:

though 120x24 is expected.

$ inxi -F
System: Kernel: 4.4.1-040401-generic x86_64 (64 bit) Desktop: Cinnamon 2.8.8
Distro: Linux Mint 17.3 Rosa
Graphics: Card: Advanced Micro Devices [AMD/ATI] Madison [Mobility Radeon HD 5650/5750 / 6530M/6550M]
Display Server: X.Org 1.15.1 drivers: ati,radeon (unloaded: fbdev,vesa) Resolution: 1920x1080@60.0hz
GLX Renderer: Gallium 0.4 on AMD REDWOOD GLX Version: 3.0 Mesa 10.1.3

[vurtun]

Correct me if I am talking about something else but as far as I know I already solved this problem in my OpenGL backends. I use SDL2 to get the framebuffer size and scale the UI to fit by calling glViewport with framebuffer width and height instead of window width and height which should render the UI with correct scaling. So if you would create a rectangle with 100x100 units and your window width and height are the same as the framebuffer width and height the 100x100 is in pixel if they are different then the UI and therefore the rectangle gets scaled. But all of this is done inside the OpenGL backends not inside the library since it is easier to do in the backend.

[ytsarko]

@vurtun As far as I understand you - setting glViewport size equal to SDL window width/height (from what I am observing in demo sources) just means that every pixel in GL viewport maps 1:1 to on-screen pixels of window. I played with it and it seems that setting glViewport to whatever values do not affect the rendered image. The 100x20 rectangle is 100x20 rectangle and nothing else. May be it has to do something with matrix transforms used in demo or some other factor (I am not a GL expert) but this is definitely not the thing I was talking about in my previous comment. I suppose if it will work as you describe then the resulting image will be scaled but blurred.

What I was talking about is described more precisely here (start reading from Direct2D and DPI paragraph):

For example, if the user's DPI setting is 144 DPI, and you ask Direct2D to draw a 200 × 100 rectangle, the rectangle will be 300 × 150 physical pixels.

This is the most convenient part - you express your UI in some "units" and tookit automatically draws shapes of appropriate size in pixels. Doing so your UI will be correctly scaled both on 4K display and on a regular 200$ 13" laptop.

[ytsarko]

This whole discussion has started with the question about some function inside nuklear (from here)

... other one uses a fractions of the window...

I was asking merely the name of this function because at this moment I am not that familiar with the implementation of the library and have not seen anything related to the function in question.

As to the testing and all - I would like to provide any help in testing the library but I myself do not have really high resolution screens (my own laptop has 1920x1080 18" screen with just 120 PPI).

[vurtun]

OK if this: https://gist.github.com/vurtun/d4ff0f8d866d07ee04b919ca49f3e55c does not help I don't care anymore and you have to life with blurry UI. Just copy & paste code into nuklear.h and set ctx->scale.x and ctx->scale.y to your scale. It could be that mouse handling does not work correctly and you have to scale mouse input for mouse motion and button events.

[ytsarko]

@vurtun I have looked at this new version of library and tried it out. Looks very good - I have checked both on Windows 10 with D3D11 and Linux with OpenGL2. Setting ctx->scaling makes everything scaled as expected.

Here are the screenshots:

Win10 - unscaled (scale 1.0):

Win10 - scaled (scale 1.25):

Linux - unscaled (scale 1.0):

Linux - unscaled (scale 1.5):

Thank you for an updated version and quick fix.

Are you going to commit these changes to mainstream?

[dumblob]

@ytsarko let me warn you a bit before using linear vector transformation scaling. This is only useful for a very specific use-case (nowadays this use-case also disappeared from embedded devices, because even there developers try hard to reuse their code for different devices having different display sizes). This very specific not-to-be-seen use-case is UI for displays of absolutely the same physical size (both physical width and height must match), but with different pixel density (horizontally or vertically or both directions).

Where this primitive linear scaling method completely fails is everywhere else (99.9 % cases), when also the physical size of the display changes. Therefore I would definitely not recommend using linear scaling, but rather specifying scalable UI in a higher abstraction - e.g. using Cassowary constraints. This higher abstraction then internally uses pixels, so there is no need for linear scaling.

[ytsarko]

@dumblob So if I understood you right you're suggesting using one of Cassowary constraints implementation to express the UI in terms of a set of constraints (for example let this widget be tied to the viewport size, that widget will be half the size of the first one and so on) and let it care about different window sizes/pixel densities? If so then this approach is a decent one, but is very high-level. And it requires you to stick with 3rdparty library for Cassowary solver (this could be pros/cons depending on your circumstances).

The beauty of nuklear is in its compactness and performance and that it doesn't depend on 3rdparty toolkits.

This primitive linear scaling method (where every element is multiplied by some scaling factor) is still good enough for majority of applications (at least desktop ones), very fast and provides quite decent results without much hurdles. Again, everything depends on your particular case (app requirements, target audience, devices and so on) and in more complex scenarios it may be better to use Cassowary or something similar.

P.S.: Btw, I haven't noticed any pure C implementations of Cassowary, only C++/Java/etc. May be you aware of any?

[dumblob]

Sorry for such a late reply - GitHub stopped sending me notifications because of bouncing (now fixed on their side) and I totally forgot to answer in this thread.

@dumblob So if I understood you right you're suggesting using one of Cassowary constraints implementation to express the UI in terms of a set of constraints (for example let this widget be tied to the viewport size, that widget will be half the size of the first one and so on) and let it care about different window sizes/pixel densities?

Correct. Specifically Cassowary is used in all genuine Apple UIs on desktops, tablets and smartphones to guarantee correct layout independently from the physical screen size and independently from resolution. There is one important thing, which most UI developers unfortunately don't realize. Namely, that the UIs they're creating will be used in the future and thus on future devices with absolutely unknown screen sizes, unknown screen ratios, and unknown resolutions. And there is only one way how to guarantee, that the UI will look as expected even on those - to use technology independent from physical screen size and independent from the screen resolution.

The beauty of nuklear is in its compactness and performance and that it doesn't depend on 3rdparty toolkits.

Yes, that's why we love Nuklear. Like font handling in Nuklear, UI objects placement and sizing can be modular.

This primitive linear scaling method (where every element is multiplied by some scaling factor) is still good enough for majority of applications (at least desktop one), very fast and provides quite decent results without much hurdles. Again, everything depends on your particular case (app, requirements, target audience, devices and so on) and in more complex scenarios it may be better to use Cassowary or something similar.

Based on my experience and experience of many UI development teams from conferences, sessions, etc. (dealing with Linux/BSD X11, Wayland, Windows, Android, Mac OS X, iOS, etc.), the major hurdle is visual incompatibility across devices caused by technological limits (not by bad UI designers). These limits are though non-existent when one starts to think outside the box and not in terms of the web plague and historical HW constraints (i.e. all the dozens of shitty linear units with overlapping functionality). It seems Apple identified it very well, that development and maintenance of a "perfect" (in terms of compliance with their HIGs) GUI across their only few products (compared to other huge IT companies) costs them a lot of resources. So they searched and found something significantly better - Cassowary.

Btw the statement "good enough for majority of desktop applications" does not hold at least nearly for 2 decades (I remember my rage already with applications on Windows 98 where some of them were for 800x600@14" and the others for 1024x768@15"). Not speaking about the fact, that Nuklear is used totally randomly for any environment (from big screens through desktop screens down to tablets, smartphones and wearables like watches).

Don't get me wrong, I'm not saying Cassowary is the only solution to the demonstrated issue. But it seems by far the simpliest and least-costly one.

P.S.: Btw, I haven't noticed any pure C implementations of Cassowary, only C++/Java/etc. May be you aware of any?

You hit the nail on the head! There are few - Amoeba, Emeus, Cassoway lite (for other implementations please refer to http://overconstrained.io/ ). I would highly recommend Amoeba as it's a single-header file in C89 with less than 1000 SLOC (as a bonus there are separate Lua bindings with a nice constraints syntax).

@vurtun, could you please add to documentation and code a huge warning with pointers to why is linear scaling a wrong idea (possibly with links to this thread and http://overconstrained.io/ )?

Also @vurtun, would you consider adding support for Amoeba for sizing and positioning in a similarly modular manner as are currently supported fonts?

[teamblubee]

what happened to the scaling? The link here is broken:
OK if this: https://gist.github.com/vurtun/d4ff0f8d866d07ee04b919ca49f3e55c does not help I don't care anymore and you have to life with blurry UI. Just copy & paste code into nuklear.h and set ctx->scale.x and ctx->scale.y to your scale. It could be that mouse handling does not work correctly and you have to scale mouse input for mouse motion and button events.

Does this library still support HiDPI screens?

[dumblob]

Does this library still support HiDPI screens?

Please read the whole thread #283 .