Added power saving mode

[corentin-plouet]

What

When enabled, this mode reduces the frame rate (potentially down to zero) by waiting for inputs (i.e. blocking/sleeping).
Both ImGui and user code can require a minimum frame rate for individual components (identified by an ImGui ID) to cater for things like blinking cursor or animations. This is dynamic; i.e. a requirement can be set when starting an animation, and set to zero if/when it's finished.

Note that I've used a fairly broad name for this feature (power saving) because in the future we might want to do other things beyond just altering the frame rate.

Why

My use case is a regular desktop application (audio) that I want to use on a laptop without too much noise, heat and power usage. This application does not need to run at the display rate/60fps, even when showing things like real-time waveform and spectrum analyzers.

How

I went for the simplest and ideally the most reliable, and efficient, implementation I could think of.
The feature is enabled setting ImGuiConfigFlags_EnablePowerSavingMode in IO.ConfigFlags.
The binding code can call ImGui::GetEventWaitingTimeout() to know how long it should wait for events; possibly 0, in which case it can poll (or just pass a 0 timeout if supported by the implementation), or infinity (when the frame rate is zero) in which case it can call a non-timeout event waiting method.

As said above, user code can add frame rate requirements. I've used an ImPool structure to store those requirements, which allows fast lookup to identify which one is currently the highest one (if a cursor requires 6fps and an animation requires 30fps, naturally the effective minimum frame rate will be the highest, 30fps).
ImGui does not share this structure; it uses its own boolean flags (currently a single one, Blinking).

Integration

I have added support for blinking cursors in ImGui.
I have added the feature in the demo app. A good test is to enable the power saving mode and observe the FPS counter when looking at the animated widgets example.

I have added support for the SDL and Glfw (OpenGL 3) bindings, and the Allegro binding.

Testing / Known issues

I have done some manual testing on Linux, with the demo (SDL, Glfw and Allegro bindings) and my own app. It seems to work well.

One known issue is that the frame rate requirement for blinking cursor is not properly reset when the input box is not visible. That should be easy to fix; I thing I can use a flag. (FIXED)

Next steps

I'd like to have your input.

I plan to test on Windows and implement support in more bindings, and use a power profiler tool to get an idea of the potential power saving.

[corentin-plouet]

An issue I forgot to mention is that the frame rate requirement is a best-effort thing, and it tends to slightly undershoot because it does not take the actual frame calculation time into account (so typically if you set 30fps requirement in practice you might get around 27 or 28).

One solution is to be more clever and dynamically adjust the returned timeout, either by subtracting the average frame processing time, or by implementing a control algorithm to track a target frame rate.

[ocornut]

Hello Corentin,

Thanks for the detailed PR and idea.
Some quick thoughts:

I don’t think using an allocating pool is required. Widgets could simply request a certain timestamp they would want to be woken up, and all of that logic can be handled with a min(), a finite amount of fields and without allocations.

Generally speaking any user inputs should trigger 3 frames of update, the speed of which can be decided by the application. I generally feel like the library should provide info about the required timeout and let the application handle framerate (vs trying to decide of framerate for the app) but I haven’t had time to think about the feature more.

For the blinking cursor specifically I would like to implement something even more custom and faster, because blinking cursor are both common and one of the worse case scenario.
#2268 (comment)
However that may require some interaction with the user main loop to skip straight to rendering, so that could be an optional feature as well.

Also maybe read #2268 for other ideas.

[corentin-plouet]

Hi Omar,

thanks for the feedback and the link to #2268.

Your first point is a good idea, it would indeed make things simpler and in fact solve an issue I'm currently facing; having two distinct calls to set/reset the requirement is not great, because it's difficult to handle resetting the value when the widget is not rendered anymore!

So I could replace the method SetFrameRateRequirement with a method named for example WakeUpBefore(double timeout).

I agree that the application should decide the frame rate.

What I have in mind for my application (a tracker-inspired digital audio workstation; I'll post screenshots at some point when it's a bit more polished 😄) is a way to be idle by default but let certain widgets temporarily increase the frame rate: when playing a song I need some auto-scrolling of the piano roll, at maybe 5fps, and when a real-time signal visualizer (waveform or spectrum) is visible, go to to 30fps (on battery) or 60fps+ on mains. Obviously this is application logic, but with the above call, the spectrum analyzer widget could say something like:

ImGui::WakeUpBefore(1 / 30.0);

and maybe the piano roll would say:

ImGui::WakeUpBefore(1 / 5.0);

and obviously the actual timeout picked would end up being the shortest one.

I will try that first (also implementing your suggestion to always render 3 frames) and see how it goes.

While I do that, I will think about your cursor fast path idea. The binding can tell ImGui whether or not it's waking up due to an event or because it's a timeout. And ImGui knows whether or not the application requested a timeout (or it's just the cursor). With those two pieces of information it knows whether or not it should enter the fast path or the normal path.

SDL_Event event;
bool got_event = 1 == SDL_WaitEventTimeout(&event, 1000.0 * ImGui::GetEventWaitingTimeout());
if (got_event)
{
    do
    {
       // Event stuff.
    }
    while (SDL_PollEvent(&event) == 1);
}
else
    // New call to let ImGui know there was timeout.
    // ImGui will know whether or not the app required a timeout
    // (and thus know if it's the fast path).
    is_imgui_fast_path = ImGui::NotifyEventTimeout();

if (is_imgui_fast_path)
{
    // Some clever stuff happens here.
}
else
{
    // Business as usual.
    
    // Start the Dear ImGui frame
    ImGui_ImplOpenGL3_NewFrame();
    ImGui_ImplSDL2_NewFrame(window);
    ImGui::NewFrame();

    ...
}

[corentin-plouet]

So, I've re-designed it following the line you suggested:

• It does not use a data structure at all; we just keep track of the MaxTimeBeforeNewFrame
• It always renders at least 3 frames after an event

I'm quite happy with the way it behaves, now.

For the cursor, aiming at 6fps, you actually end up getting something under 18 fps, because you actually get six 3-frame salvos, due to the aforementioned logic: tac-tac-tac, sleep, tac-tac-tac, sleep etc.

I have implemented and tested:

• Win32 (DirectX 11)
• Allegro, tested on Linux
• SDL / OpenGL 3, tested on Linux
• Glfw / OpenGL 3, tested on Linux

I have also done some actual power measurement.

I've used a pretty cool tool called s-tui (can be installed using pip).

This was done with nothing else running. I started the demo app, let it run for a bit in normal mode (i.e. at 60fps), then enabled the power saving mode, let it run for a bit, then clicked on a Text Input with a blinking cursor, and finally enabled the animated plots demo:

It's not a super methodical experiment but it gives an indication. Unfortunately it's only CPU power, so I can't see the overall actual impact on the entire system, including GPU (which obviously is what matters for battery life).

Next steps

1. Have a look at the macOS implementation.
2. Start thinking about the fast path for the blinking cursor
3. Refactor the event waiting code, possibly extracting it into each platform file instead of being into a platform+renderer file (because it's slightly more complex now, and it's independent from the renderer)

I will do step 3 after the other 2, because the fast path logic will constrain how the event logic works and integrates.

Call for testing

It's now in a decent state where other people can start having a look, see if they find the API convenient, see if it works for their binding/platform, etc.

[corentin-plouet]

Some more work...

A bit of cleaning and refactoring, in particular I've extracted the waiting logic into the common SDL implementation file, to keep the platform+renderer binding simple and not deviating too much from existing. I will do the same thing for the other examples.

I've also fixed the cursor logic to wait for the appropriate time to flip instead of trying to animate it as 6fps.

I repeated my not-so-scientific test from above and this time I can't see any difference at all. But, again, that's just CPU power. I might try measuring at the socket on my desktop computer to see what difference GPU power makes.

Then we can see if the more sophisticated logic to only blit the cursor is worth the effort/complexity.

Otherwise I had a look at the Emscripten example, and it won't be as simple as the other ones (or even possible?) due to the different in event model.

[corentin-plouet]

Finished refactoring all examples (i.e. Win32, SDL, Glfw and Allegro).

And now we only have 3+ frames of update after user inputs, not on things like cursor blinks (which means the cursor blink is now effectively 1.7fps).

Known issue(s):

• Need to a mouse move callback to the Glfw example.

[corentin-plouet]

So, I'm happy with how the patch looks at this stage.
It brings zero frame rate support when there are no input events, and when the window is known to be hidden (e.g. minimized) for the main platform bindings. There is low fps (< 2) when the cursor is blinking and the ability for the user to easily control the behavior on a case by case basis (e.g. when playing animations).
The change is not intrusive.

I've done some basic testing on:

• macOS (SDL, Glfw)
• Linux Ubuntu (SDL, Glfw, Allegro)
• Windows (SDL, Win32)

Note that I did not test gamepad/joystick input; just mouse and keyboard.

I've addressed all the issues I was aware of.

So I think it's good for review / testing / merging.

Regarding the rendering optimization you suggested, I'm not convinced it's super critical at this stage, and in any case can be done in a separate PR, and perhaps in a more generic way (rather than just for the cursor).

[ClaudiuHKS]

This is awesome. For example, if the user has no input activity at all, the FPS must significantly decrease. Of course, this should never happen when vSync is turned on.

[bsviglo]

@corentin-plouet, looks cool!
I have small question related to your implementation. Is it possible to enable power saving mode per window basis?
We're trying to implement game editor and one of the window is actually viewport which may and should be updated every frame while other windows like properties outliner, world outliner, asset browser and etc. may be updated much less often and even on user input or cursor blinking event. By doing this way we may update only what need to be updated with required framerate.

[corentin-plouet]

Thanks for the feedback!
@ClaudiuHKS can you clarify what you mean by "this should never happen when vSync is turned on"? Right now, you will still get reduced (or zero) FPS even when Vsync is enabled. I think that's the way it should be, Vsync is just a way to avoid tearing. Or am I missing something?

@bsviglo that's a good question indeed. This PR is on the master branch, not the viewport branch, but I need to start messing with that branch soon anyway, so I will take that into consideration. On top of my head, there is nothing preventing from doing it on a per-window basis. But as always, the devil is in the details.

[corentin-plouet]

Rebased on 1.73

[BitmaniaK69]

Any news on this thread?

[corentin-plouet]

Rebased on master.

[ocornut]

Thanks Corentin for updating this, i'm not sure when I'll be able to look at this thoroughly but I feel it's looking quite good.

As for @bsviglo suggestion, there are two things we could be later be investigating:

The obvious general "make this multi-viewport" aware.
Things like animation requests (blinking cursors) could be affecting a single viewport.
Inputs however are more delicate, I'm not sure it's safe to be considering that input on viewport A shouldn't be affecting viewport B, so we may ditch that idea of provide some viewport-specific flags.

The typical use case is as bsviglo mentioned: a running game rendered somewhere may want to request its host viewport to render at maximum rate, while other windows may be refreshed differently. The tricky thing being that imgui back-ends will end up seeing the inputs going to the game and this may erroneously wake up the imgui parts.. so this need to be solved somehow.

It's going to be difficult to make this PR in two steps (master branch then viewports). If you attempt to experiment with multi-viewports version of it, may I suggest you open a separate PR for your experiments, keep this one here as reference. Based on its contents, when we are happy with the result, we can attempt to split the non-viewport code and merge first into master then merge the rest in docking branch. Or maybe by the time I look at this PR seriously, I'll be able to provide a solution for multi-viewports.

(
Another idea mentioned elsewhere, which could be implemented unrelated to this PR, is the idea of providing refresh rate policy for imgui windows. Imagine if we have settings where e.g. Begin() would return true only 1 frame out of N unless the window is focused, and imgui figures out load-balancing so multiple low-frequency refresh windows gets their CPU-side work spread over distinct frames. Now, there are many reason I don't really like so much: per-window shearing, uneven per-frame performance, etc. but it could be useful in highly stressed systems. Anyway, the relation to this PR is the following: if we end up supporting multi-viewports on this PR, then this other idea could also be a source for flagging a viewport as "doesn't need refresh/swapping".
)

[JLMorange]

@corentin-plouet I don't understand the ImGui::WakeUpBefore API, shouldn't it be ImGui::WakeUpAt? what we want is not that the widget is refreshed Before a time out expire but at dedicated time... for example a video widget 25Fps should ask to be refreshed "At" 1/25s past the last refresh, it won't care if it is refreshed "Before" the timeout expire... or ImGui::RemainingTimeUntilRefresh()

[corentin-plouet]

Hi @JLMorange,

first, please note that the code has been through a few iterations, and my comments above reflect the entire development history (the code history has been "squashed" by git force-push though).

In fact, the API you describe is now named SetMaxWaitBeforeNextFrame, which is a longer name but more accurate. Note that it's not guaranteed scheduling: if you call it with a value of 1/25, you may end up waking up before that, simply because the user interacted (moved the mouse, etc.) So it's a "max wait" indeed.

@ocornut, I'll have a look at applying this patch to the multi-viewport / docking branch. I think the simplest and most robust solution is to have any event, on any viewport, refresh all of them. In terms of power saving, the idea is to get the CPU and GPU to be idle/sleeping as often as possible, but when they need to wake up I'm not sure we save much by having viewport-specific behavior. But I'm open to any sensible behavior; first I need to get myself familiar with that branch...

[hanatos]

i really like this! i've used something similar / less sophisticated in the past. i think our code has the same issue:

when using the gamepad nav system, how do you make sure you're staying responsive enough? glfw does not generate joystick events, so it won't wake up glfwWaitEvents*(). this is probably pending something like this: glfw/glfw#1590

does this mean with gamepads we're doomed to do power intense polling?

[ocornut]

We can still poll events at some rateand do nothing else if there aren’t gamepads changes.

[hanatos]

hmm. right, i suppose a background thread calling glfwPostEmptyEvent() after polling the joystick will do. not pretty, but should be functional.

[hanatos]

this works, thanks for the idea. for reference, here's my polling thread: https://github.com/hanatos/vkdt/blob/master/src/gui/main.c#L71

and further down the same file is the glfw main loop: https://github.com/hanatos/vkdt/blob/master/src/gui/main.c#L257

[ZaWarudo111]

Blinking cursor is not showing with ImGuiColorTextEdit, any ideas how to fix that?

TextEditor.Render() is called in the render loop

#include "TextEditor.h"

ImGui::Begin("Some window");
static TextEditor editor;
editor.Render();
ImGui::End();

and it doesnt use InputTextMultiline() but it uses DrawList to render text as i believe

[luxapana]

Hi, Any plans to merge this to master? I am also evaluating ImGui for a regular desktop app and the CPU consumption is the only issue I am facing.