Keyboard navigation feedback

[jadwallis]

Version/Branch of Dear ImGui: v1.63 WIP (master d69b2a1, 1st August)
Back-end file/Renderer/OS: Custom

After #1867 a few months ago I said I'd have some feedback on the navigation system once I'd investigated a few snags... and then I got side-tracked with other things for a while. Sorry about that! Still, better late than never, here are some thoughts.

As a quick bit of background, we first started using the navigation branch when it supported gamepad navigation but not (explicitly) keyboard navigation. We implemented keyboard support by pretending it was a gamepad (with a bit of fiddling so that the arrow keys would control navigation, switching window, window movement, or window resizing depending on what combination of Ctrl and / or Shift was held down), and put the whole thing behind a keyboard toggle (bound to F2) that turned navigation inputs on or off. It wasn't the most intuitive set of controls in the world, but it was enough for the cases where we couldn't support a mouse.

Anyway, since then we've upgraded to the latest version of ImGui, and I've been having a play with the default keyboard navigation controls that now exist. Some of our team have expressed a preference for being able to use keyboard controls in preference to mouse on PC (or at least to be less mouse reliant), and it seems like this is better supported now? Having deleted our old hacky implementation and switched keyboard navigation on, it broadly works, but there are a few issues that prevent us adopting it right away. I'm not sure how many of these would be issues for all users of ImGui - these are all from the perspective of our use case, but seem worth raising even if there turn out to be good reasons not to change them.

So, in no particular order:

• ImGui always wants to capture the keyboard when navigation mode is on. This makes some sense, but it feels like it would be nice to be able to use the keyboard controls without needing to have them behind a toggle on-off, while still being able to use the other keyboard keys to control the application underneath. Since keyboard navigation only cares about a handful of keys (arrow keys, space, enter, etc), it would be nice if I could still send the rest of the keyboard inputs (letters, numbers, etc) through to the application when they're not needed by ImGui. Perhaps some kind of separation of 'groups' or 'contexts' of keys that ImGui wants to capture could work around this? The broader question of separating inputs between ImGui and the underlying application is a tricky one without a good answer - we currently use a global toggle on-off for all of ImGui, but having a second global toggle for keyboard navigation is a bit awkward.

• The default key for 'activation' is Space. Sometimes that seems expected, but often I find that Enter / Return would make more sense. Would it be possible for those keys to also work for activation? I suppose I could just change our backend to make ImGuiKey_Space also be triggered by the Enter / Return keys, but that feels like a bit of a hack that could potentially backfire further down the line.

• The default key for toggling between a window's content and its titlebar is Alt. However, on Windows (the OS) the Alt key also selects the application window's context menu. Would it be possible to switch it out for a different key such as Ctrl; either by default or by giving us the option to rebind it? Alternatively, Stack Overflow suggests doing something like this to disable this behaviour of Windows: https://stackoverflow.com/questions/9627379/how-to-disable-normal-behaviour-of-alt-key but I'd rather avoid that if possible.

[jadwallis]

Apologies, please disregard the first bullet point! I somehow completed overlooked the existence of ImGuiConfigFlags_NavNoCaptureKeyboard, which is exactly what I want here.

The other two points still stand, but I guess really just boil down to 'can we have more options for / change the defaults for / be able to rebind these keys'.

[ocornut]

Thank you for your message @jadwallis.

This keyboard capturing behavior is relatively new, introduced in February 2018 by commit 92ee6b1
* Nav: Sets io.WantCaptureKeyboard when navigation is active. This is a little agressive but probably the best default and also a good way to get feedback. Added ImGuiNavFlags_NoCaptureKeyboard to disable this behavior. )

As hinted by the commit message, I made this semi-agressive change with the hope of getting feedback on how to best forward forwards. I would be open to change and improve it. The older behavior, which is equivalent to now setting ImGuiConfigFlags_NavNoCaptureKeyboard makes it a little tricky to decide how/where to forward keys. When I made this change my intuition is that would give us a entirely trouble-free default settings, but I'm also more interested in making the original behavior easier to work with with variety of usage scenario.

If you have any feedback of how to best tackle this and how/if imgui could provide additional information to aid the application with making better decision I'd be happy to hear them.

At the very least we could allow the user to enable/disable capturing because via the PushItemFlags/ImGuiItemFlags (which are currently private but meant to be exposed eventually).

The default key for 'activation' is Space. Sometimes that seems expected, but often I find that Enter / Return would make more sense. Would it be possible for those keys to also work for activation?

There's currently a subtlety where Space on a Drag/Slider would just directional key edit, whereas Enter would use Text Input. On a game-pad the former is more desirable. On a keyboard I feel like both are useful So I could make Enter also activate tree nodes, buttons, etc. but Space would still have an occasional use.

The default key for toggling between a window's content and its titlebar is Alt. However, on Windows (the OS) the Alt key also selects the application window's context menu. Would it be possible to switch it out for a different key such as Ctrl; either by default or by giving us the option to rebind it? Alternatively, Stack Overflow suggests doing something like this to disable this behaviour of Windows: https://stackoverflow.com/questions/9627379/how-to-disable-normal-behaviour-of-alt-key but I'd rather avoid that if possible.

This is a fairly common/known trick, why would you prefer to avoid it? Considering Alt is the key to access all menus under Windows unless you are also using a native WIn32 menu it seems more valuable to bind this key to in-app menus.

[jadwallis]

Thanks for your response. :)

I've been playing around with our backend a bit more, and what I'm currently doing is using ImGuiConfigFlags_NavNoCaptureKeyboard so that WantCaptureKeyboard is only set when ImGui wants to capture the whole keyboard, while NavActive is set when navigation mode is on. I'm then able to tell whether ImGui potentially wants to capture any keyboard key, or just those keyboard keys that are used for navigation (e.g. Ctrl, Alt, Tab, Arrow keys, etc). So the application is able to receive the keys that keyboard navigation never uses, which is 'good enough' for our purposes (at least for the moment).

I've been having a think about what extra information ImGui could provide, though unfortunately I don't have any good suggestions. This is partly impacted by my realisation we're not strictly speaking feeding our inputs to ImGui in the intended way, due to a limitation of the way our input system decides which system should receive which inputs each frame; we can't send all inputs to ImGui every frame as that prevents them being used elsewhere, so have to gate it on whether ImGui wants to use them that frame. That's entirely on us, but probably means any solution that works for us probably isn't going to be that applicable to anyone else. :/ I considered suggesting that ImGui could provide a more fine-grained indication of which inputs it intends to consume, or has actually consumed, each frame - e.g. a WantsCaptureKeys bool array that indicates which ImGuiKeys ImGui actually cares about for the next frame. However, I don't think this is necessarily a good idea: a) it would be a lot of work to get ImGui to produce that data and b) it's probably not possible to actually get this data in a form that the backend can meaningfully use, since ImGuiIO's output fields are filled in after the call to NewFrame(), at which point it's almost certainly too late for the backend's input system to do anything with the information.

The more I think about it, the more convinced I am that whichever way you approach it, there isn't going to be a nice, general solution to sharing input between ImGui and applications because at some point there's bound to be a clash - at which point giving the user some manual control over what is captured, such as with your ItemFlags suggestion, is the only way forward. At least for us, what I have now is 'good enough' - perhaps once I've subjected it to some wider user testing I might have a brainwave that results in more useful feedback than the above rambling! :)

As to the other points...

Regarding the Alt key, this is another case of me not paying close enough attention, it seems... sorry about that! I had originally assumed that the technique described in the link was a bit of a hack and therefore undesirable to introduce just for the sake of this feature. But you're quite right, it is actually a perfectly sane thing to do, and I have now done so.

Having Enter work as well as Space for whichever widgets it is possible for would definitely be nice, though you're quite right that it isn't possible for Drag/Slider widgets - I had overlooked this. If it's possible to add, that ought to be enough to close off this issue, since my other points are now resolved. :)