Feature Request: Mouse/Touch/Pointer Bindings (like middle-click paste, right-click context menu, etc.)

[carlos-zamora]

Summary of the new feature/enhancement

Expand settings to be able to define remappable mouse bindings. Arguably, different touch events should fall under this too. So let's just call this "pointer bindings" for now.

Proposed technical implementation details (optional)

Mouse bindings are a bit trickier than keybindings in that the mouse has a location where the event occurs. For example, right-clicking a tab should have a different effect on the terminal than right-clicking the terminal.

As a super early idea, consider this format:

"pointerbindings": [
{
  "device": "mouse",
  "event": "rightClick",
  "where": "tab",
  "command": "flyoutMenu"
},
{
  "device": "mouse",
  "event": "doubleClick",
  "where": "terminal",
  "command": "wordSelection"  
},
{
  "device": "touch",
  "event": "swipe",
  "where": "terminal",
  "command": "scroll"
}
]

We definitely need a spec for this because it'll be a bit hefty. We'll also need to update a decent amount of documentation (particularly settings schema) to be able to describe what combinations are acceptable (again, the JSON mentioned up here is just me rambling about a potential implementation.)

There may be overlap between some commands in keybindings. Be sure to think that through.

Mike notes:

we should go back and collect up all the threads we've said "this would be a good mouse bindings feature" below

---

spec draft

Mouse bindings

Abstract

We've had numerous requests to configure how the mouse behaves in the Terminal.
The original behavior was a simple duplication of how conhost behaved: a right
click will copy the a selection if there is one, or paste the clipboard if there
isn't. Over time, we've accumulated a number of scenarios that we believe can
all be addressed by allowing more fine-grained mouse binding support. However,
upon further review, ultimately, we don't really need deep mouse binding
support.

Scenarios

The following is a list of all the feature requests we've linked to mouse
bindings in the past, grouped into categories of related requests:

Change how mouse multi-click selects

• #7173 Multiple sets of word delimiters for double-click selection
• #9881 Limit triple-click selection to command field only
• #6511 Multi-click selection granularity
• [Feature Request: Smart Double-click Selection (regeces?) #3196] Feature Request: Smart Double-click Selection (regeces?)

Change the action that L/M/R-mouse performs

• #7646 xterm-style selection, paste on middle click, copy on mouse release
• #10802 - VK_XBUTTON1/2, etc.
• #6250 - separate "Paste Clipboard" vs "Paste Selection" actions
• #3337 - Add a right-click context menu

Other

These are smaller, independent features that could all have an individual setting (if needed)

• #11710 Request: Setting to disable zooming with ctrl+mousewheel
• #13598 Add option to configure URL clicking behavior
• #11906 Option to disable "Pinch to Zoom"
• #6182 Fast scroll by holding down Alt while scrolling with mouse wheel
• Block selection by default (without alt) (see mail thread "RE: How to disable line wrap selection in Terminal")
• #17610 Configure block select to use ctrl, not alt

Solution design

Following the above scenarios, solutions are proposed below:

Change how mouse multi-click selects

Across the requests here, we've got the following requests:

• double-click: selects a "word" between 2 same delimiters
• triple-click: selects an entire string separated by spaces
• 4-click: entire line

Currently, Ctrl+A will select the entire command/text input, however, triple
clicking (mouse version of Select All selects the entire line (including the
prompt). GIF shows selecting with mouse vs with keyboard:
...
I would like the triple click action to align to the Ctrl+A selection method.

Could we maybe add shift double click to select using alternate word
delimiters?

I was really thinking more of regex though, because it can be a good starting
point for implementing more advanced features like type-specific smart
highlighting and hyperlinking of terminal text, not just smart selection.

To boil this down, users want to be able to configure the behavior of double,
triple, and quadruple clicks. The most common request is to change the
delimiters for double-click selection. But users also want to be able to
configure the delimiters to change on
Shift/Alt/Ctrl clicks.

"mouse": {
    "clicks": {
        { "click": "double", "command": { "action": "expandSelection", "delimeters": " /\\()\"'-.,:;<>~!@#$%^&*|+=[]{}~?\u2502" } }
        { "click": "shift+double", "command": { "action": "expandSelection", "delimeters": " " } }
        { "click": "triple", "command": { "action": "expandSelection", "regex": "^.*$" } }
    }
}

Alternatively,

"mouse": {
    "doubleClick": " /\\()\"'-.,:;<>~!@#$%^&*|+=[]{}~?\u2502",
    "tripleClick": { "regex": "^.*$" }
}

[carlos-zamora]

A few things:

• Treat this as a megathread so feel free to share your ideas here 😁
• I'm a bit busy with Accessibility (TOP PRIORITY) at the moment so if anybody wants to start working on this, just let me know.

[philippeller]

Probably related, make for example scrolling work when using vim etc....used to work fine for example in WSL terminal

[DHowett-MSFT]

make .. scrolling work .. vim

That is not related. That is #376 then #545.

[schorrm]

Can I pile on here -- can we get two finger scrolling on the mousepad?

[cerebrate]

A small point that's probably obvious, but just to make sure it's said, can we make sure the default is and the defaults are to pass through mouse events within the terminal to the underlying VT, per #545?

Customizability I like, but not as much as not breaking anything that would otherwise work. If someone's using, for example, tmux mouse support in conhost, they shouldn't have to grovel through the options to figure out why it doesn't work in Terminal.

[zadjii-msft]

Though it's also being discussed in #524, I want to make sure it's tracked here that some people might not want copy-on-select. So that needs to somehow be included in these bindings. Though, that might make sense as a different setting, since I'm not sure if there are any other "onSelect" actions that would possibly make sense.

[vyevdokimov]

Add something like "lines to scroll" option - in some cases scroll is too fast in terminal using touchpad.

[getify]

@carlos-zamora thanks for that update. Further question: what about #3990, which is proposed as a shorter path to "fixing" a somewhat breaking behavior of mouse-scroll-ctrl-zooming?

It's a small subset of this larger feature request, but presumably if it's prioritized in that same queue it might be able to be tackled much more quickly (and fix our blocker problem) while this larger feature is worked on.

[zadjii-msft]

Part of me thinks I'd almost rather figure out the regression than add the escape hatch. This totally seems like the kind of thing that would have regressed in #10051. I've marked #3990 as Help Wanted and Easy Starter. I don't think we're gonna have time to get on those with the other bugs on our plate in 1.14.

That being said, we should get to the bottom of the regression. @getify mind filing a separate issue for the touchpad scrolling regression? I don't want to lose track of that for 1.14

(idly: that might be related to #9955.b)

[getify]

@zadjii-msft The "regression" is not, from what I can tell, in Terminal. I think the regression is Windows and/or the built-in touchpad drivers.

I thought initially this issue was only affecting Terminal; I hadn't experienced the problem in any other apps. That's why I came here. But another app I use does experience this same increase in the issue/sensitivity: Sublime. As far as I can tell, Terminal and Sublime are the only two apps I use which ctrl+scroll causes a zoom behavior at all, so it makes sense they're the only two where the regression/change in sensitivity has become obvious.

I've chased similar rabbit trails with Sublime, looking for some way to disable the ctrl+scroll zooming. They do have keybinding settings, and in a limited way I was able to shut off the annoying zooming behavior. However, their problem is that you can't just tell it to "ignore" the ctrl... you literally disable all actions when ctrl is held down while zooming. That's not what is desired. The outcome currently in Sublime is annoying: if I swipe to scroll, and the inertia is moving the scroll pane, and then I hit the ctrl key, all scrolling abruptly stops. There's no zoom, but I didn't want the scroll to stop like that, so it's not good.

What is desired is a setting that tells the app (Terminal or Sublime), "hey, don't treat ctrl as a special modifier while scrolling, just let scrolling continue to happen as it would if ctrl wasn't pressed".

I bring this detail up to say:

1. I don't think we need to fix a regression in Terminal
2. I hope when you add a keybinding or disabling feature into Terminal, you realize the difference between "shut all action down" and "don't pay any attention to this modifier key". The former is what Sublime did, and it sucks. The latter is what I think us users actually want.

[zadjii-msft]

Weird. Well, looking at the code for a second, I wonder if there's an easier stop gap that could be implemented: when ctrl is pressed, cancel any current momentum in the touch scrolling. I don't even know if that's something that the input APIs let us do. But like, if you did a swipe with the trackpad, then hit Ctrl, then you probably don't want the momentum from that swipe to start zooming...

I did look at the code and I'm pretty sure the regression in #9955.b would also apply to zooming here. This is admittedly psychic debugging at work, but: Precision trackpads send many scroll events with smaller incremental scroll measurements, but I think we just adjust the font size by 1 whole size each event, not based on the actual scroll delta. Unfortunately, the zooming seemed fine on my slaptop 2 so I couldn't repro this specifically.

---

FWIW, the plan I have in my head for this is a lot like how keybindings work:

[
  { "button": "mouseWheelDown", "command": { "action": "scrollDown", "rollsToScroll": "${scrollDelta}" } }, // or something like that
  { "button": "ctrl+mouseWheelDown", "command": { "action": "scrollDown", "rollsToScroll": "${scrollDelta}" } } // hey look, rebind ctrl+mouse to scroll instead
]

[getify]

when ctrl is pressed, cancel any current momentum in the touch scrolling.

I'm not sure if that's what we want or not... what I prefer is for ctrl to have no effect on scrolling behavior (either directly applied or through momentum) or zooming.

"cancel" implies it interferes in some way, and what I'd rather express is "ignore".

[zadjii-msft]

Sorry I don't mean "cancel" more, "interrupt"? Like, when the user presses ctrl, maybe we should stop any current momentum that the scrolling might have. Then the user would need to start a new scroll if they wanted to zoom. That might help mitigate the issue a bit.

A similar discussion in another (admittedly totally unrelated) thread:

#3793 (comment)

[getify]

Like, when the user presses ctrl, maybe we should stop any current momentum that the scrolling might have.

That's the behavior I have in my current hack-around in Sublime, and I dislike it. I don't want hitting ctrl to have any effect on my scrolling behavior (momentum or otherwise). I also don't want holding down ctrl before scrolling to prevent the scroll from starting (which is also how my current Sublime behavior is).

[Konstantin-Glukhov]

How about something like this?

"pointerbindings": [
{
"device": "mouse",
"event": "rightClickOnSelection",
"where": "terminal",
"command": "noAction"
},

[samheuck]

Is this issue dead? It's been... going on nearly 4 years since it was opened now?

I like using "Windows Terminal", but there's still no way I can find to disable "right-click to paste" / "QuickEdit" like you can in PowerShell.

[cloud-66]

i used to middle click paste, and it's hard for me to use windows terminal without this function. i love wt , but it still doesn't have this.

[korpa]

i used to middle click paste, and it's hard for me to use windows terminal without this function. i love wt , but it still doesn't have this.

Same here. This is the reason why I switched Windows Terminal to Alacritty a couple of weeks ago.

[amkartashov]

simple autohotkey script to paste on middle button:

; Paste with middle mouse in Windows Terminal
#IfWinActive ahk_class CASCADIA_HOSTING_WINDOW_CLASS
MButton::Send ^{vk56} ; Send Ctrl-V with middle button
#IfWinActive

[zadjii-msft]

Is this issue dead? It's been... going on nearly 4 years since it was opened now?

Nope, there's just plenty of other higher-priority work unfortunately. I've got a draft PR for an experimental setting just for the right-click context menu over in #14775. We however don't have a full spec for how we'd actually like to express the fullness of mouse bindings. That's the biggest blocker here - someone taking the time to enumerate what people have asked for and what's possible, and putting them together into a spec.

• Do folks want to bind arbitrary actions to mouse buttons, or really only a subset of actions?
• Does this become an extension of bindings, with new "keys" values to indicate mouse buttons?
• Should configuring the number of lines to scroll be a part of this? Like, rebinding mouseWheelDown to scrollDown(5)? Or is that a separate (but related) setting?
• How do we represent multi-clicks and multi-click actions? (|double|triple)(Left|Middle|Right)Click seems... verbose...

Those are the thoughts from the top of the dome. If someone wants to put together a proposal, I'd love to review it. Writing the code shouldn't be that hard, so long as we have the right design in place ☺️

[hmb]

Well, the most pressing issue is probably to just paste the clipboard with the middle mouse button. As many linux terminals, e.g. in VMs, X-Servers like Xming, even in the windows version of git-bash, "paste" is always the middle mouse click. So using the right click in ms-terminal makes it kind of random to which click has to be used:

• git-bash native -> middle click
• git-bash embedded in ms-terminal -> right click
• linux VM -> middle click

Guess what: you always use the wrong one. So in terms of a hamonized user exp would suffice for starters to "paste" with middle mouse button. The "generic covering all eventualities mouse configurator" is probably overkill for all those working in a hybrid environment.

[lucky-wolf]

Wow, 5y and not solved?

A trivial way to disable "quickedit" would suffice.

[zadjii-msft]

I mean, there is: experimental.rightClickContextMenu

[TheZetner]

simple autohotkey script to paste on middle button:

; Paste with middle mouse in Windows Terminal
#IfWinActive ahk_class CASCADIA_HOSTING_WINDOW_CLASS
MButton::Send ^{vk56} ; Send Ctrl-V with middle button
#IfWinActive

Just updated this to AHK 2.0:

; Paste with middle mouse in Windows Terminal
#HotIf WinActive("ahk_class CASCADIA_HOSTING_WINDOW_CLASS")
MButton::Send "^{vk56}" ; Send Ctrl-V with middle button

[psergiu]

It's 2024 and it's still not added :( Some companies don't allow the use of AutoHotKey on work machines. Please just add a simple "X11-style middle mouse button paste" option and it's perfect. If there's time to add more options in the future, even better.
Thank you.