Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: mouse mode 1016 (SGR-Pixels) #574

Closed
ghost opened this issue Jan 7, 2022 · 9 comments · Fixed by #579
Closed

Feature request: mouse mode 1016 (SGR-Pixels) #574

ghost opened this issue Jan 7, 2022 · 9 comments · Fixed by #579
Labels
enhancement New feature or request feature-request User requested features VT: Backend Virtual Terminal Backend (libterminal API)

Comments

@ghost
Copy link

ghost commented Jan 7, 2022

Abstract

I would like to support pixel-based mouse position in my application.

Two existing methods to do this are DEC Locator Input (DECLRP) which according to that doc is only supported for REGIS and Tektronix; and SGR-Pixels which is supported by xterm and works in normal cases.

Motivation

With SGR-Pixels, one could feasibly make terminal-based applications for image editing, support a custom bitmap/animated mouse pointer, and I'm sure plenty other things.

Other applications desire this too, example: dankamongmen/notcurses#2326

I would try my hand at a PR for you, but I cannot seem to get contour to compile on my system (Debian 11.1 bullseye) :-(

Specification

I would like SGR-Pixels (mouse mode 1016) to be supported. The difference between this and 1006 mode which is already supported are:

  • Mouse movement by pixel, not cell, triggers an event.
  • The rows/cols reported are in pixels, not cells.
  • DECRQM (CSI ? 1016 $ p) produces a corresponding DECRPM so that applications can detect it. (...and BTW thank you for teaching me about DECRQM/DECRPM :-) )

Checking xterm via:

echo -ne '\033[?1003;1016h' ; cat

...shows that click-drag outside the windows does continue to produce SGR-Pixels mouse events, i.e. xterm continues reporting for the entire mouse grab.

@ghost ghost added the bug Something isn't working label Jan 7, 2022
@christianparpart
Copy link
Member

christianparpart commented Jan 7, 2022

Cool idea @klamonte. Many thanks for the inspiration.

DECRQM (CSI ? 1016 $ p) produces a corresponding DECRPM so that applications can detect it. (...and BTW thank you for teaching me about DECRQM/DECRPM :-) )

You're welcome, man :)

I quickly tried xterm with this to get a first impression. Workable. But it seems like xterm sometimes returns negative numbers when close to the top/left border?

Also, I was super surprised to not know about this xterm feature, especially since - when I started with this project - I literally implemented as many xterm VT sequences as possible and never read about this mouse mode.

Patch #359 - 2020/08/17
...

  • add a new mouse mode 1016, which uses the same format as mode 1006, but sends the mouse's position in pixels (suggested by Igor van den Hoven).

Turns out to be added like one and half a year ago, I should have refreshed my ctlseqs.txt.

p.s.: why bug label?

EDIT: I'm currently super busy on master (and dev branches to be merged to master), so that's not as stable as 0.2.x branch and its releases, working on better resource management / perf. When that's done pretty soon, I'll certainly want to pick this one up. :)

@ghost
Copy link
Author

ghost commented Jan 7, 2022

I have no idea how the bug label got there. I swear I pressed the feature request button. 🙂

Interesting that xterm reports negative, but makes sense. It keeps the grab as long as the mouse button is down. I didn't check to see what it did with no-button mouse motion.

@j4james
Copy link

j4james commented Jan 8, 2022

DEC Locator Input (DECLRP) which according to that doc is only supported for REGIS and Tektronix

@klamonte I think you might be misreading that. REGIS and Tektronix have their own locator input modes which are different from the Text Locator extension. All the terminal emulators I know that support DECLRP do so independent of REGIS and Tektronix. This includes XTerm, MLTerm, Mintty, RLogin, and VTStar.

@christianparpart I noticed Contour is claiming to support the Text Locator extension (29 in the DA1 report), but you don't seem to implement any of the locator sequences?

@christianparpart
Copy link
Member

@j4james that is indeed a bug of wrong understanding back in the time I implemented mouse support. Will remove that. Thanks.

@christianparpart christianparpart added enhancement New feature or request VT: Backend Virtual Terminal Backend (libterminal API) feature-request User requested features and removed bug Something isn't working labels Jan 8, 2022
@christianparpart
Copy link
Member

Do we know any app already using SGR-Pixels mode so I can verify my implementation isn't bogus? (apart from looking at xterm output and reason about the comparison?) :)

@christianparpart
Copy link
Member

..shows that click-drag outside the windows does continue to produce SGR-Pixels mouse events, i.e. xterm continues reporting for the entire mouse grab.

Oh heck! I was reading this just now! Sorry.

I would consider this a bug. Because the mouse pointer is outside the window. The xterm dev maybe had a reason for this. But I consider this bad / dies not Sound right.

What do you think?

(I just remembered, xeyes knows about the mouse location outside is window too, but still, little to no use)... Hmmm

@ghost
Copy link
Author

ghost commented Jan 9, 2022

I could see it both ways. Reporting negative coordinates could be handy for something like an image cropper where you have a selection rectangle/rubberband going always to the mouse location (even when outside the window) when being dragged. But it could also just be a bug. SGR 1006 (normal coordinates) clamps everything to positive integers only. It could be that SGR-Pixels was supposed to do that too, and forgot. Or it could be we needed a few more TE's to implement to get some apps that can play with it and really nail it down.

You had asked if an application can test it. Jexer 1.5.0, released last week, can test it. Just open the Pixels window in the demo and either mouse over the field (which will show a custom i-bar icon tracking the mouse by pixel) or click the "Cute mouse" button.

( If you want to have more fun, pull from my git head, same pixels demo button: Xterm animations ;) )

@j4james
Copy link

j4james commented Jan 9, 2022

Negative numbers in a CSI-based response are almost certainly a bug, because you can't have negative parameter values. The - character is an intermediate according to ANSI/ECMA rules, and thus would indicate the end of the parameter values and the beginning of the intermediate/final control identifier.

@christianparpart
Copy link
Member

I now want a simple terminal drawing demo. Haha. Well. I know that GUIs are better suited. But knows what usernames people might come up with. I am curious.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request feature-request User requested features VT: Backend Virtual Terminal Backend (libterminal API)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants