pressing and holding Alt+f in ghci sporadically prints f

[cheater]

Steps to reproduce:

1. open ghci
2. hold down alt
3. hold down f

Expected outcome: nothing happens
Real outcome: every now and then, the character f is appended to the command line

No one I asked was able to reproduce this, but I can reproduce it here. I'm on GHCi 8.2.2 on Ubuntu 14.04, 32-bit.

[cheater]

I'm using mate-terminal which is basically Gnome 2's gnome-terminal, and bash.

[cheater]

Note: i have to hold the buttons down for 3 seconds before the f's start appearing, but the CPU here is under load (from the browser), so on a "faster" pc you might have to wait substantially longer (I'm guessing this has something to do with it)

[lspitzner]

This smells a bit similar to jtdaugherty/vty-unix#5

[lspitzner]

I can reproduce on 64bit archlinux.

[cheater]

It bears mentioning that it never happens with Ctrl-f, only with Alt-f. (is it the same for you, @lspitzner?)

[lspitzner]

yes, it is. And it aligns with the escape-sequence buffer-alignment theory, because, lets see:

(As input, in the second line there, I use: o, o, alt-o, ctrl-o, enter, ctrl-d; because other characters tend to be interpreted in some way by my emulator or bash):

> hexdump -C
oo^[o^O
00000000  6f 6f 1b 6f 0f 0a                                 |oo.o..|
00000006

That is, o is 6f, alt-o is the escape-sequence 1b 6f, ctrl-o is 0f and enter is 0a. Ctrl-d is end-of-file and we don't see that of course.

[cheater]

So maybe what happens is that at some point the input buffer ends with esc and the character it's escaping (here f, or o) isn't there yet, so code reads it off and interprets it as a literal escape. But what should be happening is that it should instead treat anything that comes after escape the same way as if Alt were pressed. I.e. when reading a buffer, the code should know if the last character of the previous read was an esc that was to be treated as an escape character, and then apply escaping properly.

In the terminal, eg on bash, if I press esc and then f, then i get moved forward one word, which is the same as alt-f. Whereas, if i press esc, then esc, then f, I get a literal f typed into the command line.

[lspitzner]

@cheater yeah, the encoding used by terminals means that we cannot distinguish between "esc f" and "alt-f", unless we consider timing. This "timing-based" approach is implicitly implemented currently by considering everything in the same buffer as "at the same time" while the next buffer are "later inputs". This is a nice approximation but breaks as we can observe.

Your proposed solution only works if your program never reacts to single esc keypresses. Might be true for ghci, but I am not sure if we should restrict the general-purpose haskeline in this fashion.

A better approach would be to fix the timing-based approach by, for example either a) reading all available input, then interpreting it (respecting cross-chunk-border escape sequences) or b) testing for available input only if the last character was a potential escape-sequence-start, and continuing if more input is present, by checking for the potential cross-chunk escape sequence.

[cheater]

OK, my solution would be:
if the last element of a buffer was an esc-as-escape-char, then wait for (timeout) miliseconds, and at that point check the buffer. if the buffer contains something, escape the first char of that buffer. if the buffer is still empty, you have a literal escape.
however... for things like ghci, you want the timeout to be infinite. Therefore, the timeout should be a Maybe Int value. If the Maybe Int is Just 0, never wait and just always interpret an unescaped esc at the end of a buffer as a literal esc. If the Maybe Int is Nothing, don't do a timeout, and instead if the last buffer ended in an unescaped esc, always escape the first character of the next buffer.

The "Nothing" case is what I would expect from a command line, and it's the way readline works.

However, that is a bit complicated. If an esc happens anywhere but at the very end of a buffer, it's always treated as an escape sequence. So programs that were detecting a lone escape this way were relying on a bug - and maybe that shouldn't be supported. The question is to the maintainer what he wants to do. I'd be happy with just the "Nothing" case happening. I.e. there's no configuration of time out, no waiting, the first char of the next buffer always gets escaped.

[lspitzner]

I would assume that if the keyboard (or whatever the next lower level software bit is) produces an escape-sequence, it never happens with any observable delay between the characters forming the sequence.

So unless you want users to be able to use "esc f" to get the alt-f effect, I see no advantage to ever having a non-zero timeout.

Note that "escape in the middle of the buffer" will practically never occur if the user actually pressed escape. Because unless the user is really quick with button-mashing, your program input processing loop will have read the "esc" before a next key is pressed. Well, unless you don't have a separate input-processing loop.. I have no idea how haskeline is implemented :D

[cheater]

If you take a thing that "practically never occurs" and you put it in a loop that iterates thousands of times a second, the thing will happen every 3 seconds.

[lspitzner]

@cheater what? do you have a way to reproduce this, that involves the escape key?

[cheater]

Yes, in fact if i do esc-f instead of alt-f it never even makes the movement, it always just skips to typing f.

[cheater]

I just tried and it only worked after typing esc-f 244 times. (I know because there are 244 f's in the command line now)

[lspitzner]

this seems to confirm the very statement that i made. None of the escape-presses landed in the middle of a buffer, because there always was a sufficient delay to the next keypress.

[cheater]

no, that's incorrect. it failed 244 times before it worked. it should have worked all 245 times, instead of printing f 244 times.

[judah]

Thank you for the report. @lspitzner, your diagnosis sounds reasonable (though see below for some open questions).

Incidentally, Haskeline does need to recognize a lone "Esc"; it's necessary for interacting with the vi-style bindings (editMode: Vi).

The relevant code is here:
https://github.com/judah/haskeline/blob/master/System/Console/Haskeline/Backend/Posix.hsc#L228

Specifically, this is the bit that tries to get as many characters as possible:
https://github.com/judah/haskeline/blob/master/System/Console/Haskeline/Backend/Posix.hsc#L238

There isn't an explicit buffer in Haskeline's code; it just keeps calling hGetChar until hReady returns false. (And in fact it sets hSetBuffering to NoBuffering.) But it's possible the Handle or terminal might have an underlying buffer somehow that makes hReady return false negatives.

Unfortunately I can't reproduce your issue at this time (using macOS Sierra Terminal). But let me know if you make any more progress in tracking it down.

[cheater]

Hey Judah, If you tell me the steps to try and debug this I'll try it on my machine.

…

On Wed, 14 Feb 2018 06:37 Judah Jacobson, ***@***.***> wrote: Thank you for the report. @lspitzner <https://github.com/lspitzner>, your diagnosis sounds reasonable (though see below for some open questions). Incidentally, Haskeline *does* need to recognize a lone "Esc"; it's necessary for interacting with the vi-style bindings (editMode: Vi). The relevant code is here: https://github.com/judah/haskeline/blob/master/System/Console/Haskeline/Backend/Posix.hsc#L228 Specifically, this is the bit that tries to get as many characters as possible: https://github.com/judah/haskeline/blob/master/System/Console/Haskeline/Backend/Posix.hsc#L238 There isn't an explicit buffer in Haskeline's code; it just keeps calling hGetChar until hReady returns false. (And in fact it sets hSetBuffering to NoBuffering.) But it's possible the Handle or terminal might have an underlying buffer somehow that makes hReady return false negatives. Unfortunately I can't reproduce your issue at this time (using macOS Sierra Terminal). But let me know if you make any more progress in tracking it down. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#77 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAL7PqjNjvLiuVoj6BMVarsI9M908ciSks5tUnEOgaJpZM4SBgiD> .

[lspitzner]

@cheater a reply to your last-but-one comment: yeah, i misunderstood what you meant earlier. I agree it is not nice to have behaviour that is inconsistent with a 1:244 ratio. I am still not convinced which of the two is the "correct" one, as that depends on our requirements.

It also appears my assumption that there would be no observable delay is wrong in practice.. would have been too easy.

@judah Yeah that implementation seems correct, so either the error is lower-level or it is something else entirely. Thanks for pointing out the code, I have made enough assumptions ignoring the actual code :-)

[bjartur]

If it's worth anything, I can not reproduce this under low CPU load, neither with GHCi.exe running in its own Windows Console nor under PowerShell cd haskeline; stack ghci. I reckon that should be obtaining haskeline from Stackage snapshot lts-13.6. Nor in stack ghci in a lts-11.3 project.

[goertzenator]

Some useful reference information on this topic:
https://stackoverflow.com/a/3219355/398021

I'm coming here from case jtdaugherty/vty#160 where a slow serial port is causing escape codes to not be recognized. I'd like to experiment with the following solution:

• Add timeout functionality to getBlockOfChars
• If an ESC appears anywhere in the block, timeout mode is activated. Checking just the last character is not sufficient because escape sequences can be of variable length.
• When timeout mode is activated, wait x milliseconds for no data to arrive before ending the block.
• I'll choose "x" to be 10 milliseconds. This should remain responsive to the user while supporting baud rates down to 1200bps. If you have a 300bps modem, time to upgrade!

I know people haven't thought about this issue in years, but I would appreciate if someone could point out any downsides to this approach.

Context: I am working on a headless appliance that is configured through a 9600bps serial port.

[Merivuokko]

I am using a refreshable braille display, and this problem really makes it difficult for me to edit command-lines with haskeline.

Background:
Most braille displays have row of cursor routing buttons – one button for every braille cell on the display. Pressing such a button causes the braille display driver to route the screen cursor to the location pointed to by the cursor routing button column number and the starting line+column of the braille window. (Braille displays can only show a small portion of the real screen contents – generally only one line, and 20-80 columns.) Cursor routing is analogous to clicking a mouse button to move the cursor to a desired location.

BRLTTY is a commonly used braille display daemon which works with text terminals. In BRLTTY, cursor routing works by sending escape sequences (representing the arrow key presses) to the TTY and observing how the running application reacts to the synthesized arrow key presses.

For example if the cursor is at the end of a line, and I press a cursor routing button at the left side of the braille display row, BRLTTY starts to (very quickly) insert the ESC [ C character sequence to the TTY (assuming the terminal type is linux).

What happens in haskeline is that the cursor mostly moves as expected, but sometimes the escape sequence gets split in haskeline's input processing, and haskeline (instead of moving the cursor) inserts a C or [C (or D if I tried to move the cursor to the right) on the line. These spurious inserts happen about every second time I use the cursor routing keys. As a result, the code that I was editing is clobbered and won't run, if I submit the input without noticing that it got unintentionally modified.

This cursor routing functionality is crucial for editing text as a braille display user – it is very tedious to try to follow cursor movements with one hand and pressing the arrow keys with the other hand.

This problem does not occur with any other TTY application that I have used during the last 20 years (including applications based on readline). I therefore prefer using rlwrap with ghci.

[cheater]

That's horrendous. Can this get fixed already?? It's been nearly 4 years and by now it's just embarrassing.