Add support for cancelling input with Escape

[PaulJuliusMartinez]

I think in certain contexts it make sense for Escape to cancel entering an input, similar to Ctrl-C or Ctrl-D. When entering a : command in vim, for example, hitting escape will return you to normal mode.

[gwenn]

But rustyline has only normal and insert mode, no command line mode.
And on unix, a single escape is always ambiguous: escape sequence versus single escape depending on how fast you type.
Do you try to rebind this key ?

[schungx]

Rebinding the Escape key works fine (you essentially make it the same as Ctrl-U) and behaves similarly to the command prompt on Windows -- you're probably running Windows if you want Escape to clear the line.

[gwenn]

But with Windows Terminal, you may have the same issue as on unix if we activate ENABLE_VIRTUAL_TERMINAL_INPUT...

[schungx]

But with Windows Terminal, you may have the same issue as on unix if we activate ENABLE_VIRTUAL_TERMINAL_INPUT...

Probably yes, but without ENABLE_VIRTUAL_TERMINAL_INPUT, I have mapped Esc to clear-line like this and it works just fine:

    // On Windows, Esc clears the input buffer
    #[cfg(target_family = "windows")]
    rl.bind_sequence(
        Event::KeySeq(smallvec![KeyEvent(KeyCode::Esc, Modifiers::empty())]),
        EventHandler::Simple(Cmd::Kill(Movement::WholeBuffer)),
    );

So, as long as you're not requiring bracketed paste, mapping the Esc should be quite safe.

With ENABLE_VIRTUAL_TERMINAL_INPUT, you can use the following trick: If it is an escape sequence, Windows Terminal always sends the whole stream at once. Therefore, it will be KeyDown(Escape), KeyDown('[') ... etc.

If it is just the Esc key, it will be KeyDown(Escape), KeyUp(Escape)) or just a lone KeyDown(Escape) without anything following (if the user holds on to the key). It is very easy to distinguish between the two.

[PaulJuliusMartinez]

But rustyline has only normal and insert mode, no command line mode.

I wasn't referring to vim-mode in Rustyline, just using vim as an example of a program that allows using Escape to cancel entering input in readline-like contexts. You can also use Escape to cancel entering a search pattern after pressing / in vim.

I understand that escapes are fundamentally ambiguous, but there are workarounds for that -- lots of programs allow configuring a timeout where, if no data is supplied after reading an Escape byte, it'll register as an Escape press. And those programs also often support setting that timeout to 0. I think the rough assumption is that you'll basically never read half an escape sequence, so if you ask to read 64 bytes, but only get a single escape byte back, it's pretty likely that's an actual Escape press.

I tried binding a sequence, but it didn't seem to work:

rustyline_editor.bind_sequence(
    KeyEvent::new('\x1B', Modifiers::empty()),
    Cmd::Interrupt,
);

Using a different character -- just 'a', for example -- does work. (Also, what's the difference between Cmd::Abort and Cmd::Interrupt? I would expect Cmd::Abort to stop editing and return an Err, but it didn't seem to do anything.)

[schungx]

I think Escape is treated differently. You have to bind via:

KeyEvent::KeySeq(smallvec![KeyEvent(KeyCode::Esc, Modifiers::empty())])

KeyEvent::new is only for ASCII.

[gwenn]

I understand that escapes are fundamentally ambiguous, but there are workarounds for that -- lots of programs allow configuring a timeout where, if no data is supplied after reading an Escape byte, it'll register as an Escape press. And those programs also often support setting that timeout to 0. I think the rough assumption is that you'll basically never read half an escape sequence, so if you ask to read 64 bytes, but only get a single escape byte back, it's pretty likely that's an actual Escape press.

See https://docs.rs/rustyline/latest/rustyline/config/struct.Config.html#method.keyseq_timeout
"By default, no timeout (-1) or 500ms if EditMode::Vi is activated."
So you can use Escape as the Meta key.

I tried binding a sequence, but it didn't seem to work:

rustyline_editor.bind_sequence(
    KeyEvent::new('\x1B', Modifiers::empty()),
    Cmd::Interrupt,
);

Using a different character -- just 'a', for example -- does work. (Also, what's the difference between Cmd::Abort and Cmd::Interrupt? I would expect Cmd::Abort to stop editing and return an Err, but it didn't seem to do anything.)

This should work by either pressing Escape twice or modifying the default timeout.
And Cmd::Abort is currently used only to abort completion or history search.

[PaulJuliusMartinez]

This does not work for me when using Behavior::PreferTerm, running on macOS 13.6.2, using rustyline 13.0.0:

use rustyline::history::MemHistory;
use rustyline::Editor;

fn main() {
    let editor_config = rustyline::config::Config::builder()
        .behavior(rustyline::config::Behavior::PreferTerm)
        .keyseq_timeout(0)
        .build();

    let mut editor = Editor::<(), MemHistory>::with_history(editor_config, MemHistory::default()).unwrap();

    editor.bind_sequence(
        rustyline::KeyEvent::new('\x1B', rustyline::Modifiers::empty()),
        rustyline::Cmd::Interrupt,
    );

    let result = editor.readline("Enter first command: ");
    println!("Initial input: {:?}", result);
}

If I delete the .behavior(rustyline::config::Behavior::PreferTerm) line, then it does work.

I am unsure what the difference could be here (the code difference seems minuscule for PreferTerm..., but I know that /dev/tty definitely receives escape key presses, because when I open it myself I can read (single) escape values.

[PaulJuliusMartinez]

I've dug more into this, and discovered the root cause. To detect single-escapes vs escape sequences, after reading a single \x1B byte, rustyline polls the input input fd to see if it has more input to read. When using Behavior::PreferTerm, the input fd is /dev/tty, and calling poll does not work on devices on MacOS.

Here are some blog posts that explain potential workarounds.

I'm not sure how feasible it is to use one of these workarounds, but there may be a more straightforward solution when keyseq_timeout is set to 0. Since the input is buffered, we can just check if there is any additional buffered input, and assume that it is a single escape if there is no buffered input, and it is an escape sequence if there is buffered input, and not bother ever calling poll at all.

It seems like this scenario is already checked for debugging purposes?

if key == E::ESC {
    if !self.tty_in.buffer().is_empty() {
        debug!(target: "rustyline", "read buffer {:?}", self.tty_in.buffer());
    }
    ...
}

I'm not sure if there's any simple workaround when keyseq_timeout is some non-zero value however.

I've made a PR to fix this here: #802