In macOS, home and end keys no longer work

[jkwuc89]

Description

In macOS (I am using Monterey 12.1), the home and end keys no longer move the cursor to the start of the line and the end of the line respectively. Instead, these keys output the escape sequences you see in the screenshot below.

This behavior is new in irb that ships with Ruby 3.1. In the previous version, these keys worked as expected.

Result of irb_info

Ruby version: 3.1.0
IRB version: irb 1.4.1 (2021-12-25)
InputMethod: ReidlineInputMethod with Reline 0.3.1
RUBY_PLATFORM: x86_64-darwin21
LANG env: en_US.UTF-8
East Asian Ambiguous Width: 1

Terminal Emulator

I am using iTerm2 Build 3.4.14 which is configured to use xterm-256color.

Setting Files

I am not using a settings file.

[jkwuc89]

I tested this using macOS's terminal app and the home and end keys also do not work there. I also retested irb under Ruby 3.0.3 to double-check that the home and end keys work properly which they do.

[jkwuc89]

I also tested this in a terminal on Ubuntu 20.04.3 LTS and the issue does not occur there.

[jkwuc89]

I managed to fix this by mapping Home and End keys in my iTerm2 profile key mappings to the hex codes 0x01 and 0x05 respectively. These hex codes map to Ctrl+A and Ctrl+E which in an iTerm2 session, function as Home (go to beginning of line) and End (go to end of line) and work as expected in irb. From my profile key mapping settings:

[aycabta]

Would you try to run the following command and press Home/End key? You can exit with Ctrl+c.

ruby -r reline -e 'while (c = Reline::IOGate.getc); pp c; end'

Please copy the result.

[jkwuc89]

ruby -r reline -e 'while (c = Reline::IOGate.getc); pp c; end'

❯ ruby -r reline -e 'while (c = Reline::IOGate.getc); pp c; end'
1
5

[jkwuc89]

I am seeing a similar issue with the Delete key. Output inside irb:

ruby -r reline -e 'while (c = Reline::IOGate.getc); pp c; end' shows the following for the Delete key.

❯ ruby -r reline -e 'while (c = Reline::IOGate.getc); pp c; end'
27
91
51
126

[sshock]

I am having the same problem with Home, End, and Delete keys not working in iTerm2 on macOS and it is a major pain point for me.

Here's the output from ruby -r reline -e 'while (c = Reline::IOGate.getc); pp c; end' for each of these:

(Home):

27
91
72

(End):

27
91
70

(Delete):

27
91
51
126

(Side note: I believe @jkwuc89 's response for Home and End resulted in just a 1 and a 5 because he had already re-mapped those to Ctrl+A and Ctrl+E.)

Here are some more problematic keys that cause weird things to show up:

(Insert on Mac):

239
157
134

(Insert on PC):

27
91
50
126

(PgUp on Mac and PC):

27
91
53
126

(PgDn on Mac and PC):

27
91
54
126

[sshock]

I think this may be a problem in reline, introduced in reline v0.2.7.

I switched to Ruby 3.0 with reline 0.2.0 and the problem went away. Then I tried various versions of reline and the problem first shows up in reline 0.2.7.

Digging further, it appears that the problem surfaced when default key bindings started using set_default_key_bindings_terminfo instead of set_default_key_bindings_comprehensive_list.

[sshock]

I added this line to the set_default_key_bindings_terminfo method in Reline::ANSI:

puts "MAPPING #{key.inspect} TO #{func}"

and it produced this:

MAPPING [27, 79, 72] TO ed_move_to_beg
MAPPING [27, 79, 70] TO ed_move_to_end
MAPPING [27, 79, 65] TO ed_prev_history
MAPPING [27, 79, 66] TO ed_next_history
MAPPING [27, 79, 67] TO ed_next_char
MAPPING [27, 79, 68] TO ed_prev_char
MAPPING [27, 91, 65] TO ed_prev_history
MAPPING [27, 91, 66] TO ed_next_history
MAPPING [27, 91, 67] TO ed_next_char
MAPPING [27, 91, 68] TO ed_prev_char

Those codes for Home and End don't match what's actually getting sent. But how could the term info be wrong? All other apps including bash and vim and you name it work totally fine in iTerm2, so I don't understand.

Also, it appears reline didn't even attempt to map the delete key...

[jitingcn]

I think this may be a problem in reline, introduced in reline v0.2.7.

I switched to Ruby 3.0 with reline 0.2.0 and the problem went away. Then I tried various versions of reline and the problem first shows up in reline 0.2.7.

Digging further, it appears that the problem surfaced when default key bindings started using set_default_key_bindings_terminfo instead of set_default_key_bindings_comprehensive_list.

I have this problem too, and I have tried to test with different versions of reline by your inspiration, and I found reline reline 74a7ffa is the first version have this problem.
I'm using Arch Linux with zsh.

[sshock]

In trying to understand terminfo better, I looked at the output of infocmp xterm -1 | grep home and noticed khome and home:

        home=\E[H,
        khome=\EOH,

khome is what should be received from the terminal when the home key is pressed

home is what could be sent to the terminal to tell it to move the cursor to the home position (upper left)

So the mapping in set_default_key_bindings_terminfo actually looks correct, since it is mapping khome == "\eOH" == [27, 79, 72] to :ed_move_to_beg.

Which means the problem seems to be on the input side of things. When I press the home key it is sending "\e[H" == [27, 91, 72]; why isn't it sending "\eOH"?

Update: I did some more research and discovered that (with xterm at least) there are two modes, normal and application. It is correct to send "\e[H" when in normal mode and "\eOH" when in application mode. And iTerm2 does switch to sending "\eOH" when I put it in application mode (e.g., by running tput smkx).

Here are a couple links I found helpful in understanding normal mode vs application mode:

• https://invisible-island.net/xterm/xterm.faq.html#xterm_arrows
• https://unix.stackexchange.com/a/383087

[sshock]

I did a bunch of testing and I am convinced that iTerm2 is sending the wrong escape sequence for Home and End I believe iTerm2 is sending the correct sequences for everything except Insert.

Update: The main problem with Home and End is that the latest version of Reline only maps the application mode codes (SS3 == escape + O) not normal mode codes (CSI == escape + [), but it doesn't put the terminal into application mode.

The main problem with Delete, PgUp, and PgDown is that Reline is not mapping them to anything.

So here are all the issues and workarounds / solutions I've come up with:

Home key (khome)

• Problem: Reline only recognizes \eOH not \e[H.
• Workaround 1: Use Key Mappings to force iTerm2 to always map Home to \EOH
  • (Probably not a good idea)
• Workaround 2: Force terminal into application mode when running IRB (and return to normal mode after)
  • See ~/.irbrc code block below
• Solution 1: Fix Reline to put terminal into application mode (and return to normal mode after)
  • See Fix Home and End keys on macOS reline#521
• Solution 2: Fix Reline to recognize both sequences like it used to
  • See Always set default key bindings in lists reline#433

End key (kend)

• Problem: Reline only recognizes \eOF not \E[F.
• Workaround 1: Use Key Mappings to force iTerm2 to always map End to \EOF
  • (Probably not a good idea)
• Workaround 2: Force terminal into application mode when running IRB (and return to normal mode after)
  • See ~/.irbrc code block below
• Solution 1: Fix Reline to put terminal into application mode (and return to normal mode after)
  • See Fix Home and End keys on macOS reline#521
• Solution 2: Fix Reline to recognize both sequences like it used to
  • See Always set default key bindings in lists reline#433

Delete key (kdch1 )

• Problem: Reline doesn't map it to anything (although somehow it works on Linux)
  • Update: I believe the reason I wasn't seeing the problem on Linux is because reline was failing to load ncurses so it wasn't even using Reline::ANSI; see my comment here; anyway, now that I have ncurses loading I am also able to reproduce all these problems on Linux
• Solution: Update Reline code to map 'kdch1' to :key_delete
  • See Add key binding for Delete (save Insert, PgUp, PgDn for later) reline#434

Insert key (kich1)

• Problem 1: iTerm2 maps it to codes [239, 157, 134], which is incorrect for xterm
• Problem 2: Reline doesn't map it to anything
• Solution 1: Change iTerm2 Key Mappings to map it to \E[2~
• Solution 2: Update Reline code to map 'kich1' to :ed_ignore
  • (Was in Add key binding for Delete (save Insert, PgUp, PgDn for later) reline#434; need a new PR now)

PgUp key (kpp)

• Problem: Reline doesn't map it to anything
• Solution: Update Reline code to map 'kpp' to :ed_ignore
  • See Add key bindings for PgUp and PgDn reline#509

PgDn key (knp)

• Problem: Reline doesn't map it to anything
• Solution: Update Reline code to map 'knp' to :ed_ignore
  • See Add key bindings for PgUp and PgDn reline#509

Here's a screenshot showing Home, End, and Insert key mappings changed in iTerm2 (not recommended, except for Insert key):

Here's what I added to ~/.irbrc to make it switch the terminal into application mode:

::IRB.conf[:USE_AUTOCOMPLETE] = false
::IRB.conf[:USE_COLORIZE] = true

# Enter application cursor mode:
print "\e[?1h"

# Leave application cursor mode:
::IRB.conf[:AT_EXIT] << proc { print "\e[?1l" }

Here's a screenshot showing how to fix the Delete key in Reline, and also how to prevent Insert, PgUp, and PgDn from printing out nasties: