AssertionError occurs using 3.130b1 interactive mode when pressed tab and up arrow

[BreezeWhite]

Bug report

Bug description:

Hi dear core developers,

I was playing with the newest REPL v3.130b1 on my M2 MacBook Pro with macOS 14.3. I built the python binary from the source at commit b62cb5234b. It's really impressive and exciting to see how much the standard REPL has evolved in this version.

However the bug occurred while I was playing with the auto-completion on the command. Says I want to see what operations are there under datetime module, type datetime. and press Tab twice, the suggestions show up correctly. But when I try to surf the suggestions with arrow keys pressed several times, I got the AssertionError:

Python 3.13.0b1+ (heads/3.13:b62cb5234b, May 10 2024, 10:17:59) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
datetime.MAXYEAR        datetime.date(          datetime.time(          datetime.tzinfo(        
datetime.MINYEAR        datetime.datetime(      datetime.timedelta(                             
Traceback (most recent call last):atetime_CAPI  datetime.timezone(                              
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/thmac-02/cpython/Lib/_pyrepl/__main__.py", line 47, in <module>
    interactive_console()
    ~~~~~~~~~~~~~~~~~~~^^
  File "/Users/thmac-02/cpython/Lib/_pyrepl/__main__.py", line 44, in interactive_console
    return run_interactive(mainmodule)
  File "/Users/thmac-02/cpython/Lib/_pyrepl/simple_interact.py", line 138, in run_multiline_interactive_console
    statement, contains_pasted_code = multiline_input(more_lines, ps1, ps2)
                                      ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/thmac-02/cpython/Lib/_pyrepl/readline.py", line 301, in multiline_input
    return reader.readline(), reader.was_paste_mode_activated
           ~~~~~~~~~~~~~~~^^
  File "/Users/thmac-02/cpython/Lib/_pyrepl/reader.py", line 652, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "/Users/thmac-02/cpython/Lib/_pyrepl/reader.py", line 635, in handle1
    self.do_cmd(cmd)
    ~~~~~~~~~~~^^^^^
  File "/Users/thmac-02/cpython/Lib/_pyrepl/reader.py", line 589, in do_cmd
    self.update_cursor()
    ~~~~~~~~~~~~~~~~~~^^
  File "/Users/thmac-02/cpython/Lib/_pyrepl/reader.py", line 499, in update_cursor
    self.cxy = self.pos2xy()
               ~~~~~~~~~~~^^
  File "/Users/thmac-02/cpython/Lib/_pyrepl/reader.py", line 471, in pos2xy
    assert 0 <= pos <= len(self.buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

Not sure if this is the intended behavior of using the new REPL, but I still hope this could be solved as this can largely improve the user experience of using the new auto-completion feature.

Thanks for all your hard works, the new python features are really awesome. Can't wait to see the final release ^^ It's my first issue sending to cpython, and sorry if there is any missing information I did not provide.

CPython versions tested on:

3.13

Operating systems tested on:

macOS

Linked PRs

• gh-118877: Fix AssertionError crash in pyrepl #118936
• [3.13] gh-118877: Fix AssertionError crash in pyrepl (GH-118936) #119363

[danielhollas]

I was able to repro this on Fedora 39. The fastest repro is if you press TAB twice to show the completion options, and then you press up arrow, followed by down arrow.

I think the up arrow should not "enter" the menu in the first place, since the menu is not currently interactive anyway right? (e.g. one cannot press Enter to select a suggested completion, left/right arrows don't do anything etc) .

CC @pablogsal

[pablogsal]

CC: @lysnikolaou @ambv

[ambv]

I'm taking this one.

[danielhollas]

I tested that the issue is not present in pypy, where pressing an up arrow goes back to in command history (instead of entering the completions menu).

[danielhollas]

I found the issue I think. :-) @ambv I can open a PR over the weekend if you'd be okay with that, it's quite fun to hack on the new REPL. :-)

[danielhollas]

The following patch seems to fix the issue

diff --git a/Lib/_pyrepl/commands.py b/Lib/_pyrepl/commands.py
index 456cba0769..aa3edce09f 100644
--- a/Lib/_pyrepl/commands.py
+++ b/Lib/_pyrepl/commands.py
@@ -245,7 +245,8 @@ def do(self) -> None:
             x, y = r.pos2xy()
             new_y = y - 1
 
-            if new_y < 0:
+            if r.bol() == 0:
                 if r.historyi > 0:
                     r.select_item(r.historyi - 1)
                     return

I took the code from the original pypy implementation. I don't yet understand why the new code doesn't work, and don't know if this fix doesn't break something else.

@BreezeWhite can you try it out?

[danielhollas]

Looks like this was introduced in commit " Fix vertical navigation with wide characters "
691c75e

@lysnikolaou could you maybe try the suggested patch above to see if your commit still does what it aimed to do?

(sorry for the noise if this is not helpful)

[pablogsal]

Looks like this was introduced in commit " Fix vertical navigation with wide characters "

691c75e

@lysnikolaou could you maybe try the suggested patch above to see if your commit still does what it aimed to do?

(sorry for the noise if this is not helpful)

I think the tests cover it but you can try to use a bunch of Chinese characters, press enter, and a bunch or regular characters and then move the cursor form one line to another and check that the cursor moves vertically and not jumping forwards in the previous line

[lysnikolaou]

@lysnikolaou could you maybe try the suggested patch above to see if your commit still does what it aimed to do?

The patch above indeed fixes this issue, but the problem is bigger than that. For example, pressing the Left arrow after the completions are shown moves the cursor three positions to the left.

It turns out reader.pos2xy uses reader.screen and reader.screeninfo to compute the position in the screen. Both of these include the two lines with completions. However, reader.pos and reader.buffer only include the things actually typed by the user (i.e. the third line datetime.). This leads to the new cursor position calculation being faulty. We'll have to think more about how to fix this.

[lysnikolaou]

Leave this open for the bug I described in #118877 (comment) above or open a new issue for that?