Skip to content

gh-111201: Support pyrepl on Windows #119559

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 70 commits into from
May 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
1061d22
Windows repl support
DinoV May 17, 2024
fa0f538
Arrow key support
DinoV May 18, 2024
6f35100
Make backspace clear char
DinoV May 18, 2024
7772787
Fix missing newline after input
DinoV May 18, 2024
aced5ae
Make insert work
DinoV May 18, 2024
fa3815f
Fix delete in middle
DinoV May 18, 2024
b30b105
Fix crash on invalid command key
DinoV May 18, 2024
df262c4
More fixes
DinoV May 20, 2024
6e57316
Colorize
DinoV May 20, 2024
18ecc2e
Use constants
DinoV May 20, 2024
60525eb
Use UnicodeChar
DinoV May 21, 2024
af79a17
Simplify
DinoV May 21, 2024
5f99256
Fix scrolling on input which is longer than screen/scrollback
DinoV May 21, 2024
bdff535
fix pager typo and refactor some unused branches (#41)
tonybaloney May 20, 2024
88d64f5
Implement screen clear for Windows (#42)
tonybaloney May 20, 2024
8a74306
Fix word wrap not being enabled in Windows Terminal
DinoV May 21, 2024
38e9c58
Fix culmitive errors in wrapping as lines proceed
DinoV May 21, 2024
9cddace
Fix issues with inputs longer than a single line
DinoV May 22, 2024
fc4efee
Resize WIP
DinoV May 23, 2024
564e6e1
set compat based on Windows build version
tonybaloney May 24, 2024
3ae4316
use escape sequence for clearing screen and setting cursor position
tonybaloney May 24, 2024
5bb02a2
Remove unused imports
tonybaloney May 24, 2024
500761e
Windows repl support
DinoV May 17, 2024
98f16b9
Arrow key support
DinoV May 18, 2024
911a76a
Make backspace clear char
DinoV May 18, 2024
77f1c42
Fix missing newline after input
DinoV May 18, 2024
243817d
Make insert work
DinoV May 18, 2024
25f51b4
Fix delete in middle
DinoV May 18, 2024
5242239
Fix crash on invalid command key
DinoV May 18, 2024
febe424
More fixes
DinoV May 20, 2024
756ac47
Colorize
DinoV May 20, 2024
50fd4c9
Use constants
DinoV May 20, 2024
ff46d66
Use UnicodeChar
DinoV May 21, 2024
fbb9f84
Simplify
DinoV May 21, 2024
683f5f6
Fix scrolling on input which is longer than screen/scrollback
DinoV May 21, 2024
2c524b8
fix pager typo and refactor some unused branches (#41)
tonybaloney May 20, 2024
44ce57b
Implement screen clear for Windows (#42)
tonybaloney May 20, 2024
5ecc8cd
Fix word wrap not being enabled in Windows Terminal
DinoV May 21, 2024
5bb90c7
Fix issues with inputs longer than a single line
DinoV May 22, 2024
35557dd
Resize WIP
DinoV May 23, 2024
e04699f
Use vt100 scrolling to avoid race conditions on resize
DinoV May 24, 2024
6230400
More win api cleanup
DinoV May 24, 2024
70a9a31
Code cleanup
DinoV May 24, 2024
4adfe77
Reformat
DinoV May 24, 2024
9c00bd1
Annotations
DinoV May 24, 2024
dc7ec86
Update news
DinoV May 24, 2024
79a3fb9
Update history
DinoV May 24, 2024
3e4fe55
Initial test cases
DinoV May 25, 2024
38800e7
📜🤖 Added by blurb_it.
blurb-it[bot] May 25, 2024
b3f7092
Fix mypy and formatting issues
DinoV May 25, 2024
a589321
More mypy fixes
DinoV May 25, 2024
9a34d8a
More MyPy
DinoV May 25, 2024
f8a9176
More MyPy fixes
DinoV May 25, 2024
ce77202
More mypy fixes, name changes in WindowsConsole
DinoV May 26, 2024
6d8e7f5
Fix up ignores
DinoV May 26, 2024
bbbb296
Avoid kernel32 on non-Windows platforms
DinoV May 26, 2024
d064124
Formatting
DinoV May 26, 2024
f244a0e
Merge remote-tracking branch 'dinov/winrepl' into winrepl_control
tonybaloney May 26, 2024
3d526ba
roll back other merge
tonybaloney May 26, 2024
b1f5688
Merge pull request #1 from tonybaloney/winrepl_control
DinoV May 29, 2024
1146dbb
Catch more specific comment, avoid typing, fix comment
DinoV May 29, 2024
972e7ec
Ignore type error on return of Any
DinoV May 29, 2024
1d300e4
Merge branch 'main' of https://github.com/python/cpython into winrepl
DinoV May 29, 2024
d783dae
Don't import traceback on exception
DinoV May 29, 2024
d1289af
Remove one more TYPE_CHECKING
DinoV May 29, 2024
db191a2
Update Lib/test/test_pyrepl/test_windows_console.py
ambv May 30, 2024
fba84aa
Use _WindowsConsoleIO for output
DinoV May 31, 2024
42e3e8e
Merge branch 'winrepl' of github.com:DinoV/cpython into winrepl
DinoV May 31, 2024
63d854e
Remove get_abs_positon
DinoV May 31, 2024
513a148
Fix tests on windows
DinoV May 31, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions Doc/whatsnew/3.13.rst
Original file line number Diff line number Diff line change
Expand Up @@ -154,10 +154,10 @@ New Features
A Better Interactive Interpreter
--------------------------------

On Unix-like systems like Linux or macOS, Python now uses a new
:term:`interactive` shell. When the user starts the :term:`REPL` from an
interactive terminal, and both :mod:`curses` and :mod:`readline` are
available, the interactive shell now supports the following new features:
On Unix-like systems like Linux or macOS as well as Windows, Python now
uses a new :term:`interactive` shell. When the user starts the
:term:`REPL` from an interactive terminal the interactive shell now
supports the following new features:

* Colorized prompts.
* Multiline editing with history preservation.
Expand All @@ -174,10 +174,13 @@ available, the interactive shell now supports the following new features:
If the new interactive shell is not desired, it can be disabled via
the :envvar:`PYTHON_BASIC_REPL` environment variable.

The new shell requires :mod:`curses` on Unix-like systems.

For more on interactive mode, see :ref:`tut-interac`.

(Contributed by Pablo Galindo Salgado, Łukasz Langa, and
Lysandros Nikolaou in :gh:`111201` based on code from the PyPy project.)
Lysandros Nikolaou in :gh:`111201` based on code from the PyPy project.
Windows support contributed by Dino Viehland and Anthony Shaw.)

.. _whatsnew313-improved-error-messages:

Expand Down
6 changes: 5 additions & 1 deletion Lib/_pyrepl/__main__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
import os
import sys

CAN_USE_PYREPL = sys.platform != "win32"
CAN_USE_PYREPL: bool
if sys.platform != "win32":
CAN_USE_PYREPL = True
else:
CAN_USE_PYREPL = sys.getwindowsversion().build >= 10586 # Windows 10 TH2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This merged as-is which is OK, but I wanted to chip in...

There is a minimal risk of cutting out terminals that would support pyrepl on earlier versions of Windows. The "most correct" (also most soapboxy, sorry!) way to determine support for this is to just try to set ENABLE_VIRTUAL_TERMINAL_PROCESSING and only fall back to basic console APIs if it fails. :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DHowett sounds like an improvement we can still make! Care to PR this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can try - at the very least, it would be a great opportunity for me to learn how to build and deploy Python 🙂

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DHowett see here and let me know if anything's missing:
https://devguide.python.org/getting-started/setup-building/#windows



def interactive_console(mainmodule=None, quiet=False, pythonstartup=False):
Expand Down
30 changes: 28 additions & 2 deletions Lib/_pyrepl/console.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,18 @@

from __future__ import annotations

import sys

from abc import ABC, abstractmethod
from dataclasses import dataclass, field


TYPE_CHECKING = False

if TYPE_CHECKING:
from typing import IO


@dataclass
class Event:
evt: str
Expand All @@ -36,6 +44,25 @@ class Console(ABC):
height: int = 25
width: int = 80

def __init__(
self,
f_in: IO[bytes] | int = 0,
f_out: IO[bytes] | int = 1,
term: str = "",
encoding: str = "",
):
self.encoding = encoding or sys.getdefaultencoding()

if isinstance(f_in, int):
self.input_fd = f_in
else:
self.input_fd = f_in.fileno()

if isinstance(f_out, int):
self.output_fd = f_out
else:
self.output_fd = f_out.fileno()

@abstractmethod
def refresh(self, screen: list[str], xy: tuple[int, int]) -> None: ...

Expand Down Expand Up @@ -108,5 +135,4 @@ def wait(self) -> None:
...

@abstractmethod
def repaint(self) -> None:
...
def repaint(self) -> None: ...
16 changes: 7 additions & 9 deletions Lib/_pyrepl/reader.py
Original file line number Diff line number Diff line change
Expand Up @@ -442,14 +442,13 @@ def get_arg(self, default: int = 1) -> int:
"""
if self.arg is None:
return default
else:
return self.arg
return self.arg

def get_prompt(self, lineno: int, cursor_on_line: bool) -> str:
"""Return what should be in the left-hand margin for line
'lineno'."""
if self.arg is not None and cursor_on_line:
prompt = "(arg: %s) " % self.arg
prompt = f"(arg: {self.arg}) "
elif self.paste_mode:
prompt = "(paste) "
elif "\n" in self.buffer:
Expand Down Expand Up @@ -515,12 +514,12 @@ def pos2xy(self) -> tuple[int, int]:
offset = l - 1 if in_wrapped_line else l # need to remove backslash
if offset >= pos:
break

if p + sum(l2) >= self.console.width:
pos -= l - 1 # -1 cause backslash is not in buffer
else:
if p + sum(l2) >= self.console.width:
pos -= l - 1 # -1 cause backslash is not in buffer
else:
pos -= l + 1 # +1 cause newline is in buffer
y += 1
pos -= l + 1 # +1 cause newline is in buffer
y += 1
return p + sum(l2[:pos]), y

def insert(self, text: str | list[str]) -> None:
Expand Down Expand Up @@ -582,7 +581,6 @@ def suspend(self) -> SimpleContextManager:
for arg in ("msg", "ps1", "ps2", "ps3", "ps4", "paste_mode"):
setattr(self, arg, prev_state[arg])
self.prepare()
pass

def finish(self) -> None:
"""Called when a command signals that we're finished."""
Expand Down
11 changes: 9 additions & 2 deletions Lib/_pyrepl/readline.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,14 @@

from . import commands, historical_reader
from .completing_reader import CompletingReader
from .unix_console import UnixConsole, _error
from .console import Console as ConsoleType

Console: type[ConsoleType]
_error: tuple[type[Exception], ...] | type[Exception]
try:
from .unix_console import UnixConsole as Console, _error
except ImportError:
from .windows_console import WindowsConsole as Console, _error

ENCODING = sys.getdefaultencoding() or "latin1"

Expand Down Expand Up @@ -328,7 +335,7 @@ def __post_init__(self) -> None:

def get_reader(self) -> ReadlineAlikeReader:
if self.reader is None:
console = UnixConsole(self.f_in, self.f_out, encoding=ENCODING)
console = Console(self.f_in, self.f_out, encoding=ENCODING)
self.reader = ReadlineAlikeReader(console=console, config=self.config)
return self.reader

Expand Down
6 changes: 5 additions & 1 deletion Lib/_pyrepl/simple_interact.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,12 @@
from types import ModuleType

from .readline import _get_reader, multiline_input
from .unix_console import _error

_error: tuple[type[Exception], ...] | type[Exception]
try:
from .unix_console import _error
except ModuleNotFoundError:
from .windows_console import _error

def check() -> str:
"""Returns the error message if there is a problem initializing the state."""
Expand Down
28 changes: 13 additions & 15 deletions Lib/_pyrepl/unix_console.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,18 +143,7 @@ def __init__(
- term (str): Terminal name.
- encoding (str): Encoding to use for I/O operations.
"""

self.encoding = encoding or sys.getdefaultencoding()

if isinstance(f_in, int):
self.input_fd = f_in
else:
self.input_fd = f_in.fileno()

if isinstance(f_out, int):
self.output_fd = f_out
else:
self.output_fd = f_out.fileno()
super().__init__(f_in, f_out, term, encoding)

self.pollob = poll()
self.pollob.register(self.input_fd, select.POLLIN)
Expand Down Expand Up @@ -592,14 +581,19 @@ def __write_changed_line(self, y, oldline, newline, px_coord):
px_pos = 0
j = 0
for c in oldline:
if j >= px_coord: break
if j >= px_coord:
break
j += wlen(c)
px_pos += 1

# reuse the oldline as much as possible, but stop as soon as we
# encounter an ESCAPE, because it might be the start of an escape
# sequene
while x_coord < minlen and oldline[x_pos] == newline[x_pos] and newline[x_pos] != "\x1b":
while (
x_coord < minlen
and oldline[x_pos] == newline[x_pos]
and newline[x_pos] != "\x1b"
):
x_coord += wlen(newline[x_pos])
x_pos += 1

Expand All @@ -619,7 +613,11 @@ def __write_changed_line(self, y, oldline, newline, px_coord):
self.__posxy = x_coord + character_width, y

# if it's a single character change in the middle of the line
elif x_coord < minlen and oldline[x_pos + 1 :] == newline[x_pos + 1 :] and wlen(oldline[x_pos]) == wlen(newline[x_pos]):
elif (
x_coord < minlen
and oldline[x_pos + 1 :] == newline[x_pos + 1 :]
and wlen(oldline[x_pos]) == wlen(newline[x_pos])
):
character_width = wlen(newline[x_pos])
self.__move(x_coord, y)
self.__write(newline[x_pos])
Expand Down
Loading
Loading