Skip to content

Commit

Permalink
gh-111201: A new Python REPL (GH-111567)
Browse files Browse the repository at this point in the history
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Marta Gómez Macías <mgmacias@google.com>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
  • Loading branch information
6 people authored May 5, 2024
1 parent 40cc809 commit f27f8c7
Show file tree
Hide file tree
Showing 41 changed files with 5,328 additions and 170 deletions.
4 changes: 3 additions & 1 deletion .github/workflows/mypy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ on:
pull_request:
paths:
- ".github/workflows/mypy.yml"
- "Lib/_pyrepl/**"
- "Lib/test/libregrtest/**"
- "Tools/build/generate_sbom.py"
- "Tools/cases_generator/**"
Expand Down Expand Up @@ -35,8 +36,9 @@ jobs:
strategy:
matrix:
target: [
"Lib/_pyrepl",
"Lib/test/libregrtest",
"Tools/build/",
"Tools/build",
"Tools/cases_generator",
"Tools/clinic",
"Tools/jit",
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/reusable-macos.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ jobs:
HOMEBREW_NO_INSTALL_CLEANUP: 1
HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK: 1
PYTHONSTRICTEXTENSIONBUILD: 1
TERM: linux
strategy:
fail-fast: false
matrix:
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/reusable-ubuntu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ jobs:
FORCE_COLOR: 1
OPENSSL_VER: 3.0.13
PYTHONSTRICTEXTENSIONBUILD: 1
TERM: linux
steps:
- uses: actions/checkout@v4
- name: Register gcc problem matcher
Expand Down
14 changes: 10 additions & 4 deletions Doc/glossary.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,14 @@ Glossary
.. glossary::

``>>>``
The default Python prompt of the interactive shell. Often seen for code
examples which can be executed interactively in the interpreter.
The default Python prompt of the :term:`interactive` shell. Often
seen for code examples which can be executed interactively in the
interpreter.

``...``
Can refer to:

* The default Python prompt of the interactive shell when entering the
* The default Python prompt of the :term:`interactive` shell when entering the
code for an indented code block, when within a pair of matching left and
right delimiters (parentheses, square brackets, curly braces or triple
quotes), or after specifying a decorator.
Expand Down Expand Up @@ -620,7 +621,8 @@ Glossary
execute them and see their results. Just launch ``python`` with no
arguments (possibly by selecting it from your computer's main
menu). It is a very powerful way to test out new ideas or inspect
modules and packages (remember ``help(x)``).
modules and packages (remember ``help(x)``). For more on interactive
mode, see :ref:`tut-interac`.

interpreted
Python is an interpreted language, as opposed to a compiled one,
Expand Down Expand Up @@ -1084,6 +1086,10 @@ Glossary

See also :term:`namespace package`.

REPL
An acronym for the "read–eval–print loop", another name for the
:term:`interactive` interpreter shell.

__slots__
A declaration inside a class that saves memory by pre-declaring space for
instance attributes and eliminating instance dictionaries. Though
Expand Down
22 changes: 22 additions & 0 deletions Doc/tutorial/appendix.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,28 @@ Appendix
Interactive Mode
================

There are two variants of the interactive :term:`REPL`. The classic
basic interpreter is supported on all platforms with minimal line
control capabilities.

On Unix-like systems (e.g. Linux or macOS) with :mod:`curses` and
:mod:`readline` support, a new interactive shell is used by default.
This one supports color, multiline editing, history browsing, and
paste mode. To disable color, see :ref:`using-on-controlling-color` for
details. Function keys provide some additional functionality.
:kbd:`F1` enters the interactive help browser :mod:`pydoc`.
:kbd:`F2` allows for browsing command-line history without output nor the
:term:`>>>` and :term:`...` prompts. :kbd:`F3` enters "paste mode", which
makes pasting larger blocks of code easier. Press :kbd:`F3` to return to
the regular prompt.

When using the new interactive shell, exit the shell by typing :kbd:`exit`
or :kbd:`quit`. Adding call parentheses after those commands is not
required.

If the new interactive shell is not desired, it can be disabled via
the :envvar:`PYTHON_BASIC_REPL` environment variable.

.. _tut-error:

Error Handling
Expand Down
10 changes: 10 additions & 0 deletions Doc/using/cmdline.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ additional methods of invocation:
* When called with standard input connected to a tty device, it prompts for
commands and executes them until an EOF (an end-of-file character, you can
produce that with :kbd:`Ctrl-D` on UNIX or :kbd:`Ctrl-Z, Enter` on Windows) is read.
For more on interactive mode, see :ref:`tut-interac`.
* When called with a file name argument or with a file as standard input, it
reads and executes a script from that file.
* When called with a directory name argument, it reads and executes an
Expand Down Expand Up @@ -1182,6 +1183,15 @@ conflict.

.. versionadded:: 3.13

.. envvar:: PYTHON_BASIC_REPL

If this variable is set to ``1``, the interpreter will not attempt to
load the Python-based :term:`REPL` that requires :mod:`curses` and
:mod:`readline`, and will instead use the traditional parser-based
:term:`REPL`.

.. versionadded:: 3.13

.. envvar:: PYTHON_HISTORY

This environment variable can be used to set the location of a
Expand Down
28 changes: 28 additions & 0 deletions Doc/whatsnew/3.13.rst
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,34 @@ New typing features:
New Features
============

A Better Interactive Interpreter
--------------------------------

On Unix-like systems like Linux or macOS, Python now uses a new
:term:`interactive` shell. When the user starts the :term:`REPL`
from a tty, and both :mod:`curses` and :mod:`readline` are available,
the interactive shell now supports the following new features:

* colorized prompts;
* multiline editing with history preservation;
* interactive help browsing using :kbd:`F1` with a separate command
history;
* history browsing using :kbd:`F2` that skips output as well as the
:term:`>>>` and :term:`...` prompts;
* "paste mode" with :kbd:`F3` that makes pasting larger blocks of code
easier (press :kbd:`F3` again to return to the regular prompt);
* ability to issue REPL-specific commands like :kbd:`help`, :kbd:`exit`,
and :kbd:`quit` without the need to use call parentheses after the
command name.

If the new interactive shell is not desired, it can be disabled via
the :envvar:`PYTHON_BASIC_REPL` environment variable.

For more on interactive mode, see :ref:`tut-interac`.

(Contributed by Pablo Galindo Salgado, Łukasz Langa, and
Lysandros Nikolaou in :gh:`111201` based on code from the PyPy project.)

Improved Error Messages
-----------------------

Expand Down
19 changes: 19 additions & 0 deletions Lib/_pyrepl/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Copyright 2000-2008 Michael Hudson-Doyle <micahel@gmail.com>
# Armin Rigo
#
# All Rights Reserved
#
#
# Permission to use, copy, modify, and distribute this software and
# its documentation for any purpose is hereby granted without fee,
# provided that the above copyright notice appear in all copies and
# that both that copyright notice and this permission notice appear in
# supporting documentation.
#
# THE AUTHOR MICHAEL HUDSON DISCLAIMS ALL WARRANTIES WITH REGARD TO
# THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
# AND FITNESS, IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL,
# INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER
# RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF
# CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
# CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
43 changes: 43 additions & 0 deletions Lib/_pyrepl/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
import os
import sys

CAN_USE_PYREPL = True

def interactive_console(mainmodule=None, quiet=False, pythonstartup=False):
global CAN_USE_PYREPL
if not CAN_USE_PYREPL:
return sys._baserepl()

startup_path = os.getenv("PYTHONSTARTUP")
if pythonstartup and startup_path:
import tokenize
with tokenize.open(startup_path) as f:
startup_code = compile(f.read(), startup_path, "exec")
exec(startup_code)

# set sys.{ps1,ps2} just before invoking the interactive interpreter. This
# mimics what CPython does in pythonrun.c
if not hasattr(sys, "ps1"):
sys.ps1 = ">>> "
if not hasattr(sys, "ps2"):
sys.ps2 = "... "
#
run_interactive = None
try:
import errno
if not os.isatty(sys.stdin.fileno()):
raise OSError(errno.ENOTTY, "tty required", "stdin")
from .simple_interact import check
if err := check():
raise RuntimeError(err)
from .simple_interact import run_multiline_interactive_console
run_interactive = run_multiline_interactive_console
except Exception as e:
print(f"warning: can't use pyrepl: {e}", file=sys.stderr)
CAN_USE_PYREPL = False
if run_interactive is None:
return sys._baserepl()
return run_interactive(mainmodule)

if __name__ == "__main__":
interactive_console()
68 changes: 68 additions & 0 deletions Lib/_pyrepl/_minimal_curses.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
"""Minimal '_curses' module, the low-level interface for curses module
which is not meant to be used directly.
Based on ctypes. It's too incomplete to be really called '_curses', so
to use it, you have to import it and stick it in sys.modules['_curses']
manually.
Note that there is also a built-in module _minimal_curses which will
hide this one if compiled in.
"""

import ctypes
import ctypes.util


class error(Exception):
pass


def _find_clib():
trylibs = ["ncursesw", "ncurses", "curses"]

for lib in trylibs:
path = ctypes.util.find_library(lib)
if path:
return path
raise ModuleNotFoundError("curses library not found", name="_pyrepl._minimal_curses")


_clibpath = _find_clib()
clib = ctypes.cdll.LoadLibrary(_clibpath)

clib.setupterm.argtypes = [ctypes.c_char_p, ctypes.c_int, ctypes.POINTER(ctypes.c_int)]
clib.setupterm.restype = ctypes.c_int

clib.tigetstr.argtypes = [ctypes.c_char_p]
clib.tigetstr.restype = ctypes.POINTER(ctypes.c_char)

clib.tparm.argtypes = [ctypes.c_char_p] + 9 * [ctypes.c_int] # type: ignore[operator]
clib.tparm.restype = ctypes.c_char_p

OK = 0
ERR = -1

# ____________________________________________________________


def setupterm(termstr, fd):
err = ctypes.c_int(0)
result = clib.setupterm(termstr, fd, ctypes.byref(err))
if result == ERR:
raise error("setupterm() failed (err=%d)" % err.value)


def tigetstr(cap):
if not isinstance(cap, bytes):
cap = cap.encode("ascii")
result = clib.tigetstr(cap)
if ctypes.cast(result, ctypes.c_void_p).value == ERR:
return None
return ctypes.cast(result, ctypes.c_char_p).value


def tparm(str, i1=0, i2=0, i3=0, i4=0, i5=0, i6=0, i7=0, i8=0, i9=0):
result = clib.tparm(str, i1, i2, i3, i4, i5, i6, i7, i8, i9)
if result is None:
raise error("tparm() returned NULL")
return result
Loading

0 comments on commit f27f8c7

Please sign in to comment.