Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-40222: "Zero cost" exception handling #25729

Merged
merged 45 commits into from
May 7, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
a414fa6
Add SETUP_CLEANUP bytecode to track lasti for re-raising exceptions i…
markshannon Apr 19, 2021
902690d
Move some handling of exceptions to bytecode from unwind code.
markshannon Apr 21, 2021
21646b1
Move setting up except block out of unwind code into bytecode.
markshannon Apr 21, 2021
9c9575a
Move cleanup code for try-finally and try-except in bytecode.
markshannon Apr 21, 2021
3be89db
Move cleanup code for with and sync with into bytecode.
markshannon Apr 21, 2021
cf664f7
Simplify exit code for async for loops.
markshannon Apr 21, 2021
722b722
Remove SETUP_EXCEPT opcode.
markshannon Apr 21, 2021
caa355f
Remove EXCEPT_HANDLER as a type of exception handling block.
markshannon Apr 21, 2021
4c21ddd
Factor out all block pops and pushes into simple opcodes.
markshannon Apr 21, 2021
6a8d90c
Generate exception handling table
markshannon Apr 22, 2021
9115cea
Parse and verify exception table when unwinding exceptions.
markshannon Apr 23, 2021
350334c
Update test to account for new code object constructor.
markshannon Apr 26, 2021
d6da9cb
Push lasti to stack when cleaning up exception handlers, and use it t…
markshannon Apr 26, 2021
a6b9292
Remove block stack from frames add convert SETUP/POP_BLOCK instructio…
markshannon Apr 26, 2021
9d0a6c1
Make dis understand the exception table.
markshannon Apr 29, 2021
ca482d0
Merge branch 'master' into zero-cost-except
markshannon Apr 29, 2021
4629a97
Set line numbers at start of exception handlers in try-except and try…
markshannon Apr 30, 2021
58b4a5a
Handle exceptions in trace functions correctly.
markshannon Apr 30, 2021
e7a9221
Fix syntax error test.
markshannon Apr 30, 2021
062ffa4
Get frame.setlineno working again.
markshannon Apr 30, 2021
71d95c1
Merge branch 'master' into zero-cost-except
markshannon Apr 30, 2021
e5c2c3e
Remove debugging prints
markshannon Apr 30, 2021
7571e90
Fix test_dis after merge
markshannon Apr 30, 2021
812e78f
Layout frameobject in most compact form.
markshannon Apr 30, 2021
5801089
Add NEWS.
markshannon Apr 30, 2021
8cb6366
Remove SETUP_FINALLY, SETUP_ASYNC_WITH, SETUP_CLEANUP, POP_BLOCK inst…
markshannon Apr 30, 2021
4ec3417
Update dis.rst
markshannon Apr 30, 2021
07cd46d
fix NEWS formatting
markshannon Apr 30, 2021
3b52a3f
Fix typos and clarify text.
markshannon Apr 30, 2021
594a636
Restore dis after discarded experiment.
markshannon Apr 30, 2021
585f306
Bump magic number for 3.11
markshannon Apr 30, 2021
0f11d34
Fix typo
markshannon Apr 30, 2021
63a9b0e
Merge branch 'master' into zero-cost-except
markshannon May 3, 2021
133f74f
Re-order optimization passes to remove more NOPs.
markshannon May 3, 2021
299e051
Update importlib yet again.
markshannon May 3, 2021
43c199d
Assorted clarifications.
markshannon May 6, 2021
630e755
Rename frame.setlineno assistant code to clarify that we now model th…
markshannon May 6, 2021
6333f6c
Revert mistaken change.
markshannon May 6, 2021
e1d6e1e
Update Include/opcode.h
markshannon May 6, 2021
73b600a
Extend notes on exception handling.
markshannon May 6, 2021
f263ee2
Fix refleaks
markshannon May 6, 2021
2f31c84
Merge branch 'main' into zero-cost-except
markshannon May 7, 2021
0c3fd95
Document removal frame push/pop C-API functions.
markshannon May 7, 2021
b92ada2
Update importlib
markshannon May 7, 2021
1878117
Clarify exception handling notes
markshannon May 7, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 27 additions & 31 deletions Doc/library/dis.rst
Original file line number Diff line number Diff line change
Expand Up @@ -616,13 +616,6 @@ the original TOS1.
.. versionadded:: 3.5


.. opcode:: SETUP_ASYNC_WITH

Creates a new frame object.

.. versionadded:: 3.5



**Miscellaneous opcodes**

Expand Down Expand Up @@ -692,28 +685,29 @@ iterations of the loop.
opcode implements ``from module import *``.


.. opcode:: POP_BLOCK

Removes one block from the block stack. Per frame, there is a stack of
blocks, denoting :keyword:`try` statements, and such.


.. opcode:: POP_EXCEPT

Removes one block from the block stack. The popped block must be an exception
handler block, as implicitly created when entering an except handler. In
addition to popping extraneous values from the frame stack, the last three
popped values are used to restore the exception state.
Pops three values from the stack, which are used to restore the exception state.


.. opcode:: RERAISE

Re-raises the exception currently on top of the stack. If oparg is non-zero,
restores ``f_lasti`` of the current frame to its value when the exception was raised.
pops an additional value from the stack which is used to set ``f_lasti``
of the current frame.

.. versionadded:: 3.9


.. opcode:: PUSH_EXC_INFO

Pops the three values from the stack. Pushes the current exception to the top of the stack.
Pushes the three values originally popped back to the stack.
Used in exception handlers.

.. versionadded:: 3.11


.. opcode:: WITH_EXCEPT_START

Calls the function in position 7 on the stack with the top three
Expand All @@ -724,6 +718,17 @@ iterations of the loop.
.. versionadded:: 3.9


.. opcode:: POP_EXCEPT_AND_RERAISE

Pops the exception currently on top of the stack. Pops the integer value on top
of the stack and sets the ``f_lasti`` attribute of the frame with that value.
Then pops the next exception from the stack uses it to restore the current exception.
Finally it re-raises the originally popped exception.
Used in excpetion handler cleanup.

.. versionadded:: 3.11


.. opcode:: LOAD_ASSERTION_ERROR

Pushes :exc:`AssertionError` onto the stack. Used by the :keyword:`assert`
Expand All @@ -738,18 +743,15 @@ iterations of the loop.
by :opcode:`CALL_FUNCTION` to construct a class.


.. opcode:: SETUP_WITH (delta)
.. opcode:: BEFORE_WITH (delta)

This opcode performs several operations before a with block starts. First,
it loads :meth:`~object.__exit__` from the context manager and pushes it onto
the stack for later use by :opcode:`WITH_EXCEPT_START`. Then,
:meth:`~object.__enter__` is called, and a finally block pointing to *delta*
is pushed. Finally, the result of calling the ``__enter__()`` method is pushed onto
the stack. The next opcode will either ignore it (:opcode:`POP_TOP`), or
store it in (a) variable(s) (:opcode:`STORE_FAST`, :opcode:`STORE_NAME`, or
:opcode:`UNPACK_SEQUENCE`).
:meth:`~object.__enter__` is called. Finally, the result of calling the
``__enter__()`` method is pushed onto the stack.

.. versionadded:: 3.2
.. versionadded:: 3.11


.. opcode:: COPY_DICT_WITHOUT_KEYS
Expand Down Expand Up @@ -1039,12 +1041,6 @@ All of the following opcodes use their arguments.
Loads the global named ``co_names[namei]`` onto the stack.


.. opcode:: SETUP_FINALLY (delta)

Pushes a try block from a try-finally or try-except clause onto the block
stack. *delta* points to the finally block or the first except block.


.. opcode:: LOAD_FAST (var_num)

Pushes a reference to the local ``co_varnames[var_num]`` onto the stack.
Expand Down
4 changes: 3 additions & 1 deletion Doc/whatsnew/3.11.rst
Original file line number Diff line number Diff line change
Expand Up @@ -91,10 +91,12 @@ Optimizations
=============




Build and C API Changes
=======================


* :c:func:`PyFrame_BlockSetup` and :c:func:`PyFrame_BlockPop` have been removed.

Deprecated
==========
Expand Down
5 changes: 3 additions & 2 deletions Include/cpython/code.h
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ struct PyCodeObject {
PyObject *co_name; /* unicode (name, for reference) */
PyObject *co_linetable; /* string (encoding addr<->lineno mapping) See
Objects/lnotab_notes.txt for details. */
PyObject *co_exceptiontable; /* Byte string encoding exception handling table */
void *co_zombieframe; /* for optimization only (see frameobject.c) */
PyObject *co_weakreflist; /* to support weakrefs to code objects */
/* Scratch space for extra data relating to the code object.
Expand Down Expand Up @@ -117,12 +118,12 @@ PyAPI_DATA(PyTypeObject) PyCode_Type;
PyAPI_FUNC(PyCodeObject *) PyCode_New(
int, int, int, int, int, PyObject *, PyObject *,
PyObject *, PyObject *, PyObject *, PyObject *,
PyObject *, PyObject *, int, PyObject *);
PyObject *, PyObject *, int, PyObject *, PyObject *);

PyAPI_FUNC(PyCodeObject *) PyCode_NewWithPosOnlyArgs(
int, int, int, int, int, int, PyObject *, PyObject *,
PyObject *, PyObject *, PyObject *, PyObject *,
PyObject *, PyObject *, int, PyObject *);
PyObject *, PyObject *, int, PyObject *, PyObject *);
Comment on lines 118 to +126
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This C-API change seems worth mentioning somewhere.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check the discussion in https://bugs.python.org/issue40222

/* same as struct above */

/* Creates a new empty code object with the specified source location. */
Expand Down
15 changes: 3 additions & 12 deletions Include/cpython/frameobject.h
Original file line number Diff line number Diff line change
Expand Up @@ -34,18 +34,14 @@ struct _frame {
PyObject *f_locals; /* local symbol table (any mapping) */
PyObject **f_valuestack; /* points after the last local */
PyObject *f_trace; /* Trace function */
int f_stackdepth; /* Depth of value stack */
char f_trace_lines; /* Emit per-line trace events? */
char f_trace_opcodes; /* Emit per-opcode trace events? */

/* Borrowed reference to a generator, or NULL */
PyObject *f_gen;

int f_stackdepth; /* Depth of value stack */
int f_lasti; /* Last instruction if called */
int f_lineno; /* Current line number. Only valid if non-zero */
int f_iblock; /* index in f_blockstack */
PyFrameState f_state; /* What state the frame is in */
PyTryBlock f_blockstack[CO_MAXBLOCKS]; /* for try and loop blocks */
char f_trace_lines; /* Emit per-line trace events? */
char f_trace_opcodes; /* Emit per-opcode trace events? */
PyObject *f_localsplus[1]; /* locals+stack, dynamically sized */
};

Expand Down Expand Up @@ -77,11 +73,6 @@ _PyFrame_New_NoTrack(PyThreadState *, PyFrameConstructor *, PyObject *);

/* The rest of the interface is specific for frame objects */

/* Block management functions */

PyAPI_FUNC(void) PyFrame_BlockSetup(PyFrameObject *, int, int, int);
PyAPI_FUNC(PyTryBlock *) PyFrame_BlockPop(PyFrameObject *);
Comment on lines -82 to -83
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that these were part of the public C-API, it would probably be worth adding a note to the What's New doc about their removal. I suppose the same may be true of f_blockstack, though it's less likely that matters.


/* Conversions between "fast locals" and locals in dictionary */

PyAPI_FUNC(void) PyFrame_LocalsToFast(PyFrameObject *, int);
Expand Down
25 changes: 11 additions & 14 deletions Include/opcode.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions Lib/ctypes/test/test_values.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,9 @@ class struct_frozen(Structure):
continue
items.append((entry.name.decode("ascii"), entry.size))

expected = [("__hello__", 137),
("__phello__", -137),
("__phello__.spam", 137),
expected = [("__hello__", 142),
("__phello__", -142),
("__phello__.spam", 142),
]
self.assertEqual(items, expected, "PyImport_FrozenModules example "
"in Doc/library/ctypes.rst may be out of date")
Expand Down
Loading