languages/cpython
languages/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython synced 2024-10-06 13:47:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

bpo-40222: "Zero cost" exception handling (GH-25729)

Browse source 
"Zero cost" exception handling.

* Uses a lookup table to determine how to handle exceptions.
* Removes SETUP_FINALLY and POP_TOP block instructions, eliminating (most of) the runtime overhead of try statements.
* Reduces the size of the frame object by about 60%.
This commit is contained in:
Mark Shannon 2021-05-07 15:19:19 +01:00 committed by GitHub
parent b32c8e9795
commit adcd220556
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
30 changed files with 6614 additions and 5687 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

58
Doc/library/dis.rst
View file

@ -616,13 +616,6 @@ the original TOS1.
.. versionadded:: 3.5
.. opcode:: SETUP_ASYNC_WITH
Creates a new frame object.
.. versionadded:: 3.5
**Miscellaneous opcodes**
@ -692,28 +685,29 @@ iterations of the loop.
opcode implements ``from module import *``.
.. opcode:: POP_BLOCK
Removes one block from the block stack. Per frame, there is a stack of
blocks, denoting :keyword:`try` statements, and such.
.. opcode:: POP_EXCEPT
Removes one block from the block stack. The popped block must be an exception
handler block, as implicitly created when entering an except handler. In
addition to popping extraneous values from the frame stack, the last three
popped values are used to restore the exception state.
Pops three values from the stack, which are used to restore the exception state.
.. opcode:: RERAISE
Re-raises the exception currently on top of the stack. If oparg is non-zero,
restores ``f_lasti`` of the current frame to its value when the exception was raised.
pops an additional value from the stack which is used to set ``f_lasti``
of the current frame.
.. versionadded:: 3.9
.. opcode:: PUSH_EXC_INFO
Pops the three values from the stack. Pushes the current exception to the top of the stack.
Pushes the three values originally popped back to the stack.
Used in exception handlers.
.. versionadded:: 3.11
.. opcode:: WITH_EXCEPT_START
Calls the function in position 7 on the stack with the top three
@ -724,6 +718,17 @@ iterations of the loop.
.. versionadded:: 3.9
.. opcode:: POP_EXCEPT_AND_RERAISE
Pops the exception currently on top of the stack. Pops the integer value on top
of the stack and sets the ``f_lasti`` attribute of the frame with that value.
Then pops the next exception from the stack uses it to restore the current exception.
Finally it re-raises the originally popped exception.
Used in excpetion handler cleanup.
.. versionadded:: 3.11
.. opcode:: LOAD_ASSERTION_ERROR
Pushes :exc:`AssertionError` onto the stack. Used by the :keyword:`assert`
@ -738,18 +743,15 @@ iterations of the loop.
by :opcode:`CALL_FUNCTION` to construct a class.
.. opcode:: SETUP_WITH (delta)
.. opcode:: BEFORE_WITH (delta)
This opcode performs several operations before a with block starts. First,
it loads :meth:`~object.__exit__` from the context manager and pushes it onto
the stack for later use by :opcode:`WITH_EXCEPT_START`. Then,
:meth:`~object.__enter__` is called, and a finally block pointing to *delta*
is pushed. Finally, the result of calling the ``__enter__()`` method is pushed onto
the stack. The next opcode will either ignore it (:opcode:`POP_TOP`), or
store it in (a) variable(s) (:opcode:`STORE_FAST`, :opcode:`STORE_NAME`, or
:opcode:`UNPACK_SEQUENCE`).
:meth:`~object.__enter__` is called. Finally, the result of calling the
``__enter__()`` method is pushed onto the stack.
.. versionadded:: 3.2
.. versionadded:: 3.11
.. opcode:: COPY_DICT_WITHOUT_KEYS
@ -1039,12 +1041,6 @@ All of the following opcodes use their arguments.
Loads the global named ``co_names[namei]`` onto the stack.
.. opcode:: SETUP_FINALLY (delta)
Pushes a try block from a try-finally or try-except clause onto the block
stack. *delta* points to the finally block or the first except block.
.. opcode:: LOAD_FAST (var_num)
Pushes a reference to the local ``co_varnames[var_num]`` onto the stack.

4
Doc/whatsnew/3.11.rst
View file

@ -91,10 +91,12 @@ Optimizations
=============
Build and C API Changes
=======================
* :c:func:`PyFrame_BlockSetup` and :c:func:`PyFrame_BlockPop` have been removed.
Deprecated
==========

5
Include/cpython/code.h
View file

@ -40,6 +40,7 @@ struct PyCodeObject {
PyObject *co_name; /* unicode (name, for reference) */
PyObject *co_linetable; /* string (encoding addr<->lineno mapping) See
Objects/lnotab_notes.txt for details. */
PyObject *co_exceptiontable; /* Byte string encoding exception handling table */
void *co_zombieframe; /* for optimization only (see frameobject.c) */
PyObject *co_weakreflist; /* to support weakrefs to code objects */
/* Scratch space for extra data relating to the code object.
@ -117,12 +118,12 @@ PyAPI_DATA(PyTypeObject) PyCode_Type;
PyAPI_FUNC(PyCodeObject *) PyCode_New(
int, int, int, int, int, PyObject *, PyObject *,
PyObject *, PyObject *, PyObject *, PyObject *,
PyObject *, PyObject *, int, PyObject *);
PyObject *, PyObject *, int, PyObject *, PyObject *);
PyAPI_FUNC(PyCodeObject *) PyCode_NewWithPosOnlyArgs(
int, int, int, int, int, int, PyObject *, PyObject *,
PyObject *, PyObject *, PyObject *, PyObject *,
PyObject *, PyObject *, int, PyObject *);
PyObject *, PyObject *, int, PyObject *, PyObject *);
/* same as struct above */
/* Creates a new empty code object with the specified source location. */

15
Include/cpython/frameobject.h
View file

@ -34,18 +34,14 @@ struct _frame {
PyObject *f_locals; /* local symbol table (any mapping) */
PyObject **f_valuestack; /* points after the last local */
PyObject *f_trace; /* Trace function */
int f_stackdepth; /* Depth of value stack */
char f_trace_lines; /* Emit per-line trace events? */
char f_trace_opcodes; /* Emit per-opcode trace events? */
/* Borrowed reference to a generator, or NULL */
PyObject *f_gen;
int f_stackdepth; /* Depth of value stack */
int f_lasti; /* Last instruction if called */
int f_lineno; /* Current line number. Only valid if non-zero */
int f_iblock; /* index in f_blockstack */
PyFrameState f_state; /* What state the frame is in */
PyTryBlock f_blockstack[CO_MAXBLOCKS]; /* for try and loop blocks */
char f_trace_lines; /* Emit per-line trace events? */
char f_trace_opcodes; /* Emit per-opcode trace events? */
PyObject *f_localsplus[1]; /* locals+stack, dynamically sized */
};
@ -77,11 +73,6 @@ _PyFrame_New_NoTrack(PyThreadState *, PyFrameConstructor *, PyObject *);
/* The rest of the interface is specific for frame objects */
/* Block management functions */
PyAPI_FUNC(void) PyFrame_BlockSetup(PyFrameObject *, int, int, int);
PyAPI_FUNC(PyTryBlock *) PyFrame_BlockPop(PyFrameObject *);
/* Conversions between "fast locals" and locals in dictionary */
PyAPI_FUNC(void) PyFrame_LocalsToFast(PyFrameObject *, int);

25
Include/opcode.h generated
View file

@ -35,10 +35,13 @@ extern "C" {
#define MATCH_SEQUENCE 32
#define MATCH_KEYS 33
#define COPY_DICT_WITHOUT_KEYS 34
#define PUSH_EXC_INFO 35
#define POP_EXCEPT_AND_RERAISE 37
#define WITH_EXCEPT_START 49
#define GET_AITER 50
#define GET_ANEXT 51
#define BEFORE_ASYNC_WITH 52
#define BEFORE_WITH 53
#define END_ASYNC_FOR 54
#define INPLACE_ADD 55
#define INPLACE_SUBTRACT 56
@ -69,7 +72,6 @@ extern "C" {
#define IMPORT_STAR 84
#define SETUP_ANNOTATIONS 85
#define YIELD_VALUE 86
#define POP_BLOCK 87
#define POP_EXCEPT 89
#define HAVE_ARGUMENT 90
#define STORE_NAME 90
@ -103,7 +105,6 @@ extern "C" {
#define CONTAINS_OP 118
#define RERAISE 119
#define JUMP_IF_NOT_EXC_MATCH 121
#define SETUP_FINALLY 122
#define LOAD_FAST 124
#define STORE_FAST 125
#define DELETE_FAST 126
@ -118,14 +119,12 @@ extern "C" {
#define DELETE_DEREF 138
#define CALL_FUNCTION_KW 141
#define CALL_FUNCTION_EX 142
#define SETUP_WITH 143
#define EXTENDED_ARG 144
#define LIST_APPEND 145
#define SET_ADD 146
#define MAP_ADD 147
#define LOAD_CLASSDEREF 148
#define MATCH_CLASS 152
#define SETUP_ASYNC_WITH 154
#define FORMAT_VALUE 155
#define BUILD_CONST_KEY_MAP 156
#define BUILD_STRING 157
@ -140,8 +139,8 @@ static uint32_t _PyOpcode_RelativeJump[8] = {
0U,
0U,
536870912U,
67125248U,
67141632U,
16384U,
0U,
0U,
0U,
0U,
@ -150,22 +149,20 @@ static uint32_t _PyOpcode_Jump[8] = {
0U,
0U,
536870912U,
101695488U,
67141632U,
34586624U,
0U,
0U,
0U,
0U,
};
#endif /* OPCODE_TABLES */
/* EXCEPT_HANDLER is a special, implicit block type which is created when
entering an except handler. It is not an opcode but we define it here
as we want it to be available to both frameobject.c and ceval.c, while
remaining private.*/
#define EXCEPT_HANDLER 257
#define HAS_ARG(op) ((op) >= HAVE_ARGUMENT)
/* Reserve some bytecodes for internal use in the compiler.
* The value of 240 is arbitrary. */
#define IS_ARTIFICIAL(op) ((op) > 240)
#ifdef __cplusplus
}
#endif

6
Lib/ctypes/test/test_values.py
View file

@ -80,9 +80,9 @@ class struct_frozen(Structure):
continue
items.append((entry.name.decode("ascii"), entry.size))
expected = [("__hello__", 137),
("__phello__", -137),
("__phello__.spam", 137),
expected = [("__hello__", 142),
("__phello__", -142),
("__phello__.spam", 142),
]
self.assertEqual(items, expected, "PyImport_FrozenModules example "
"in Doc/library/ctypes.rst may be out of date")

56
Lib/dis.py
View file

@ -203,6 +203,9 @@ def show_code(co, *, file=None):
_Instruction.starts_line.__doc__ = "Line started by this opcode (if any), otherwise None"
_Instruction.is_jump_target.__doc__ = "True if other code jumps to here, otherwise False"
_ExceptionTableEntry = collections.namedtuple("_ExceptionTableEntry",
"start end target depth lasti")
_OPNAME_WIDTH = 20
_OPARG_WIDTH = 5
@ -308,8 +311,33 @@ def _get_name_info(name_index, name_list):
return argval, argrepr
def parse_varint(iterator):
b = next(iterator)
val = b & 63
while b&64:
val <<= 6
b = next(iterator)
val |= b&63
return val
def parse_exception_table(code):
iterator = iter(code.co_exceptiontable)
entries = []
try:
while True:
start = parse_varint(iterator)*2
length = parse_varint(iterator)*2
end = start + length
target = parse_varint(iterator)*2
dl = parse_varint(iterator)
depth = dl >> 1
lasti = bool(dl&1)
entries.append(_ExceptionTableEntry(start, end, target, depth, lasti))
except StopIteration:
return entries
def _get_instructions_bytes(code, varnames=None, names=None, constants=None,
cells=None, linestarts=None, line_offset=0):
cells=None, linestarts=None, line_offset=0, exception_entries=()):
"""Iterate over the instructions in a bytecode string.
Generates a sequence of Instruction namedtuples giving the details of each
@ -318,7 +346,10 @@ def _get_instructions_bytes(code, varnames=None, names=None, constants=None,
arguments.
"""
labels = findlabels(code)
labels = set(findlabels(code))
for start, end, target, _, _ in exception_entries:
for i in range(start, end):
labels.add(target)
starts_line = None
for offset, op, arg in _unpack_opargs(code):
if linestarts is not None:
@ -369,8 +400,10 @@ def disassemble(co, lasti=-1, *, file=None):
"""Disassemble a code object."""
cell_names = co.co_cellvars + co.co_freevars
linestarts = dict(findlinestarts(co))
exception_entries = parse_exception_table(co)
_disassemble_bytes(co.co_code, lasti, co.co_varnames, co.co_names,
co.co_consts, cell_names, linestarts, file=file)
co.co_consts, cell_names, linestarts, file=file,
exception_entries=exception_entries)
def _disassemble_recursive(co, *, file=None, depth=None):
disassemble(co, file=file)
@ -385,7 +418,7 @@ def _disassemble_recursive(co, *, file=None, depth=None):
def _disassemble_bytes(code, lasti=-1, varnames=None, names=None,
constants=None, cells=None, linestarts=None,
*, file=None, line_offset=0):
*, file=None, line_offset=0, exception_entries=()):
# Omit the line number column entirely if we have no line number info
show_lineno = bool(linestarts)
if show_lineno:
@ -403,7 +436,7 @@ def _disassemble_bytes(code, lasti=-1, varnames=None, names=None,
offset_width = 4
for instr in _get_instructions_bytes(code, varnames, names,
constants, cells, linestarts,
line_offset=line_offset):
line_offset=line_offset, exception_entries=exception_entries):
new_source_line = (show_lineno and
instr.starts_line is not None and
instr.offset > 0)
@ -412,6 +445,12 @@ def _disassemble_bytes(code, lasti=-1, varnames=None, names=None,
is_current_instr = instr.offset == lasti
print(instr._disassemble(lineno_width, is_current_instr, offset_width),
file=file)
if exception_entries:
print("ExceptionTable:", file=file)
for entry in exception_entries:
lasti = " lasti" if entry.lasti else ""
end = entry.end-2
print(f" {entry.start} to {end} -> {entry.target} [{entry.depth}]{lasti}", file=file)
def _disassemble_str(source, **kwargs):
"""Compile the source string, then disassemble the code object."""
@ -482,13 +521,15 @@ def __init__(self, x, *, first_line=None, current_offset=None):
self._linestarts = dict(findlinestarts(co))
self._original_object = x
self.current_offset = current_offset
self.exception_entries = parse_exception_table(co)
def __iter__(self):
co = self.codeobj
return _get_instructions_bytes(co.co_code, co.co_varnames, co.co_names,
co.co_consts, self._cell_names,
self._linestarts,
line_offset=self._line_offset)
line_offset=self._line_offset,
exception_entries=self.exception_entries)
def __repr__(self):
return "{}({!r})".format(self.__class__.__name__,
@ -519,7 +560,8 @@ def dis(self):
linestarts=self._linestarts,
line_offset=self._line_offset,
file=output,
lasti=offset)
lasti=offset,
exception_entries=self.exception_entries)
return output.getvalue()

3
Lib/importlib/_bootstrap_external.py
View file

@ -352,6 +352,7 @@ def _write_atomic(path, data, mode=0o666):
# Python 3.10b1 3437 (Undo making 'annotations' future by default - We like to dance among core devs!)
# Python 3.10b1 3438 Safer line number table handling.
# Python 3.10b1 3439 (Add ROT_N)
# Python 3.11a1 3450 Use exception table for unwinding ("zero cost" exception handling)
#
# MAGIC must change whenever the bytecode emitted by the compiler may no
@ -361,7 +362,7 @@ def _write_atomic(path, data, mode=0o666):
# Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array
# in PC/launcher.c must also be updated.
MAGIC_NUMBER = (3439).to_bytes(2, 'little') + b'\r\n'
MAGIC_NUMBER = (3450).to_bytes(2, 'little') + b'\r\n'
_RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c
_PYCACHE = '__pycache__'

9
Lib/opcode.py
View file

@ -86,11 +86,15 @@ def jabs_op(name, op):
def_op('MATCH_SEQUENCE', 32)
def_op('MATCH_KEYS', 33)
def_op('COPY_DICT_WITHOUT_KEYS', 34)
def_op('PUSH_EXC_INFO', 35)
def_op('POP_EXCEPT_AND_RERAISE', 37)
def_op('WITH_EXCEPT_START', 49)
def_op('GET_AITER', 50)
def_op('GET_ANEXT', 51)
def_op('BEFORE_ASYNC_WITH', 52)
def_op('BEFORE_WITH', 53)
def_op('END_ASYNC_FOR', 54)
def_op('INPLACE_ADD', 55)
@ -124,7 +128,6 @@ def jabs_op(name, op):
def_op('IMPORT_STAR', 84)
def_op('SETUP_ANNOTATIONS', 85)
def_op('YIELD_VALUE', 86)
def_op('POP_BLOCK', 87)
def_op('POP_EXCEPT', 89)
@ -164,7 +167,6 @@ def jabs_op(name, op):
def_op('RERAISE', 119)
jabs_op('JUMP_IF_NOT_EXC_MATCH', 121)
jrel_op('SETUP_FINALLY', 122) # Distance to target address
def_op('LOAD_FAST', 124) # Local variable number
haslocal.append(124)
@ -190,7 +192,7 @@ def jabs_op(name, op):
def_op('CALL_FUNCTION_KW', 141) # #args + #kwargs
def_op('CALL_FUNCTION_EX', 142) # Flags
jrel_op('SETUP_WITH', 143)
def_op('EXTENDED_ARG', 144)
EXTENDED_ARG = 144
def_op('LIST_APPEND', 145)
@ -201,7 +203,6 @@ def jabs_op(name, op):
def_op('MATCH_CLASS', 152)
jrel_op('SETUP_ASYNC_WITH', 154)
def_op('FORMAT_VALUE', 155)
def_op('BUILD_CONST_KEY_MAP', 156)
def_op('BUILD_STRING', 157)

1
Lib/test/test_code.py
View file

@ -227,6 +227,7 @@ def func(): pass
co.co_name,
co.co_firstlineno,
co.co_lnotab,
co.co_exceptiontable,
co.co_freevars,
co.co_cellvars)

179
Lib/test/test_dis.py
View file

@ -282,42 +282,47 @@ def bug42562():
"""
dis_traceback = """\
%3d 0 SETUP_FINALLY 7 (to 16)
%3d 0 NOP
%3d 2 LOAD_CONST 1 (1)
4 LOAD_CONST 2 (0)
--> 6 BINARY_TRUE_DIVIDE
8 POP_TOP
10 POP_BLOCK
%3d 12 LOAD_FAST 1 (tb)
14 RETURN_VALUE
%3d 10 LOAD_FAST 1 (tb)
12 RETURN_VALUE
>> 14 PUSH_EXC_INFO
%3d >> 16 DUP_TOP
%3d 16 DUP_TOP
18 LOAD_GLOBAL 0 (Exception)
20 JUMP_IF_NOT_EXC_MATCH 29 (to 58)
20 JUMP_IF_NOT_EXC_MATCH 28 (to 56)
22 POP_TOP
24 STORE_FAST 0 (e)
26 POP_TOP
28 SETUP_FINALLY 10 (to 50)
%3d 30 LOAD_FAST 0 (e)
32 LOAD_ATTR 1 (__traceback__)
34 STORE_FAST 1 (tb)
36 POP_BLOCK
38 POP_EXCEPT
40 LOAD_CONST 0 (None)
42 STORE_FAST 0 (e)
44 DELETE_FAST 0 (e)
%3d 28 LOAD_FAST 0 (e)
30 LOAD_ATTR 1 (__traceback__)
32 STORE_FAST 1 (tb)
34 POP_EXCEPT
36 LOAD_CONST 0 (None)
38 STORE_FAST 0 (e)
40 DELETE_FAST 0 (e)
%3d 46 LOAD_FAST 1 (tb)
48 RETURN_VALUE
>> 50 LOAD_CONST 0 (None)
52 STORE_FAST 0 (e)
54 DELETE_FAST 0 (e)
56 RERAISE 1
%3d 42 LOAD_FAST 1 (tb)
44 RETURN_VALUE
>> 46 LOAD_CONST 0 (None)
48 STORE_FAST 0 (e)
50 DELETE_FAST 0 (e)
52 RERAISE 1
>> 54 POP_EXCEPT_AND_RERAISE
%3d >> 58 RERAISE 0
%3d >> 56 RERAISE 0
ExceptionTable:
2 to 8 -> 14 [0]
14 to 26 -> 54 [3] lasti
28 to 32 -> 46 [3] lasti
46 to 52 -> 54 [3] lasti
56 to 56 -> 54 [3] lasti
""" % (TRACEBACK_CODE.co_firstlineno + 1,
TRACEBACK_CODE.co_firstlineno + 2,
TRACEBACK_CODE.co_firstlineno + 5,
@ -360,38 +365,46 @@ def _tryfinallyconst(b):
b()
dis_tryfinally = """\
%3d 0 SETUP_FINALLY 6 (to 14)
%3d 0 NOP
%3d 2 LOAD_FAST 0 (a)
4 POP_BLOCK
%3d 6 LOAD_FAST 1 (b)
8 CALL_FUNCTION 0
10 POP_TOP
12 RETURN_VALUE
>> 14 LOAD_FAST 1 (b)
%3d 4 LOAD_FAST 1 (b)
6 CALL_FUNCTION 0
8 POP_TOP
10 RETURN_VALUE
>> 12 PUSH_EXC_INFO
14 LOAD_FAST 1 (b)
16 CALL_FUNCTION 0
18 POP_TOP
20 RERAISE 0
>> 22 POP_EXCEPT_AND_RERAISE
ExceptionTable:
2 to 2 -> 12 [0]
12 to 20 -> 22 [3] lasti
""" % (_tryfinally.__code__.co_firstlineno + 1,
_tryfinally.__code__.co_firstlineno + 2,
_tryfinally.__code__.co_firstlineno + 4,
)
dis_tryfinallyconst = """\
%3d 0 SETUP_FINALLY 6 (to 14)
%3d 0 NOP
%3d 2 POP_BLOCK
%3d 2 NOP
%3d 4 LOAD_FAST 0 (b)
6 CALL_FUNCTION 0
8 POP_TOP
10 LOAD_CONST 1 (1)
12 RETURN_VALUE
>> 14 LOAD_FAST 0 (b)
16 CALL_FUNCTION 0
18 POP_TOP
20 RERAISE 0
14 PUSH_EXC_INFO
16 LOAD_FAST 0 (b)
18 CALL_FUNCTION 0
20 POP_TOP
22 RERAISE 0
>> 24 POP_EXCEPT_AND_RERAISE
ExceptionTable:
14 to 22 -> 24 [3] lasti
""" % (_tryfinallyconst.__code__.co_firstlineno + 1,
_tryfinallyconst.__code__.co_firstlineno + 2,
_tryfinallyconst.__code__.co_firstlineno + 4,
@ -833,7 +846,7 @@ async def async_def():
Positional-only arguments: 0
Kw-only arguments: 0
Number of locals: 2
Stack size: 9
Stack size: 10
Flags: OPTIMIZED, NEWLOCALS, NOFREE, COROUTINE
Constants:
0: None
@ -1059,61 +1072,64 @@ def jumpy():
Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Who let lolcatz into this test suite?', argrepr="'Who let lolcatz into this test suite?'", offset=98, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=100, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=102, starts_line=None, is_jump_target=False),
Instruction(opname='SETUP_FINALLY', opcode=122, arg=48, argval=202, argrepr='to 202', offset=104, starts_line=20, is_jump_target=True),
Instruction(opname='SETUP_FINALLY', opcode=122, arg=6, argval=120, argrepr='to 120', offset=106, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=108, starts_line=21, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval=0, argrepr='0', offset=110, starts_line=None, is_jump_target=False),
Instruction(opname='BINARY_TRUE_DIVIDE', opcode=27, arg=None, argval=None, argrepr='', offset=112, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=114, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=116, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=12, argval=144, argrepr='to 144', offset=118, starts_line=None, is_jump_target=False),
Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=120, starts_line=22, is_jump_target=True),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=2, argval='ZeroDivisionError', argrepr='ZeroDivisionError', offset=122, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_IF_NOT_EXC_MATCH', opcode=121, arg=106, argval=212, argrepr='to 212', offset=124, starts_line=None, is_jump_target=False),
Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=104, starts_line=20, is_jump_target=True),
Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=106, starts_line=21, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval=0, argrepr='0', offset=108, starts_line=None, is_jump_target=False),
Instruction(opname='BINARY_TRUE_DIVIDE', opcode=27, arg=None, argval=None, argrepr='', offset=110, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=112, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=14, argval=144, argrepr='to 144', offset=114, starts_line=None, is_jump_target=False),
Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=116, starts_line=None, is_jump_target=False),
Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=118, starts_line=22, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=2, argval='ZeroDivisionError', argrepr='ZeroDivisionError', offset=120, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_IF_NOT_EXC_MATCH', opcode=121, arg=109, argval=218, argrepr='to 218', offset=122, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=124, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=126, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=128, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=130, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=132, starts_line=23, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=134, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=136, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=138, starts_line=None, is_jump_target=False),
Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=140, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=22, argval=188, argrepr='to 188', offset=142, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=130, starts_line=23, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=132, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=134, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=136, starts_line=None, is_jump_target=False),
Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=138, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=25, argval=192, argrepr='to 192', offset=140, starts_line=None, is_jump_target=False),
Instruction(opname='POP_EXCEPT_AND_RERAISE', opcode=37, arg=None, argval=None, argrepr='', offset=142, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=144, starts_line=25, is_jump_target=True),
Instruction(opname='SETUP_WITH', opcode=143, arg=12, argval=172, argrepr='to 172', offset=146, starts_line=None, is_jump_target=False),
Instruction(opname='BEFORE_WITH', opcode=53, arg=None, argval=None, argrepr='', offset=146, starts_line=None, is_jump_target=False),
Instruction(opname='STORE_FAST', opcode=125, arg=1, argval='dodgy', argrepr='dodgy', offset=148, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=150, starts_line=26, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Never reach this', argrepr="'Never reach this'", offset=152, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=154, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=156, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=158, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=160, starts_line=25, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=158, starts_line=25, is_jump_target=False),
Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=160, starts_line=None, is_jump_target=False),
Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=162, starts_line=None, is_jump_target=False),
Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=164, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=3, argval=3, argrepr='', offset=166, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=168, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=8, argval=188, argrepr='to 188', offset=170, starts_line=None, is_jump_target=False),
Instruction(opname='WITH_EXCEPT_START', opcode=49, arg=None, argval=None, argrepr='', offset=172, starts_line=None, is_jump_target=True),
Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=89, argval=178, argrepr='to 178', offset=174, starts_line=None, is_jump_target=False),
Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=176, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=178, starts_line=None, is_jump_target=True),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=180, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=3, argval=3, argrepr='', offset=164, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=166, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=11, argval=192, argrepr='to 192', offset=168, starts_line=None, is_jump_target=False),
Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=170, starts_line=None, is_jump_target=False),
Instruction(opname='WITH_EXCEPT_START', opcode=49, arg=None, argval=None, argrepr='', offset=172, starts_line=None, is_jump_target=False),
Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=90, argval=180, argrepr='to 180', offset=174, starts_line=None, is_jump_target=False),
Instruction(opname='RERAISE', opcode=119, arg=4, argval=4, argrepr='', offset=176, starts_line=None, is_jump_target=False),
Instruction(opname='POP_EXCEPT_AND_RERAISE', opcode=37, arg=None, argval=None, argrepr='', offset=178, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=180, starts_line=None, is_jump_target=True),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=182, starts_line=None, is_jump_target=False),
Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=184, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=186, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=188, starts_line=None, is_jump_target=True),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=190, starts_line=28, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=192, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=194, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=196, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=198, starts_line=None, is_jump_target=False),
Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=200, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=202, starts_line=None, is_jump_target=True),
Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=204, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=206, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=208, starts_line=None, is_jump_target=False),
Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=210, starts_line=None, is_jump_target=False),
Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=212, starts_line=22, is_jump_target=True),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=184, starts_line=None, is_jump_target=False),
Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=186, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=188, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=190, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=192, starts_line=28, is_jump_target=True),
Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=194, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=196, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=198, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=200, starts_line=None, is_jump_target=False),
Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=202, starts_line=None, is_jump_target=False),
Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=204, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=206, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=208, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=210, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=212, starts_line=None, is_jump_target=False),
Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=214, starts_line=None, is_jump_target=False),
Instruction(opname='POP_EXCEPT_AND_RERAISE', opcode=37, arg=None, argval=None, argrepr='', offset=216, starts_line=None, is_jump_target=False),
Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=218, starts_line=22, is_jump_target=True),
]
# One last piece of inspect fodder to check the default line number handling
@ -1211,6 +1227,7 @@ def test_from_traceback(self):
self.assertEqual(b.current_offset, tb.tb_lasti)
def test_from_traceback_dis(self):
self.maxDiff = None
tb = get_tb()
b = dis.Bytecode.from_traceback(tb)
self.assertEqual(b.dis(), dis_traceback)

2
Lib/test/test_exceptions.py
View file

@ -218,7 +218,7 @@ def testSyntaxErrorOffset(self):
check('class foo:return 1', 1, 11)
check('def f():\n continue', 2, 3)
check('def f():\n break', 2, 3)
check('try:\n pass\nexcept:\n pass\nexcept ValueError:\n pass', 2, 3)
check('try:\n pass\nexcept:\n pass\nexcept ValueError:\n pass', 3, 1)
# Errors thrown by tokenizer.c
check('(0x+1)', 1, 3)

7
Lib/test/test_sys.py
View file

@ -1273,13 +1273,12 @@ class C(object): pass
check(sys.float_info, vsize('') + self.P * len(sys.float_info))
# frame
import inspect
CO_MAXBLOCKS = 20
x = inspect.currentframe()
ncells = len(x.f_code.co_cellvars)
nfrees = len(x.f_code.co_freevars)
extras = x.f_code.co_stacksize + x.f_code.co_nlocals +\
ncells + nfrees - 1
check(x, vsize('4Pi2c4P3ic' + CO_MAXBLOCKS*'3i' + 'P' + extras*'P'))
localsplus = x.f_code.co_stacksize + x.f_code.co_nlocals +\
ncells + nfrees
check(x, vsize('8P3i3c' + localsplus*'P'))
# function
def func(): pass
check(func, size('14P'))

56
Lib/test/test_sys_settrace.py
View file

@ -1253,7 +1253,7 @@ def test_jump_out_of_block_backwards(output):
output.append(6)
output.append(7)
@async_jump_test(4, 5, [3, 5])
@async_jump_test(4, 5, [3], (ValueError, 'into'))
async def test_jump_out_of_async_for_block_forwards(output):
for i in [1]:
async for i in asynciter([1, 2]):
@ -1295,7 +1295,7 @@ def test_jump_in_nested_finally(output):
output.append(8)
output.append(9)
@jump_test(6, 7, [2, 7], (ZeroDivisionError, ''))
@jump_test(6, 7, [2], (ValueError, 'within'))
def test_jump_in_nested_finally_2(output):
try:
output.append(2)
@ -1306,7 +1306,7 @@ def test_jump_in_nested_finally_2(output):
output.append(7)
output.append(8)
@jump_test(6, 11, [2, 11], (ZeroDivisionError, ''))
@jump_test(6, 11, [2], (ValueError, 'within'))
def test_jump_in_nested_finally_3(output):
try:
output.append(2)
@ -1321,7 +1321,7 @@ def test_jump_in_nested_finally_3(output):
output.append(11)
output.append(12)
@jump_test(5, 11, [2, 4], (ValueError, 'after'))
@jump_test(5, 11, [2, 4], (ValueError, 'exception'))
def test_no_jump_over_return_try_finally_in_finally_block(output):
try:
output.append(2)
@ -1417,8 +1417,8 @@ def test_jump_backwards_out_of_try_except_block(output):
output.append(5)
raise
@jump_test(5, 7, [4, 7, 8])
def test_jump_between_except_blocks(output):
@jump_test(5, 7, [4], (ValueError, 'within'))
def test_no_jump_between_except_blocks(output):
try:
1/0
except ZeroDivisionError:
@ -1428,8 +1428,8 @@ def test_jump_between_except_blocks(output):
output.append(7)
output.append(8)
@jump_test(5, 6, [4, 6, 7])
def test_jump_within_except_block(output):
@jump_test(5, 6, [4], (ValueError, 'within'))
def test_no_jump_within_except_block(output):
try:
1/0
except:
@ -1654,54 +1654,54 @@ def test_no_jump_backwards_into_for_block(output):
output.append(2)
output.append(3)
@async_jump_test(3, 2, [2, 2], (ValueError, 'into'))
@async_jump_test(3, 2, [2, 2], (ValueError, 'within'))
async def test_no_jump_backwards_into_async_for_block(output):
async for i in asynciter([1, 2]):
output.append(2)
output.append(3)
@jump_test(1, 3, [], (ValueError, 'into'))
@jump_test(1, 3, [], (ValueError, 'depth'))
def test_no_jump_forwards_into_with_block(output):
output.append(1)
with tracecontext(output, 2):
output.append(3)
@async_jump_test(1, 3, [], (ValueError, 'into'))
@async_jump_test(1, 3, [], (ValueError, 'depth'))
async def test_no_jump_forwards_into_async_with_block(output):
output.append(1)
async with asynctracecontext(output, 2):
output.append(3)
@jump_test(3, 2, [1, 2, -1], (ValueError, 'into'))
@jump_test(3, 2, [1, 2, -1], (ValueError, 'depth'))
def test_no_jump_backwards_into_with_block(output):
with tracecontext(output, 1):
output.append(2)
output.append(3)
@async_jump_test(3, 2, [1, 2, -1], (ValueError, 'into'))
@async_jump_test(3, 2, [1, 2, -1], (ValueError, 'depth'))
async def test_no_jump_backwards_into_async_with_block(output):
async with asynctracecontext(output, 1):
output.append(2)
output.append(3)
@jump_test(1, 3, [], (ValueError, 'into'))
def test_no_jump_forwards_into_try_finally_block(output):
@jump_test(1, 3, [3, 5])
def test_jump_forwards_into_try_finally_block(output):
output.append(1)
try:
output.append(3)
finally:
output.append(5)
@jump_test(5, 2, [2, 4], (ValueError, 'into'))
def test_no_jump_backwards_into_try_finally_block(output):
@jump_test(5, 2, [2, 4, 2, 4, 5])
def test_jump_backwards_into_try_finally_block(output):
try:
output.append(2)
finally:
output.append(4)
output.append(5)
@jump_test(1, 3, [], (ValueError, 'into'))
def test_no_jump_forwards_into_try_except_block(output):
@jump_test(1, 3, [3])
def test_jump_forwards_into_try_except_block(output):
output.append(1)
try:
output.append(3)
@ -1709,8 +1709,8 @@ def test_no_jump_forwards_into_try_except_block(output):
output.append(5)
raise
@jump_test(6, 2, [2], (ValueError, 'into'))
def test_no_jump_backwards_into_try_except_block(output):
@jump_test(6, 2, [2, 2, 6])
def test_jump_backwards_into_try_except_block(output):
try:
output.append(2)
except:
@ -1719,7 +1719,7 @@ def test_no_jump_backwards_into_try_except_block(output):
output.append(6)
# 'except' with a variable creates an implicit finally block
@jump_test(5, 7, [4], (ValueError, 'into'))
@jump_test(5, 7, [4], (ValueError, 'within'))
def test_no_jump_between_except_blocks_2(output):
try:
1/0
@ -1756,7 +1756,7 @@ def test_jump_out_of_finally_block(output):
finally:
output.append(5)
@jump_test(1, 5, [], (ValueError, "into an 'except'"))
@jump_test(1, 5, [], (ValueError, "into an exception"))
def test_no_jump_into_bare_except_block(output):
output.append(1)
try:
@ -1764,7 +1764,7 @@ def test_no_jump_into_bare_except_block(output):
except:
output.append(5)
@jump_test(1, 5, [], (ValueError, "into an 'except'"))
@jump_test(1, 5, [], (ValueError, "into an exception"))
def test_no_jump_into_qualified_except_block(output):
output.append(1)
try:
@ -1772,7 +1772,7 @@ def test_no_jump_into_qualified_except_block(output):
except Exception:
output.append(5)
@jump_test(3, 6, [2, 5, 6], (ValueError, "into an 'except'"))
@jump_test(3, 6, [2, 5, 6], (ValueError, "into an exception"))
def test_no_jump_into_bare_except_block_from_try_block(output):
try:
output.append(2)
@ -1783,7 +1783,7 @@ def test_no_jump_into_bare_except_block_from_try_block(output):
raise
output.append(8)
@jump_test(3, 6, [2], (ValueError, "into an 'except'"))
@jump_test(3, 6, [2], (ValueError, "into an exception"))
def test_no_jump_into_qualified_except_block_from_try_block(output):
try:
output.append(2)
@ -1794,7 +1794,7 @@ def test_no_jump_into_qualified_except_block_from_try_block(output):
raise
output.append(8)
@jump_test(7, 1, [1, 3, 6], (ValueError, "out of an 'except'"))
@jump_test(7, 1, [1, 3, 6], (ValueError, "within"))
def test_no_jump_out_of_bare_except_block(output):
output.append(1)
try:
@ -1804,7 +1804,7 @@ def test_no_jump_out_of_bare_except_block(output):
output.append(6)
output.append(7)
@jump_test(7, 1, [1, 3, 6], (ValueError, "out of an 'except'"))
@jump_test(7, 1, [1, 3, 6], (ValueError, "within"))
def test_no_jump_out_of_qualified_except_block(output):
output.append(1)
try:

7
Misc/NEWS.d/next/Core and Builtins/2021-04-30-15-48-36.bpo-40222.j3VxeX.rst Normal file
View file

@ -0,0 +1,7 @@
"Zero cost" exception handling.
* Uses a lookup table to determine how to handle exceptions.
* Removes SETUP_FINALLY and POP_TOP block instructions, eliminating the runtime overhead of try statements.
* Reduces the size of the frame object by about 60%.
Patch by Mark Shannon

7
Modules/_opcode.c
View file

@ -59,7 +59,12 @@ _opcode_stack_effect_impl(PyObject *module, int opcode, PyObject *oparg,
"stack_effect: jump must be False, True or None");
return -1;
}
effect = PyCompile_OpcodeStackEffectWithJump(opcode, oparg_int, jump_int);
if (IS_ARTIFICIAL(opcode)) {
effect = PY_INVALID_STACK_EFFECT;
}
else {
effect = PyCompile_OpcodeStackEffectWithJump(opcode, oparg_int, jump_int);
}
if (effect == PY_INVALID_STACK_EFFECT) {
PyErr_SetString(PyExc_ValueError,
"invalid opcode or oparg");

61
Objects/clinic/codeobject.c.h generated
View file

@ -5,7 +5,8 @@ preserve
PyDoc_STRVAR(code_new__doc__,
"code(argcount, posonlyargcount, kwonlyargcount, nlocals, stacksize,\n"
" flags, codestring, constants, names, varnames, filename, name,\n"
" firstlineno, linetable, freevars=(), cellvars=(), /)\n"
" firstlineno, linetable, exceptiontable, freevars=(), cellvars=(),\n"
" /)\n"
"--\n"
"\n"
"Create a code object. Not for the faint of heart.");
@ -15,8 +16,8 @@ code_new_impl(PyTypeObject *type, int argcount, int posonlyargcount,
int kwonlyargcount, int nlocals, int stacksize, int flags,
PyObject *code, PyObject *consts, PyObject *names,
PyObject *varnames, PyObject *filename, PyObject *name,
int firstlineno, PyObject *linetable, PyObject *freevars,
PyObject *cellvars);
int firstlineno, PyObject *linetable, PyObject *exceptiontable,
PyObject *freevars, PyObject *cellvars);
static PyObject *
code_new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
@ -36,6 +37,7 @@ code_new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
PyObject *name;
int firstlineno;
PyObject *linetable;
PyObject *exceptiontable;
PyObject *freevars = NULL;
PyObject *cellvars = NULL;
@ -43,7 +45,7 @@ code_new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
!_PyArg_NoKeywords("code", kwargs)) {
goto exit;
}
if (!_PyArg_CheckPositional("code", PyTuple_GET_SIZE(args), 14, 16)) {
if (!_PyArg_CheckPositional("code", PyTuple_GET_SIZE(args), 15, 17)) {
goto exit;
}
argcount = _PyLong_AsInt(PyTuple_GET_ITEM(args, 0));
@ -115,14 +117,11 @@ code_new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
goto exit;
}
linetable = PyTuple_GET_ITEM(args, 13);
if (PyTuple_GET_SIZE(args) < 15) {
goto skip_optional;
}
if (!PyTuple_Check(PyTuple_GET_ITEM(args, 14))) {
_PyArg_BadArgument("code", "argument 15", "tuple", PyTuple_GET_ITEM(args, 14));
if (!PyBytes_Check(PyTuple_GET_ITEM(args, 14))) {
_PyArg_BadArgument("code", "argument 15", "bytes", PyTuple_GET_ITEM(args, 14));
goto exit;
}
freevars = PyTuple_GET_ITEM(args, 14);
exceptiontable = PyTuple_GET_ITEM(args, 14);
if (PyTuple_GET_SIZE(args) < 16) {
goto skip_optional;
}
@ -130,9 +129,17 @@ code_new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
_PyArg_BadArgument("code", "argument 16", "tuple", PyTuple_GET_ITEM(args, 15));
goto exit;
}
cellvars = PyTuple_GET_ITEM(args, 15);
freevars = PyTuple_GET_ITEM(args, 15);
if (PyTuple_GET_SIZE(args) < 17) {
goto skip_optional;
}
if (!PyTuple_Check(PyTuple_GET_ITEM(args, 16))) {
_PyArg_BadArgument("code", "argument 17", "tuple", PyTuple_GET_ITEM(args, 16));
goto exit;
}
cellvars = PyTuple_GET_ITEM(args, 16);
skip_optional:
return_value = code_new_impl(type, argcount, posonlyargcount, kwonlyargcount, nlocals, stacksize, flags, code, consts, names, varnames, filename, name, firstlineno, linetable, freevars, cellvars);
return_value = code_new_impl(type, argcount, posonlyargcount, kwonlyargcount, nlocals, stacksize, flags, code, consts, names, varnames, filename, name, firstlineno, linetable, exceptiontable, freevars, cellvars);
exit:
return return_value;
@ -144,7 +151,7 @@ PyDoc_STRVAR(code_replace__doc__,
" co_flags=-1, co_firstlineno=-1, co_code=None, co_consts=None,\n"
" co_names=None, co_varnames=None, co_freevars=None,\n"
" co_cellvars=None, co_filename=None, co_name=None,\n"
" co_linetable=None)\n"
" co_linetable=None, co_exceptiontable=None)\n"
"--\n"
"\n"
"Return a copy of the code object with new values for the specified fields.");
@ -160,15 +167,16 @@ code_replace_impl(PyCodeObject *self, int co_argcount,
PyObject *co_consts, PyObject *co_names,
PyObject *co_varnames, PyObject *co_freevars,
PyObject *co_cellvars, PyObject *co_filename,
PyObject *co_name, PyBytesObject *co_linetable);
PyObject *co_name, PyBytesObject *co_linetable,
PyBytesObject *co_exceptiontable);
static PyObject *
code_replace(PyCodeObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
{
PyObject *return_value = NULL;
static const char * const _keywords[] = {"co_argcount", "co_posonlyargcount", "co_kwonlyargcount", "co_nlocals", "co_stacksize", "co_flags", "co_firstlineno", "co_code", "co_consts", "co_names", "co_varnames", "co_freevars", "co_cellvars", "co_filename", "co_name", "co_linetable", NULL};
static const char * const _keywords[] = {"co_argcount", "co_posonlyargcount", "co_kwonlyargcount", "co_nlocals", "co_stacksize", "co_flags", "co_firstlineno", "co_code", "co_consts", "co_names", "co_varnames", "co_freevars", "co_cellvars", "co_filename", "co_name", "co_linetable", "co_exceptiontable", NULL};
static _PyArg_Parser _parser = {NULL, _keywords, "replace", 0};
PyObject *argsbuf[16];
PyObject *argsbuf[17];
Py_ssize_t noptargs = nargs + (kwnames ? PyTuple_GET_SIZE(kwnames) : 0) - 0;
int co_argcount = self->co_argcount;
int co_posonlyargcount = self->co_posonlyargcount;
@ -186,6 +194,7 @@ code_replace(PyCodeObject *self, PyObject *const *args, Py_ssize_t nargs, PyObje
PyObject *co_filename = self->co_filename;
PyObject *co_name = self->co_name;
PyBytesObject *co_linetable = (PyBytesObject *)self->co_linetable;
PyBytesObject *co_exceptiontable = (PyBytesObject *)self->co_exceptiontable;
args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 0, 0, 0, argsbuf);
if (!args) {
@ -343,15 +352,25 @@ code_replace(PyCodeObject *self, PyObject *const *args, Py_ssize_t nargs, PyObje
goto skip_optional_kwonly;
}
}
if (!PyBytes_Check(args[15])) {
_PyArg_BadArgument("replace", "argument 'co_linetable'", "bytes", args[15]);
if (args[15]) {
if (!PyBytes_Check(args[15])) {
_PyArg_BadArgument("replace", "argument 'co_linetable'", "bytes", args[15]);
goto exit;
}
co_linetable = (PyBytesObject *)args[15];
if (!--noptargs) {
goto skip_optional_kwonly;
}
}
if (!PyBytes_Check(args[16])) {
_PyArg_BadArgument("replace", "argument 'co_exceptiontable'", "bytes", args[16]);
goto exit;
}
co_linetable = (PyBytesObject *)args[15];
co_exceptiontable = (PyBytesObject *)args[16];
skip_optional_kwonly:
return_value = code_replace_impl(self, co_argcount, co_posonlyargcount, co_kwonlyargcount, co_nlocals, co_stacksize, co_flags, co_firstlineno, co_code, co_consts, co_names, co_varnames, co_freevars, co_cellvars, co_filename, co_name, co_linetable);
return_value = code_replace_impl(self, co_argcount, co_posonlyargcount, co_kwonlyargcount, co_nlocals, co_stacksize, co_flags, co_firstlineno, co_code, co_consts, co_names, co_varnames, co_freevars, co_cellvars, co_filename, co_name, co_linetable, co_exceptiontable);
exit:
return return_value;
}
/*[clinic end generated code: output=e3091c7baaaaa420 input=a9049054013a1b77]*/
/*[clinic end generated code: output=a272b22f63ea002e input=a9049054013a1b77]*/

35
Objects/codeobject.c
View file

@ -119,7 +119,7 @@ PyCode_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount,
PyObject *code, PyObject *consts, PyObject *names,
PyObject *varnames, PyObject *freevars, PyObject *cellvars,
PyObject *filename, PyObject *name, int firstlineno,
PyObject *linetable)
PyObject *linetable, PyObject *exceptiontable)
{
PyCodeObject *co;
Py_ssize_t *cell2arg = NULL;
@ -137,7 +137,8 @@ PyCode_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount,
cellvars == NULL || !PyTuple_Check(cellvars) ||
name == NULL || !PyUnicode_Check(name) ||
filename == NULL || !PyUnicode_Check(filename) ||
linetable == NULL || !PyBytes_Check(linetable)) {
linetable == NULL || !PyBytes_Check(linetable) ||
exceptiontable == NULL || !PyBytes_Check(exceptiontable)) {
PyErr_BadInternalCall();
return NULL;
}
@ -260,6 +261,8 @@ PyCode_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount,
co->co_firstlineno = firstlineno;
Py_INCREF(linetable);
co->co_linetable = linetable;
Py_INCREF(exceptiontable);
co->co_exceptiontable = exceptiontable;
co->co_zombieframe = NULL;
co->co_weakreflist = NULL;
co->co_extra = NULL;
@ -277,12 +280,12 @@ PyCode_New(int argcount, int kwonlyargcount,
PyObject *code, PyObject *consts, PyObject *names,
PyObject *varnames, PyObject *freevars, PyObject *cellvars,
PyObject *filename, PyObject *name, int firstlineno,
PyObject *linetable)
PyObject *linetable, PyObject *exceptiontable)
{
return PyCode_NewWithPosOnlyArgs(argcount, 0, kwonlyargcount, nlocals,
stacksize, flags, code, consts, names,
varnames, freevars, cellvars, filename,
name, firstlineno, linetable);
name, firstlineno, linetable, exceptiontable);
}
int
@ -369,7 +372,8 @@ PyCode_NewEmpty(const char *filename, const char *funcname, int firstlineno)
filename_ob, /* filename */
funcname_ob, /* name */
firstlineno, /* firstlineno */
emptystring /* linetable */
emptystring, /* linetable */
emptystring /* exception table */
);
failed:
@ -397,6 +401,7 @@ static PyMemberDef code_memberlist[] = {
{"co_name", T_OBJECT, OFF(co_name), READONLY},
{"co_firstlineno", T_INT, OFF(co_firstlineno), READONLY},
{"co_linetable", T_OBJECT, OFF(co_linetable), READONLY},
{"co_exceptiontable", T_OBJECT, OFF(co_exceptiontable), READONLY},
{NULL} /* Sentinel */
};
@ -538,6 +543,7 @@ code.__new__ as code_new
name: unicode
firstlineno: int
linetable: object(subclass_of="&PyBytes_Type")
exceptiontable: object(subclass_of="&PyBytes_Type")
freevars: object(subclass_of="&PyTuple_Type", c_default="NULL") = ()
cellvars: object(subclass_of="&PyTuple_Type", c_default="NULL") = ()
/
@ -550,9 +556,9 @@ code_new_impl(PyTypeObject *type, int argcount, int posonlyargcount,
int kwonlyargcount, int nlocals, int stacksize, int flags,
PyObject *code, PyObject *consts, PyObject *names,
PyObject *varnames, PyObject *filename, PyObject *name,
int firstlineno, PyObject *linetable, PyObject *freevars,
PyObject *cellvars)
/*[clinic end generated code: output=42c1839b082ba293 input=0ec80da632b99f57]*/
int firstlineno, PyObject *linetable, PyObject *exceptiontable,
PyObject *freevars, PyObject *cellvars)
/*[clinic end generated code: output=a3899259c3b4cace input=f823c686da4b3a03]*/
{
PyObject *co = NULL;
PyObject *ournames = NULL;
@ -618,7 +624,9 @@ code_new_impl(PyTypeObject *type, int argcount, int posonlyargcount,
code, consts, ournames,
ourvarnames, ourfreevars,
ourcellvars, filename,
name, firstlineno, linetable);
name, firstlineno, linetable,
exceptiontable
);
cleanup:
Py_XDECREF(ournames);
Py_XDECREF(ourvarnames);
@ -663,6 +671,7 @@ code_dealloc(PyCodeObject *co)
Py_XDECREF(co->co_filename);
Py_XDECREF(co->co_name);
Py_XDECREF(co->co_linetable);
Py_XDECREF(co->co_exceptiontable);
if (co->co_cell2arg != NULL)
PyMem_Free(co->co_cell2arg);
if (co->co_zombieframe != NULL)
@ -715,6 +724,7 @@ code.replace
co_filename: unicode(c_default="self->co_filename") = None
co_name: unicode(c_default="self->co_name") = None
co_linetable: PyBytesObject(c_default="(PyBytesObject *)self->co_linetable") = None
co_exceptiontable: PyBytesObject(c_default="(PyBytesObject *)self->co_exceptiontable") = None
Return a copy of the code object with new values for the specified fields.
[clinic start generated code]*/
@ -727,8 +737,9 @@ code_replace_impl(PyCodeObject *self, int co_argcount,
PyObject *co_consts, PyObject *co_names,
PyObject *co_varnames, PyObject *co_freevars,
PyObject *co_cellvars, PyObject *co_filename,
PyObject *co_name, PyBytesObject *co_linetable)
/*[clinic end generated code: output=50d77e668d3b449b input=a5f997b173d7f636]*/
PyObject *co_name, PyBytesObject *co_linetable,
PyBytesObject *co_exceptiontable)
/*[clinic end generated code: output=80957472b7f78ed6 input=38376b1193efbbae]*/
{
#define CHECK_INT_ARG(ARG) \
if (ARG < 0) { \
@ -758,7 +769,7 @@ code_replace_impl(PyCodeObject *self, int co_argcount,
co_argcount, co_posonlyargcount, co_kwonlyargcount, co_nlocals,
co_stacksize, co_flags, (PyObject*)co_code, co_consts, co_names,
co_varnames, co_freevars, co_cellvars, co_filename, co_name,
co_firstlineno, (PyObject*)co_linetable);
co_firstlineno, (PyObject*)co_linetable, (PyObject*)co_exceptiontable);
}
static PyObject *

179
Objects/exception_handling_notes.txt Normal file
View file

@ -0,0 +1,179 @@
Description of exception handling in Python 3.11
------------------------------------------------
Python 3.11 uses what is known as "zero-cost" exception handling.
Prior to 3.11, exceptions were handled by a runtime stack of "blocks".
In zero-cost exception handling, the cost of supporting exceptions is minimized.
In the common case (where no exception is raised) the cost is reduced
to zero (or close to zero).
The cost of raising an exception is increased, but not by much.
The following code:
def f():
try:
g(0)
except:
return "fail"
compiles as follows in 3.10:
2 0 SETUP_FINALLY 7 (to 16)
3 2 LOAD_GLOBAL 0 (g)
4 LOAD_CONST 1 (0)
6 CALL_FUNCTION 1
8 POP_TOP
10 POP_BLOCK
12 LOAD_CONST 0 (None)
14 RETURN_VALUE
4 >> 16 POP_TOP
18 POP_TOP
20 POP_TOP
5 22 POP_EXCEPT
24 LOAD_CONST 3 ('fail')
26 RETURN_VALUE
Note the explicit instructions to push and pop from the "block" stack:
SETUP_FINALLY and POP_BLOCK.
In 3.11, the SETUP_FINALLY and POP_BLOCK are eliminated, replaced with
a table to determine where to jump to when an exception is raised.
2 0 NOP
3 2 LOAD_GLOBAL 0 (g)
4 LOAD_CONST 1 (0)
6 CALL_FUNCTION 1
8 POP_TOP
10 LOAD_CONST 0 (None)
12 RETURN_VALUE
>> 14 PUSH_EXC_INFO
4 16 POP_TOP
18 POP_TOP
20 POP_TOP
5 22 POP_EXCEPT
24 LOAD_CONST 2 ('fail')
26 RETURN_VALUE
>> 28 POP_EXCEPT_AND_RERAISE
ExceptionTable:
2 to 8 -> 14 [0]
14 to 20 -> 28 [3] lasti
(Note this code is from an early 3.11 alpha, the NOP may well have be removed before release).
If an instruction raises an exception then its offset is used to find the target to jump to.
For example, the CALL_FUNCTION at offset 6, falls into the range 2 to 8.
So, if g() raises an exception, then control jumps to offset 14.
Unwinding
---------
When an exception is raised, the current instruction offset is used to find following:
target to jump to, stack depth, and 'lasti', which determines whether the instruction
offset of the raising instruction should be pushed.
This information is stored in the exception table, described below.
If there is no relevant entry, the exception bubbles up to the caller.
If there is an entry, then:
1. pop values from the stack until it matches the stack depth for the handler,
2. if 'lasti' is true, then push the offset that the exception was raised at.
3. push the exception to the stack as three values: traceback, value, type,
4. jump to the target offset and resume execution.
Format of the exception table
-----------------------------
Conceptually, the exception table consists of a sequence of 5-tuples:
1. start-offset (inclusive)
2. end-offset (exclusive)
3. target
4. stack-depth
5. push-lasti (boolean)
All offsets and lengths are in instructions, not bytes.
We want the format to be compact, but quickly searchable.
For it to be compact, it needs to have variable sized entries so that we can store common (small) offsets compactly, but handle large offsets if needed.
For it to be searchable quickly, we need to support binary search giving us log(n) performance in all cases.
Binary search typically assumes fixed size entries, but that is not necesary, as long as we can identify the start of an entry.
It is worth noting that the size (end-start) is always smaller than the end, so we encode the entries as:
start, size, target, depth, push-lasti
Also, sizes are limited to 2**30 as the code length cannot exceed 2**31 and each instruction takes 2 bytes.
It also happens that depth is generally quite small.
So, we need to encode:
start (up to 30 bits)
size (up to 30 bits)
target (up to 30 bits)
depth (up to ~8 bits)
lasti (1 bit)
We need a marker for the start of the entry, so the first byte of entry will have the most significant bit set.
Since the most significant bit is reserved for marking the start of an entry, we have 7 bits per byte to encode offsets.
Encoding uses a standard varint encoding, but with only 7 bits instead of the usual 8.
The 8 bits of a bit are (msb left) SXdddddd where S is the start bit. X is the extend bit meaning that the next byte is required to extend the offset.
In addition, we will combine depth and lasti into a single value, ((depth<<1)+lasti), before encoding.
For example, the exception entry:
start: 20
end: 28
target: 100
depth: 3
lasti: False
is encoded first by converting to the more compact four value form:
start: 20
size: 8
target: 100
depth<<1+lasti: 6
which is then encoded as:
148 (MSB + 20 for start)
8 (size)
65 (Extend bit + 1)
36 (Remainder of target, 100 == (1<<6)+36)
6
for a total of five bytes.
Script to parse the exception table
-----------------------------------
def parse_varint(iterator):
b = next(iterator)
val = b & 63
while b&64:
val <<= 6
b = next(iterator)
val |= b&63
return val
def parse_exception_table(code):
iterator = iter(code.co_exceptiontable)
try:
while True:
start = parse_varint(iterator)*2
length = parse_varint(iterator)*2
end = start + length - 2 # Present as inclusive, not exclusive
target = parse_varint(iterator)*2
dl = parse_varint(iterator)
depth = dl >> 1
lasti = bool(dl&1)
yield start, end, target, depth, lasti
except StopIteration:
return

327
Objects/frameobject.c
View file

@ -91,56 +91,71 @@ get_arg(const _Py_CODEUNIT *codestr, Py_ssize_t i)
return oparg;
}
/* Model the evaluation stack, to determine which jumps
* are safe and how many values needs to be popped.
* The stack is modelled by a 64 integer, treating any
* stack that can't fit into 64 bits as "overflowed".
*/
typedef enum kind {
With = 1,
Loop = 2,
Try = 3,
Except = 4,
Iterator = 1,
Except = 2,
Object = 3,
} Kind;
#define BITS_PER_BLOCK 3
#define BITS_PER_BLOCK 2
#define UNINITIALIZED -2
#define OVERFLOWED -1
#define MAX_STACK_ENTRIES (63/BITS_PER_BLOCK)
#define WILL_OVERFLOW (1ULL<<((MAX_STACK_ENTRIES-1)*BITS_PER_BLOCK))
static inline int64_t
push_block(int64_t stack, Kind kind)
push_value(int64_t stack, Kind kind)
{
assert(stack < ((int64_t)1)<<(BITS_PER_BLOCK*CO_MAXBLOCKS));
return (stack << BITS_PER_BLOCK) | kind;
if (((uint64_t)stack) >= WILL_OVERFLOW) {
return OVERFLOWED;
}
else {
return (stack << BITS_PER_BLOCK) | kind;
}
}
static inline int64_t
pop_block(int64_t stack)
pop_value(int64_t stack)
{
assert(stack > 0);
return stack >> BITS_PER_BLOCK;
return Py_ARITHMETIC_RIGHT_SHIFT(int64_t, stack, BITS_PER_BLOCK);
}
static inline Kind
top_block(int64_t stack)
top_of_stack(int64_t stack)
{
return stack & ((1<<BITS_PER_BLOCK)-1);
}
static int64_t *
markblocks(PyCodeObject *code_obj, int len)
mark_stacks(PyCodeObject *code_obj, int len)
{
const _Py_CODEUNIT *code =
(const _Py_CODEUNIT *)PyBytes_AS_STRING(code_obj->co_code);
int64_t *blocks = PyMem_New(int64_t, len+1);
int64_t *stacks = PyMem_New(int64_t, len+1);
int i, j, opcode;
if (blocks == NULL) {
if (stacks == NULL) {
PyErr_NoMemory();
return NULL;
}
memset(blocks, -1, (len+1)*sizeof(int64_t));
blocks[0] = 0;
for (int i = 1; i <= len; i++) {
stacks[i] = UNINITIALIZED;
}
stacks[0] = 0;
int todo = 1;
while (todo) {
todo = 0;
for (i = 0; i < len; i++) {
int64_t block_stack = blocks[i];
int64_t except_stack;
if (block_stack == -1) {
int64_t next_stack = stacks[i];
if (next_stack == UNINITIALIZED) {
continue;
}
opcode = _Py_OPCODE(code[i]);
@ -150,109 +165,153 @@ markblocks(PyCodeObject *code_obj, int len)
case POP_JUMP_IF_FALSE:
case POP_JUMP_IF_TRUE:
case JUMP_IF_NOT_EXC_MATCH:
j = get_arg(code, i);
{
int64_t target_stack;
int j = get_arg(code, i);
assert(j < len);
if (blocks[j] == -1 && j < i) {
if (stacks[j] == UNINITIALIZED && j < i) {
todo = 1;
}
assert(blocks[j] == -1 || blocks[j] == block_stack);
blocks[j] = block_stack;
blocks[i+1] = block_stack;
if (opcode == JUMP_IF_NOT_EXC_MATCH) {
next_stack = pop_value(pop_value(next_stack));
target_stack = next_stack;
}
else if (opcode == JUMP_IF_FALSE_OR_POP ||
opcode == JUMP_IF_TRUE_OR_POP)
{
target_stack = next_stack;
next_stack = pop_value(next_stack);
}
else {
next_stack = pop_value(next_stack);
target_stack = next_stack;
}
assert(stacks[j] == UNINITIALIZED || stacks[j] == target_stack);
stacks[j] = target_stack;
stacks[i+1] = next_stack;
break;
}
case JUMP_ABSOLUTE:
j = get_arg(code, i);
assert(j < len);
if (blocks[j] == -1 && j < i) {
if (stacks[j] == UNINITIALIZED && j < i) {
todo = 1;
}
assert(blocks[j] == -1 || blocks[j] == block_stack);
blocks[j] = block_stack;
assert(stacks[j] == UNINITIALIZED || stacks[j] == next_stack);
stacks[j] = next_stack;
break;
case SETUP_FINALLY:
j = get_arg(code, i) + i + 1;
assert(j < len);
except_stack = push_block(block_stack, Except);
assert(blocks[j] == -1 || blocks[j] == except_stack);
blocks[j] = except_stack;
block_stack = push_block(block_stack, Try);
blocks[i+1] = block_stack;
break;
case SETUP_WITH:
case SETUP_ASYNC_WITH:
j = get_arg(code, i) + i + 1;
assert(j < len);
except_stack = push_block(block_stack, Except);
assert(blocks[j] == -1 || blocks[j] == except_stack);
blocks[j] = except_stack;
block_stack = push_block(block_stack, With);
blocks[i+1] = block_stack;
case POP_EXCEPT:
next_stack = pop_value(pop_value(pop_value(next_stack)));
stacks[i+1] = next_stack;
break;
case JUMP_FORWARD:
j = get_arg(code, i) + i + 1;
assert(j < len);
assert(blocks[j] == -1 || blocks[j] == block_stack);
blocks[j] = block_stack;
assert(stacks[j] == UNINITIALIZED || stacks[j] == next_stack);
stacks[j] = next_stack;
break;
case GET_ITER:
case GET_AITER:
block_stack = push_block(block_stack, Loop);
blocks[i+1] = block_stack;
next_stack = push_value(pop_value(next_stack), Iterator);
stacks[i+1] = next_stack;
break;
case FOR_ITER:
blocks[i+1] = block_stack;
block_stack = pop_block(block_stack);
{
int64_t target_stack = pop_value(next_stack);
stacks[i+1] = push_value(next_stack, Object);
j = get_arg(code, i) + i + 1;
assert(j < len);
assert(blocks[j] == -1 || blocks[j] == block_stack);
blocks[j] = block_stack;
break;
case POP_BLOCK:
case POP_EXCEPT:
block_stack = pop_block(block_stack);
blocks[i+1] = block_stack;
assert(stacks[j] == UNINITIALIZED || stacks[j] == target_stack);
stacks[j] = target_stack;
break;
}
case END_ASYNC_FOR:
block_stack = pop_block(pop_block(block_stack));
blocks[i+1] = block_stack;
next_stack = pop_value(pop_value(pop_value(next_stack)));
stacks[i+1] = next_stack;
break;
case PUSH_EXC_INFO:
next_stack = push_value(next_stack, Except);
next_stack = push_value(next_stack, Except);
next_stack = push_value(next_stack, Except);
stacks[i+1] = next_stack;
case RETURN_VALUE:
case RAISE_VARARGS:
case RERAISE:
case POP_EXCEPT_AND_RERAISE:
/* End of block */
break;
case GEN_START:
stacks[i+1] = next_stack;
break;
default:
blocks[i+1] = block_stack;
{
int delta = PyCompile_OpcodeStackEffect(opcode, _Py_OPARG(code[i]));
while (delta < 0) {
next_stack = pop_value(next_stack);
delta++;
}
while (delta > 0) {
next_stack = push_value(next_stack, Object);
delta--;
}
stacks[i+1] = next_stack;
}
}
}
}
return blocks;
return stacks;
}
static int
compatible_block_stack(int64_t from_stack, int64_t to_stack)
compatible_kind(Kind from, Kind to) {
if (to == 0) {
return 0;
}
if (to == Object) {
return 1;
}
return from == to;
}
static int
compatible_stack(int64_t from_stack, int64_t to_stack)
{
if (to_stack < 0) {
if (from_stack < 0 || to_stack < 0) {
return 0;
}
while(from_stack > to_stack) {
from_stack = pop_block(from_stack);
from_stack = pop_value(from_stack);
}
return from_stack == to_stack;
while(from_stack) {
Kind from_top = top_of_stack(from_stack);
Kind to_top = top_of_stack(to_stack);
if (!compatible_kind(from_top, to_top)) {
return 0;
}
from_stack = pop_value(from_stack);
to_stack = pop_value(to_stack);
}
return to_stack == 0;
}
static const char *
explain_incompatible_block_stack(int64_t to_stack)
explain_incompatible_stack(int64_t to_stack)
{
Kind target_kind = top_block(to_stack);
assert(to_stack != 0);
if (to_stack == OVERFLOWED) {
return "stack is too deep to analyze";
}
if (to_stack == UNINITIALIZED) {
return "can't jump into an exception handler, or code may be unreachable";
}
Kind target_kind = top_of_stack(to_stack);
switch(target_kind) {
case Except:
return "can't jump into an 'except' block as there's no exception";
case Try:
return "can't jump into the body of a try statement";
case With:
return "can't jump into the body of a with statement";
case Loop:
case Object:
return "differing stack depth";
case Iterator:
return "can't jump into the body of a for loop";
default:
Py_UNREACHABLE();
@ -299,27 +358,12 @@ first_line_not_before(int *lines, int len, int line)
static void
frame_stack_pop(PyFrameObject *f)
{
assert(f->f_stackdepth >= 0);
assert(f->f_stackdepth > 0);
f->f_stackdepth--;
PyObject *v = f->f_valuestack[f->f_stackdepth];
Py_DECREF(v);
}
static void
frame_block_unwind(PyFrameObject *f)
{
assert(f->f_stackdepth >= 0);
assert(f->f_iblock > 0);
f->f_iblock--;
PyTryBlock *b = &f->f_blockstack[f->f_iblock];
intptr_t delta = f->f_stackdepth - b->b_level;
while (delta > 0) {
frame_stack_pop(f);
delta--;
}
}
/* Setter for f_lineno - you can set f_lineno from within a trace function in
* order to jump to a given line of code, subject to some restrictions. Most
* lines are OK to jump to because they don't make any assumptions about the
@ -327,13 +371,7 @@ frame_block_unwind(PyFrameObject *f)
* would still work without any stack errors), but there are some constructs
* that limit jumping:
*
* o Lines with an 'except' statement on them can't be jumped to, because
* they expect an exception to be on the top of the stack.
* o Lines that live in a 'finally' block can't be jumped from or to, since
* we cannot be sure which state the interpreter was in or would be in
* during execution of the finally block.
* o 'try', 'with' and 'async with' blocks can't be jumped into because
* the blockstack needs to be set up before their code runs.
* o Any excpetion handlers.
* o 'for' and 'async for' loops can't be jumped into because the
* iterator needs to be on the stack.
* o Jumps cannot be made from within a trace function invoked with a
@ -428,67 +466,56 @@ frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno, void *Py_UNUSED(ignore
return -1;
}
int64_t *blocks = markblocks(f->f_code, len);
if (blocks == NULL) {
int64_t *stacks = mark_stacks(f->f_code, len);
if (stacks == NULL) {
PyMem_Free(lines);
return -1;
}
int64_t target_block_stack = -1;
int64_t best_block_stack = -1;
int64_t best_stack = OVERFLOWED;
int best_addr = -1;
int64_t start_block_stack = blocks[f->f_lasti];
int64_t start_stack = stacks[f->f_lasti];
int err = -1;
const char *msg = "cannot find bytecode for specified line";
for (int i = 0; i < len; i++) {
if (lines[i] == new_lineno) {
target_block_stack = blocks[i];
if (compatible_block_stack(start_block_stack, target_block_stack)) {
msg = NULL;
if (target_block_stack > best_block_stack) {
best_block_stack = target_block_stack;
int64_t target_stack = stacks[i];
if (compatible_stack(start_stack, target_stack)) {
err = 0;
if (target_stack > best_stack) {
best_stack = target_stack;
best_addr = i;
}
}
else if (msg) {
if (target_block_stack >= 0) {
msg = explain_incompatible_block_stack(target_block_stack);
else if (err < 0) {
if (start_stack == OVERFLOWED) {
msg = "stack to deep to analyze";
}
else if (start_stack == UNINITIALIZED) {
msg = "can't jump from within an exception handler";
}
else {
msg = "code may be unreachable.";
msg = explain_incompatible_stack(target_stack);
err = 1;
}
}
}
}
PyMem_Free(blocks);
PyMem_Free(stacks);
PyMem_Free(lines);
if (msg != NULL) {
if (err) {
PyErr_SetString(PyExc_ValueError, msg);
return -1;
}
/* Unwind block stack. */
while (start_block_stack > best_block_stack) {
Kind kind = top_block(start_block_stack);
switch(kind) {
case Loop:
frame_stack_pop(f);
break;
case Try:
frame_block_unwind(f);
break;
case With:
frame_block_unwind(f);
// Pop the exit function
frame_stack_pop(f);
break;
case Except:
PyErr_SetString(PyExc_ValueError,
"can't jump out of an 'except' block");
return -1;
}
start_block_stack = pop_block(start_block_stack);
if (f->f_state == FRAME_SUSPENDED) {
/* Account for value popped by yield */
start_stack = pop_value(start_stack);
}
while (start_stack > best_stack) {
frame_stack_pop(f);
start_stack = pop_value(start_stack);
}
/* Finally set the new f_lasti and return OK. */
f->f_lineno = 0;
f->f_lasti = best_addr;
@ -852,7 +879,6 @@ _PyFrame_New_NoTrack(PyThreadState *tstate, PyFrameConstructor *con, PyObject *l
f->f_gen = NULL;
f->f_lasti = -1;
f->f_lineno = 0;
f->f_iblock = 0;
f->f_state = FRAME_CREATED;
// f_blockstack and f_localsplus initialized by frame_alloc()
return f;
@ -884,33 +910,6 @@ PyFrame_New(PyThreadState *tstate, PyCodeObject *code,
return f;
}
/* Block management */
void
PyFrame_BlockSetup(PyFrameObject *f, int type, int handler, int level)
{
PyTryBlock *b;
if (f->f_iblock >= CO_MAXBLOCKS) {
Py_FatalError("block stack overflow");
}
b = &f->f_blockstack[f->f_iblock++];
b->b_type = type;
b->b_level = level;
b->b_handler = handler;
}
PyTryBlock *
PyFrame_BlockPop(PyFrameObject *f)
{
PyTryBlock *b;
if (f->f_iblock <= 0) {
Py_FatalError("block stack underflow");
}
b = &f->f_blockstack[--f->f_iblock];
return b;
}
/* Convert between "fast" version of locals and dictionary version.
map and values are input arguments. map is a tuple of strings.

3
PC/launcher.c
View file

@ -1258,7 +1258,8 @@ static PYC_MAGIC magic_values[] = {
{ 3390, 3399, L"3.7" },
{ 3400, 3419, L"3.8" },
{ 3420, 3429, L"3.9" },
{ 3430, 3439, L"3.10" },
{ 3430, 3449, L"3.10" },
{ 3450, 3469, L"3.11" },
{ 0 }
};

371
Python/ceval.c
View file

@ -95,6 +95,7 @@ static PyObject * special_lookup(PyThreadState *, PyObject *, _Py_Identifier *);
static int check_args_iterable(PyThreadState *, PyObject *func, PyObject *vararg);
static void format_kwargs_error(PyThreadState *, PyObject *func, PyObject *kwargs);
static void format_awaitable_error(PyThreadState *, PyTypeObject *, int, int);
static PyTryBlock get_exception_handler(PyCodeObject *, int);
#define NAME_ERROR_MSG \
"name '%.200s' is not defined"
@ -1448,34 +1449,6 @@ eval_frame_handle_pending(PyThreadState *tstate)
GETLOCAL(i) = value; \
Py_XDECREF(tmp); } while (0)
#define UNWIND_BLOCK(b) \
while (STACK_LEVEL() > (b)->b_level) { \
PyObject *v = POP(); \
Py_XDECREF(v); \
}
#define UNWIND_EXCEPT_HANDLER(b) \
do { \
PyObject *type, *value, *traceback; \
_PyErr_StackItem *exc_info; \
assert(STACK_LEVEL() >= (b)->b_level + 3); \
while (STACK_LEVEL() > (b)->b_level + 3) { \
value = POP(); \
Py_XDECREF(value); \
} \
exc_info = tstate->exc_info; \
type = exc_info->exc_type; \
value = exc_info->exc_value; \
traceback = exc_info->exc_traceback; \
exc_info->exc_type = POP(); \
exc_info->exc_value = POP(); \
exc_info->exc_traceback = POP(); \
Py_XDECREF(type); \
Py_XDECREF(value); \
Py_XDECREF(traceback); \
} while(0)
/* macros for opcode cache */
#define OPCACHE_CHECK() \
do { \
@ -1738,7 +1711,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
assert(!_PyErr_Occurred(tstate));
#endif
main_loop:
for (;;) {
assert(stack_pointer >= f->f_valuestack); /* else underflow */
assert(STACK_LEVEL() <= co->co_stacksize); /* else overflow */
@ -1754,9 +1726,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
if (_Py_atomic_load_relaxed(eval_breaker)) {
opcode = _Py_OPCODE(*next_instr);
if (opcode != SETUP_FINALLY &&
opcode != SETUP_WITH &&
opcode != BEFORE_ASYNC_WITH &&
if (opcode != BEFORE_ASYNC_WITH &&
opcode != YIELD_FROM) {
/* Few cases where we skip running signal handlers and other
pending calls:
@ -1800,14 +1770,14 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
tstate->c_traceobj,
tstate, f,
&trace_info);
/* Reload possibly changed frame fields */
JUMPTO(f->f_lasti);
stack_pointer = f->f_valuestack+f->f_stackdepth;
f->f_stackdepth = -1;
if (err) {
/* trace function raised an exception */
goto error;
}
/* Reload possibly changed frame fields */
JUMPTO(f->f_lasti);
stack_pointer = f->f_valuestack+f->f_stackdepth;
f->f_stackdepth = -1;
NEXTOPARG();
}
@ -2425,7 +2395,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
case TARGET(RETURN_VALUE): {
retval = POP();
assert(f->f_iblock == 0);
assert(EMPTY());
f->f_state = FRAME_RETURNED;
f->f_stackdepth = 0;
@ -2664,14 +2633,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
case TARGET(POP_EXCEPT): {
PyObject *type, *value, *traceback;
_PyErr_StackItem *exc_info;
PyTryBlock *b = PyFrame_BlockPop(f);
if (b->b_type != EXCEPT_HANDLER) {
_PyErr_SetString(tstate, PyExc_SystemError,
"popped block is not an except handler");
goto error;
}
assert(STACK_LEVEL() >= (b)->b_level + 3 &&
STACK_LEVEL() <= (b)->b_level + 4);
exc_info = tstate->exc_info;
type = exc_info->exc_type;
value = exc_info->exc_value;
@ -2685,15 +2646,48 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
DISPATCH();
}
case TARGET(POP_BLOCK): {
PyFrame_BlockPop(f);
DISPATCH();
case TARGET(POP_EXCEPT_AND_RERAISE): {
PyObject *lasti = PEEK(4);
if (PyLong_Check(lasti)) {
f->f_lasti = PyLong_AsLong(lasti);
assert(!_PyErr_Occurred(tstate));
}
else {
_PyErr_SetString(tstate, PyExc_SystemError, "lasti is not an int");
goto error;
}
PyObject *type, *value, *traceback;
_PyErr_StackItem *exc_info;
type = POP();
value = POP();
traceback = POP();
Py_DECREF(POP()); /* lasti */
_PyErr_Restore(tstate, type, value, traceback);
exc_info = tstate->exc_info;
type = exc_info->exc_type;
value = exc_info->exc_value;
traceback = exc_info->exc_traceback;
exc_info->exc_type = POP();
exc_info->exc_value = POP();
exc_info->exc_traceback = POP();
Py_XDECREF(type);
Py_XDECREF(value);
Py_XDECREF(traceback);
goto exception_unwind;
}
case TARGET(RERAISE): {
assert(f->f_iblock > 0);
if (oparg) {
f->f_lasti = f->f_blockstack[f->f_iblock-1].b_handler;
PyObject *lasti = PEEK(oparg+3);
if (PyLong_Check(lasti)) {
f->f_lasti = PyLong_AsLong(lasti);
assert(!_PyErr_Occurred(tstate));
}
else {
assert(PyLong_Check(lasti));
_PyErr_SetString(tstate, PyExc_SystemError, "lasti is not an int");
goto error;
}
}
PyObject *exc = POP();
PyObject *val = POP();
@ -2705,19 +2699,17 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
case TARGET(END_ASYNC_FOR): {
PyObject *exc = POP();
PyObject *val = POP();
PyObject *tb = POP();
assert(PyExceptionClass_Check(exc));
if (PyErr_GivenExceptionMatches(exc, PyExc_StopAsyncIteration)) {
PyTryBlock *b = PyFrame_BlockPop(f);
assert(b->b_type == EXCEPT_HANDLER);
Py_DECREF(exc);
UNWIND_EXCEPT_HANDLER(b);
Py_DECREF(val);
Py_DECREF(tb);
Py_DECREF(POP());
JUMPBY(oparg);
DISPATCH();
}
else {
PyObject *val = POP();
PyObject *tb = POP();
_PyErr_Restore(tstate, exc, val, tb);
goto exception_unwind;
}
@ -4022,12 +4014,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
DISPATCH();
}
case TARGET(SETUP_FINALLY): {
PyFrame_BlockSetup(f, SETUP_FINALLY, INSTR_OFFSET() + oparg,
STACK_LEVEL());
DISPATCH();
}
case TARGET(BEFORE_ASYNC_WITH): {
_Py_IDENTIFIER(__aenter__);
_Py_IDENTIFIER(__aexit__);
@ -4053,17 +4039,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
DISPATCH();
}
case TARGET(SETUP_ASYNC_WITH): {
PyObject *res = POP();
/* Setup the finally block before pushing the result
of __aenter__ on the stack. */
PyFrame_BlockSetup(f, SETUP_FINALLY, INSTR_OFFSET() + oparg,
STACK_LEVEL());
PUSH(res);
DISPATCH();
}
case TARGET(SETUP_WITH): {
case TARGET(BEFORE_WITH): {
_Py_IDENTIFIER(__enter__);
_Py_IDENTIFIER(__exit__);
PyObject *mgr = TOP();
@ -4081,23 +4057,20 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
Py_DECREF(mgr);
res = _PyObject_CallNoArg(enter);
Py_DECREF(enter);
if (res == NULL)
if (res == NULL) {
goto error;
/* Setup the finally block before pushing the result
of __enter__ on the stack. */
PyFrame_BlockSetup(f, SETUP_FINALLY, INSTR_OFFSET() + oparg,
STACK_LEVEL());
}
PUSH(res);
DISPATCH();
}
case TARGET(WITH_EXCEPT_START): {
/* At the top of the stack are 7 values:
/* At the top of the stack are 8 values:
- (TOP, SECOND, THIRD) = exc_info()
- (FOURTH, FIFTH, SIXTH) = previous exception for EXCEPT_HANDLER
- SEVENTH: the context.__exit__ bound method
We call SEVENTH(TOP, SECOND, THIRD).
- (FOURTH, FIFTH, SIXTH) = previous exception
- SEVENTH: lasti of exception in exc_info()
- EIGHTH: the context.__exit__ bound method
We call EIGHTH(TOP, SECOND, THIRD).
Then we push again the TOP exception and the __exit__
return value.
*/
@ -4109,7 +4082,8 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
tb = THIRD();
assert(!Py_IsNone(exc));
assert(!PyLong_Check(exc));
exit_func = PEEK(7);
assert(PyLong_Check(PEEK(7)));
exit_func = PEEK(8);
PyObject *stack[4] = {NULL, exc, val, tb};
res = PyObject_Vectorcall(exit_func, stack + 1,
3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL);
@ -4120,6 +4094,37 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
DISPATCH();
}
case TARGET(PUSH_EXC_INFO): {
PyObject *type = TOP();
PyObject *value = SECOND();
PyObject *tb = THIRD();
_PyErr_StackItem *exc_info = tstate->exc_info;
SET_THIRD(exc_info->exc_traceback);
SET_SECOND(exc_info->exc_value);
if (exc_info->exc_type != NULL) {
SET_TOP(exc_info->exc_type);
}
else {
Py_INCREF(Py_None);
SET_TOP(Py_None);
}
Py_INCREF(tb);
PUSH(tb);
exc_info->exc_traceback = tb;
Py_INCREF(value);
PUSH(value);
assert(PyExceptionInstance_Check(value));
exc_info->exc_value = value;
Py_INCREF(type);
PUSH(type);
assert(PyExceptionClass_Check(type));
exc_info->exc_type = type;
DISPATCH();
}
case TARGET(LOAD_METHOD): {
/* Designed to work in tandem with CALL_METHOD. */
PyObject *name = GETITEM(names, oparg);
@ -4455,64 +4460,54 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
}
exception_unwind:
f->f_state = FRAME_UNWINDING;
/* Unwind stacks if an exception occurred */
while (f->f_iblock > 0) {
/* Pop the current block. */
PyTryBlock *b = &f->f_blockstack[--f->f_iblock];
/* We can't use f->f_lasti here, as RERAISE may have set it */
int lasti = INSTR_OFFSET()-1;
PyTryBlock from_table = get_exception_handler(co, lasti);
if (from_table.b_handler < 0) {
// No handlers, so exit.
break;
}
if (b->b_type == EXCEPT_HANDLER) {
UNWIND_EXCEPT_HANDLER(b);
continue;
assert(STACK_LEVEL() >= from_table.b_level);
while (STACK_LEVEL() > from_table.b_level) {
PyObject *v = POP();
Py_XDECREF(v);
}
PyObject *exc, *val, *tb;
int handler = from_table.b_handler;
if (from_table.b_type) {
PyObject *lasti = PyLong_FromLong(f->f_lasti);
if (lasti == NULL) {
goto exception_unwind;
}
UNWIND_BLOCK(b);
if (b->b_type == SETUP_FINALLY) {
PyObject *exc, *val, *tb;
int handler = b->b_handler;
_PyErr_StackItem *exc_info = tstate->exc_info;
/* Beware, this invalidates all b->b_* fields */
PyFrame_BlockSetup(f, EXCEPT_HANDLER, f->f_lasti, STACK_LEVEL());
PUSH(exc_info->exc_traceback);
PUSH(exc_info->exc_value);
if (exc_info->exc_type != NULL) {
PUSH(exc_info->exc_type);
}
else {
Py_INCREF(Py_None);
PUSH(Py_None);
}
_PyErr_Fetch(tstate, &exc, &val, &tb);
/* Make the raw exception data
available to the handler,
so a program can emulate the
Python main loop. */
_PyErr_NormalizeException(tstate, &exc, &val, &tb);
if (tb != NULL)
PyException_SetTraceback(val, tb);
else
PyException_SetTraceback(val, Py_None);
Py_INCREF(exc);
exc_info->exc_type = exc;
Py_INCREF(val);
exc_info->exc_value = val;
exc_info->exc_traceback = tb;
if (tb == NULL)
tb = Py_None;
Py_INCREF(tb);
PUSH(tb);
PUSH(val);
PUSH(exc);
JUMPTO(handler);
if (trace_info.cframe.use_tracing) {
trace_info.instr_prev = INT_MAX;
}
/* Resume normal execution */
f->f_state = FRAME_EXECUTING;
goto main_loop;
}
} /* unwind stack */
/* End the loop as we still have an error */
break;
PUSH(lasti);
}
_PyErr_Fetch(tstate, &exc, &val, &tb);
/* Make the raw exception data
available to the handler,
so a program can emulate the
Python main loop. */
_PyErr_NormalizeException(tstate, &exc, &val, &tb);
if (tb != NULL)
PyException_SetTraceback(val, tb);
else
PyException_SetTraceback(val, Py_None);
if (tb == NULL) {
tb = Py_None;
Py_INCREF(Py_None);
}
PUSH(tb);
PUSH(val);
PUSH(exc);
JUMPTO(handler);
if (trace_info.cframe.use_tracing) {
trace_info.instr_prev = INT_MAX;
}
/* Resume normal execution */
f->f_state = FRAME_EXECUTING;
f->f_lasti = handler;
NEXTOPARG();
goto dispatch_opcode;
} /* main loop */
assert(retval == NULL);
@ -4777,6 +4772,102 @@ positional_only_passed_as_keyword(PyThreadState *tstate, PyCodeObject *co,
}
/* Exception table parsing code.
* See Objects/exception_table_notes.txt for details.
*/
static inline unsigned char *
parse_varint(unsigned char *p, int *result) {
int val = p[0] & 63;
while (p[0] & 64) {
p++;
val = (val << 6) | (p[0] & 63);
}
*result = val;
return p+1;
}
static inline unsigned char *
scan_back_to_entry_start(unsigned char *p) {
for (; (p[0]&128) == 0; p--);
return p;
}
static inline unsigned char *
skip_to_next_entry(unsigned char *p) {
for (; (p[0]&128) == 0; p++);
return p;
}
static inline unsigned char *
parse_range(unsigned char *p, int *start, int*end)
{
p = parse_varint(p, start);
int size;
p = parse_varint(p, &size);
*end = *start + size;
return p;
}
static inline void
parse_block(unsigned char *p, PyTryBlock *block) {
int depth_and_lasti;
p = parse_varint(p, &block->b_handler);
p = parse_varint(p, &depth_and_lasti);
block->b_level = depth_and_lasti >> 1;
block->b_type = depth_and_lasti & 1;
}
#define MAX_LINEAR_SEARCH 40
static PyTryBlock
get_exception_handler(PyCodeObject *code, int index)
{
PyTryBlock res;
unsigned char *start = (unsigned char *)PyBytes_AS_STRING(code->co_exceptiontable);
unsigned char *end = start + PyBytes_GET_SIZE(code->co_exceptiontable);
/* Invariants:
* start_table == end_table OR
* start_table points to a legal entry and end_table points
* beyond the table or to a legal entry that is after index.
*/
if (end - start > MAX_LINEAR_SEARCH) {
int offset;
parse_varint(start, &offset);
if (offset > index) {
res.b_handler = -1;
return res;
}
do {
unsigned char * mid = start + ((end-start)>>1);
mid = scan_back_to_entry_start(mid);
parse_varint(mid, &offset);
if (offset > index) {
end = mid;
}
else {
start = mid;
}
} while (end - start > MAX_LINEAR_SEARCH);
}
unsigned char *scan = start;
while (scan < end) {
int start_offset, size;
scan = parse_varint(scan, &start_offset);
if (start_offset > index) {
break;
}
scan = parse_varint(scan, &size);
if (start_offset + size > index) {
parse_block(scan, &res);
return res;
}
scan = skip_to_next_entry(scan);
}
res.b_handler = -1;
return res;
}
PyFrameObject *
_PyEval_MakeFrameVector(PyThreadState *tstate,

484
Python/compile.c
View file

@ -67,6 +67,14 @@
*/
#define MAX_ALLOWED_STACK_USE (STACK_USE_GUIDELINE * 100)
/* Pseudo-instructions used in the compiler,
* but turned into NOPs by the assembler. */
#define SETUP_FINALLY 255
#define SETUP_CLEANUP 254
#define SETUP_WITH 253
#define POP_BLOCK 252
#define IS_TOP_LEVEL_AWAIT(c) ( \
(c->c_flags->cf_flags & PyCF_ALLOW_TOP_LEVEL_AWAIT) \
&& (c->u->u_ste->ste_type == ModuleBlock))
@ -74,10 +82,23 @@
struct instr {
unsigned char i_opcode;
int i_oparg;
struct basicblock_ *i_target; /* target block (if jump instruction) */
/* target block (if jump instruction) */
struct basicblock_ *i_target;
/* target block when exception is raised, should not be set by front-end. */
struct basicblock_ *i_except;
int i_lineno;
};
typedef struct excepthandler {
struct instr *setup;
int offset;
} ExceptHandler;
typedef struct exceptstack {
struct basicblock_ *handlers[CO_MAXBLOCKS+1];
int depth;
} ExceptStack;
#define LOG_BITS_PER_INT 5
#define MASK_LOW_LOG_BITS 31
@ -101,7 +122,7 @@ is_relative_jump(struct instr *i)
static inline int
is_jump(struct instr *i)
{
return is_bit_set_in_table(_PyOpcode_Jump, i->i_opcode);
return i->i_opcode >= SETUP_WITH || is_bit_set_in_table(_PyOpcode_Jump, i->i_opcode);
}
typedef struct basicblock_ {
@ -124,12 +145,18 @@ typedef struct basicblock_ {
int b_predecessors;
/* Basic block has no fall through (it ends with a return, raise or jump) */
unsigned b_nofallthrough : 1;
/* Basic block is an exception handler that preserves lasti */
unsigned b_preserve_lasti : 1;
/* Used by compiler passes to mark whether they have visited a basic block. */
unsigned b_visited : 1;
/* Basic block exits scope (it ends with a return or raise) */
unsigned b_exit : 1;
/* depth of stack upon entry of block, computed by stackdepth() */
int b_startdepth;
/* instruction offset for block, computed by assemble_jump_offsets() */
int b_offset;
/* Exception stack at start of block, used by assembler to create the exception handling table */
ExceptStack *b_exceptstack;
} basicblock;
/* fblockinfo tracks the current frame block.
@ -305,6 +332,8 @@ static int compiler_match(struct compiler *, stmt_ty);
static int compiler_pattern_subpattern(struct compiler *, pattern_ty,
pattern_context *);
static void clean_basic_block(basicblock *bb);
static PyCodeObject *assemble(struct compiler *, int addNone);
static PyObject *__doc__, *__annotations__;
@ -1029,11 +1058,6 @@ stack_effect(int opcode, int oparg, int jump)
case INPLACE_OR:
return -1;
case SETUP_WITH:
/* 1 in the normal flow.
* Restore the stack position and push 6 values before jumping to
* the handler if an exception be raised. */
return jump ? 6 : 1;
case RETURN_VALUE:
return -1;
case IMPORT_STAR:
@ -1048,6 +1072,8 @@ stack_effect(int opcode, int oparg, int jump)
return 0;
case POP_EXCEPT:
return -3;
case POP_EXCEPT_AND_RERAISE:
return -7;
case STORE_NAME:
return -1;
@ -1111,14 +1137,26 @@ stack_effect(int opcode, int oparg, int jump)
case LOAD_GLOBAL:
return 1;
/* Exception handling */
/* Exception handling pseudo-instructions */
case SETUP_FINALLY:
/* 0 in the normal flow.
* Restore the stack position and push 6 values before jumping to
* Restore the stack position and push 3 values before jumping to
* the handler if an exception be raised. */
return jump ? 6 : 0;
return jump ? 3 : 0;
case SETUP_CLEANUP:
/* As SETUP_FINALLY, but pushes lasti as well */
return jump ? 4 : 0;
case SETUP_WITH:
/* 0 in the normal flow.
* Restore the stack position to the position before the result
* of __(a)enter__ and push 4 values before jumping to the handler
* if an exception be raised. */
return jump ? -1 + 4 : 0;
case RERAISE:
return -3;
case PUSH_EXC_INFO:
return 3;
case WITH_EXCEPT_START:
return 1;
@ -1165,13 +1203,9 @@ stack_effect(int opcode, int oparg, int jump)
/* Iterators and generators */
case GET_AWAITABLE:
return 0;
case SETUP_ASYNC_WITH:
/* 0 in the normal flow.
* Restore the stack position to the position before the result
* of __aenter__ and push 6 values before jumping to the handler
* if an exception be raised. */
return jump ? -1 + 6 : 0;
case BEFORE_ASYNC_WITH:
case BEFORE_WITH:
return 1;
case GET_AITER:
return 0;
@ -1180,7 +1214,7 @@ stack_effect(int opcode, int oparg, int jump)
case GET_YIELD_FROM_ITER:
return 0;
case END_ASYNC_FOR:
return -7;
return -4;
case FORMAT_VALUE:
/* If there's a fmt_spec on the stack, we go from 2->1,
else 1->1. */
@ -1238,7 +1272,7 @@ compiler_addop_line(struct compiler *c, int opcode, int line)
basicblock *b;
struct instr *i;
int off;
assert(!HAS_ARG(opcode));
assert(!HAS_ARG(opcode) || IS_ARTIFICIAL(opcode));
off = compiler_next_instr(c->u->u_curblock);
if (off < 0)
return 0;
@ -1815,6 +1849,7 @@ compiler_unwind_fblock(struct compiler *c, struct fblockinfo *info,
if (preserve_tos) {
ADDOP(c, ROT_FOUR);
}
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_EXCEPT);
return 1;
@ -1845,6 +1880,7 @@ compiler_unwind_fblock(struct compiler *c, struct fblockinfo *info,
if (preserve_tos) {
ADDOP(c, ROT_FOUR);
}
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_EXCEPT);
if (info->fb_datum) {
ADDOP_LOAD_CONST(c, Py_None);
@ -3072,14 +3108,15 @@ compiler_continue(struct compiler *c)
static int
compiler_try_finally(struct compiler *c, stmt_ty s)
{
basicblock *body, *end, *exit;
basicblock *body, *end, *exit, *cleanup;
body = compiler_new_block(c);
end = compiler_new_block(c);
exit = compiler_new_block(c);
if (body == NULL || end == NULL || exit == NULL)
cleanup = compiler_new_block(c);
if (body == NULL || end == NULL || exit == NULL || cleanup == NULL) {
return 0;
}
/* `try` block */
ADDOP_JUMP(c, SETUP_FINALLY, end);
compiler_use_next_block(c, body);
@ -3098,11 +3135,17 @@ compiler_try_finally(struct compiler *c, stmt_ty s)
ADDOP_JUMP_NOLINE(c, JUMP_FORWARD, exit);
/* `finally` block */
compiler_use_next_block(c, end);
c->u->u_lineno = -1;
ADDOP_JUMP(c, SETUP_CLEANUP, cleanup);
ADDOP(c, PUSH_EXC_INFO);
if (!compiler_push_fblock(c, FINALLY_END, end, NULL, NULL))
return 0;
VISIT_SEQ(c, stmt, s->v.Try.finalbody);
compiler_pop_fblock(c, FINALLY_END, end);
ADDOP_I(c, RERAISE, 0);
compiler_use_next_block(c, cleanup);
ADDOP(c, POP_EXCEPT_AND_RERAISE);
compiler_use_next_block(c, exit);
return 1;
}
@ -3140,14 +3183,15 @@ compiler_try_finally(struct compiler *c, stmt_ty s)
static int
compiler_try_except(struct compiler *c, stmt_ty s)
{
basicblock *body, *orelse, *except, *end;
basicblock *body, *orelse, *except, *end, *cleanup;
Py_ssize_t i, n;
body = compiler_new_block(c);
except = compiler_new_block(c);
orelse = compiler_new_block(c);
end = compiler_new_block(c);
if (body == NULL || except == NULL || orelse == NULL || end == NULL)
cleanup = compiler_new_block(c);
if (body == NULL || except == NULL || orelse == NULL || end == NULL || cleanup == NULL)
return 0;
ADDOP_JUMP(c, SETUP_FINALLY, except);
compiler_use_next_block(c, body);
@ -3159,15 +3203,20 @@ compiler_try_except(struct compiler *c, stmt_ty s)
ADDOP_JUMP_NOLINE(c, JUMP_FORWARD, orelse);
n = asdl_seq_LEN(s->v.Try.handlers);
compiler_use_next_block(c, except);
c->u->u_lineno = -1;
ADDOP_JUMP(c, SETUP_CLEANUP, cleanup);
ADDOP(c, PUSH_EXC_INFO);
/* Runtime will push a block here, so we need to account for that */
if (!compiler_push_fblock(c, EXCEPTION_HANDLER, NULL, NULL, NULL))
return 0;
for (i = 0; i < n; i++) {
excepthandler_ty handler = (excepthandler_ty)asdl_seq_GET(
s->v.Try.handlers, i);
if (!handler->v.ExceptHandler.type && i < n-1)
return compiler_error(c, "default 'except:' must be last");
SET_LOC(c, handler);
if (!handler->v.ExceptHandler.type && i < n-1) {
return compiler_error(c, "default 'except:' must be last");
}
except = compiler_new_block(c);
if (except == NULL)
return 0;
@ -3202,7 +3251,7 @@ compiler_try_except(struct compiler *c, stmt_ty s)
*/
/* second try: */
ADDOP_JUMP(c, SETUP_FINALLY, cleanup_end);
ADDOP_JUMP(c, SETUP_CLEANUP, cleanup_end);
compiler_use_next_block(c, cleanup_body);
if (!compiler_push_fblock(c, HANDLER_CLEANUP, cleanup_body, NULL, handler->v.ExceptHandler.name))
return 0;
@ -3211,6 +3260,7 @@ compiler_try_except(struct compiler *c, stmt_ty s)
VISIT_SEQ(c, stmt, handler->v.ExceptHandler.body);
compiler_pop_fblock(c, HANDLER_CLEANUP, cleanup_body);
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_EXCEPT);
/* name = None; del name; # Mark as artificial */
c->u->u_lineno = -1;
@ -3224,6 +3274,7 @@ compiler_try_except(struct compiler *c, stmt_ty s)
/* name = None; del name; # Mark as artificial */
c->u->u_lineno = -1;
ADDOP_LOAD_CONST(c, Py_None);
compiler_nameop(c, handler->v.ExceptHandler.name, Store);
compiler_nameop(c, handler->v.ExceptHandler.name, Del);
@ -3246,15 +3297,18 @@ compiler_try_except(struct compiler *c, stmt_ty s)
compiler_pop_fblock(c, HANDLER_CLEANUP, cleanup_body);
/* name = None; del name; # Mark as artificial */
c->u->u_lineno = -1;
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_EXCEPT);
ADDOP_JUMP(c, JUMP_FORWARD, end);
}
compiler_use_next_block(c, except);
}
compiler_pop_fblock(c, EXCEPTION_HANDLER, NULL);
/* Mark as artificial */
c->u->u_lineno = -1;
compiler_pop_fblock(c, EXCEPTION_HANDLER, NULL);
ADDOP_I(c, RERAISE, 0);
compiler_use_next_block(c, cleanup);
ADDOP(c, POP_EXCEPT_AND_RERAISE);
compiler_use_next_block(c, orelse);
VISIT_SEQ(c, stmt, s->v.Try.orelse);
compiler_use_next_block(c, end);
@ -4764,6 +4818,8 @@ compiler_async_comprehension_generator(struct compiler *c,
compiler_pop_fblock(c, ASYNC_COMPREHENSION_GENERATOR, start);
compiler_use_next_block(c, except);
//c->u->u_lineno = -1;
ADDOP(c, END_ASYNC_FOR);
return 1;
@ -4944,20 +5000,24 @@ compiler_visit_keyword(struct compiler *c, keyword_ty k)
*/
static int
compiler_with_except_finish(struct compiler *c) {
compiler_with_except_finish(struct compiler *c, basicblock * cleanup) {
basicblock *exit;
exit = compiler_new_block(c);
if (exit == NULL)
return 0;
ADDOP_JUMP(c, POP_JUMP_IF_TRUE, exit);
NEXT_BLOCK(c);
ADDOP_I(c, RERAISE, 1);
ADDOP_I(c, RERAISE, 4);
compiler_use_next_block(c, cleanup);
ADDOP(c, POP_EXCEPT_AND_RERAISE);
compiler_use_next_block(c, exit);
ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP);
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_EXCEPT);
ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP);
return 1;
}
@ -4988,7 +5048,7 @@ compiler_with_except_finish(struct compiler *c) {
static int
compiler_async_with(struct compiler *c, stmt_ty s, int pos)
{
basicblock *block, *final, *exit;
basicblock *block, *final, *exit, *cleanup;
withitem_ty item = asdl_seq_GET(s->v.AsyncWith.items, pos);
assert(s->kind == AsyncWith_kind);
@ -5001,7 +5061,8 @@ compiler_async_with(struct compiler *c, stmt_ty s, int pos)
block = compiler_new_block(c);
final = compiler_new_block(c);
exit = compiler_new_block(c);
if (!block || !final || !exit)
cleanup = compiler_new_block(c);
if (!block || !final || !exit || !cleanup)
return 0;
/* Evaluate EXPR */
@ -5012,9 +5073,9 @@ compiler_async_with(struct compiler *c, stmt_ty s, int pos)
ADDOP_LOAD_CONST(c, Py_None);
ADDOP(c, YIELD_FROM);
ADDOP_JUMP(c, SETUP_ASYNC_WITH, final);
ADDOP_JUMP(c, SETUP_WITH, final);
/* SETUP_ASYNC_WITH pushes a finally block. */
/* SETUP_WITH pushes a finally block. */
compiler_use_next_block(c, block);
if (!compiler_push_fblock(c, ASYNC_WITH, block, final, s)) {
return 0;
@ -5055,13 +5116,17 @@ compiler_async_with(struct compiler *c, stmt_ty s, int pos)
/* For exceptional outcome: */
compiler_use_next_block(c, final);
c->u->u_lineno = -1;
ADDOP_JUMP(c, SETUP_CLEANUP, cleanup);
ADDOP(c, PUSH_EXC_INFO);
ADDOP(c, WITH_EXCEPT_START);
ADDOP(c, GET_AWAITABLE);
ADDOP_LOAD_CONST(c, Py_None);
ADDOP(c, YIELD_FROM);
compiler_with_except_finish(c);
compiler_with_except_finish(c, cleanup);
compiler_use_next_block(c, exit);
compiler_use_next_block(c, exit);
return 1;
}
@ -5090,7 +5155,7 @@ compiler_use_next_block(c, exit);
static int
compiler_with(struct compiler *c, stmt_ty s, int pos)
{
basicblock *block, *final, *exit;
basicblock *block, *final, *exit, *cleanup;
withitem_ty item = asdl_seq_GET(s->v.With.items, pos);
assert(s->kind == With_kind);
@ -5098,12 +5163,14 @@ compiler_with(struct compiler *c, stmt_ty s, int pos)
block = compiler_new_block(c);
final = compiler_new_block(c);
exit = compiler_new_block(c);
if (!block || !final || !exit)
cleanup = compiler_new_block(c);
if (!block || !final || !exit || !cleanup)
return 0;
/* Evaluate EXPR */
VISIT(c, expr, item->context_expr);
/* Will push bound __exit__ */
ADDOP(c, BEFORE_WITH);
ADDOP_JUMP(c, SETUP_WITH, final);
/* SETUP_WITH pushes a finally block. */
@ -5146,8 +5213,12 @@ compiler_with(struct compiler *c, stmt_ty s, int pos)
/* For exceptional outcome: */
compiler_use_next_block(c, final);
c->u->u_lineno = -1;
ADDOP_JUMP(c, SETUP_CLEANUP, cleanup);
ADDOP(c, PUSH_EXC_INFO);
ADDOP(c, WITH_EXCEPT_START);
compiler_with_except_finish(c);
compiler_with_except_finish(c, cleanup);
compiler_use_next_block(c, exit);
return 1;
@ -6383,11 +6454,13 @@ compiler_match(struct compiler *c, stmt_ty s)
*/
struct assembler {
PyObject *a_bytecode; /* string containing bytecode */
PyObject *a_bytecode; /* bytes containing bytecode */
int a_offset; /* offset into bytecode */
int a_nblocks; /* number of reachable blocks */
PyObject *a_lnotab; /* string containing lnotab */
PyObject *a_lnotab; /* bytes containing lnotab */
int a_lnotab_off; /* offset into lnotab */
PyObject *a_except_table; /* bytes containing exception table */
int a_except_table_off; /* offset into exception table */
int a_prevlineno; /* lineno of last emitted line in line table */
int a_lineno; /* lineno of last emitted instruction */
int a_lineno_start; /* bytecode start offset of current lineno */
@ -6466,7 +6539,8 @@ stackdepth(struct compiler *c)
instr->i_opcode == JUMP_FORWARD ||
instr->i_opcode == RETURN_VALUE ||
instr->i_opcode == RAISE_VARARGS ||
instr->i_opcode == RERAISE)
instr->i_opcode == RERAISE ||
instr->i_opcode == POP_EXCEPT_AND_RERAISE)
{
/* remaining code is dead */
next = NULL;
@ -6488,6 +6562,7 @@ assemble_init(struct assembler *a, int nblocks, int firstlineno)
memset(a, 0, sizeof(struct assembler));
a->a_prevlineno = a->a_lineno = firstlineno;
a->a_lnotab = NULL;
a->a_except_table = NULL;
a->a_bytecode = PyBytes_FromStringAndSize(NULL, DEFAULT_CODE_SIZE);
if (a->a_bytecode == NULL) {
goto error;
@ -6496,6 +6571,10 @@ assemble_init(struct assembler *a, int nblocks, int firstlineno)
if (a->a_lnotab == NULL) {
goto error;
}
a->a_except_table = PyBytes_FromStringAndSize(NULL, DEFAULT_LNOTAB_SIZE);
if (a->a_except_table == NULL) {
goto error;
}
if ((size_t)nblocks > SIZE_MAX / sizeof(basicblock *)) {
PyErr_NoMemory();
goto error;
@ -6504,6 +6583,7 @@ assemble_init(struct assembler *a, int nblocks, int firstlineno)
error:
Py_XDECREF(a->a_bytecode);
Py_XDECREF(a->a_lnotab);
Py_XDECREF(a->a_except_table);
return 0;
}
@ -6512,6 +6592,7 @@ assemble_free(struct assembler *a)
{
Py_XDECREF(a->a_bytecode);
Py_XDECREF(a->a_lnotab);
Py_XDECREF(a->a_except_table);
}
static int
@ -6541,6 +6622,253 @@ assemble_emit_linetable_pair(struct assembler *a, int bdelta, int ldelta)
return 1;
}
static int
is_block_push(struct instr *instr)
{
int opcode = instr->i_opcode;
return opcode == SETUP_FINALLY || opcode == SETUP_WITH || opcode == SETUP_CLEANUP;
}
static basicblock *
push_except_block(ExceptStack *stack, struct instr *setup) {
assert(is_block_push(setup));
int opcode = setup->i_opcode;
basicblock * target = setup->i_target;
if (opcode == SETUP_WITH || opcode == SETUP_CLEANUP) {
target->b_preserve_lasti = 1;
}
stack->handlers[++stack->depth] = target;
return target;
}
static basicblock *
pop_except_block(ExceptStack *stack) {
assert(stack->depth > 0);
return stack->handlers[--stack->depth];
}
static basicblock *
except_stack_top(ExceptStack *stack) {
return stack->handlers[stack->depth];
}
static ExceptStack *
make_except_stack(void) {
ExceptStack *new = PyMem_Malloc(sizeof(ExceptStack));
if (new == NULL) {
PyErr_NoMemory();
return NULL;
}
new->depth = 0;
new->handlers[0] = NULL;
return new;
}
static ExceptStack *
copy_except_stack(ExceptStack *stack) {
ExceptStack *copy = PyMem_Malloc(sizeof(ExceptStack));
if (copy == NULL) {
PyErr_NoMemory();
return NULL;
}
memcpy(copy, stack, sizeof(ExceptStack));
return copy;
}
static int
label_exception_targets(basicblock *entry) {
int nblocks = 0;
for (basicblock *b = entry; b != NULL; b = b->b_next) {
b->b_visited = 0;
nblocks++;
}
basicblock **todo_stack = PyMem_Malloc(sizeof(basicblock *)*nblocks);
if (todo_stack == NULL) {
PyErr_NoMemory();
return -1;
}
ExceptStack *except_stack = make_except_stack();
if (except_stack == NULL) {
PyMem_Free(todo_stack);
PyErr_NoMemory();
return -1;
}
except_stack->depth = 0;
todo_stack[0] = entry;
entry->b_visited = 1;
entry->b_exceptstack = except_stack;
basicblock **todo = &todo_stack[1];
basicblock *handler = NULL;
while (todo > todo_stack) {
todo--;
basicblock *b = todo[0];
assert(b->b_visited == 1);
except_stack = b->b_exceptstack;
assert(except_stack != NULL);
b->b_exceptstack = NULL;
handler = except_stack_top(except_stack);
for (int i = 0; i < b->b_iused; i++) {
struct instr *instr = &b->b_instr[i];
if (is_block_push(instr)) {
if (!instr->i_target->b_visited) {
ExceptStack *copy = copy_except_stack(except_stack);
if (copy == NULL) {
goto error;
}
instr->i_target->b_exceptstack = copy;
todo[0] = instr->i_target;
instr->i_target->b_visited = 1;
todo++;
}
handler = push_except_block(except_stack, instr);
}
else if (instr->i_opcode == POP_BLOCK) {
handler = pop_except_block(except_stack);
}
else if (is_jump(instr)) {
instr->i_except = handler;
assert(i == b->b_iused -1);
if (!instr->i_target->b_visited) {
if (b->b_nofallthrough == 0) {
ExceptStack *copy = copy_except_stack(except_stack);
if (copy == NULL) {
goto error;
}
instr->i_target->b_exceptstack = copy;
}
else {
instr->i_target->b_exceptstack = except_stack;
except_stack = NULL;
}
todo[0] = instr->i_target;
instr->i_target->b_visited = 1;
todo++;
}
}
else {
instr->i_except = handler;
}
}
if (b->b_nofallthrough == 0 && !b->b_next->b_visited) {
assert(except_stack != NULL);
b->b_next->b_exceptstack = except_stack;
todo[0] = b->b_next;
b->b_next->b_visited = 1;
todo++;
}
else if (except_stack != NULL) {
PyMem_Free(except_stack);
}
}
#ifdef Py_DEBUG
for (basicblock *b = entry; b != NULL; b = b->b_next) {
assert(b->b_exceptstack == NULL);
}
#endif
PyMem_Free(todo_stack);
return 0;
error:
PyMem_Free(todo_stack);
PyMem_Free(except_stack);
return -1;
}
static void
convert_exception_handlers_to_nops(basicblock *entry) {
for (basicblock *b = entry; b != NULL; b = b->b_next) {
for (int i = 0; i < b->b_iused; i++) {
struct instr *instr = &b->b_instr[i];
if (is_block_push(instr) || instr->i_opcode == POP_BLOCK) {
instr->i_opcode = NOP;
}
}
}
}
static inline void
write_except_byte(struct assembler *a, int byte) {
unsigned char *p = (unsigned char *) PyBytes_AS_STRING(a->a_except_table);
p[a->a_except_table_off++] = byte;
}
#define CONTINUATION_BIT 64
static void
assemble_emit_exception_table_item(struct assembler *a, int value, int msb)
{
assert ((msb | 128) == 128);
assert(value >= 0 && value < (1 << 30));
if (value >= 1 << 24) {
write_except_byte(a, (value >> 24) | CONTINUATION_BIT | msb);
msb = 0;
}
if (value >= 1 << 18) {
write_except_byte(a, ((value >> 18)&0x3f) | CONTINUATION_BIT | msb);
msb = 0;
}
if (value >= 1 << 12) {
write_except_byte(a, ((value >> 12)&0x3f) | CONTINUATION_BIT | msb);
msb = 0;
}
if (value >= 1 << 6) {
write_except_byte(a, ((value >> 6)&0x3f) | CONTINUATION_BIT | msb);
msb = 0;
}
write_except_byte(a, (value&0x3f) | msb);
}
/* See Objects/exception_table_notes.txt for details of layout */
#define MAX_SIZE_OF_ENTRY 20
static int
assemble_emit_exception_table_entry(struct assembler *a, int start, int end, basicblock *handler)
{
Py_ssize_t len = PyBytes_GET_SIZE(a->a_except_table);
if (a->a_except_table_off + MAX_SIZE_OF_ENTRY >= len) {
if (_PyBytes_Resize(&a->a_except_table, len * 2) < 0)
return 0;
}
int size = end-start;
assert(end > start);
int target = handler->b_offset;
int depth = handler->b_preserve_lasti ? handler->b_startdepth-4 : handler->b_startdepth-3;
assert(depth >= 0);
int depth_lasti = (depth<<1) | handler->b_preserve_lasti;
assemble_emit_exception_table_item(a, start, (1<<7));
assemble_emit_exception_table_item(a, size, 0);
assemble_emit_exception_table_item(a, target, 0);
assemble_emit_exception_table_item(a, depth_lasti, 0);
return 1;
}
static int
assemble_exception_table(struct assembler *a)
{
basicblock *b;
int ioffset = 0;
basicblock *handler = NULL;
int start = -1;
for (b = a->a_entry; b != NULL; b = b->b_next) {
ioffset = b->b_offset;
for (int i = 0; i < b->b_iused; i++) {
struct instr *instr = &b->b_instr[i];
if (instr->i_except != handler) {
if (handler != NULL) {
RETURN_IF_FALSE(assemble_emit_exception_table_entry(a, start, ioffset, handler));
}
start = ioffset;
handler = instr->i_except;
}
ioffset += instrsize(instr->i_oparg);
}
}
if (handler != NULL) {
RETURN_IF_FALSE(assemble_emit_exception_table_entry(a, start, ioffset, handler));
}
return 1;
}
/* Appends a range to the end of the line number table. See
* Objects/lnotab_notes.txt for the description of the line number table. */
@ -6793,7 +7121,7 @@ merge_const_one(struct compiler *c, PyObject **obj)
}
static PyCodeObject *
makecode(struct compiler *c, struct assembler *a, PyObject *consts)
makecode(struct compiler *c, struct assembler *a, PyObject *consts, int maxdepth)
{
PyCodeObject *co = NULL;
PyObject *names = NULL;
@ -6804,7 +7132,7 @@ makecode(struct compiler *c, struct assembler *a, PyObject *consts)
Py_ssize_t nlocals;
int nlocals_int;
int flags;
int posorkeywordargcount, posonlyargcount, kwonlyargcount, maxdepth;
int posorkeywordargcount, posonlyargcount, kwonlyargcount;
names = dict_keys_inorder(c->u->u_names, 0);
varnames = dict_keys_inorder(c->u->u_varnames, 0);
@ -6846,23 +7174,11 @@ makecode(struct compiler *c, struct assembler *a, PyObject *consts)
posonlyargcount = Py_SAFE_DOWNCAST(c->u->u_posonlyargcount, Py_ssize_t, int);
posorkeywordargcount = Py_SAFE_DOWNCAST(c->u->u_argcount, Py_ssize_t, int);
kwonlyargcount = Py_SAFE_DOWNCAST(c->u->u_kwonlyargcount, Py_ssize_t, int);
maxdepth = stackdepth(c);
if (maxdepth < 0) {
Py_DECREF(consts);
goto error;
}
if (maxdepth > MAX_ALLOWED_STACK_USE) {
PyErr_Format(PyExc_SystemError,
"excessive stack use: stack is %d deep",
maxdepth);
Py_DECREF(consts);
goto error;
}
co = PyCode_NewWithPosOnlyArgs(posonlyargcount+posorkeywordargcount,
posonlyargcount, kwonlyargcount, nlocals_int,
maxdepth, flags, a->a_bytecode, consts, names,
varnames, freevars, cellvars, c->c_filename,
c->u->u_name, c->u->u_firstlineno, a->a_lnotab);
c->u->u_name, c->u->u_firstlineno, a->a_lnotab, a->a_except_table);
Py_DECREF(consts);
error:
Py_XDECREF(names);
@ -7015,6 +7331,25 @@ assemble(struct compiler *c, int addNone)
goto error;
}
int maxdepth = stackdepth(c);
if (maxdepth < 0) {
goto error;
}
if (maxdepth > MAX_ALLOWED_STACK_USE) {
PyErr_Format(PyExc_SystemError,
"excessive stack use: stack is %d deep",
maxdepth);
goto error;
}
if (label_exception_targets(entryblock)) {
goto error;
}
convert_exception_handlers_to_nops(entryblock);
for (basicblock *b = a.a_entry; b != NULL; b = b->b_next) {
clean_basic_block(b);
}
/* Can't modify the bytecode after computing jump offsets. */
assemble_jump_offsets(&a, c);
@ -7024,6 +7359,10 @@ assemble(struct compiler *c, int addNone)
if (!assemble_emit(&a, &b->b_instr[j]))
goto error;
}
if (!assemble_exception_table(&a)) {
return 0;
}
if (!assemble_line_range(&a)) {
return 0;
}
@ -7031,6 +7370,11 @@ assemble(struct compiler *c, int addNone)
if (_PyBytes_Resize(&a.a_lnotab, a.a_lnotab_off) < 0) {
goto error;
}
if (_PyBytes_Resize(&a.a_except_table, a.a_except_table_off) < 0) {
goto error;
}
if (!merge_const_one(c, &a.a_lnotab)) {
goto error;
}
@ -7041,7 +7385,7 @@ assemble(struct compiler *c, int addNone)
goto error;
}
co = makecode(c, &a, consts);
co = makecode(c, &a, consts, maxdepth);
error:
Py_XDECREF(consts);
assemble_free(&a);
@ -7417,9 +7761,10 @@ optimize_basic_block(struct compiler *c, basicblock *bb, PyObject *consts)
static void
clean_basic_block(basicblock *bb, int prev_lineno) {
clean_basic_block(basicblock *bb) {
/* Remove NOPs when legal to do so. */
int dest = 0;
int prev_lineno = -1;
for (int src = 0; src < bb->b_iused; src++) {
int lineno = bb->b_instr[src].i_lineno;
if (bb->b_instr[src].i_opcode == NOP) {
@ -7451,7 +7796,6 @@ clean_basic_block(basicblock *bb, int prev_lineno) {
}
}
}
}
if (dest != src) {
bb->b_instr[dest] = bb->b_instr[src];
@ -7472,6 +7816,7 @@ normalize_basic_block(basicblock *bb) {
case RETURN_VALUE:
case RAISE_VARARGS:
case RERAISE:
case POP_EXCEPT_AND_RERAISE:
bb->b_exit = 1;
bb->b_nofallthrough = 1;
break;
@ -7588,9 +7933,9 @@ propogate_line_numbers(struct assembler *a) {
if (is_jump(&b->b_instr[b->b_iused-1])) {
switch (b->b_instr[b->b_iused-1].i_opcode) {
/* Note: Only actual jumps, not exception handlers */
case SETUP_ASYNC_WITH:
case SETUP_WITH:
case SETUP_FINALLY:
case SETUP_CLEANUP:
continue;
}
basicblock *target = b->b_instr[b->b_iused-1].i_target;
@ -7619,7 +7964,7 @@ optimize_cfg(struct compiler *c, struct assembler *a, PyObject *consts)
if (optimize_basic_block(c, b, consts)) {
return -1;
}
clean_basic_block(b, -1);
clean_basic_block(b);
assert(b->b_predecessors == 0);
}
if (mark_reachable(a)) {
@ -7632,16 +7977,10 @@ optimize_cfg(struct compiler *c, struct assembler *a, PyObject *consts)
b->b_nofallthrough = 0;
}
}
basicblock *pred = NULL;
for (basicblock *b = a->a_entry; b != NULL; b = b->b_next) {
int prev_lineno = -1;
if (pred && pred->b_iused) {
prev_lineno = pred->b_instr[pred->b_iused-1].i_lineno;
}
clean_basic_block(b, prev_lineno);
pred = b->b_nofallthrough ? NULL : b;
}
eliminate_empty_basic_blocks(a->a_entry);
for (basicblock *b = a->a_entry; b != NULL; b = b->b_next) {
clean_basic_block(b);
}
/* Delete jump instructions made redundant by previous step. If a non-empty
block ends with a jump instruction, check if the next non-empty block
reached through normal flow control is the target of that jump. If it
@ -7657,7 +7996,6 @@ optimize_cfg(struct compiler *c, struct assembler *a, PyObject *consts)
assert(b->b_next->b_iused);
b->b_nofallthrough = 0;
b_last_instr->i_opcode = NOP;
clean_basic_block(b, -1);
maybe_empty_blocks = 1;
}
}
@ -7691,12 +8029,13 @@ ensure_exits_have_lineno(struct compiler *c)
/* Copy all exit blocks without line number that are targets of a jump.
*/
for (basicblock *b = c->u->u_blocks; b != NULL; b = b->b_list) {
entry = b;
if (b->b_iused > 0 && is_jump(&b->b_instr[b->b_iused-1])) {
switch (b->b_instr[b->b_iused-1].i_opcode) {
/* Note: Only actual jumps, not exception handlers */
case SETUP_ASYNC_WITH:
case SETUP_WITH:
case SETUP_FINALLY:
case SETUP_CLEANUP:
continue;
}
basicblock *target = b->b_instr[b->b_iused-1].i_target;
@ -7709,7 +8048,6 @@ ensure_exits_have_lineno(struct compiler *c)
b->b_instr[b->b_iused-1].i_target = new_target;
}
}
entry = b;
}
assert(entry != NULL);
if (is_exit_without_lineno(entry)) {

2
Python/frozen_hello.h
View file

@ -8,5 +8,5 @@ const unsigned char _Py_M__hello[] = {
5,112,114,105,110,116,169,0,114,1,0,0,0,114,1,0,
0,0,122,14,60,102,114,111,122,101,110,32,104,101,108,108,
111,62,218,8,60,109,111,100,117,108,101,62,1,0,0,0,
115,4,0,0,0,4,0,12,1,
115,4,0,0,0,4,0,12,1,243,0,0,0,0,
};

3507
Python/importlib.h generated
View file

File diff suppressed because it is too large Load diff

4981
Python/importlib_external.h generated
View file

File diff suppressed because it is too large Load diff

1878
Python/importlib_zipimport.h
View file

File diff suppressed because it is too large Load diff

9
Python/marshal.c
View file

@ -525,6 +525,7 @@ w_complex_object(PyObject *v, char flag, WFILE *p)
w_object(co->co_name, p);
w_long(co->co_firstlineno, p);
w_object(co->co_linetable, p);
w_object(co->co_exceptiontable, p);
}
else if (PyObject_CheckBuffer(v)) {
/* Write unknown bytes-like objects as a bytes object */
@ -1313,6 +1314,7 @@ r_object(RFILE *p)
PyObject *name = NULL;
int firstlineno;
PyObject *linetable = NULL;
PyObject *exceptiontable = NULL;
idx = r_ref_reserve(flag, p);
if (idx < 0)
@ -1370,6 +1372,10 @@ r_object(RFILE *p)
linetable = r_object(p);
if (linetable == NULL)
goto code_error;
exceptiontable = r_object(p);
if (exceptiontable == NULL)
goto code_error;
if (PySys_Audit("code.__new__", "OOOiiiiii",
code, filename, name, argcount, posonlyargcount,
@ -1382,7 +1388,7 @@ r_object(RFILE *p)
nlocals, stacksize, flags,
code, consts, names, varnames,
freevars, cellvars, filename, name,
firstlineno, linetable);
firstlineno, linetable, exceptiontable);
v = r_ref_insert(v, idx, flag, p);
code_error:
@ -1395,6 +1401,7 @@ r_object(RFILE *p)
Py_XDECREF(filename);
Py_XDECREF(name);
Py_XDECREF(linetable);
Py_XDECREF(exceptiontable);
}
retval = v;
break;

14
Python/opcode_targets.h generated
View file

@ -34,9 +34,9 @@ static void *opcode_targets[256] = {
&&TARGET_MATCH_SEQUENCE,
&&TARGET_MATCH_KEYS,
&&TARGET_COPY_DICT_WITHOUT_KEYS,
&&TARGET_PUSH_EXC_INFO,
&&_unknown_opcode,
&&_unknown_opcode,
&&_unknown_opcode,
&&TARGET_POP_EXCEPT_AND_RERAISE,
&&_unknown_opcode,
&&_unknown_opcode,
&&_unknown_opcode,
@ -52,7 +52,7 @@ static void *opcode_targets[256] = {
&&TARGET_GET_AITER,
&&TARGET_GET_ANEXT,
&&TARGET_BEFORE_ASYNC_WITH,
&&_unknown_opcode,
&&TARGET_BEFORE_WITH,
&&TARGET_END_ASYNC_FOR,
&&TARGET_INPLACE_ADD,
&&TARGET_INPLACE_SUBTRACT,
@ -86,7 +86,7 @@ static void *opcode_targets[256] = {
&&TARGET_IMPORT_STAR,
&&TARGET_SETUP_ANNOTATIONS,
&&TARGET_YIELD_VALUE,
&&TARGET_POP_BLOCK,
&&_unknown_opcode,
&&_unknown_opcode,
&&TARGET_POP_EXCEPT,
&&TARGET_STORE_NAME,
@ -121,7 +121,7 @@ static void *opcode_targets[256] = {
&&TARGET_RERAISE,
&&_unknown_opcode,
&&TARGET_JUMP_IF_NOT_EXC_MATCH,
&&TARGET_SETUP_FINALLY,
&&_unknown_opcode,
&&_unknown_opcode,
&&TARGET_LOAD_FAST,
&&TARGET_STORE_FAST,
@ -142,7 +142,7 @@ static void *opcode_targets[256] = {
&&_unknown_opcode,
&&TARGET_CALL_FUNCTION_KW,
&&TARGET_CALL_FUNCTION_EX,
&&TARGET_SETUP_WITH,
&&_unknown_opcode,
&&TARGET_EXTENDED_ARG,
&&TARGET_LIST_APPEND,
&&TARGET_SET_ADD,
@ -153,7 +153,7 @@ static void *opcode_targets[256] = {
&&_unknown_opcode,
&&TARGET_MATCH_CLASS,
&&_unknown_opcode,
&&TARGET_SETUP_ASYNC_WITH,
&&_unknown_opcode,
&&TARGET_FORMAT_VALUE,
&&TARGET_BUILD_CONST_KEY_MAP,
&&TARGET_BUILD_STRING,

10
Tools/scripts/generate_opcode_h.py
View file

@ -16,14 +16,12 @@
""".lstrip()
footer = """
/* EXCEPT_HANDLER is a special, implicit block type which is created when
entering an except handler. It is not an opcode but we define it here
as we want it to be available to both frameobject.c and ceval.c, while
remaining private.*/
#define EXCEPT_HANDLER 257
#define HAS_ARG(op) ((op) >= HAVE_ARGUMENT)
/* Reserve some bytecodes for internal use in the compiler.
* The value of 240 is arbitrary. */
#define IS_ARTIFICIAL(op) ((op) > 240)
#ifdef __cplusplus
}
#endif