321 commits

Author	SHA1	Message	Date
Sam Gross	5aa91c56bf	gh-124296: Remove private dictionary version tag (PEP 699) (#124472)	2024-10-01 12:39:56 -04:00
Ken Jin	198756b0f6	gh-117376: Fix off-by-ones in conversion functions (GH-124301) ... Fix off-by-ones in conversion function	2024-09-26 02:41:07 +08:00
Sam Gross	f4997bb3ac	gh-123923: Defer refcounting for f_funcobj in _PyInterpreterFrame (#124026) ... Use a `_PyStackRef` and defer the reference to `f_funcobj` when possible. This avoids some reference count contention in the common case of executing the same code object from multiple threads concurrently in the free-threaded build.	2024-09-24 20:08:18 +00:00
Ken Jin	8810e286fa	gh-121459: Deferred LOAD_GLOBAL (GH-123128) ... Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Sam Gross <655866+colesbury@users.noreply.github.com>	2024-09-14 00:23:51 +08:00
Sam Gross	b2afe2aae4	gh-123923: Defer refcounting for f_executable in _PyInterpreterFrame (#123924) ... Use a `_PyStackRef` and defer the reference to `f_executable` when possible. This avoids some reference count contention in the common case of executing the same code object from multiple threads concurrently in the free-threaded build.	2024-09-12 12:37:06 -04:00
Mark Shannon	4ed7d1d6ac	GH-123996: Explicitly mark 'self_or_null' as an array of size 1 to ensure that it is kept in memory for calls (GH-124003)	2024-09-12 15:32:45 +01:00
Victor Stinner	f1a0d96f41	gh-123091: Use _Py_IsImmortalLoose() (#123511) ... Use _Py_IsImmortalLoose() in bytesobject.c, typeobject.c and ceval.c.	2024-09-02 14:25:19 +02:00
Mark Shannon	54a05a4600	GH-123232: Factor BINARY_SLICE and STORE_SLICE to handle stats properly for tier 2. (GH-123381)	2024-08-27 10:49:39 +01:00
Kirill Podoprigora	67f2c84bff	gh-123205: Python/bytecodes.c: Fix compiler warning (#123206) ... Fix MSVC warning "conversion from '__int64' to 'int'"	2024-08-23 15:35:25 -04:00
Mark Shannon	0b0f7befad	GH-123232: Fix "not specialized" stats (GH-123236)	2024-08-23 10:46:03 +01:00
Mark Shannon	5d3201fe3f	GH-123040: Specialize shadowed LOAD_ATTR. (GH-123219)	2024-08-23 10:22:35 +01:00
Donghee Na	297f2e093e	gh-123083: Fix a potential use-after-free in `STORE_ATTR_WITH_HINT` (gh-123092)	2024-08-22 23:49:09 +09:00
Mark Shannon	a3d8c0542e	GH-123197: Only count an instruction as deferred if it hasn't deopted first. (GH-123222) ... Only count an instruction as deferred if hasn't deopted first.	2024-08-22 14:17:10 +01:00
Mark Shannon	a4fd7aa4a6	GH-115776: Allow any fixed sized object to have inline values (GH-123192)	2024-08-21 15:52:04 +01:00
Mark Shannon	7b26c4d1e3	GH-123197: Increment correct stat for CALL_KW (GH-123200)	2024-08-21 12:52:28 +01:00
Mark Shannon	1eba8bae92	GH-123185: Check for NULL after calling _PyEvalFramePushAndInit (GH-123194)	2024-08-21 12:44:56 +01:00
Mark Shannon	bb1d30336e	GH-118093: Make CALL_ALLOC_AND_ENTER_INIT suitable for tier 2. (GH-123140) ... * Convert CALL_ALLOC_AND_ENTER_INIT to micro-ops such that tier 2 supports it * Allow inexact arguments for CALL_ALLOC_AND_ENTER_INIT.	2024-08-20 16:52:58 +01:00
Mark Shannon	c13e7d98fb	GH-118093: Specialize CALL_KW (GH-123006)	2024-08-16 17:11:24 +01:00
Brandt Bucher	f84754b705	GH-118093: Turn some DEOPT_IFs into EXIT_IFs (GH-122998)	2024-08-14 07:54:42 -07:00
Mark Shannon	eec7bdaf01	GH-120024: Remove CHECK_EVAL_BREAKER macro. (GH-122968) ... * Factor some instructions into micro-ops to isolate CHECK_EVAL_BREAKER for escape analysis * Eliminate CHECK_EVAL_BREAKER macro	2024-08-14 12:04:05 +01:00
Brandt Bucher	9621a7d017	GH-118093: Handle some polymorphism before requiring progress in tier two (GH-122843)	2024-08-12 12:39:31 -07:00
Sam Gross	ab094d1b2b	gh-117139: Replace _PyList_FromArraySteal with stack ref variant (#122830) ... This replaces `_PyList_FromArraySteal` with `_PyList_FromStackRefSteal`. It's functionally equivalent, but takes a `_PyStackRef` array instead of an array of `PyObject` pointers. Co-authored-by: Ken Jin <kenjin@python.org>	2024-08-12 14:49:49 -04:00
Sam Gross	736fe4d23e	gh-117139: Fix a few _PyStackRef related bugs (#122831) ... `BUILD_SET` should use a borrow instead of a steal. The cleanup in `_DO_CALL` `CONVERSION_FAILED` was incorrect. Co-authored-by: Ken Jin <kenjin@python.org>	2024-08-12 14:49:33 -04:00
Mark Shannon	81c739e2dc	GH-120024: Tidy up case generator code a bit. (GH-122780)	2024-08-08 10:57:59 +01:00
Sam Gross	3e753c689a	gh-118926: Spill deferred references to stack in cases generator (#122748) ... This automatically spills the results from `_PyStackRef_FromPyObjectNew` to the in-memory stack so that the deferred references are visible to the GC before we make any possibly escaping call. Co-authored-by: Ken Jin <kenjin@python.org>	2024-08-07 13:23:53 -04:00
sobolevn	61a8bf2853	gh-122759: Remove assert from RERAISE error handling (#122760)	2024-08-07 17:25:25 +03:00
Sam Gross	674a50ef2f	gh-117139: Fix an incorrect borrow in bytecodes.c (#122318) ... `_PyDict_SetItem_Take2` steals both the key (i.e., `sub`) and the value.	2024-08-07 19:06:19 +05:30
Mark Shannon	4c31791848	GH-120024: Move three more escaping calls out of conditional statements (GH-122734)	2024-08-06 14:14:52 +01:00
Mark Shannon	a8be8fc6c4	GH-120024: Refactor code a bit so that escaping calls can be wrapped in spill code in code generator (GH-122693)	2024-08-06 08:40:39 +01:00
Mark Shannon	5bd72912a1	GH-122616: Simplify LOAD_ATTR_WITH_HINT and STORE_ATTR_WITH_HINT (GH-122620)	2024-08-05 16:27:48 +01:00
Mark Shannon	7aca84e557	GH-117224: Move the body of a few large-ish micro-ops into helper functions (GH-122601)	2024-08-02 16:31:17 +01:00
Mark Shannon	df13a1821a	GH-118095: Add tier two support for BINARY_SUBSCR_GETITEM (GH-120793)	2024-08-01 16:19:05 -07:00
Mark Shannon	a9d56e38a0	GH-122155: Track local variables between pops and pushes in cases generator (GH-122286)	2024-08-01 09:27:26 +01:00
Brandt Bucher	15d4cd0967	GH-116090: Fire RAISE events from _FOR_ITER_TIER_TWO (GH-122413)	2024-07-29 12:17:47 -07:00
Mark Shannon	95a73917cd	GH-122029: Break INSTRUMENTED_CALL into micro-ops, so that its behavior is consistent with CALL (GH-122177)	2024-07-26 14:35:57 +01:00
Mark Shannon	afb0aa6ed2	GH-121131: Clean up and fix some instrumented instructions. (GH-121132) ... * Add support for 'prev_instr' to code generator and refactor some INSTRUMENTED instructions	2024-07-26 12:24:12 +01:00
Brandt Bucher	d9efa45d74	GH-118093: Add tier two support for BINARY_OP_INPLACE_ADD_UNICODE (GH-122253)	2024-07-25 14:45:07 -07:00
Brandt Bucher	5f6001130f	GH-118093: Add tier two support for LOAD_ATTR_PROPERTY (GH-122283)	2024-07-25 10:45:28 -07:00
Mark Shannon	5e686ff57d	GH-122034: Add StackRef variants of type checks to reduce the number of PyStackRef_AsPyObjectBorrow calls (GH-122037)	2024-07-25 18:32:43 +01:00
Mark Shannon	2e14a52cce	GH-122160: Remove BUILD_CONST_KEY_MAP opcode. (GH-122164)	2024-07-25 16:24:29 +01:00
Brandt Bucher	7b36b67b1e	GH-118093: Add tier two support to several instructions (GH-121884)	2024-07-18 14:24:58 -07:00
Mark Shannon	3eacfc1a4d	GH-121784: Generate an error during code gen if a variable is marked unused, but is used and thus cached in a prior uop. (#121788) ... * Reject uop definitions that declare values as 'unused' that are already cached by prior uops * Track which variables are defined and only load from memory when needed * Support explicit `flush` in macro definitions. * Make sure stack is flushed in where needed.	2024-07-18 12:49:24 +01:00
Sam Gross	8e8d202f55	gh-117139: Add _PyTuple_FromStackRefSteal and use it (#121244) ... Avoids the extra conversion from stack refs to PyObjects.	2024-07-02 12:30:14 -04:00
Ken Jin	e6543daf12	gh-117139: Fix a few wrong steals in bytecodes.c (GH-121127) ... Fix a few wrong steals in bytecodes.c	2024-06-29 02:14:48 +08:00
Victor Stinner	12af8ec864	gh-121040: Use __attribute__((fallthrough)) (#121044) ... Fix warnings when using -Wimplicit-fallthrough compiler flag. Annotate explicitly "fall through" switch cases with a new _Py_FALLTHROUGH macro which uses __attribute__((fallthrough)) if available. Replace "fall through" comments with _Py_FALLTHROUGH. Add _Py__has_attribute() macro. No longer define __has_attribute() macro if it's not defined. Move also _Py__has_builtin() at the top of pyport.h. Co-Authored-By: Nikita Sobolev <mail@sobolevn.me>	2024-06-27 09:58:44 +00:00
Ken Jin	22b0de2755	gh-117139: Convert the evaluation stack to stack refs (#118450) ... This PR sets up tagged pointers for CPython. The general idea is to create a separate struct _PyStackRef for everything on the evaluation stack to store the bits. This forces the C compiler to warn us if we try to cast things or pull things out of the struct directly. Only for free threading: We tag the low bit if something is deferred - that means we skip incref and decref operations on it. This behavior may change in the future if Mark's plans to defer all objects in the interpreter loop pans out. This implies a strict stack reference discipline is required. ALL incref and decref operations on stackrefs must use the stackref variants. It is unsafe to untag something then do normal incref/decref ops on it. The new incref and decref variants are called dup and close. They mimic a "handle" API operating on these stackrefs. Please read Include/internal/pycore_stackref.h for more information! --------- Co-authored-by: Mark Shannon <9448417+markshannon@users.noreply.github.com>	2024-06-27 03:10:43 +08:00
Mark Shannon	8f5a01707f	GH-120982: Add stack check assertions to generated interpreter code (GH-120992)	2024-06-25 16:42:29 +01:00
Irit Katriel	65a12c559c	gh-120834: fix type of *_iframe field in _PyGenObject_HEAD declaration (#120835)	2024-06-24 10:23:38 +01:00
Mark Shannon	9cefcc0ee7	GH-120507: Lower the BEFORE_WITH and BEFORE_ASYNC_WITH instructions. (#120640) ... * Remove BEFORE_WITH and BEFORE_ASYNC_WITH instructions. * Add LOAD_SPECIAL instruction * Reimplement `with` and `async with` statements using LOAD_SPECIAL	2024-06-18 12:17:46 +01:00
Mark Shannon	274f844830	GH-120619: Clean up RETURN_VALUE instruction (GH-120624) ... * Rename _POP_FRAME to _RETURN_VALUE as it returns a value as well as popping a frame. * Remove remaining _POP_FRAMEs	2024-06-17 14:40:11 +01:00