Commit graph

5032 commits

Author SHA1 Message Date
Donghee Na c744dbe9ac
gh-112535: Update _Py_ThreadId() to support s390/s390x (gh-112751) 2023-12-08 23:28:07 +09:00
Mark Shannon aefdebdef1
GH-111485: Factor out opcode ID generator from the main cases generator. (GH-112831) 2023-12-08 11:48:30 +00:00
Ronald Oussoren 15a80b15af
gh-110820: Make sure processor specific defines are correct for Universal 2 build on macOS (#112828)
* gh-110820: Make sure processor specific defines are correct for Universal 2 build on macOS

A number of processor specific defines are different for x86-64 and
arm64, and need to be adjusted in pymacconfig.h.

* remove debug stuf
2023-12-08 10:09:34 +01:00
Sam Gross cf6110ba13
gh-111924: Use PyMutex for Runtime-global Locks. (gh-112207)
This replaces some usages of PyThread_type_lock with PyMutex, which does not require memory allocation to initialize.

This simplifies some of the runtime initialization and is also one step towards avoiding changing the default raw memory allocator during initialize/finalization, which can be non-thread-safe in some circumstances.
2023-12-07 12:33:40 -07:00
Sam Gross db460735af
gh-112538: Add internal-only _PyThreadStateImpl "wrapper" for PyThreadState (gh-112560)
Every PyThreadState instance is now actually a _PyThreadStateImpl.
It is safe to cast from `PyThreadState*` to `_PyThreadStateImpl*` and back.
The _PyThreadStateImpl will contain fields that we do not want to expose
in the public C API.
2023-12-07 12:11:45 -07:00
Sam Gross 2d76be251d
gh-111962: Make dtoa thread-safe in --disable-gil builds. (#112049)
This updates `dtoa.c` to avoid using the Bigint free-list in --disable-gil builds and
to pre-computes the needed powers of 5 during interpreter initialization.

* gh-111962: Make dtoa thread-safe in `--disable-gil` builds.

This avoids using the Bigint free-list in `--disable-gil` builds
and pre-computes the needed powers of 5 during interpreter initialization.

* Fix size of cached powers of 5 array.

We need the powers of 5 up to 5**512 because we only jump straight to
underflow when the exponent is less than -512 (or larger than 308).

* Rename Py_NOGIL to Py_GIL_DISABLED

* Changes from review

* Fix assertion placement
2023-12-07 13:47:55 +00:00
Christopher Chavez 7576534f4a
bpo-42519: Remove outdated sentence in comment (#112822)
Update objimpl.h
2023-12-07 14:24:11 +01:00
andrewluotechnologies 9c3458e058
gh-112125: Fix None.__ne__(None) returning NotImplemented instead of False (#112504) 2023-12-07 13:56:01 +01:00
Mark Shannon b449415b2f
GH-111485: Separate out parsing, analysis and code-gen phases of tier 1 code generator (GH-112299) 2023-12-07 12:49:40 +00:00
Victor Stinner 828451dfde
gh-111545: Add Py_HashPointer() function (#112096)
* Implement _Py_HashPointerRaw() as a static inline function.
* Add Py_HashPointer() tests to test_capi.test_hash.
* Keep _Py_HashPointer() function as an alias to Py_HashPointer().
2023-12-06 15:09:22 +01:00
Donghee Na 5aa317e4ca
gh-112535: Add comment for ppc32/64 registers (gh-112746) 2023-12-05 19:44:19 +09:00
Donghee Na d824512059
gh-112535: Update _Py_ThreadId() to support PowerPC (gh-112624) 2023-12-05 18:03:32 +09:00
Serhiy Storchaka da6760bdf5
gh-65210: Add const qualifiers in PyArg_VaParseTupleAndKeywords() (GH-105958)
Change the declaration of the keywords parameter in functions
PyArg_ParseTupleAndKeywords() and PyArg_VaParseTupleAndKeywords() from `char **`
to `char * const *` in C and `const char * const *` in C++.

It makes these functions compatible with argument of type `const char * const *`,
`const char **` or `char * const *` in C++ and `char * const *` in C
without explicit type cast.

Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
2023-12-04 13:14:56 +02:00
Victor Stinner a74902a14c
gh-106550: Fix sign conversion in pycore_code.h (#112613)
Fix sign conversion in pycore_code.h: use unsigned integers and cast
explicitly when needed.
2023-12-04 11:42:58 +01:00
Victor Stinner 1f2a676785
gh-106560: Fix redundant declarations in Include/ (#112611)
Don't declare PyBool_Type, PyLong_Type and PySys_Audit() twice, but
only once.

Compiler warnings seen by building Python with gcc -Wredundant-decls.
2023-12-03 12:16:31 +01:00
Victor Stinner 5c5022b862
gh-112567: Add _PyTimeFraction C API (#112568)
Use a fraction internally in the _PyTime API to reduce the risk of
integer overflow: simplify the fraction using Greatest Common
Divisor (GCD). The fraction API is used by time functions:
perf_counter(), monotonic() and process_time().

For example, QueryPerformanceFrequency() usually returns 10 MHz on
Windows 10 and newer. The fraction SEC_TO_NS / frequency =
1_000_000_000 / 10_000_000 can be simplified to 100 / 1.

* Add _PyTimeFraction type.
* Add functions:

  * _PyTimeFraction_Set()
  * _PyTimeFraction_Mul()
  * _PyTimeFraction_Resolution()

* No longer check "numer * denom <= _PyTime_MAX" in
  _PyTimeFraction_Set(). _PyTimeFraction_Mul() uses _PyTime_Mul()
  which handles integer overflow.
2023-12-01 19:50:10 +01:00
Victor Stinner 05a370abd6
gh-112567: Add _Py_GetTicksPerSecond() function (#112587)
* Move _PyRuntimeState.time to _posixstate.ticks_per_second and
  time_module_state.ticks_per_second.
* Add time_module_state.clocks_per_second.
* Rename _PyTime_GetClockWithInfo() to py_clock().
* Rename _PyTime_GetProcessTimeWithInfo() to py_process_time().
* Add process_time_times() helper function, called by
  py_process_time().
* os.times() is now always built: no longer rely on HAVE_TIMES.
2023-12-01 17:05:56 +01:00
Victor Stinner a9073564ee
gh-110481: Fix typo in Py_SET_REFCNT() (#112595) 2023-12-01 15:54:40 +00:00
Victor Stinner 5f6ac2d88a
gh-110481: Fix Py_SET_REFCNT() integer overflow (#112174)
If Py_NOGIL is defined and Py_SET_REFCNT() is called with a reference
count larger than UINT32_MAX, make the object immortal.

Set _Py_IMMORTAL_REFCNT constant type to Py_ssize_t to fix the
following compiler warning:

Include/internal/pycore_global_objects_fini_generated.h:14:24:
warning: comparison of integers of different signs: 'Py_ssize_t'
(aka 'long') and 'unsigned int' [-Wsign-compare]

    if (Py_REFCNT(obj) < _Py_IMMORTAL_REFCNT) {
        ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~
2023-12-01 15:50:16 +01:00
Pablo Galindo Salgado a73aa48e6b
gh-112367: Only free perf trampoline arenas at shutdown (#112368)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2023-12-01 13:20:51 +00:00
Irit Katriel 07ebd46f9e
gh-112519: Make it possible to specify instruction flags for pseudo instructions in bytecodes.c (#112520) 2023-11-30 11:03:30 +00:00
Kirill Podoprigora 0785c68559
gh-111972: Make Unicode name C APIcapsule initialization thread-safe (#112249) 2023-11-30 11:12:49 +01:00
Guido van Rossum e723700190
Rename ...Uop... to ...UOp... (uppercase O) for consistency (#112327)
* Rename _PyUopExecute to _PyUOpExecute (uppercase O) for consistency
* Also rename _PyUopName and _PyUOp_Replacements, and some output strings
2023-11-28 17:10:11 -08:00
Grant Ramsay e954ac7205
gh-63284: Add support for TLS-PSK (pre-shared key) to the ssl module (#103181)
Add support for TLS-PSK (pre-shared key) to the ssl module.

---------

Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-11-27 04:01:44 +00:00
Eric Snow 9e56eedd01
gh-76785: Return an "excinfo" Object From Interpreter.run() (gh-111573) 2023-11-23 00:55:00 +00:00
Eric Snow 790db85c77
gh-76785: Add _PyType_GetModuleName() to the Internal C-API (gh-112323)
The new function corresponds to the existing (public) PyType_GetName() and PyType_GetQualName().
2023-11-22 15:03:33 -07:00
Thomas Wouters 1c5279b9d6 Post 3.13.0a2 2023-11-22 22:24:48 +01:00
Thomas Wouters 9c4347ef8b Python 3.13.0a2 2023-11-22 12:20:24 +01:00
Guido van Rossum 8deb8bc2e5
gh-112287: Speed up Tier 2 (uop) interpreter a little (#112286)
This makes the Tier 2 interpreter a little faster.
I calculated by about 3%,
though I hesitate to claim an exact number.

This starts by doubling the trace size limit (to 512),
making it more likely that loops fit in a trace.

The rest of the approach is to only load
`oparg` and `operand` in cases that use them.
The code generator know when these are used.

For `oparg`, it will conditionally emit
```
oparg = CURRENT_OPARG();
```
at the top of the case block.
(The `oparg` variable may be referenced multiple times
by the instructions code block, so it must be in a variable.)

For `operand`, it will use `CURRENT_OPERAND()` directly
instead of referencing the `operand` variable,
which no longer exists.
(There is only one place where this will be used.)
2023-11-20 11:25:32 -08:00
Guido van Rossum 1995955173
gh-106529: Make FOR_ITER a viable uop (#112134)
This uses the new mechanism whereby certain uops
are replaced by others during translation,
using the `_PyUop_Replacements` table.
We further special-case the `_FOR_ITER_TIER_TWO` uop
to update the deoptimization target to point
just past the corresponding `END_FOR` opcode.

Two tiny code cleanups are also part of this PR.
2023-11-20 10:08:53 -08:00
Hugo van Kemenade 3b3ec0d77f
gh-111863: Rename Py_NOGIL to Py_GIL_DISABLED (#111864)
Rename Py_NOGIL to Py_GIL_DISABLED
2023-11-20 15:52:00 +02:00
Donghee Na 7c9f2677fb
gh-111926: Update _PyWeakref_IS_DEAD to be thread-safe (gh-112267) 2023-11-20 07:36:45 +09:00
Guido van Rossum be0bd54c6b
gh-106529: Cleanups split off gh-112134 (#112214)
- Double max trace size to 256
- Add a dependency on executor_cases.c.h for ceval.o
- Mark `_SPECIALIZE_UNPACK_SEQUENCE` as `TIER_ONE_ONLY`
- Add debug output back showing the optimized trace
- Bunch of cleanups to Tools/cases_generator/
2023-11-17 11:49:42 -08:00
Sam Gross 446f18a911
gh-111956: Add thread-safe one-time initialization. (gh-111960) 2023-11-16 12:19:54 -07:00
Victor Stinner bd89bca9e2
gh-111798: Use lower Py_C_RECURSION_LIMIT in debug mode (#112124)
* Run again test_ast_recursion_limit() on WASI platform.
* Add _testinternalcapi.get_c_recursion_remaining().
* Fix test_ast and test_sys_settrace: test_ast_recursion_limit() and
  test_trace_unpack_long_sequence() now adjust the maximum recursion
  depth depending on the the remaining C recursion.
2023-11-16 13:52:33 +00:00
Victor Stinner 7e2308aaa2
gh-112026: Add again _PyThreadState_UncheckedGet() function (#112121)
Add again the private _PyThreadState_UncheckedGet() function as an
alias to the new public PyThreadState_GetUnchecked() function.
2023-11-15 19:15:03 +01:00
Victor Stinner bd2f1485b0
gh-112026: Restore removed _PyDict_GetItemStringWithError() (#112119)
Restore the removed _PyDict_GetItemStringWithError() function. It is
used by numpy.
2023-11-15 17:10:06 +00:00
Victor Stinner 58469244be
gh-112026: Restore removed private C API (#112115)
Restore removed private C API functions, macros and structures which
have no simple replacement for now:

* _PyDict_GetItem_KnownHash()
* _PyDict_NewPresized()
* _PyHASH_BITS
* _PyHASH_IMAG
* _PyHASH_INF
* _PyHASH_MODULUS
* _PyHASH_MULTIPLIER
* _PyLong_Copy()
* _PyLong_FromDigits()
* _PyLong_New()
* _PyLong_Sign()
* _PyObject_CallMethodId()
* _PyObject_CallMethodNoArgs()
* _PyObject_CallMethodOneArg()
* _PyObject_CallOneArg()
* _PyObject_EXTRA_INIT
* _PyObject_FastCallDict()
* _PyObject_GetAttrId()
* _PyObject_Vectorcall()
* _PyObject_VectorcallMethod()
* _PyStack_AsDict()
* _PyThread_CurrentFrames()
* _PyUnicodeWriter structure
* _PyUnicodeWriter_Dealloc()
* _PyUnicodeWriter_Finish()
* _PyUnicodeWriter_Init()
* _PyUnicodeWriter_Prepare()
* _PyUnicodeWriter_PrepareKind()
* _PyUnicodeWriter_WriteASCIIString()
* _PyUnicodeWriter_WriteChar()
* _PyUnicodeWriter_WriteLatin1String()
* _PyUnicodeWriter_WriteStr()
* _PyUnicodeWriter_WriteSubstring()
* _PyUnicode_AsString()
* _PyUnicode_FromId()
* _PyVectorcall_Function()
* _Py_HashDouble()
* _Py_HashPointer()
* _Py_IDENTIFIER()
* _Py_c_abs()
* _Py_c_diff()
* _Py_c_neg()
* _Py_c_pow()
* _Py_c_prod()
* _Py_c_quot()
* _Py_c_sum()
* _Py_static_string()
* _Py_static_string_init()
2023-11-15 16:38:31 +00:00
Victor Stinner b338ffa4bc
gh-112026: Add again <unistd.h> include in Python.h (#112046)
Add again <ctype.h> and <unistd.h> includes in Python.h, but don't
include them in the limited C API version 3.13 and newer.
2023-11-15 16:59:32 +01:00
Mark Shannon 4bbb367ba6
GH-111848: Set the IP when de-optimizing (GH-112065)
* Replace jumps with deopts in tier 2

* Fewer special cases of uop names

* Add target field to uop IR

* Remove more redundant SET_IP and _CHECK_VALIDITY micro-ops

* Extend whitelist of non-escaping API functions.
2023-11-15 15:48:58 +00:00
Victor Stinner 62802b6228
gh-111545: Add Include/cpython/pyhash.h header file (#112063)
Move non-limited C API to a new Include/cpython/pyhash.h header file.
2023-11-15 01:19:20 +01:00
Mark Shannon a519b87958
GH-111848: Convert remaining jumps to deopts into tier 2 code. (GH-112045) 2023-11-14 15:30:33 +00:00
Victor Stinner 4f04172c92
gh-111262: Add PyDict_Pop() function (#112028)
_PyDict_Pop_KnownHash(): remove the default value and the return type
becomes an int.

Co-authored-by: Stefan Behnel <stefan_ml@behnel.de>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2023-11-14 12:51:00 +00:00
Irit Katriel 36aab34fab
gh-107149: make new opcode util functions private rather than public and unstable (#112042) 2023-11-14 00:31:02 +00:00
Victor Stinner babb787047
gh-111138: Add PyList_Extend() and PyList_Clear() functions (#111862)
* Split list_extend() into two sub-functions: list_extend_fast() and
  list_extend_iter().
* list_inplace_concat() no longer has to call Py_DECREF() on the
  list_extend() result, since list_extend() now returns an int.
2023-11-13 16:14:56 +00:00
Serhiy Storchaka 771bd3c94a
Add private _PyUnicode_AsUTF8NoNUL() function (GH-111957)
Like PyUnicode_AsUTF8(), but check for embedded null characters.
2023-11-10 21:31:36 +02:00
Sam Gross 289af86122
gh-111569: Fix critical sections test on WebAssembly (GH-111897)
This adds a macro `Py_CAN_START_THREADS` that corresponds to the Python
function `test.support.threading_helper.can_start_thread()`. WASI and
some Emscripten builds do not have a working pthread implementation.

This macro is used to guard the critical sections C API tests that
require a working threads implementation.
2023-11-09 15:37:11 -08:00
Samuel Thibault 0802fd6c8e
gh-81925: Implement native thread ids for kFreeBSD (#111761)
---------

Co-authored-by: Antoine Pitrou <antoine@python.org>
2023-11-09 21:02:30 +01:00
Mark Shannon 34a03e951b
GH-111843: Tier 2 exponential backoff (GH-111850) 2023-11-09 13:49:51 +00:00
Mark Shannon 25c4956488
GH-109369: Exit tier 2 if executor is invalid (GH-111657) 2023-11-09 11:19:51 +00:00