Commit graph

11760 commits

Author SHA1 Message Date
Victor Stinner 04fc4f2a46
bpo-40989: PyObject_INIT() becomes an alias to PyObject_Init() (GH-20901)
The PyObject_INIT() and PyObject_INIT_VAR() macros become aliases to,
respectively, PyObject_Init() and PyObject_InitVar() functions.

Rename _PyObject_INIT() and _PyObject_INIT_VAR() static inline
functions to, respectively, _PyObject_Init() and _PyObject_InitVar(),
and move them to pycore_object.h. Remove their return value:
their return type becomes void.

The _datetime module is now built with the Py_BUILD_CORE_MODULE macro
defined.

Remove an outdated comment on _Py_tracemalloc_config.
2020-06-16 01:28:07 +02:00
Victor Stinner e822e37946
bpo-36020: Remove snprintf macro in pyerrors.h (GH-20889)
On Windows, #include "pyerrors.h" no longer defines "snprintf" and
"vsnprintf" macros.

PyOS_snprintf() and PyOS_vsnprintf() should be used to get portable
behavior.

Replace snprintf() calls with PyOS_snprintf() and replace vsnprintf()
calls with PyOS_vsnprintf().
2020-06-15 21:59:47 +02:00
Dong-hee Na bf69a8f99f
bpo-1635741: Port _dbm module to multiphase initialization (GH-20848) 2020-06-16 01:20:54 +09:00
Niklas Fiekas 794e7d1ab2
bpo-29782: Consolidate _Py_Bit_Length() (GH-20739)
In GH-2866, _Py_Bit_Length() was added to pymath.h for lack of a better
location. GH-20518 added a more appropriate header file for bit utilities. It
also shows how to properly use intrinsics. This allows reconsidering bpo-29782.

* Move the function to the new header.
* Changed return type to match __builtin_clzl() and reviewed usage.
* Use intrinsics where available.
* Pick a fallback implementation suitable for inlining.
2020-06-15 14:33:48 +02:00
Christian Heimes 0d3350daa8
bpo-40955: Fix memory leak in subprocess module (GH-20825)
```
Direct leak of 8 byte(s) in 1 object(s) allocated from:
    #0 0x7f008bf19667 in __interceptor_malloc (/lib64/libasan.so.6+0xb0667)
    #1 0x7f007a0bee4a in subprocess_fork_exec /home/heimes/dev/python/cpython/Modules/_posixsubprocess.c:774
    #2 0xe0305b in cfunction_call Objects/methodobject.c:546
```

Signed-off-by: Christian Heimes <christian@python.org>
2020-06-12 09:18:43 -07:00
Dong-hee Na ddef3bdc7b
bpo-40950: Port nis module to multiphase initialization (GH-20811) 2020-06-12 11:26:00 +09:00
Pablo Galindo 1ed83adb0e
bpo-40939: Remove the old parser (GH-20768)
This commit removes the old parser, the deprecated parser module, the old parser compatibility flags and environment variables and all associated support code and documentation.
2020-06-11 17:30:46 +01:00
Victor Stinner d72b9644a3
bpo-40947: getpath.c uses PyConfig.platlibdir (GH-20807)
Followup of bpo-40854, there is one remaining usage of PLATLIBDIR
which should be replaced by config->platlibdir.

test_sys checks that sys.platlibdir attribute exists and is a string.

Update Makefile: getpath.c and sysmodule.c no longer need PLATLIBDIR
macro, PyConfig.platlibdir member is used instead.

Co-authored-by: Sandro Mani <manisandro@gmail.com>
2020-06-11 17:28:52 +02:00
Victor Stinner 1bcc32f062
bpo-39465: Use _PyInterpreterState_GET() (GH-20788)
Replace _PyThreadState_GET() with _PyInterpreterState_GET() in:

* get_small_int()
* gcmodule.c: add also get_gc_state() function
* _PyTrash_deposit_object()
* _PyTrash_destroy_chain()
* warnings_get_state()
* Py_GetRecursionLimit()

Cleanup listnode.c: add 'parser' variable.
2020-06-10 20:08:26 +02:00
Victor Stinner d36cf5f1d2
bpo-40943: Replace PY_FORMAT_SIZE_T with "z" (GH-20781)
The PEP 353, written in 2005, introduced PY_FORMAT_SIZE_T. Python no
longer supports macOS 10.4 and Visual Studio 2010, but requires more
recent macOS and Visual Studio versions. In 2020 with Python 3.10, it
is now safe to use directly "%zu" to format size_t and "%zi" to
format Py_ssize_t.
2020-06-10 18:38:05 +02:00
Dong-hee Na 0383be4666
Remove usesless function from csv module (GH-20762) 2020-06-10 00:33:43 +09:00
Stefan Krah 951d680d56
Add multicore support to deccheck.py. (GH-20731) 2020-06-08 19:33:12 +02:00
Sandro Mani 8f023a2f66
bpo-40854: Allow overriding sys.platlibdir via PYTHONPLATLIBDIR env-var (GH-20605) 2020-06-08 17:28:11 +02:00
Victor Stinner c6b292cdee
bpo-29882: Add _Py_popcount32() function (GH-20518)
* Rename pycore_byteswap.h to pycore_bitutils.h.
* Move popcount_digit() to pycore_bitutils.h as _Py_popcount32().
* _Py_popcount32() uses GCC and clang builtin function if available.
* Add unit tests to _Py_popcount32().
2020-06-08 16:30:33 +02:00
Hai Shi 47a23fc63f
bpo-40898: Remove redundant if statements in tp_traverse (GH-20692) 2020-06-07 21:05:36 +09:00
Rémi Lapeyre b8867e5d5a
Fix return type of test helper function heapctypewithbuffer_releasebuffer() (GH-20685) 2020-06-07 09:05:33 +02:00
scoder f7c4e23642
bpo-40724: Support setting buffer slots from type specs (GH-20648)
This is not part of the limited API but makes the buffer slots available for type specs.
2020-06-06 21:35:10 +02:00
Lysandros Nikolaou ba6fd87e41
Refactor scripts in Tools/peg_generator/scripts (GH-20401) 2020-06-05 21:21:40 -07:00
Erlend Egeberg Aasland 45af786e11
bpo-40867: Remove unused include from Module/_randommodule.c (GH-20635) 2020-06-05 14:32:09 -07:00
Ram Rachum 235f918f44
bpo-40876: Clarify error message in the csv module (GH-20653) 2020-06-05 17:56:06 -03:00
Stefan Krah 087d612efe
bpo-40874: Update to libmpdec-2.5.0 (GH-20652) 2020-06-05 19:43:01 +02:00
Dong-hee Na 3ad52e366f
bpo-1635741: Port mmap module to multiphase initialization (GH-19459) 2020-06-06 00:01:02 +09:00
Victor Stinner e005ead49b
bpo-40521: Make context free list per-interpreter (GH-20644)
Each interpreter now has its own context free list:

* Move context free list into PyInterpreterState.
* Add _Py_context_state structure.
* Add tstate parameter to _PyContext_ClearFreeList()
  and _PyContext_Fini().
* Pass tstate to clear_freelists().
2020-06-05 02:56:37 +02:00
Victor Stinner 78a02c2568
bpo-40521: Make async gen free lists per-interpreter (GH-20643)
Each interpreter now has its own asynchronous generator free lists:

* Move async gen free lists into PyInterpreterState.
* Move _PyAsyncGen_MAXFREELIST define to pycore_interp.h
* Add _Py_async_gen_state structure.
* Add tstate parameter to _PyAsyncGen_ClearFreeLists
  and _PyAsyncGen_Fini().
2020-06-05 02:34:14 +02:00
Victor Stinner 88ec919010
bpo-40521: Make list free list per-interpreter (GH-20642)
Each interpreter now has its own list free list:

* Move list numfree and free_list into PyInterpreterState.
* Add _Py_list_state structure.
* Add tstate parameter to _PyList_ClearFreeList()
  and _PyList_Fini().
* Remove "#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS".
* _PyGC_Fini() clears gcstate->garbage list which can be stored in
  the list free list. Call _PyGC_Fini() before _PyList_Fini() to
  prevent leaking this list.
2020-06-05 02:05:41 +02:00
Victor Stinner 3744ed2c9c
bpo-40521: Make frame free list per-interpreter (GH-20638)
Each interpreter now has its own frame free list:

* Move frame free list into PyInterpreterState.
* Add _Py_frame_state structure.
* Add tstate parameter to _PyFrame_ClearFreeList()
  and _PyFrame_Fini().
* Remove "#if PyFrame_MAXFREELIST > 0".
* Remove "#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS".
2020-06-05 01:39:24 +02:00
Victor Stinner 2ba59370c3
bpo-40521: Make float free list per-interpreter (GH-20636)
Each interpreter now has its own float free list:

* Move tuple numfree and free_list into PyInterpreterState.
* Add _Py_float_state structure.
* Add tstate parameter to _PyFloat_ClearFreeList()
  and _PyFloat_Fini().
2020-06-05 00:50:05 +02:00
Victor Stinner 69ac6e58fd
bpo-40521: Make tuple free list per-interpreter (GH-20247)
Each interpreter now has its own tuple free lists:

* Move tuple numfree and free_list arrays into PyInterpreterState.
* Define PyTuple_MAXSAVESIZE and PyTuple_MAXFREELIST macros in
  pycore_interp.h.
* Add _Py_tuple_state structure. Pass it explicitly to tuple_alloc().
* Add tstate parameter to _PyTuple_ClearFreeList()
* Each interpreter now has its own empty tuple singleton.
2020-06-04 23:38:36 +02:00
Erlend Egeberg Aasland 6ed578f6db
bpo-40865: Remove unused insint() macro from hash modules (GH-20627)
Automerge-Triggered-By: @tiran
2020-06-04 13:08:42 -07:00
Dong-hee Na 586be6f3ff
Remove unused ReaderObject_Check macro (#20614) 2020-06-04 00:43:46 +09:00
Victor Stinner fa7ab6aa0f
bpo-40826: Add _PyOS_InterruptOccurred(tstate) function (GH-20599)
my_fgets() now calls _PyOS_InterruptOccurred(tstate) to check for
pending signals, rather calling PyOS_InterruptOccurred().

my_fgets() is called with the GIL released, whereas
PyOS_InterruptOccurred() must be called with the GIL held.

test_repl: use text=True and avoid SuppressCrashReport in
test_multiline_string_parsing().

Fix my_fgets() on Windows: fgets(fp) does crash if fileno(fp) is closed.
2020-06-03 14:39:59 +02:00
Victor Stinner 18a90248fd
bpo-32604: Fix reference leak in select module (GH-20600)
Fix reference leak in PyInit_select() of the select module:
remove Py_INCREF(poll_Type).
2020-06-03 14:36:46 +02:00
Victor Stinner 317bab0bf6
PyOS_AfterFork_Child() pass tstate to _PyEval_ReInitThreads() (GH-20598) 2020-06-02 18:44:54 +02:00
Victor Stinner 26881c8fae
PyOS_AfterFork_Child() uses PyStatus (GH-20596)
PyOS_AfterFork_Child() helper functions now return a PyStatus:
PyOS_AfterFork_Child() is now responsible to handle errors.

* Move _PySignal_AfterFork() to the internal C API
* Add #ifdef HAVE_FORK on _PyGILState_Reinit(), _PySignal_AfterFork()
  and _PyInterpreterState_DeleteExceptMain().
2020-06-02 15:51:37 +02:00
Victor Stinner cbe1296922
bpo-40826: PyOS_InterruptOccurred() requires GIL (GH-20578)
PyOS_InterruptOccurred() now fails with a fatal error if it is called
with the GIL released.
2020-06-01 20:34:15 +02:00
Ammar Askar 06e3a27a3c
bpo-39943: Fix MSVC warnings in sre extension (GH-20508) 2020-06-01 19:21:43 +02:00
Hai Shi a97011b9b8
bpo-39593: Add test on ctypes cfield.c s_set() (GH-18424) 2020-06-01 18:54:18 +02:00
Dong-hee Na e9684fac5a
bpo-1635741: Port fcntl module to multiphase initialization (GH-20540) 2020-06-02 01:12:24 +09:00
Christian Heimes a871f692b4
bpo-30008: Fix OpenSSL no-deprecated compilation (GH-20397)
Fix :mod:`ssl`` code to be compatible with OpenSSL 1.1.x builds that use
``no-deprecated`` and ``--api=1.1.0``.

Note: Tests assume full OpenSSL API and fail with limited API.

Signed-off-by: Christian Heimes <christian@python.org>
Co-authored-by: Mark Wright <gienah@gentoo.org>
2020-06-01 08:58:14 +02:00
Zackery Spytz 7b78e7f9fd
bpo-40061: Fix a possible refleak in _asynciomodule.c (GH-19748)
tup should be decrefed in the unlikely event of a PyList_New()
failure.
2020-05-30 01:22:02 -07:00
Petr Viktorin 459acc5516
bpo-40777: Initialize PyDateTime_IsoCalendarDateType.tp_base at run-time (GH-20493)
Recent changes to _datetimemodule broke compilation on mingw; see the comments in this change for details.

FWIW, @corona10: this issue is why `PyType_FromModuleAndSpec` & friends take the `bases` argument at run time.
2020-05-28 09:14:46 -07:00
Victor Stinner 0de437de62
bpo-25920: Remove socket.getaddrinfo() lock on macOS (GH-20177)
On macOS, socket.getaddrinfo() no longer uses an internal lock to
prevent race conditions when calling getaddrinfo(). getaddrinfo is
thread-safe is macOS 10.5, whereas Python 3.9 requires macOS 10.6 or
newer.

The lock was also used on FreeBSD older than 5.3, OpenBSD older than
201311 and NetBSD older than 4.
2020-05-28 17:23:39 +02:00
Serhiy Storchaka 5f4b229df7
bpo-40792: Make the result of PyNumber_Index() always having exact type int. (GH-20443)
Previously, the result could have been an instance of a subclass of int.

Also revert bpo-26202 and make attributes start, stop and step of the range
object having exact type int.

Add private function _PyNumber_Index() which preserves the old behavior
of PyNumber_Index() for performance to use it in the conversion functions
like PyLong_AsLong().
2020-05-28 10:33:45 +03:00
Victor Stinner 10228bad04
bpo-40795: ctypes calls unraisablehook with an exception (GH-20452)
If ctypes fails to convert the result of a callback or if a ctypes
callback function raises an exception, sys.unraisablehook is now
called with an exception set. Previously, the error was logged into
stderr by PyErr_Print().
2020-05-28 00:38:12 +02:00
YoSTEALTH 76ef255bde
bpo-37129: Add os.RWF_APPEND flag for os.pwritev() (GH-20336) 2020-05-27 23:32:22 +02:00
Pablo Galindo e4799b9594
Fix compiler warnings in _zoneinfo.c (GH-20342)
```
D:\a\cpython\cpython\Modules\_zoneinfo.c(903,52): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data [D:\a\cpython\cpython\PCbuild\_zoneinfo.vcxproj]
D:\a\cpython\cpython\Modules\_zoneinfo.c(904,44): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data [D:\a\cpython\cpython\PCbuild\_zoneinfo.vcxproj]
D:\a\cpython\cpython\Modules\_zoneinfo.c(1772,31): warning C4244: '=': conversion from 'ssize_t' to 'uint8_t', possible loss of data [D:\a\cpython\cpython\PCbuild\_zoneinfo.vcxproj]
```
2020-05-27 13:48:12 -07:00
Christian Heimes db5aed931f
bpo-40791: Use CRYPTO_memcmp() for compare_digest (#20456)
hashlib.compare_digest uses OpenSSL's CRYPTO_memcmp() function
when OpenSSL is available.

Note: The _operator module is a builtin module. I don't want to add
libcrypto dependency to libpython. Therefore I duplicated the wrapper
function and added a copy to _hashopenssl.c.
2020-05-27 21:50:06 +02:00
Sean Gillespie 29a1384c04
bpo-13097: ctypes: limit callback to 1024 arguments (GH-19914)
ctypes now raises an ArgumentError when a callback
is invoked with more than 1024 arguments.

The ctypes module allocates arguments on the stack in
ctypes_callproc() using alloca(), which is problematic
when large numbers of arguments are passed. Instead
of a stack overflow, this commit raises an ArgumentError
if more than 1024 parameters are passed.
2020-05-27 17:22:07 +02:00
Victor Stinner fe2978b3b9
bpo-39573: Convert Py_REFCNT and Py_SIZE to functions (GH-20429)
Convert Py_REFCNT() and Py_SIZE() macros to static inline functions.
They cannot be used as l-value anymore: use Py_SET_REFCNT() and
Py_SET_SIZE() to set an object reference count and size.

Replace &Py_SIZE(self) with &((PyVarObject*)self)->ob_size
in arraymodule.c.

This change is backward incompatible on purpose, to prepare the C API
for an opaque PyObject structure.
2020-05-27 14:55:10 +02:00
Pablo Galindo 1cf15af9a6 bpo-40217: Ensure Py_VISIT(Py_TYPE(self)) is always called for PyType_FromSpec types (reverts GH-19414) (GH-20264)
Heap types now always visit the type in tp_traverse. See added docs for details.

This reverts commit 0169d3003b.

Automerge-Triggered-By: @encukou
2020-05-27 02:03:38 -07:00