Commit graph

118399 commits

Author SHA1 Message Date
Victor Stinner 4e5a7284ee
gh-108444: Argument Clinic uses PyLong_AsInt() (#108458)
Argument Clinic now uses the new public PyLong_AsInt(), rather than
the old name _PyLong_AsInt().
2023-08-25 00:51:22 +02:00
Nikita Sobolev be800f4be7
Run mypy when Tools/requirements-dev.txt changes (#108457) 2023-08-24 22:05:23 +00:00
Victor Stinner be436e08b8
gh-108444: Add PyLong_AsInt() public function (#108445)
* Rename _PyLong_AsInt() to PyLong_AsInt().
* Add documentation.
* Add test.
* For now, keep _PyLong_AsInt() as an alias to PyLong_AsInt().
2023-08-24 23:55:30 +02:00
Filipe Laíns feb9a49c9c
GH-103247: bypass the import cache on the _require_loader helper 2023-08-24 21:05:03 +00:00
Samuel Thibault edb569e8e1
gh-108447: Detect platform triplets for x86_64 GNU/Hurd (#108045) 2023-08-24 22:37:20 +02:00
Victor Stinner a071ecb4d1
gh-106320: Remove private _PySys functions (#108452)
Move private functions to the internal C API (pycore_sysmodule.h):

* _PySys_GetAttr()
* _PySys_GetSizeOf()

No longer export most of these functions.

Fix also a typo in Include/cpython/optimizer.h: add a missing space.
2023-08-24 20:02:09 +00:00
Victor Stinner 26893016a7
gh-106320: Remove private _PyDict functions (#108449)
Move private functions to the internal C API (pycore_dict.h):

* _PyDictView_Intersect()
* _PyDictView_New()
* _PyDict_ContainsId()
* _PyDict_DelItemId()
* _PyDict_DelItem_KnownHash()
* _PyDict_GetItemIdWithError()
* _PyDict_GetItem_KnownHash()
* _PyDict_HasSplitTable()
* _PyDict_NewPresized()
* _PyDict_Next()
* _PyDict_Pop()
* _PyDict_SetItemId()
* _PyDict_SetItem_KnownHash()
* _PyDict_SizeOf()

No longer export most of these functions.

Move also the _PyDictViewObject structure to the internal C API.

Move dict_getitem_knownhash() function from _testcapi to the
_testinternalcapi extension. Update test_capi.test_dict for this
change.
2023-08-24 20:01:50 +00:00
Victor Stinner c3d580b238
gh-106320: Remove private _PyList functions (#108451)
Move private functions to the internal C API (pycore_list.h):

* _PyList_Extend()
* _PyList_DebugMallocStats()

No longer export these functions.
2023-08-24 19:44:34 +00:00
Guido van Rossum 88941d665f
gh-106581: Fix two bugs in the code generator's copy optimization (#108380)
I was comparing the last preceding poke with the *last* peek,
rather than the *first* peek.

Unfortunately this bug obscured another bug:
When the last preceding poke is UNUSED, the first peek disappears,
leaving the variable unassigned. This is how I fixed it:

- Rename CopyEffect to CopyItem.
- Change CopyItem to contain StackItems instead of StackEffects.
- Update those StackItems when adjusting the manager higher or lower.
- Assert that those StackItems' offsets are equivalent.
- Other clever things.

---------

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2023-08-24 19:10:51 +00:00
Victor Stinner c494fb333b
gh-106320: Remove private _PyEval function (#108433)
Move private _PyEval functions to the internal C API
(pycore_ceval.h):

* _PyEval_GetBuiltin()
* _PyEval_GetBuiltinId()
* _PyEval_GetSwitchInterval()
* _PyEval_MakePendingCalls()
* _PyEval_SetProfile()
* _PyEval_SetSwitchInterval()
* _PyEval_SetTrace()

No longer export most of these functions.
2023-08-24 20:25:22 +02:00
Victor Stinner 995f4c48e1
gh-80527: Change support.requires_legacy_unicode_capi() (#108438)
The decorator now requires to be called with parenthesis:

    @support.requires_legacy_unicode_capi()

instead of:

    @support.requires_legacy_unicode_capi

The implementation now only imports _testcapi when the decorator is
called, so "import test.support" no longer imports the _testcapi
extension.
2023-08-24 20:09:23 +02:00
Victor Stinner fa6933e035
gh-107211: Fix test_peg_generator (#108435)
Export these internal functions for test_peg_generator

* _PyArena_AddPyObject()
* _PyArena_Free()
* _PyArena_Malloc()
* _PyArena_New()
2023-08-24 17:47:44 +00:00
Victor Stinner fa626b8ca0
gh-107178: Remove _testcapi.test_dict_capi() (#108436)
The new _testcapi.test_dict tests are more complete,
_testcapi.test_dict_capi() became redundant.
2023-08-24 17:43:29 +00:00
Victor Stinner 480a337366
gh-106320: Remove private _PyContext_NewHamtForTests() (#108434)
Move the function to the internal C API.
2023-08-24 19:37:41 +02:00
Victor Stinner aa6f787faa
gh-108388: Convert test_concurrent_futures to package (#108401)
Convert test_concurrent_futures to a package of sub-tests.
2023-08-24 19:21:44 +02:00
Victor Stinner bbbe1faf7b
gh-106320: Remove private float C API functions (#108430)
106320: Remove private float C API functions

Remove private C API functions:

* _Py_parse_inf_or_nan()
* _Py_string_to_number_with_underscores()

Move these functions to the internal C API and no longer export them.
2023-08-24 19:09:49 +02:00
Victor Stinner 773b803c02
gh-106320: Remove private _PyManagedBuffer_Type (#108431)
Remove private _PyManagedBuffer_Type variable. Move it to the
internal C API and no longer export it.
2023-08-24 19:07:54 +02:00
Victor Stinner c55e73112c
gh-106320: Remove private PyLong C API functions (#108429)
Remove private PyLong C API functions:

* _PyLong_AsByteArray()
* _PyLong_DivmodNear()
* _PyLong_Format()
* _PyLong_Frexp()
* _PyLong_FromByteArray()
* _PyLong_FromBytes()
* _PyLong_GCD()
* _PyLong_Lshift()
* _PyLong_Rshift()

Move these functions to the internal C API. No longer export
_PyLong_FromBytes() function.
2023-08-24 18:53:50 +02:00
A 7f31676340
Fix a code snippet typo in asyncio docs (#108427) 2023-08-24 16:27:54 +00:00
Victor Stinner b7808820b1
gh-107211: No longer export internal functions (5) (#108423)
No longer export _PyCompile_AstOptimize() internal C API function.

Change comment style to "// comment" and add comment explaining why
other functions have to be exported.
2023-08-24 16:06:53 +00:00
Victor Stinner f1ae706ca5
gh-107211: No longer export internal functions (7) (#108425)
No longer export _PyUnicode_FromId() internal C API function.

Change comment style to "// comment" and add comment explaining why
other functions have to be exported.

Update Tools/build/generate_token.py to update Include/internal/pycore_token.h
comments.
2023-08-24 17:40:56 +02:00
Victor Stinner 52c6a6e48a
gh-108308: Remove _PyDict_GetItemStringWithError() function (#108426)
Remove the internal _PyDict_GetItemStringWithError() function. It can
now be replaced with the new public PyDict_ContainsString() and
PyDict_GetItemStringRef() functions.

getargs.c now now uses a strong reference for current_arg.
find_keyword() returns a strong reference.
2023-08-24 17:34:22 +02:00
Victor Stinner ea871c9b0f
gh-107211: No longer export internal functions (6) (#108424)
No longer export these 5 internal C API functions:

* _PyArena_AddPyObject()
* _PyArena_Free()
* _PyArena_Malloc()
* _PyArena_New()
* _Py_FatalRefcountErrorFunc()

Change comment style to "// comment" and add comment explaining why
other functions have to be exported.
2023-08-24 17:28:35 +02:00
Victor Stinner 3f7e93be51
gh-107211: No longer export PyTime internal functions (#108422)
No longer export these 2 internal C API functions:

* _PyTime_AsNanoseconds()
* _PyTime_GetSystemClockWithInfo()

Change comment style to "// comment" and add comment explaining why
other functions have to be exported.
2023-08-24 17:17:40 +02:00
Victor Stinner a35d48d4bd
gh-108240: _PyCapsule_SetTraverse() rejects NULL callbacks (#108417) 2023-08-24 16:37:59 +02:00
Victor Stinner 6726626646
gh-108314: Add PyDict_ContainsString() function (#108323) 2023-08-24 15:59:12 +02:00
Jeffery To c163d7f0b6
gh-95855: Refactor platform triplet detection code, add detection for MIPS soft float and musl libc (#107221)
- Move platform triplet detection code into Misc/platform_triplet.c
- Refactor MIPS detection, use defined(__mips64) to detect MIPS64
- Compute libc values in separate section
- Add detection for MIPS soft float
- Add detection for musl

musl supports SPE with its soft-float ABI:
https://git.musl-libc.org/cgit/musl/commit/?id=7be59733d71ada3a32a98622507399253f1d5e48

Original patch by Christian Heimes.

Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-08-24 14:22:50 +02:00
Daniele Procida 809ea7c4b6
gh-107432 Update Porting Python 2 Code to Python 3 how-to (GH-107434)
https://docs.python.org/3/howto/pyporting.html#porting-python-2-code-to-python-3 was written for another time. In this patch:

- material that frames Python 3 as "new" is removed
- descriptions and directions have been trimmed
2023-08-24 11:28:23 +02:00
Chris Markiewicz 2eb60c1934
gh-108111: Flush gzip write buffer before seeking, fixing bad writes (#108341)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-08-24 10:23:01 +02:00
Victor Stinner aa9a359ca2
gh-108388: Split test_multiprocessing_spawn (#108396)
Split test_multiprocessing_fork, test_multiprocessing_forkserver and
test_multiprocessing_spawn into test packages. Each package is made
of 4 sub-tests: processes, threads, manager and misc. It allows
running more tests in parallel and so reduce the total test duration.
2023-08-24 05:35:39 +02:00
Victor Stinner 174e9da083
gh-108388: regrtest splits test_asyncio package (#108393)
Currently, test_asyncio package is only splitted into sub-tests when
using command "./python -m test". With this change, it's also
splitted when passing it on the command line:
"./python -m test test_asyncio".

Remove the concept of "STDTESTS". Python is now mature enough to not
have to bother with that anymore. Removing STDTESTS simplify the
code.
2023-08-24 02:44:58 +00:00
Victor Stinner 7a6cc3eb66
test_peg_generator and test_freeze require cpu (#108386)
The test_peg_generator and test_tools.test_freeze tests now require
the 'cpu' resource. Skip these tests unless the 'cpu' resource is
enabled (it is disabled by default).

These tests are no longer skipped if Python is built with ASAN or
MSAN sanitizer.
2023-08-24 00:52:48 +02:00
Guido van Rossum 3107b453bc
gh-108253: Fix reads of uninitialized memory in funcobject.c (#108383) 2023-08-23 22:36:19 +00:00
Victor Stinner 513c89d012
gh-108240: Add _PyCapsule_SetTraverse() internal function (#108339)
The _socket extension uses _PyCapsule_SetTraverse() to visit and clear
the socket type in the garbage collector. So the _socket.socket type
can be cleared in some corner cases when it wasn't possible before.
2023-08-24 00:19:11 +02:00
Gregory P. Smith b6be18812c
gh-70766: Mention the object getstate caveat in 3.11 What's new. (#108379) 2023-08-23 15:00:09 -07:00
Victor Stinner 4890f65ecf
gh-108308: Use PyDict_GetItemRef() in moduleobject.c (#108381)
Replace PyDict_GetItemWithError() with PyDict_GetItemRef() which
returns a strong reference.

Cleanup also the function: move variable definition to their first
assignation, rename variable names to use name longer than 1
character.
2023-08-23 21:57:40 +00:00
Victor Stinner 592bacb6fc
gh-108342: Make ssl TestPreHandshakeClose more reliable (#108370)
* In preauth tests of test_ssl, explicitly break reference cycles
  invoving SingleConnectionTestServerThread to make sure that the
  thread is deleted. Otherwise, the test marks the environment as
  altered because the threading module sees a "dangling thread"
  (SingleConnectionTestServerThread). This test leak was introduced
  by the test added for the fix of issue gh-108310.
* Use support.SHORT_TIMEOUT instead of hardcoded 1.0 or 2.0 seconds
  timeout.
* SingleConnectionTestServerThread.run() catchs TimeoutError
* Fix a race condition (missing synchronization) in
  test_preauth_data_to_tls_client(): the server now waits until the
  client connect() completed in call_after_accept().
* test_https_client_non_tls_response_ignored() calls server.join()
  explicitly.
* Replace "localhost" with server.listener.getsockname()[0].
2023-08-23 21:57:11 +00:00
Victor Stinner ec3527d196
gh-108308: config_dict_get() uses PyDict_GetItemRef() (#108371)
Replace _PyDict_GetItemStringWithError() with PyDict_GetItemRef() in
config_dict_get() to get a strong reference to the item.
2023-08-23 23:12:08 +02:00
Victor Stinner 4dc9f48930
gh-108308: Replace _PyDict_GetItemStringWithError() (#108372)
Replace _PyDict_GetItemStringWithError() calls with
PyDict_GetItemStringRef() which returns a strong reference to the
item.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-23 22:59:00 +02:00
albanD 1700d34d31
gh-77377: Ensure multiprocessing SemLock is valid for spawn-based Process before serializing it (#107275)
Ensure multiprocessing SemLock is valid for spawn Process before serializing it.

Creating a multiprocessing SemLock with a fork context, and then trying to pass it to a spawn-created Process, would segfault if not detected early.

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2023-08-23 20:27:35 +00:00
Petr Viktorin 5d18715765
gh-107811: tarfile: treat overflow in UID/GID as failure to set it (#108369) 2023-08-23 20:00:07 +02:00
Irit Katriel 72119d16a5
gh-105481: remove regen-opcode. Generated _PyOpcode_Caches in regen-cases. (#108367) 2023-08-23 18:39:00 +01:00
Serhiy Storchaka 422f81b5d2
gh-107298: Fix a few more refs in the C API docs (GH-108361)
gh-107298: Fix few more refs in the C API docs
2023-08-23 20:12:42 +03:00
Dong-hee Na 2135bcd3ca
gh-107265: Ensure de_instrument does not handle ENTER_EXECUTOR (#108366) 2023-08-23 08:45:20 -07:00
Nikita Sobolev 3f61cf646d
gh-108303: Move ann_module*.py files to typinganndata/ folder (#108354) 2023-08-23 17:42:08 +02:00
Victor Stinner f5559f38d9
gh-108308: Replace PyDict_GetItem() with PyDict_GetItemRef() (#108309)
Replace PyDict_GetItem() calls with PyDict_GetItemRef()
or PyDict_GetItemWithError() to handle errors.

* Replace PyLong_AS_LONG() with _PyLong_AsInt()
  and check for errors.
* Check for PyDict_Contains() error.
* pycore_init_builtins() checks for _PyType_Lookup() failure.
2023-08-23 17:40:26 +02:00
Serhiy Storchaka 154477be72
gh-50002: xml.dom.minidom now preserves whitespaces in attributes (GH-107947)
Also double quotes (") are now only quoted in attributes.
2023-08-23 15:23:41 +03:00
Mariusz Felisiak 29bc6165ab
gh-105539: Fix ResourceWarning from unclosed SQLite connections in test_sqlite3 (#108360)
Follow up to 1a1bfc2891.

Explicitly manage connections in:

- test_audit.test_sqlite3
- test_sqlite3.test_audit
- test_sqlite3.test_backup

Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-08-23 10:10:08 +00:00
Petr Viktorin 31b61d19ab
gh-108294: Add time.sleep audit event (GH-108298) 2023-08-23 11:00:22 +02:00
Irit Katriel 2dfbd4f36d
gh-108113: Make it possible to optimize an AST (#108282) 2023-08-23 09:01:17 +01:00