Commit graph

25351 commits

Author SHA1 Message Date
Jakub Kulík 7431448b81
bpo-43498: Fix dictionary iteration error in _ExecutorManagerThread (GH-24868) 2021-11-29 14:02:56 +02:00
Erlend Egeberg Aasland b3f443a35e
bpo-40280: Disable unusable core extension modules on emscripten (GH-29834)
Co-authored-by: Christian Heimes <christian@python.org>
2021-11-29 12:39:14 +01:00
Sam Bull 934a826237
bpo-37658: Actually return result in race condition (GH-29202) 2021-11-29 10:12:57 +02:00
Christian Heimes f87ea03502
bpo-45915: use fcntl(fd, F_GETFD) in is_valid_fd() (GH-29821) 2021-11-28 19:40:27 +01:00
180909 46c8d91571
bpo-19460: Add test for MIMENonMultipart (GH-29817) 2021-11-28 11:24:41 +02:00
Irit Katriel 4dfae6f38e
bpo-45614: Fix traceback display for exceptions with invalid module name (GH-29726) 2021-11-27 22:00:10 +00:00
Pablo Galindo Salgado e71c12efcd
bpo-42268: Fail the configure step if the selected compiler doesn't support memory sanitizer (GH-29806) 2021-11-27 18:04:06 +00:00
Raymond Hettinger af9ee57b96
bpo-45876: Improve accuracy for stdev() and pstdev() in statistics (GH-29736)
* Inlined code from variance functions

* Added helper functions for the float square root of a fraction

* Call helper functions

* Add blurb

* Fix over-specified test

* Add a test for the _sqrt_frac() helper function

* Increase the tested range

* Add type hints to the internal function.

* Fix test for correct rounding

* Simplify ⌊√(n/m)⌋ calculation

Co-authored-by: Mark Dickinson <dickinsm@gmail.com>

* Add comment and beef-up tests

* Test for zero denominator

* Add algorithmic references

* Add test for the _isqrt_frac_rto() helper function.

* Compute the 109 instead of hard-wiring it

* Stronger test for _isqrt_frac_rto()

* Bigger range

* Bigger range

* Replace float() call with int/int division to be parallel with the other code path.

* Factor out division. Update proof link. Remove internal type declaration

Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2021-11-26 22:54:50 -07:00
Steve Dower db55f3faba
bpo-44530: Reverts a change to the 'code.__new__' audit event (GH-29809) 2021-11-27 00:26:45 +00:00
Ma Lin 7edb6270a7
bpo-41735: Fix thread lock in zlib.Decompress.flush() may go wrong (GH-29587)
* Fix thread lock in zlib.Decompress.flush() may go wrong

Getting `.unconsumed_tail` before acquiring the thread lock may mix up decompress state.
2021-11-26 16:18:17 -08:00
Steve Dower 4841e694ee
bpo-45901: Fixes argument passing when invoking .py files directly through the Store package (GH-29799) 2021-11-26 23:08:20 +00:00
Victor Stinner 9a7611a7c4
bpo-45866: Fix typo in the NEWS entry (GH-29798) 2021-11-26 22:56:25 +01:00
Christian Heimes 4ebde73b8e
bpo-40280: Move hard-coded feature checks to configure (GH-29789)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-11-26 19:36:48 +01:00
Christian Heimes 8caceb7a47
bpo-40280: Add configure check for socket shutdown (GH-29795) 2021-11-26 15:16:54 +01:00
Victor Stinner 253b7a0a9f
bpo-45866: pegen strips directory of "generated from" header (GH-29777)
"make regen-all" now produces the same output when run from a
directory other than the source tree: when building Python out of the
source tree.
2021-11-26 11:50:34 +01:00
Simon McVittie 97dcab7832
bpo-43137: webbrowser: Replace gvfs-open and gnome-open with "gio open" (GH-29154)
* [bpo-43137](): webbrowser: Prefer gio open over gvfs-open
    
    gvfs-open(1) was superseded by gio(1) in 2015, and removed from GNOME
    releases in 2018. Debian and its derivatives like Ubuntu currently still
    have a compatibility shim for gvfs-open, but we plan to remove it.
    
    webbrowser prefers xdg-settings and xdg-open over gvfs-open, so this
    will only have any practical effect on systems where the xdg-utils
    package is not installed.
    
    Note that we don't check for GNOME_DESKTOP_SESSION_ID before using gio.
    gio does the right thing on any desktop environment that follows
    freedesktop.org specifications, similar to xdg-settings, so it's
    unnecessary to guard in this way. GNOME_DESKTOP_SESSION_ID was deprecated
    in 2008 and removed from upstream gnome-session in 2018 (it's still
    present in Debian/Ubuntu for backward compatibility, but probably
    shouldn't be). The replacement way to detect a desktop environment is
    the XDG_CURRENT_DESKTOP environment variable, which is a colon-separated
    sequence where the first item is the current desktop environment and the
    second and subsequent items (if present) are other desktop environments
    that it resembles or is based on.
    
    Resolves: 

* [bpo-43137](): webbrowser: Never invoke gnome-open
    
    gnome-open was part of GNOME 2, which was superseded in around 2010 and
    is unmaintained. The replacement was gvfs-open, which was subsequently
    replaced by gio(1) (as used in the previous commit).

* [bpo-43137](): webbrowser: Don't run gvfs-open on GNOME
    
    gvfs-open was deprecated in 2015 and removed in 2018. The replacement
    is gio(1) (as used in a previous commit).
    
    GNOME_DESKTOP_SESSION_ID was deprecated in 2008 and removed in 2018.
    The replacement is XDG_CURRENT_DESKTOP (as mentioned in a previous
    commit).

---

To test this on a typical modern Linux system, it is necessary to disable the `xdg-settings` and `xdg-open` code paths, for example with this hack:

<details><summary>Hack to disable use of xdg-settings and xdg-open</summary>

```diff
diff --git a/Lib/webbrowser.py b/Lib/webbrowser.py
index 3244f206aa..8f6c09d1d2 100755
--- a/Lib/webbrowser.py
+++ b/Lib/webbrowser.py
@@ -459,7 +459,7 @@ def open(self, url, new=0, autoraise=True):
 def register_X_browsers():
 
     # use xdg-open if around
-    if shutil.which("xdg-open"):
+    if 0 and shutil.which("xdg-open"):
         register("xdg-open", None, BackgroundBrowser("xdg-open"))
 
     # Opens an appropriate browser for the URL scheme according to
@@ -549,7 +549,7 @@ def register_standard_browsers():
         # Prefer X browsers if present
         if os.environ.get("DISPLAY") or os.environ.get("WAYLAND_DISPLAY"):
             try:
-                cmd = "xdg-settings get default-web-browser".split()
+                cmd = "false xdg-settings get default-web-browser".split()
                 raw_result = subprocess.check_output(cmd, stderr=subprocess.DEVNULL)
                 result = raw_result.decode().strip()
             except (FileNotFoundError, subprocess.CalledProcessError, PermissionError, NotADirectoryError) :
```

</details>

I haven't attempted to assess which of the specific web browsers such as Galeon are still extant, and which ones disappeared years ago. They could almost certainly be cleaned up, but that's beyond the scope of this PR.
2021-11-25 12:49:48 -08:00
Christian Heimes 1052a39b76
bpo-40280: Add wasm cross build targets (GH-29771) 2021-11-25 21:24:40 +01:00
Christian Heimes dfcc6ff36f
bpo-33393: Update config.guess and config.sub (GH-29781) 2021-11-25 20:55:29 +01:00
Christian Heimes dc19e86983
bpo-41498: Fix build on platforms without sigset_t (GH-29770) 2021-11-25 12:28:41 +01:00
Pablo Galindo Salgado 24c10d2943
bpo-45727: Only trigger the 'did you forgot a comma' error suggestion if inside parentheses (GH-29757) 2021-11-24 22:21:23 +00:00
Christian Heimes b30bf4520a
bpo-45881: Use CC from env first for cross building (GH-29752) 2021-11-24 18:53:33 +01:00
Jason R. Coombs d5cd2effa6
bpo-45514: Deprecate importlib resources legacy functions. (GH-29036)
* bpo-45514: Apply changes from importlib_resources@a3ef4128c6

* Mark legacy functions as deprecated in the docs and link to the migration docs in importlib_resources docs.

* Apply changes from importlib_resources@329ae9d5f2c.

* Indicate importlib.resources as a module.

Co-authored-by: Filipe Laíns <lains@riseup.net>
2021-11-24 02:51:37 -05:00
Christian Heimes dd8ce9ea8d
bpo-45886: Allow overriding freeze command for cross compiling (GH-29735) 2021-11-24 08:07:15 +01:00
Zachary Ware f9de97aae5
bpo-45616: Let py.exe distinguish between v3.1 and v3.10 (GH-29731) 2021-11-23 22:41:04 -06:00
Victor Stinner 4ae26b9c1d
bpo-39026: Fix Python.h when building with Xcode (GH-29488)
Fix Python.h to build C extensions with Xcode: remove a relative
include from Include/cpython/pystate.h.
2021-11-23 18:58:57 +01:00
Guido van Rossum 5be98e57b3
bpo-45873: Get rid of bootstrap_python (#29717)
Instead we use $(PYTHON_FOR_REGEN) .../deepfreeze.py with the
frozen .h file as input, as we did for Windows in bpo-45850.

We also get rid of the code that generates the .h files
when make regen-frozen is run (i.e., .../make_frozen.py),
and the MANIFEST file.

Restore Python 3.8 and 3.9 as Windows host Python again

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2021-11-23 08:56:06 -08:00
Miro Hrončok ae1965ccb4
bpo-45703: Invalidate _NamespacePath cache on importlib.invalidate_ca… (GH-29384)
Consider the following directory structure:

    .
    └── PATH1
        └── namespace
            └── sub1
                └── __init__.py

And both PATH1 and PATH2 in sys path:

    $ PYTHONPATH=PATH1:PATH2 python3.11
    >>> import namespace
    >>> import namespace.sub1
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> ...

While this interpreter still runs, PATH2/namespace/sub2 is created:

    .
    ├── PATH1
    │   └── namespace
    │       └── sub1
    │           └── __init__.py
    └── PATH2
        └── namespace
            └── sub2
                └── __init__.py

The newly created module cannot be imported:

    >>> ...
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> import namespace.sub2
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ModuleNotFoundError: No module named 'namespace.sub2'

Calling importlib.invalidate_caches() now newly allows to import it:

    >>> import importlib
    >>> importlib.invalidate_caches()
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> import namespace.sub2
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace', '.../PATH2/namespace'])

This was not previously possible.
2021-11-23 07:38:02 -08:00
Eric Snow 8ed1495ad9
bpo-45783: Preserve file moves and deletions in the tests for the freeze tool. (GH-29527)
Use shutil.copytree rather than Git, which might be missing (or configured
differently) when testing Python built from a source release.
2021-11-23 14:43:40 +01:00
Nikita Sobolev b48ac6fe38
bpo-45878: convert try/except to self.assertRaises in Lib/ctypes/test/test_functions.py (GH-29721) 2021-11-23 13:12:13 +02:00
Mark Shannon 135cabd328
bpo-44525: Copy free variables in bytecode to allow calls to inner functions to be specialized (GH-29595)
* Make internal APIs that take PyFrameConstructor take a PyFunctionObject instead.

* Add reference to function to frame, borrow references to builtins and globals.

* Add COPY_FREE_VARS instruction to allow specialization of calls to inner functions.
2021-11-23 09:53:24 +00:00
Neil Schemenauer 457e6a6e96
bpo-45561: Run smelly.py tool from $(srcdir) (GH-29138) 2021-11-23 08:51:02 +01:00
Julien Palard 024209401e
bpo-42238: [doc] Announce the future removal of make suspicous. (GH-29652)
* bpo-42238: [doc] Announce the future removal of make suspicous.

* Add a news entry.
2021-11-22 15:17:54 -08:00
Guido van Rossum 1037ca5a8e
bpo-45850: Implement deep-freeze on Windows (#29648)
Implement changes to build with deep-frozen modules on Windows.
Note that we now require Python 3.10 as the "bootstrap" or "host" Python.
This causes a modest startup speed (around 7%) on Windows.
2021-11-22 10:09:48 -08:00
Mark Shannon 7fd92a8b7e
bpo-45813: Make sure that frame->generator is NULLed when generator is deallocated. (GH-29700) 2021-11-22 14:01:23 +00:00
Eric V. Smith d3062f672c
bpo-44649: Fix dataclasses(slots=True) with a field with a default, but init=False (GH-29692)
Special handling is needed, because for non-slots dataclasses the instance attributes are not set: reading from a field just references the class's attribute of the same name, which contains the default value. But this doesn't work for classes using __slots__: they don't read the class's attribute. So in that case (and that case only), initialize the instance attribute. Handle this for both normal defaults, and for fields using default_factory.
2021-11-22 08:26:12 -05:00
Erlend Egeberg Aasland db2277a114
bpo-45723: Add helpers for save/restore env (GH-29637) 2021-11-22 09:05:06 +01:00
Raymond Hettinger d2b55b07d2
bpo-45766: Add direct proportion option to linear_regression(). (#29490)
* bpo-45766: Add direct proportion option to linear_regression().

* Update 2021-11-09-09-18-06.bpo-45766.dvbcMf.rst

* Use ellipsis to avoid round-off issues.

* Update Misc/NEWS.d/next/Library/2021-11-09-09-18-06.bpo-45766.dvbcMf.rst

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>

* Update signature in main docs

* Fix missing comma

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-11-21 08:39:26 -06:00
Eric V. Smith f7638dd0f9
Added kw_only parameter to make_dataclasses. (GH-29679) 2021-11-20 18:25:56 -05:00
Logan Jones fdc0e09c33
bpo-44733: Add max_tasks_per_child to ProcessPoolExecutor (GH-27373)
Co-authored-by: Antoine Pitrou <antoine@python.org>
2021-11-20 21:19:41 +01:00
Pablo Galindo Salgado 81f4e116ef
bpo-45811: Improve error message when source code contains invisible control characters (GH-29654) 2021-11-20 18:28:28 +00:00
Pablo Galindo Salgado 7a1d932528
bpo-45450: Improve syntax error for parenthesized arguments (GH-28906) 2021-11-20 18:27:40 +00:00
Dylan Van Assche ef5305819f
bpo-42158: Add MIME types for n-triples, n-quads, n3 and trig (GH-23230)
Co-authored-by: Éric Araujo <merwok@netwok.org>
2021-11-20 16:52:00 +01:00
Pablo Galindo Salgado fdcc46d955
bpo-45848: Allow the parser to get error lines from encoded files (GH-29646) 2021-11-20 15:36:07 +01:00
Brett Cannon be36e06340
bpo-45250: fix docs regarding __iter__ and iterators being inconsistently required by CPython (GH-29170)
It is now considered a historical accident that e.g. `for` loops and the `iter()` built-in function do not require the iterators they work with to define `__iter__`, only `__next__`.
2021-11-19 16:40:34 -08:00
Christian Heimes c8c21bdd19
bpo-45847: Port builtin hashlib extensions to PY_STDLIB_MOD (GH-29642) 2021-11-19 20:20:32 +01:00
Ruben Vorderman 0ff3d95b98
bpo-45507: EOFErrors should be thrown for truncated gzip members (GH-29029) 2021-11-19 19:07:05 +01:00
Christian Heimes 7e44dc0ba7
bpo-45573: Move mandatory core modules to Modules/Setup.bootstrap (GH-29616) 2021-11-19 16:40:57 +01:00
Mark Shannon 337cb480e9
bpo-45709: Fix tracing when exception is handled. (GH-29638) 2021-11-19 15:16:49 +00:00
Erlend Egeberg Aasland 29e5874d5a
bpo-45774: Autoconfiscate SQLite detection (GH-29507)
Co-authored-by: Christian Heimes <christian@python.org>
2021-11-19 15:10:41 +01:00
Dennis Sweeney 036fead695
bpo-45609: Specialize STORE_SUBSCR (GH-29242)
* Specialize STORE_SUBSCR for list[int], and dict[object]

* Adds _PyDict_SetItem_Take2 which consumes references to the key and values.
2021-11-19 10:30:37 +00:00