Commit graph

4092 commits

Author SHA1 Message Date
Brett Cannon 482b0ee8f6
Clarify that WASI tool requirements are included in the devcontainer (GH-112561) 2023-11-30 13:01:07 -08:00
Irit Katriel 07ebd46f9e
gh-112519: Make it possible to specify instruction flags for pseudo instructions in bytecodes.c (#112520) 2023-11-30 11:03:30 +00:00
Donghee Na 7eeea13403
gh-112205: Support @getter annotation from AC (gh-112396) 2023-11-30 19:40:53 +09:00
Brett Cannon 37589d76bb
GH-103065, GH-106704, GH-105253: Provide a Tools/wasm/wasi.py script to simplify doing a WASI build (GH-112473) 2023-11-29 16:18:25 -08:00
Donghee Na 4ec849bba8
gh-112213: Add missing declaration of target_critical_section (gh-112374)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-11-25 00:58:39 +09:00
Mark Shannon 1619f4350e
GH-111485: Sort cases in the case generator output (GH-112315) 2023-11-22 15:19:50 +00:00
Guido van Rossum 8deb8bc2e5
gh-112287: Speed up Tier 2 (uop) interpreter a little (#112286)
This makes the Tier 2 interpreter a little faster.
I calculated by about 3%,
though I hesitate to claim an exact number.

This starts by doubling the trace size limit (to 512),
making it more likely that loops fit in a trace.

The rest of the approach is to only load
`oparg` and `operand` in cases that use them.
The code generator know when these are used.

For `oparg`, it will conditionally emit
```
oparg = CURRENT_OPARG();
```
at the top of the case block.
(The `oparg` variable may be referenced multiple times
by the instructions code block, so it must be in a variable.)

For `operand`, it will use `CURRENT_OPERAND()` directly
instead of referencing the `operand` variable,
which no longer exists.
(There is only one place where this will be used.)
2023-11-20 11:25:32 -08:00
Brett Cannon 56e59a49ae
GH-111807: Lower the parser stack depth under WASI debug builds (#112225) 2023-11-20 13:27:33 +00:00
Donghee Na b8c952af72
gh-111903: Update AC to support "pycore_critical_section.h" header (gh-112251) 2023-11-19 10:13:58 +09:00
Donghee Na e52cc80f7f
gh-112213: Add @critical_section target directive to Argument Clinic (gh-112232)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-11-19 08:05:49 +09:00
Guido van Rossum eb3c94ea66
gh-110319: Assert type_version != 0 before using it (#112226)
- Ensure that `assert(type_version != 0);` always comes *before* using `type_version`

Also:
- In cases_generator, rename `-v` to from `--verbose` to `--viable`
2023-11-17 20:58:13 -08:00
Guido van Rossum da314f7c8d
A few more cases_generator cleanups (#112220) 2023-11-17 22:36:37 +00:00
Guido van Rossum be0bd54c6b
gh-106529: Cleanups split off gh-112134 (#112214)
- Double max trace size to 256
- Add a dependency on executor_cases.c.h for ceval.o
- Mark `_SPECIALIZE_UNPACK_SEQUENCE` as `TIER_ONE_ONLY`
- Add debug output back showing the optimized trace
- Bunch of cleanups to Tools/cases_generator/
2023-11-17 11:49:42 -08:00
Nikita Sobolev 762eb58220
Remove imp_dummy_def from Tools/c-analyzer/cpython/ignored.tsv (gh-112122)
It was removed in 3.12, no need to keep the ignore.
2023-11-16 12:21:23 -07:00
Victor Stinner d9fd33a869
gh-112088: Run autoreconf in GHA check_generated_files (#112090)
The "Check if generated files are up to date" job of GitHub Actions
now runs the "autoreconf -ivf -Werror" command instead of the "make
regen-configure" command to avoid depending on the external quay.io
server.

Add Tools/build/regen-configure.sh script to regenerate the configure
with an Ubuntu container image. The
"quay.io/tiran/cpython_autoconf:271" container image
(https://github.com/tiran/cpython_autoconf) is no longer used.
2023-11-15 21:47:14 +01:00
Mark Shannon 4bbb367ba6
GH-111848: Set the IP when de-optimizing (GH-112065)
* Replace jumps with deopts in tier 2

* Fewer special cases of uop names

* Add target field to uop IR

* Remove more redundant SET_IP and _CHECK_VALIDITY micro-ops

* Extend whitelist of non-escaping API functions.
2023-11-15 15:48:58 +00:00
Sam Gross 324531df90
gh-111903: Add @critical_section directive to Argument Clinic. (#111904)
The `@critical_section` directive instructs Argument Clinic to generate calls
to `Py_BEGIN_CRITICAL_SECTION()` and `Py_END_CRITICAL_SECTION()` around the
bound function. In `--disable-gil` builds, these calls will lock and unlock
the `self` object. They are no-ops in the default build.

This is used in one place (`_io._Buffered.close`) as a demonstration.
Subsequent PRs will use it more widely in the `_io.Buffered` bindings.
2023-11-14 10:47:46 +00:00
Brandt Bucher 31ad7e061e
GH-111520: Add back the operand local (GH-111813) 2023-11-13 17:27:19 -08:00
Alex Waygood 2e7f070080
Bump mypy to 1.7.0 (#111961) 2023-11-10 16:00:50 +00:00
Mark Shannon 25c4956488
GH-109369: Exit tier 2 if executor is invalid (GH-111657) 2023-11-09 11:19:51 +00:00
Mark Shannon 06efb60264
GH-111848: Tidy up tier 2 handling of FOR_ITER specialization by using DEOPT_IF instead of jumps. (GH-111849) 2023-11-08 13:31:55 +00:00
Victor Stinner 11e83488c5
gh-111089: Revert PyUnicode_AsUTF8() changes (#111833)
* Revert "gh-111089: Use PyUnicode_AsUTF8() in Argument Clinic (#111585)"

This reverts commit d9b606b3d0.

* Revert "gh-111089: Use PyUnicode_AsUTF8() in getargs.c (#111620)"

This reverts commit cde1071b2a.

* Revert "gh-111089: PyUnicode_AsUTF8() now raises on embedded NUL (#111091)"

This reverts commit d731579bfb.

* Revert "gh-111089: Add PyUnicode_AsUTF8() to the limited C API (#111121)"

This reverts commit d8f32be5b6.

* Revert "gh-111089: Use PyUnicode_AsUTF8() in sqlite3 (#111122)"

This reverts commit 37e4e20eaa.
2023-11-07 22:36:13 +00:00
Mark Shannon 931f4438c9
GH-111485: Allow arbitrary annotations on instructions and micro-ops. (GH-111697) 2023-11-07 09:42:39 +00:00
CF Bolz-Tereick 9573d14215
gh-96954: use a directed acyclic word graph for storing the unicodedata codepoint names (#97906)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
Co-authored-by: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com>
2023-11-04 15:56:58 +01:00
Brad Larsen f21b23058e
Add a fuzz target for _elementtree.XMLParser._parse_whole (#111477)
* Add a fuzzer for `_elementtree.XMLParser._parse_whole`
2023-11-03 14:01:56 -07:00
dependabot[bot] 7215f173f5
build(deps): bump hypothesis from 6.87.1 to 6.88.1 in /Tools (#111596)
Bumps [hypothesis](https://github.com/HypothesisWorks/hypothesis) from 6.87.1 to 6.88.1.
- [Release notes](https://github.com/HypothesisWorks/hypothesis/releases)
- [Commits](https://github.com/HypothesisWorks/hypothesis/compare/hypothesis-python-6.87.1...hypothesis-python-6.88.1)

---
updated-dependencies:
- dependency-name: hypothesis
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-02 10:53:40 +00:00
Guido van Rossum 7e135a48d6
gh-111520: Integrate the Tier 2 interpreter in the Tier 1 interpreter (#111428)
- There is no longer a separate Python/executor.c file.
- Conventions in Python/bytecodes.c are slightly different -- don't use `goto error`,
  you must use `GOTO_ERROR(error)` (same for others like `unused_local_error`).
- The `TIER_ONE` and `TIER_TWO` symbols are only valid in the generated (.c.h) files.
- In Lib/test/support/__init__.py, `Py_C_RECURSION_LIMIT` is imported from `_testcapi`.
- On Windows, in debug mode, stack allocation grows from 8MiB to 12MiB.
- **Beware!** This changes the env vars to enable uops and their debugging
  to `PYTHON_UOPS` and `PYTHON_LLTRACE`.
2023-11-01 13:13:02 -07:00
Victor Stinner d9b606b3d0
gh-111089: Use PyUnicode_AsUTF8() in Argument Clinic (#111585)
Replace PyUnicode_AsUTF8AndSize() with PyUnicode_AsUTF8() to remove
the explicit check for embedded null characters.

The change avoids to have to include explicitly <string.h> to get the
strlen() function when using a recent version of the limited C API.
2023-11-01 16:34:42 +01:00
Mark Shannon b14e882428
GH-111485: Use micro-ops to split specialization code from base action (GH-111561) 2023-11-01 10:53:27 +00:00
Mark Shannon 2904d99839
GH-111485: Remove some special cases from the code generator and bytecodes.c (GH-111540) 2023-10-31 13:21:07 +00:00
Mark Shannon d27acd4461
GH-111485: Increment next_instr consistently at the start of the instruction. (GH-111486) 2023-10-31 10:09:54 +00:00
Michael Droettboom 9495bcaf59
gh-111277: In summarize_stats.py, don't fail fast on invalid ratios (#111278) 2023-10-30 17:10:07 -07:00
Michael Droettboom 84b4533e84
gh-109329: Count tier2 opcode misses (#110561)
This keeps a separate 'miss' counter for each micro-opcode, incremented whenever a guard uop takes a deoptimization side exit.
2023-10-30 17:02:45 -07:00
Riccardo Ghetta 8eaa206fec
gh-109515: When generating deep frozen modules on Windows, use a list file instead of arguments (GH-109516) 2023-10-30 16:14:26 +00:00
Sam Gross 6dfb8fe023
gh-110481: Implement biased reference counting (gh-110764) 2023-10-30 16:06:09 +00:00
Dino Viehland 05f2f0ac92
gh-90815: Add mimalloc memory allocator (#109914)
* Add mimalloc v2.12

Modified src/alloc.c to remove include of alloc-override.c and not
compile new handler.

Did not include the following files:

 - include/mimalloc-new-delete.h
 - include/mimalloc-override.h
 - src/alloc-override-osx.c
 - src/alloc-override.c
 - src/static.c
 - src/region.c

mimalloc is thread safe and shares a single heap across all runtimes,
therefore finalization and getting global allocated blocks across all
runtimes is different.

* mimalloc: minimal changes for use in Python:

 - remove debug spam for freeing large allocations
 - use same bytes (0xDD) for freed allocations in CPython and mimalloc
   This is important for the test_capi debug memory tests

* Don't export mimalloc symbol in libpython.
* Enable mimalloc as Python allocator option.
* Add mimalloc MIT license.
* Log mimalloc in Lib/test/pythoninfo.py.
* Document new mimalloc support.
* Use macro defs for exports as done in:
  https://github.com/python/cpython/pull/31164/

Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
2023-10-30 15:43:11 +00:00
Savannah Ostrowski 4a929d432b
GH-111339: Fix initialization and finalization of static optimizer types (GH-111430) 2023-10-29 13:53:25 -07:00
Irit Katriel 67a91f78e4
gh-109094: replace frame->prev_instr by frame->instr_ptr (#109095) 2023-10-26 13:43:10 +00:00
Mark Shannon b0699aa544
GH-111213: Fix a few broken stats (GH-111216) 2023-10-26 11:33:12 +01:00
Artyom Romanov 0d1cbff833
Bump test deps: ruff and mypy (#111288) 2023-10-25 01:05:37 -07:00
Michael Droettboom 81eba76450
gh-110019: Refactor summarize_stats (GH-110398) 2023-10-24 09:57:39 +01:00
Nikita Sobolev 4fbf20605b
gh-110572: Remove test_* from _testcapi/getargs.c (GH-111214) 2023-10-24 08:50:11 +03:00
Furkan Onder 32c37fe1ba
gh-67565: Remove redundant C-contiguity checks (GH-105521)
Co-authored-by: Stefan Krah <skrah@bytereef.org>
2023-10-23 12:54:46 +03:00
Victor Stinner 0937b11b89
gh-110964: clinic: refactor output_templates() (#110982) 2023-10-20 23:42:02 +02:00
Victor Stinner be5e8a0103
gh-110964: Remove private _PyArg functions (#110966)
Move the following private functions and structures to
pycore_modsupport.h internal C API:

* _PyArg_BadArgument()
* _PyArg_CheckPositional()
* _PyArg_NoKeywords()
* _PyArg_NoPositional()
* _PyArg_ParseStack()
* _PyArg_ParseStackAndKeywords()
* _PyArg_Parser structure
* _PyArg_UnpackKeywords()
* _PyArg_UnpackKeywordsWithVararg()
* _PyArg_UnpackStack()
* _Py_ANY_VARARGS()

Changes:

* Python/getargs.h now includes pycore_modsupport.h to export
  functions.
* clinic.py now adds pycore_modsupport.h when one of these functions
  is used.
* Add pycore_modsupport.h includes when a C extension uses one of
  these functions.
* Define Py_BUILD_CORE_MODULE in C extensions which now include
  directly or indirectly (via code generated by Argument Clinic)
  pycore_modsupport.h:

  * _csv
  * _curses_panel
  * _dbm
  * _gdbm
  * _multiprocessing.posixshmem
  * _sqlite.row
  * _statistics
  * grp
  * resource
  * syslog

* _testcapi: bad_get() no longer uses METH_FASTCALL calling
  convention but METH_VARARGS. Replace _PyArg_UnpackStack() with
  PyArg_ParseTuple().
* _testcapi: add PYTESTCAPI_NEED_INTERNAL_API macro which is defined
  by _testcapi sub-modules which need the internal C API
  (pycore_modsupport.h): exceptions.c, float.c, vectorcall.c,
  watchers.c.
* Remove Include/cpython/modsupport.h header file.
  Include/modsupport.h no longer includes the removed header file.
* Fix mypy clinic.py
2023-10-17 14:30:31 +02:00
Victor Stinner 054f496bd4
gh-85283: Fix Argument Clinic for md5 extension (#110976)
Limited C API supports the defining class under some conditions.
2023-10-17 13:46:16 +02:00
Victor Stinner 4dba0a6d87
gh-85283: Build md5 extension with limited C API (#110967)
* Replace _Py_strhex() with few lines of code.
* Replace _PyType_GetModuleState() with PyType_GetModuleState().
* Fix make check-c-globals.
2023-10-17 10:57:41 +00:00
Nikita Sobolev b75b1f389f
Bump test deps: ruff and pre-commit-hooks (#110972) 2023-10-17 10:32:29 +00:00
Nikita Sobolev bad7a35055
gh-110907: AC: Disallow using * with vararg (#110908) 2023-10-16 15:26:11 +01:00
Pablo Galindo Salgado 8316769d83
gh-110805: Fix test_peg_generator after the change in the parser for REPL tracebacks (#110814)
Fix test_peg_generator after the change in the parser for REPL tracebacks
2023-10-13 13:04:23 +02:00