Commit graph

1209 commits

Author SHA1 Message Date
Pablo Galindo Salgado 45d648597b
gh-112387: Fix error positions for decoded strings with backwards tokenize errors (#112409)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2023-11-27 18:37:48 +00:00
Pablo Galindo Salgado 2c8b191742
gh-112388: Fix an error that was causing the parser to try to overwrite tokenizer errors (#112410)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2023-11-27 18:36:11 +00:00
Pablo Galindo Salgado d59feb5dbe
gh-112243: Don't include comments in f-string debug expressions (#112284) 2023-11-20 15:18:24 +00:00
Crowthebird 1c8f912ebd
bpo-45759: Better error messages for non-matching 'elif'/'else' statements (#29513) 2023-11-20 13:27:53 +00:00
Brett Cannon 56e59a49ae
GH-111807: Lower the parser stack depth under WASI debug builds (#112225) 2023-11-20 13:27:33 +00:00
Sam Gross 446f18a911
gh-111956: Add thread-safe one-time initialization. (gh-111960) 2023-11-16 12:19:54 -07:00
Markus Mohrhard 1447af7970
gh-106905: avoid incorrect SystemError about recursion depth mismatch (#106906)
* gh-106905: avoid incorrect SystemError about recursion depth mismatch

* Update Misc/NEWS.d/next/Core and Builtins/2023-07-20-11-41-16.gh-issue-106905.AyZpuB.rst

---------

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-11-13 13:05:17 +00:00
Serhiy Storchaka 937872e8ea
Simplify _PyPegen_join_names_with_dot() (GH-111602) 2023-11-01 16:25:36 +02:00
Tomas R 453e96e302
gh-111420: Allow type comments in parenthesized with statements (#111468) 2023-10-31 21:02:42 +00:00
Pablo Galindo Salgado 3d2f1f0b83
gh-111380: Show SyntaxWarnings only once when parsing if invalid syntax is encouintered (#111381) 2023-10-27 12:19:34 +09:00
Shantanu 3156d193b8
gh-100445: Improve error message for unterminated strings with escapes (#100446) 2023-10-18 13:58:51 +01:00
Pablo Galindo Salgado 24e4ec7766
gh-110938: Fix error messages for indented blocks with functions and classes with generic type parameters (#110973) 2023-10-17 13:45:13 +01:00
Lysandros Nikolaou f46333b9f5
gh-107450: Remove unnecessary overflow check in parser error handler (#110940) 2023-10-16 22:41:01 +02:00
Lysandros Nikolaou a1ac5590e0
gh-107450: Check for overflow in the tokenizer and fix overflow test (#110832)
Co-authored-by: Filipe Laíns <lains@riseup.net>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-10-16 16:42:49 +02:00
Pablo Galindo Salgado e1d8c65e1d
gh-110805: Allow the repl to show source code and complete tracebacks (#110775) 2023-10-13 09:25:37 +00:00
Lysandros Nikolaou fb7843ee89
gh-107450: Raise OverflowError when parser column offset overflows (#110754) 2023-10-12 09:34:12 +00:00
Pablo Galindo Salgado 3d180347ae
gh-110696: Fix incorrect syntax error message for incorrect argument unpacking (#110706) 2023-10-12 09:02:02 +00:00
Lysandros Nikolaou 17d65547df
gh-104169: Fix test_peg_generator after tokenizer refactoring (#110727)
* Fix test_peg_generator after tokenizer refactoring
* Remove references to tokenizer.c in comments etc.
2023-10-12 09:34:35 +02:00
Filipe Laíns 23645420dc
GH-110749: fix unistd.h import in file_tokenizer.c (#110750) 2023-10-12 07:52:13 +02:00
Lysandros Nikolaou 01481f2dc1
gh-104169: Refactor tokenizer into lexer and wrappers (#110684)
* The lexer, which include the actual lexeme producing logic, goes into
  the `lexer` directory.
* The wrappers, one wrapper per input mode (file, string, utf-8, and
  readline), go into the `tokenizer` directory and include logic for
  creating a lexer instance and managing the buffer for different modes.
---------

Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-10-11 15:14:44 +00:00
sunmy2019 2cb62c6437
gh-110309: Prune empty constant in format specs (#110320)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-10-05 14:08:42 +00:00
Pablo Galindo Salgado cc389ef627
gh-110259: Fix f-strings with multiline expressions and format specs (#110271)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2023-10-05 14:26:44 +01:00
Victor Stinner 7513994c92
gh-110014: Include explicitly <unistd.h> header (#110155)
* Remove unused <locale.h> includes.
* Remove unused <fcntl.h> include in traceback.h.
* Remove redundant <assert.h> and <stddef.h> includes. They  are already
  included by "Python.h".
* Remove <object.h> include in faulthandler.c. Python.h already includes it.
* Add missing <stdbool.h> in pycore_pythread.h if HAVE_PTHREAD_STUBS
  is defined.
* Fix also warnings in pthread_stubs.h: don't redefine macros if they
  are already defined, like the __NEED_pthread_t macro.
2023-09-30 20:06:45 +00:00
Pablo Galindo Salgado b28ffaa193
gh-109596: Ensure repeated rules in the grammar are not allowed and fix incorrect soft keywords (#109606) 2023-09-22 19:03:23 +01:00
Pablo Galindo Salgado 5bda2f637e
gh-109114: Relax the check for invalid lambdas inside f-strings to avoid false positives (#109121) 2023-09-08 17:00:23 +00:00
Victor Stinner b0edf3b98e
GH-91079: Rename C_RECURSION_LIMIT to Py_C_RECURSION_LIMIT (#108507)
Symbols of the C API should be prefixed by "Py_" to avoid conflict
with existing names in 3rd party C extensions on "#include <Python.h>".

test.pythoninfo now logs Py_C_RECURSION_LIMIT constant and other
_testcapi and _testinternalcapi constants.
2023-09-08 09:48:28 +00:00
Serhiy Storchaka b2729e93e9
gh-88943: Improve syntax error for non-ASCII character that follows a numerical literal (GH-109081)
It now points on the invalid non-ASCII character, not on the valid numerical literal.
2023-09-07 17:00:13 +03:00
Victor Stinner 578ebc5d5f
gh-108767: Replace ctype.h functions with pyctype.h functions (#108772)
Replace <ctype.h> locale dependent functions with Python "pyctype.h"
locale independent functions:

* Replace isalpha() with Py_ISALPHA().
* Replace isdigit() with Py_ISDIGIT().
* Replace isxdigit() with Py_ISXDIGIT().
* Replace tolower() with Py_TOLOWER().

Leave Modules/_sre/sre.c unchanged, it uses locale dependent
functions on purpose.

Include explicitly <ctype.h> in _decimal.c to get isascii().
2023-09-01 18:36:53 +02:00
Victor Stinner e59a95238b
gh-108444: Remove _PyLong_AsInt() function (#108461)
* Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
* Remove _PyLong_AsInt() alias to PyLong_AsInt().
2023-08-25 11:13:59 +02:00
Dennis Sweeney 86617518c4
gh-108179: Add error message for parser stack overflows (#108256) 2023-08-22 08:41:50 +01:00
Victor Stinner 0dd3fc2a64
gh-108216: Cleanup #include in internal header files (#108228)
* Add missing includes.
* Remove unused includes.
* Update old include/symbol names to newer names.
* Mention at least one included symbol.
* Sort includes.
* Update Tools/cases_generator/generate_cases.py used to generated
  pycore_opcode_metadata.h.
* Update Parser/asdl_c.py used to generate pycore_ast.h.
* Cleanup also includes in _testcapimodule.c and _testinternalcapi.c.
2023-08-21 18:05:59 +00:00
Irit Katriel 10a91d7e98
gh-108113: Make it possible to create an optimized AST (#108154) 2023-08-21 16:31:30 +00:00
Lysandros Nikolaou d66bc9e8a7
gh-107967: Fix infinite recursion on invalid escape sequence warning (#107968) 2023-08-15 11:26:42 +00:00
Mark Shannon fa45958450
GH-107263: Increase C stack limit for most functions, except _PyEval_EvalFrameDefault() (GH-107535)
* Set C recursion limit to 1500, set cost of eval loop to 2 frames, and compiler mutliply to 2.
2023-08-04 10:10:29 +01:00
Pablo Galindo Salgado da8f87b7ea
gh-107015: Remove async_hacks from the tokenizer (#107018) 2023-07-26 16:34:15 +01:00
Victor Stinner 1a3faba9f1
gh-106869: Use new PyMemberDef constant names (#106871)
* Remove '#include "structmember.h"'.
* If needed, add <stddef.h> to get offsetof() function.
* Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
* Replace:

  * T_SHORT => Py_T_SHORT
  * T_INT => Py_T_INT
  * T_LONG => Py_T_LONG
  * T_FLOAT => Py_T_FLOAT
  * T_DOUBLE => Py_T_DOUBLE
  * T_STRING => Py_T_STRING
  * T_OBJECT => _Py_T_OBJECT
  * T_CHAR => Py_T_CHAR
  * T_BYTE => Py_T_BYTE
  * T_UBYTE => Py_T_UBYTE
  * T_USHORT => Py_T_USHORT
  * T_UINT => Py_T_UINT
  * T_ULONG => Py_T_ULONG
  * T_STRING_INPLACE => Py_T_STRING_INPLACE
  * T_BOOL => Py_T_BOOL
  * T_OBJECT_EX => Py_T_OBJECT_EX
  * T_LONGLONG => Py_T_LONGLONG
  * T_ULONGLONG => Py_T_ULONGLONG
  * T_PYSSIZET => Py_T_PYSSIZET
  * T_NONE => _Py_T_NONE
  * READONLY => Py_READONLY
  * PY_AUDIT_READ => Py_AUDIT_READ
  * READ_RESTRICTED => Py_AUDIT_READ
  * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED
  * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
2023-07-25 15:28:30 +02:00
Victor Stinner 7d41ead919
gh-106320: Remove _PyBytes_Join() C API (#107144)
Move private _PyBytes functions to the internal C API
(pycore_bytesobject.h):

* _PyBytes_DecodeEscape()
* _PyBytes_FormatEx()
* _PyBytes_FromHex()
* _PyBytes_Join()

No longer export these functions.
2023-07-23 20:10:12 +00:00
Victor Stinner d228825e08
gh-106320: Remove _PyOS_ReadlineTState API (#107034)
Remove _PyOS_ReadlineTState variable from the public C API.
The symbol is still exported for the readline shared extension.
2023-07-22 14:45:56 +00:00
Menelaos Kotoglou 76e20c361c
gh-106989: Remove tok report warnings (#106993) 2023-07-22 14:23:23 +02:00
Serhiy Storchaka be1b968dc1
gh-106521: Remove _PyObject_LookupAttr() function (GH-106642) 2023-07-12 08:57:10 +03:00
Lysandros Nikolaou dfe4de2038
gh-106396: Special-case empty format spec to gen empty JoinedStr node (#106401) 2023-07-04 14:19:08 +02:00
Victor Stinner d8c5d76da2
gh-106320: Remove private _PyUnicode codecs C API functions (#106385)
Remove private _PyUnicode codecs C API functions: move them to the
internal C API (pycore_unicodeobject.h). No longer export most of
these functions.
2023-07-04 07:29:52 +00:00
Victor Stinner c5afc97fc2
gh-106320: Remove private _PyErr C API functions (#106356)
Remove private _PyErr C API functions: move them to the internal
C API (pycore_pyerrors.h).
2023-07-03 10:48:50 +00:00
Inada Naoki d5bd32fb48
gh-104922: remove PY_SSIZE_T_CLEAN (#106315) 2023-07-02 15:07:46 +09:00
Nikita Sobolev 46c1097868
gh-106145: Make end_{lineno,col_offset} required on type_param nodes (#106224) 2023-06-30 23:45:08 +00:00
Victor Stinner 8c5f74fc89
gh-106023: Update code using _PyObject_FastCall() (#106257)
Replace _PyObject_FastCall() calls with PyObject_Vectorcall().
2023-06-30 01:05:01 +00:00
Pablo Galindo Salgado 13237a2da8
gh-98931: Add custom error messages to invalid import/from with multiple targets (#105985)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-06-22 15:56:40 +00:00
Lysandros Nikolaou 6586cee27f
gh-105938: Emit a SyntaxWarning for escaped braces in an f-string (#105939) 2023-06-20 12:38:46 +00:00
Brandt Bucher a4056c8f9c
GH-105588: Add missing error checks to some obj2ast_* converters (GH-105589) 2023-06-15 15:45:13 -07:00
Lysandros Nikolaou d382ad4915
gh-105820: Fix tok_mode expression buffer in file & readline tokenizer (#105828) 2023-06-15 16:21:24 +00:00