languages/cpython
languages/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython synced 2024-09-05 08:17:45 +00:00
Code Issues Packages Projects Releases Wiki Activity
cpython/Python/symtable.c

3003 lines
103 KiB
C
Raw Normal View History

Relax the rules for using 'from ... import *' and exec in the presence of nested functions. Either is allowed in a function if it contains no defs or lambdas or the defs and lambdas it contains have no free variables. If a function is itself nested and has free variables, either is illegal. Revise the symtable to use a PySymtableEntryObject, which holds all the revelent information for a scope, rather than using a bunch of st_cur_XXX pointers in the symtable struct. The changes simplify the internal management of the current symtable scope and of the stack. Added new C source file: Python/symtable.c. (Does the Windows build process need to be updated?) As part of these changes, the initial _symtable module interface introduced in 2.1a2 is replaced. A dictionary of PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
#include "Python.h"
gh-108765: Cleanup #include in Python/*.c files (#108977) Mention one symbol imported by each #include.
2023-09-06 13:56:08 +00:00
#include "pycore_ast.h" // stmt_ty
bpo-43244: Remove parser_interface.h header file (GH-25001) Remove parser functions using the "struct _mod" type, because the AST C API was removed: * PyParser_ASTFromFile() * PyParser_ASTFromFileObject() * PyParser_ASTFromFilename() * PyParser_ASTFromString() * PyParser_ASTFromStringObject() These functions were undocumented and excluded from the limited C API. Add pycore_parser.h internal header file. Rename functions: * PyParser_ASTFromFileObject() => _PyParser_ASTFromFile() * PyParser_ASTFromStringObject() => _PyParser_ASTFromString() These functions are no longer exported (replace PyAPI_FUNC() with extern). Remove also _PyPegen_run_parser_from_file() function. Update test_peg_generator to use _PyPegen_run_parser_from_file_pointer() instead.
2021-03-24 00:29:09 +00:00
#include "pycore_parser.h" // _PyParser_ASTFromString()
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
#include "pycore_pystate.h" // _PyThreadState_GET()
#include "pycore_symtable.h" // PySTEntryObject
gh-106869: Use new PyMemberDef constant names (#106871) * Remove '#include "structmember.h"'. * If needed, add <stddef.h> to get offsetof() function. * Update Parser/asdl_c.py to regenerate Python/Python-ast.c. * Replace: * T_SHORT => Py_T_SHORT * T_INT => Py_T_INT * T_LONG => Py_T_LONG * T_FLOAT => Py_T_FLOAT * T_DOUBLE => Py_T_DOUBLE * T_STRING => Py_T_STRING * T_OBJECT => _Py_T_OBJECT * T_CHAR => Py_T_CHAR * T_BYTE => Py_T_BYTE * T_UBYTE => Py_T_UBYTE * T_USHORT => Py_T_USHORT * T_UINT => Py_T_UINT * T_ULONG => Py_T_ULONG * T_STRING_INPLACE => Py_T_STRING_INPLACE * T_BOOL => Py_T_BOOL * T_OBJECT_EX => Py_T_OBJECT_EX * T_LONGLONG => Py_T_LONGLONG * T_ULONGLONG => Py_T_ULONGLONG * T_PYSSIZET => Py_T_PYSSIZET * T_NONE => _Py_T_NONE * READONLY => Py_READONLY * PY_AUDIT_READ => Py_AUDIT_READ * READ_RESTRICTED => Py_AUDIT_READ * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
2023-07-25 13:28:30 +00:00
gh-109390: add dump_symtable utility under #if 0 (#109391) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-09-20 16:55:56 +00:00
// Set this to 1 to dump all symtables to stdout for debugging
#define _PY_DUMP_SYMTABLE 0
Relax the rules for using 'from ... import *' and exec in the presence of nested functions. Either is allowed in a function if it contains no defs or lambdas or the defs and lambdas it contains have no free variables. If a function is itself nested and has free variables, either is illegal. Revise the symtable to use a PySymtableEntryObject, which holds all the revelent information for a scope, rather than using a bunch of st_cur_XXX pointers in the symtable struct. The changes simplify the internal management of the current symtable scope and of the stack. Added new C source file: Python/symtable.c. (Does the Windows build process need to be updated?) As part of these changes, the initial _symtable module interface introduced in 2.1a2 is replaced. A dictionary of PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
Bug #889500, fix line number on SyntaxWarning for global declarations.
2005-12-19 04:27:42 +00:00
/* error strings used for warnings */
bpo-28936: Detect lexically first syntax error first (#4097) Lexically first global and nonlocal syntax errors at given scope should be detected first.
2017-10-26 21:28:35 +00:00
#define GLOBAL_PARAM \
"name '%U' is parameter and global"
#define NONLOCAL_PARAM \
"name '%U' is parameter and nonlocal"
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
#define GLOBAL_AFTER_ASSIGN \
Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. Patch by Ivan Levkivskyi.
2016-09-09 16:36:26 +00:00
"name '%U' is assigned to before global declaration"
Add a comment explaining the st_symbols cache.
2001-12-10 00:53:18 +00:00
Provisional implementation of PEP 3104. Add nonlocal_stmt to Grammar and Nonlocal node to AST. They both parallel the definitions for globals. The symbol table treats variables declared as nonlocal just like variables that are free implicitly. This change is missing the language spec changes, but makes some decisions about what the spec should say via the unittests. The PEP is silent on a number of decisions, so we should review those before claiming that nonlocal is complete. Thomas Wouters made the grammer and ast changes. Jeremy Hylton added the symbol table changes and the tests. Pete Shinners and Neal Norwitz helped review the code.
2007-02-27 06:50:52 +00:00
#define NONLOCAL_AFTER_ASSIGN \
Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. Patch by Ivan Levkivskyi.
2016-09-09 16:36:26 +00:00
"name '%U' is assigned to before nonlocal declaration"
Provisional implementation of PEP 3104. Add nonlocal_stmt to Grammar and Nonlocal node to AST. They both parallel the definitions for globals. The symbol table treats variables declared as nonlocal just like variables that are free implicitly. This change is missing the language spec changes, but makes some decisions about what the spec should say via the unittests. The PEP is silent on a number of decisions, so we should review those before claiming that nonlocal is complete. Thomas Wouters made the grammer and ast changes. Jeremy Hylton added the symbol table changes and the tests. Pete Shinners and Neal Norwitz helped review the code.
2007-02-27 06:50:52 +00:00
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
#define GLOBAL_AFTER_USE \
Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. Patch by Ivan Levkivskyi.
2016-09-09 16:36:26 +00:00
"name '%U' is used prior to global declaration"
Add a comment explaining the st_symbols cache.
2001-12-10 00:53:18 +00:00
Provisional implementation of PEP 3104. Add nonlocal_stmt to Grammar and Nonlocal node to AST. They both parallel the definitions for globals. The symbol table treats variables declared as nonlocal just like variables that are free implicitly. This change is missing the language spec changes, but makes some decisions about what the spec should say via the unittests. The PEP is silent on a number of decisions, so we should review those before claiming that nonlocal is complete. Thomas Wouters made the grammer and ast changes. Jeremy Hylton added the symbol table changes and the tests. Pete Shinners and Neal Norwitz helped review the code.
2007-02-27 06:50:52 +00:00
#define NONLOCAL_AFTER_USE \
Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. Patch by Ivan Levkivskyi.
2016-09-09 16:36:26 +00:00
"name '%U' is used prior to nonlocal declaration"
#define GLOBAL_ANNOT \
"annotated name '%U' can't be global"
#define NONLOCAL_ANNOT \
"annotated name '%U' can't be nonlocal"
Provisional implementation of PEP 3104. Add nonlocal_stmt to Grammar and Nonlocal node to AST. They both parallel the definitions for globals. The symbol table treats variables declared as nonlocal just like variables that are free implicitly. This change is missing the language spec changes, but makes some decisions about what the spec should say via the unittests. The PEP is silent on a number of decisions, so we should review those before claiming that nonlocal is complete. Thomas Wouters made the grammer and ast changes. Jeremy Hylton added the symbol table changes and the tests. Pete Shinners and Neal Norwitz helped review the code.
2007-02-27 06:50:52 +00:00
Bug #889500, fix line number on SyntaxWarning for global declarations.
2005-12-19 04:27:42 +00:00
#define IMPORT_STAR_WARNING "import * only allowed at module level"
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
#define NAMED_EXPR_COMP_IN_CLASS \
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
"assignment expression within a comprehension cannot be used in a class body"
gh-104640: Disallow walrus in comprehension within type scopes (#104641)
2023-05-19 14:31:09 +00:00
#define NAMED_EXPR_COMP_IN_TYPEVAR_BOUND \
"assignment expression within a comprehension cannot be used in a TypeVar bound"
#define NAMED_EXPR_COMP_IN_TYPEALIAS \
"assignment expression within a comprehension cannot be used in a type alias"
#define NAMED_EXPR_COMP_IN_TYPEPARAM \
"assignment expression within a comprehension cannot be used within the definition of a generic"
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
#define NAMED_EXPR_COMP_CONFLICT \
"assignment expression cannot rebind comprehension iteration variable '%U'"
#define NAMED_EXPR_COMP_INNER_LOOP_CONFLICT \
"comprehension inner loop cannot rebind assignment expression target '%U'"
#define NAMED_EXPR_COMP_ITER_EXPR \
"assignment expression cannot be used in a comprehension iterable expression"
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
#define ANNOTATION_NOT_ALLOWED \
gh-104572: Improve error messages for invalid constructs in PEP 695 contexts (#104573)
2023-05-17 13:05:42 +00:00
"%s cannot be used within an annotation"
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
#define EXPR_NOT_ALLOWED_IN_TYPE_VARIABLE \
"%s cannot be used within %s"
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
#define EXPR_NOT_ALLOWED_IN_TYPE_ALIAS \
gh-104572: Improve error messages for invalid constructs in PEP 695 contexts (#104573)
2023-05-17 13:05:42 +00:00
"%s cannot be used within a type alias"
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
#define EXPR_NOT_ALLOWED_IN_TYPE_PARAMETERS \
gh-104572: Improve error messages for invalid constructs in PEP 695 contexts (#104573)
2023-05-17 13:05:42 +00:00
"%s cannot be used within the definition of a generic"
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
#define DUPLICATE_TYPE_PARAM \
"duplicate type parameter '%U'"
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
#define LOCATION(x) \
(x)->lineno, (x)->col_offset, (x)->end_lineno, (x)->end_col_offset
#define ST_LOCATION(x) \
(x)->ste_lineno, (x)->ste_col_offset, (x)->ste_end_lineno, (x)->ste_end_col_offset
No need to export PySTEntry_New, it is only used in symtable.c
2006-02-28 22:36:46 +00:00
static PySTEntryObject *
Merged revisions 65715,65724,65726,65732,65736-65739,65775 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r65715 | benjamin.peterson | 2008-08-16 16:04:16 -0500 (Sat, 16 Aug 2008) | 1 line add some documentation for symtable ........ r65724 | benjamin.peterson | 2008-08-16 17:11:33 -0500 (Sat, 16 Aug 2008) | 2 lines include filename and line number in SyntaxError ........ r65726 | georg.brandl | 2008-08-16 17:37:05 -0500 (Sat, 16 Aug 2008) | 2 lines Review symtable docs. ........ r65732 | benjamin.peterson | 2008-08-16 18:29:40 -0500 (Sat, 16 Aug 2008) | 1 line PySTEntry's constructor is static; there's no point in a fancy API name ........ r65736 | benjamin.peterson | 2008-08-16 20:09:17 -0500 (Sat, 16 Aug 2008) | 1 line expose PySTEntry.nested so the symtable module will work ........ r65737 | benjamin.peterson | 2008-08-16 20:17:15 -0500 (Sat, 16 Aug 2008) | 1 line a few improvements ........ r65738 | benjamin.peterson | 2008-08-16 20:27:30 -0500 (Sat, 16 Aug 2008) | 1 line fix compile errors ........ r65739 | benjamin.peterson | 2008-08-16 21:23:43 -0500 (Sat, 16 Aug 2008) | 1 line uhh PySTEntry->ste_unoptimized has to be exposed too ........ r65775 | benjamin.peterson | 2008-08-17 12:13:26 -0500 (Sun, 17 Aug 2008) | 5 lines get the symtable module back in working order - Fix broken functions - Add (hopefully) extensive tests - Modernize a little ........
2008-08-17 18:02:44 +00:00
ste_new(struct symtable *st, identifier name, _Py_block_ty block,
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
void *key, int lineno, int col_offset,
int end_lineno, int end_col_offset)
Relax the rules for using 'from ... import *' and exec in the presence of nested functions. Either is allowed in a function if it contains no defs or lambdas or the defs and lambdas it contains have no free variables. If a function is itself nested and has free variables, either is illegal. Revise the symtable to use a PySymtableEntryObject, which holds all the revelent information for a scope, rather than using a bunch of st_cur_XXX pointers in the symtable struct. The changes simplify the internal management of the current symtable scope and of the stack. Added new C source file: Python/symtable.c. (Does the Windows build process need to be updated?) As part of these changes, the initial _symtable module interface introduced in 2.1a2 is replaced. A dictionary of PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PySTEntryObject *ste = NULL;
Closed reference leak of variable 'k' in function ste_new which wasn't decrefed in error cases
2012-09-10 01:08:46 +00:00
PyObject *k = NULL;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
k = PyLong_FromVoidPtr(key);
if (k == NULL)
goto fail;
ste = PyObject_New(PySTEntryObject, &PySTEntry_Type);
Cleanup 'k' when the creation of PySTEntryObject fails. ad3824a90261 used to decref 'k' in too many error cases.
2012-09-12 15:58:10 +00:00
if (ste == NULL) {
Py_DECREF(k);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
goto fail;
Cleanup 'k' when the creation of PySTEntryObject fails. ad3824a90261 used to decref 'k' in too many error cases.
2012-09-12 15:58:10 +00:00
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
ste->ste_table = st;
Partly revert ad3824a90261 and add comment about reference ownership
2012-09-12 15:52:46 +00:00
ste->ste_id = k; /* ste owns reference to k */
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
gh-99300: Use Py_NewRef() in Python/ directory (#99302) Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and Py_XNewRef() in C files of the Python/ directory.
2022-11-10 08:03:39 +00:00
ste->ste_name = Py_NewRef(name);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
ste->ste_symbols = NULL;
ste->ste_varnames = NULL;
ste->ste_children = NULL;
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
ste->ste_directives = NULL;
gh-119311: Fix name mangling with PEP 695 generic classes (#119464) Fixes #119311. Fixes #119395.
2024-05-28 02:50:38 +00:00
ste->ste_mangled_names = NULL;
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
ste->ste_type = block;
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
ste->ste_scope_info = NULL;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
ste->ste_nested = 0;
ste->ste_free = 0;
ste->ste_varargs = 0;
ste->ste_varkeywords = 0;
ste->ste_opt_lineno = 0;
add column offset to all syntax errors
2010-09-20 23:02:10 +00:00
ste->ste_opt_col_offset = 0;
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
ste->ste_annotations_used = 0;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
ste->ste_lineno = lineno;
add column offset to all syntax errors
2010-09-20 23:02:10 +00:00
ste->ste_col_offset = col_offset;
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
ste->ste_end_lineno = end_lineno;
ste->ste_end_col_offset = end_col_offset;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (st->st_cur != NULL &&
(st->st_cur->ste_nested ||
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
_PyST_IsFunctionLike(st->st_cur)))
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
ste->ste_nested = 1;
ste->ste_child_free = 0;
ste->ste_generator = 0;
Issue #28003: Implement PEP 525 -- Asynchronous Generators.
2016-09-09 05:01:51 +00:00
ste->ste_coroutine = 0;
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
ste->ste_comprehension = NoComprehension;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
ste->ste_returns_value = 0;
hide the __class__ closure from the class body (#12370)
2013-05-15 20:26:42 +00:00
ste->ste_needs_class_closure = 0;
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
ste->ste_comp_inlined = 0;
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
ste->ste_comp_iter_target = 0;
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
ste->ste_can_see_class_scope = 0;
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
ste->ste_comp_iter_expr = 0;
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
ste->ste_needs_classdict = 0;
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
ste->ste_annotation_block = NULL;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Issue #18408: ste_new() initialize all attributes before handling error If an attribute is not initialized, the destructor can crash
2013-07-11 20:49:00 +00:00
ste->ste_symbols = PyDict_New();
ste->ste_varnames = PyList_New(0);
ste->ste_children = PyList_New(0);
if (ste->ste_symbols == NULL
|| ste->ste_varnames == NULL
|| ste->ste_children == NULL)
goto fail;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (PyDict_SetItem(st->st_blocks, ste->ste_id, (PyObject *)ste) < 0)
goto fail;
return ste;
Relax the rules for using 'from ... import *' and exec in the presence of nested functions. Either is allowed in a function if it contains no defs or lambdas or the defs and lambdas it contains have no free variables. If a function is itself nested and has free variables, either is illegal. Revise the symtable to use a PySymtableEntryObject, which holds all the revelent information for a scope, rather than using a bunch of st_cur_XXX pointers in the symtable struct. The changes simplify the internal management of the current symtable scope and of the stack. Added new C source file: Python/symtable.c. (Does the Windows build process need to be updated?) As part of these changes, the initial _symtable module interface introduced in 2.1a2 is replaced. A dictionary of PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
fail:
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Py_XDECREF(ste);
return NULL;
Relax the rules for using 'from ... import *' and exec in the presence of nested functions. Either is allowed in a function if it contains no defs or lambdas or the defs and lambdas it contains have no free variables. If a function is itself nested and has free variables, either is illegal. Revise the symtable to use a PySymtableEntryObject, which holds all the revelent information for a scope, rather than using a bunch of st_cur_XXX pointers in the symtable struct. The changes simplify the internal management of the current symtable scope and of the stack. Added new C source file: Python/symtable.c. (Does the Windows build process need to be updated?) As part of these changes, the initial _symtable module interface introduced in 2.1a2 is replaced. A dictionary of PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
}
static PyObject *
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
ste_repr(PySTEntryObject *ste)
Relax the rules for using 'from ... import *' and exec in the presence of nested functions. Either is allowed in a function if it contains no defs or lambdas or the defs and lambdas it contains have no free variables. If a function is itself nested and has free variables, either is illegal. Revise the symtable to use a PySymtableEntryObject, which holds all the revelent information for a scope, rather than using a bunch of st_cur_XXX pointers in the symtable struct. The changes simplify the internal management of the current symtable scope and of the stack. Added new C source file: Python/symtable.c. (Does the Windows build process need to be updated?) As part of these changes, the initial _symtable module interface introduced in 2.1a2 is replaced. A dictionary of PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
{
gh-109207: Fix SystemError when printing symtable entry object. (GH-109225)
2023-09-10 12:04:24 +00:00
return PyUnicode_FromFormat("<symtable entry %U(%R), line %d>",
ste->ste_name, ste->ste_id, ste->ste_lineno);
Relax the rules for using 'from ... import *' and exec in the presence of nested functions. Either is allowed in a function if it contains no defs or lambdas or the defs and lambdas it contains have no free variables. If a function is itself nested and has free variables, either is illegal. Revise the symtable to use a PySymtableEntryObject, which holds all the revelent information for a scope, rather than using a bunch of st_cur_XXX pointers in the symtable struct. The changes simplify the internal management of the current symtable scope and of the stack. Added new C source file: Python/symtable.c. (Does the Windows build process need to be updated?) As part of these changes, the initial _symtable module interface introduced in 2.1a2 is replaced. A dictionary of PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
}
static void
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
ste_dealloc(PySTEntryObject *ste)
Relax the rules for using 'from ... import *' and exec in the presence of nested functions. Either is allowed in a function if it contains no defs or lambdas or the defs and lambdas it contains have no free variables. If a function is itself nested and has free variables, either is illegal. Revise the symtable to use a PySymtableEntryObject, which holds all the revelent information for a scope, rather than using a bunch of st_cur_XXX pointers in the symtable struct. The changes simplify the internal management of the current symtable scope and of the stack. Added new C source file: Python/symtable.c. (Does the Windows build process need to be updated?) As part of these changes, the initial _symtable module interface introduced in 2.1a2 is replaced. A dictionary of PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
ste->ste_table = NULL;
Py_XDECREF(ste->ste_id);
Py_XDECREF(ste->ste_name);
Py_XDECREF(ste->ste_symbols);
Py_XDECREF(ste->ste_varnames);
Py_XDECREF(ste->ste_children);
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
Py_XDECREF(ste->ste_directives);
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
Py_XDECREF(ste->ste_annotation_block);
gh-119311: Fix name mangling with PEP 695 generic classes (#119464) Fixes #119311. Fixes #119395.
2024-05-28 02:50:38 +00:00
Py_XDECREF(ste->ste_mangled_names);
bpo-42519: Replace PyObject_MALLOC() with PyObject_Malloc() (GH-23587) No longer use deprecated aliases to functions: * Replace PyObject_MALLOC() with PyObject_Malloc() * Replace PyObject_REALLOC() with PyObject_Realloc() * Replace PyObject_FREE() with PyObject_Free() * Replace PyObject_Del() with PyObject_Free() * Replace PyObject_DEL() with PyObject_Free()
2020-12-01 09:37:39 +00:00
PyObject_Free(ste);
Relax the rules for using 'from ... import *' and exec in the presence of nested functions. Either is allowed in a function if it contains no defs or lambdas or the defs and lambdas it contains have no free variables. If a function is itself nested and has free variables, either is illegal. Revise the symtable to use a PySymtableEntryObject, which holds all the revelent information for a scope, rather than using a bunch of st_cur_XXX pointers in the symtable struct. The changes simplify the internal management of the current symtable scope and of the stack. Added new C source file: Python/symtable.c. (Does the Windows build process need to be updated?) As part of these changes, the initial _symtable module interface introduced in 2.1a2 is replaced. A dictionary of PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
}
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
#define OFF(x) offsetof(PySTEntryObject, x)
Relax the rules for using 'from ... import *' and exec in the presence of nested functions. Either is allowed in a function if it contains no defs or lambdas or the defs and lambdas it contains have no free variables. If a function is itself nested and has free variables, either is illegal. Revise the symtable to use a PySymtableEntryObject, which holds all the revelent information for a scope, rather than using a bunch of st_cur_XXX pointers in the symtable struct. The changes simplify the internal management of the current symtable scope and of the stack. Added new C source file: Python/symtable.c. (Does the Windows build process need to be updated?) As part of these changes, the initial _symtable module interface introduced in 2.1a2 is replaced. A dictionary of PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
Add optional docstrings to member descriptors. For backwards compatibility, this required all places where an array of "struct memberlist" structures was declared that is referenced from a type's tp_members slot to change the type of the structure to PyMemberDef; "struct memberlist" is now only used by old code that still calls PyMember_Get/Set. The code in PyObject_GenericGetAttr/SetAttr now calls the new APIs PyMember_GetOne/SetOne, which take a PyMemberDef argument. As examples, I added actual docstrings to the attributes of a few types: file, complex, instance method, super, and xxsubtype.spamlist. Also converted the symtable to new style getattr.
2001-09-20 20:46:19 +00:00
static PyMemberDef ste_memberlist[] = {
gh-106869: Use new PyMemberDef constant names (#106871) * Remove '#include "structmember.h"'. * If needed, add <stddef.h> to get offsetof() function. * Update Parser/asdl_c.py to regenerate Python/Python-ast.c. * Replace: * T_SHORT => Py_T_SHORT * T_INT => Py_T_INT * T_LONG => Py_T_LONG * T_FLOAT => Py_T_FLOAT * T_DOUBLE => Py_T_DOUBLE * T_STRING => Py_T_STRING * T_OBJECT => _Py_T_OBJECT * T_CHAR => Py_T_CHAR * T_BYTE => Py_T_BYTE * T_UBYTE => Py_T_UBYTE * T_USHORT => Py_T_USHORT * T_UINT => Py_T_UINT * T_ULONG => Py_T_ULONG * T_STRING_INPLACE => Py_T_STRING_INPLACE * T_BOOL => Py_T_BOOL * T_OBJECT_EX => Py_T_OBJECT_EX * T_LONGLONG => Py_T_LONGLONG * T_ULONGLONG => Py_T_ULONGLONG * T_PYSSIZET => Py_T_PYSSIZET * T_NONE => _Py_T_NONE * READONLY => Py_READONLY * PY_AUDIT_READ => Py_AUDIT_READ * READ_RESTRICTED => Py_AUDIT_READ * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
2023-07-25 13:28:30 +00:00
{"id", _Py_T_OBJECT, OFF(ste_id), Py_READONLY},
{"name", _Py_T_OBJECT, OFF(ste_name), Py_READONLY},
{"symbols", _Py_T_OBJECT, OFF(ste_symbols), Py_READONLY},
{"varnames", _Py_T_OBJECT, OFF(ste_varnames), Py_READONLY},
{"children", _Py_T_OBJECT, OFF(ste_children), Py_READONLY},
{"nested", Py_T_INT, OFF(ste_nested), Py_READONLY},
{"type", Py_T_INT, OFF(ste_type), Py_READONLY},
{"lineno", Py_T_INT, OFF(ste_lineno), Py_READONLY},
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
{NULL}
Relax the rules for using 'from ... import *' and exec in the presence of nested functions. Either is allowed in a function if it contains no defs or lambdas or the defs and lambdas it contains have no free variables. If a function is itself nested and has free variables, either is illegal. Revise the symtable to use a PySymtableEntryObject, which holds all the revelent information for a scope, rather than using a bunch of st_cur_XXX pointers in the symtable struct. The changes simplify the internal management of the current symtable scope and of the stack. Added new C source file: Python/symtable.c. (Does the Windows build process need to be updated?) As part of these changes, the initial _symtable module interface introduced in 2.1a2 is replaced. A dictionary of PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
};
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
PyTypeObject PySTEntry_Type = {
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PyVarObject_HEAD_INIT(&PyType_Type, 0)
"symtable entry",
sizeof(PySTEntryObject),
0,
(destructor)ste_dealloc, /* tp_dealloc */
bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async (GH-13464) Automatically replace tp_print -> tp_vectorcall_offset tp_compare -> tp_as_async tp_reserved -> tp_as_async
2019-05-31 02:13:39 +00:00
0, /* tp_vectorcall_offset */
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
0, /* tp_getattr */
0, /* tp_setattr */
bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async (GH-13464) Automatically replace tp_print -> tp_vectorcall_offset tp_compare -> tp_as_async tp_reserved -> tp_as_async
2019-05-31 02:13:39 +00:00
0, /* tp_as_async */
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
(reprfunc)ste_repr, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
PyObject_GenericGetAttr, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT, /* tp_flags */
0, /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
ste_memberlist, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
0, /* tp_init */
0, /* tp_alloc */
0, /* tp_new */
Relax the rules for using 'from ... import *' and exec in the presence of nested functions. Either is allowed in a function if it contains no defs or lambdas or the defs and lambdas it contains have no free variables. If a function is itself nested and has free variables, either is illegal. Revise the symtable to use a PySymtableEntryObject, which holds all the revelent information for a scope, rather than using a bunch of st_cur_XXX pointers in the symtable struct. The changes simplify the internal management of the current symtable scope and of the stack. Added new C source file: Python/symtable.c. (Does the Windows build process need to be updated?) As part of these changes, the initial _symtable module interface introduced in 2.1a2 is replaced. A dictionary of PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
};
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
static int symtable_analyze(struct symtable *st);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
static int symtable_enter_block(struct symtable *st, identifier name,
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
_Py_block_ty block, void *ast,
int lineno, int col_offset,
int end_lineno, int end_col_offset);
closes bpo-39872: Remove unused args from symtable_exit_block and symtable_visit_annotations. (GH-18800)
2020-03-06 15:46:04 +00:00
static int symtable_exit_block(struct symtable *st);
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
static int symtable_visit_stmt(struct symtable *st, stmt_ty s);
static int symtable_visit_expr(struct symtable *st, expr_ty s);
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
static int symtable_visit_type_param(struct symtable *st, type_param_ty s);
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
static int symtable_visit_genexp(struct symtable *st, expr_ty s);
Hide list comp variables and support set comprehensions
2007-04-15 12:05:43 +00:00
static int symtable_visit_listcomp(struct symtable *st, expr_ty s);
static int symtable_visit_setcomp(struct symtable *st, expr_ty s);
Merged revisions 56154-56264 via svnmerge from svn+ssh://pythondev@svn.python.org/python/branches/p3yk ................ r56155 | neal.norwitz | 2007-07-03 08:59:08 +0300 (Tue, 03 Jul 2007) | 1 line Get this test working after converting map to return an iterator ................ r56202 | neal.norwitz | 2007-07-09 04:30:09 +0300 (Mon, 09 Jul 2007) | 37 lines Merged revisions 56124-56201 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r56129 | georg.brandl | 2007-06-30 04:01:01 -0700 (Sat, 30 Jun 2007) | 2 lines Document smtp.SMTPAuthenticationError. ........ r56137 | georg.brandl | 2007-07-01 01:11:35 -0700 (Sun, 01 Jul 2007) | 2 lines Fix a few webbrowser.py problems. ........ r56143 | georg.brandl | 2007-07-02 04:54:28 -0700 (Mon, 02 Jul 2007) | 2 lines Remove duplicate sentence from alarm() doc. ........ r56170 | mark.hammond | 2007-07-03 19:03:10 -0700 (Tue, 03 Jul 2007) | 3 lines copy built files to the PCBuild directory, where tools like distutils or external build processes can find them. ........ r56176 | kurt.kaiser | 2007-07-05 15:03:39 -0700 (Thu, 05 Jul 2007) | 10 lines Many calls to tk.call involve an arglist containing a single tuple. Calls using METH_OLDARGS unpack this tuple; calls using METH_VARARG don't. Tcl's concatenation of args was affected; IDLE doesn't start. Modify Tkapp_Call() to unpack single tuple arglists. Bug 1733943 Ref http://mail.python.org/pipermail/python-checkins/2007-May/060454.html ........ r56177 | neal.norwitz | 2007-07-05 21:13:39 -0700 (Thu, 05 Jul 2007) | 1 line Fix typo in comment ........ ................ r56251 | neal.norwitz | 2007-07-11 10:01:01 +0300 (Wed, 11 Jul 2007) | 1 line Get working with map returning an iterator (had to fix whitespace too) ................ r56255 | thomas.wouters | 2007-07-11 13:41:37 +0300 (Wed, 11 Jul 2007) | 6 lines Clean up merge glitch or copy-paste error (the entire module was duplicated, except the first half even had some more copy-paste errors, referring to listcomps and genexps instead of setcomps) ................ r56256 | thomas.wouters | 2007-07-11 15:16:01 +0300 (Wed, 11 Jul 2007) | 14 lines Dict comprehensions. Still needs doc changes (like many python-3000 features ;-). It generates bytecode similar to: x = {} for k, v in (generator here): x[k] = v except there is no tuple-packing and -unpacking involved. Trivial measurement suggests it's significantly faster than dict(generator here) (in the order of 2 to 3 times as fast) but I have not done extensive measurements. ................ r56263 | guido.van.rossum | 2007-07-11 15:36:26 +0300 (Wed, 11 Jul 2007) | 3 lines Patch 1724999 by Ali Gholami Rudi -- avoid complaints about dict size change during iter in destroy call. ................
2007-07-11 13:09:30 +00:00
static int symtable_visit_dictcomp(struct symtable *st, expr_ty s);
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
static int symtable_visit_arguments(struct symtable *st, arguments_ty);
static int symtable_visit_excepthandler(struct symtable *st, excepthandler_ty);
static int symtable_visit_alias(struct symtable *st, alias_ty);
static int symtable_visit_comprehension(struct symtable *st, comprehension_ty);
static int symtable_visit_keyword(struct symtable *st, keyword_ty);
bpo-41746: Add type information to asdl_seq objects (GH-22223) * Add new capability to the PEG parser to type variable assignments. For instance: ``` | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a } ``` * Add new sequence types from the asdl definition (automatically generated) * Make `asdl_seq` type a generic aliasing pointer type. * Create a new `asdl_generic_seq` for the generic case using `void*`. * The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed. * New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences. * Changes all possible `asdl_seq` types to use specific versions everywhere.
2020-09-16 18:42:00 +00:00
static int symtable_visit_params(struct symtable *st, asdl_arg_seq *args);
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
static int symtable_visit_annotation(struct symtable *st, expr_ty annotation, void *key);
bpo-41746: Add type information to asdl_seq objects (GH-22223) * Add new capability to the PEG parser to type variable assignments. For instance: ``` | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a } ``` * Add new sequence types from the asdl definition (automatically generated) * Make `asdl_seq` type a generic aliasing pointer type. * Create a new `asdl_generic_seq` for the generic case using `void*`. * The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed. * New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences. * Changes all possible `asdl_seq` types to use specific versions everywhere.
2020-09-16 18:42:00 +00:00
static int symtable_visit_argannotations(struct symtable *st, asdl_arg_seq *args);
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
static int symtable_implicit_arg(struct symtable *st, int pos);
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
static int symtable_visit_annotations(struct symtable *st, stmt_ty, arguments_ty, expr_ty,
struct _symtable_entry *parent_ste);
reflect with statements with multiple items in the AST (closes #12106)
2011-05-27 18:58:08 +00:00
static int symtable_visit_withitem(struct symtable *st, withitem_ty item);
bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917) Co-authored-by: Guido van Rossum <guido@python.org> Co-authored-by: Talin <viridia@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-02-26 22:51:55 +00:00
static int symtable_visit_match_case(struct symtable *st, match_case_ty m);
bpo-43892: Make match patterns explicit in the AST (GH-25585) Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2021-04-29 05:58:44 +00:00
static int symtable_visit_pattern(struct symtable *st, pattern_ty s);
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
static int symtable_raise_if_annotation_block(struct symtable *st, const char *, expr_ty);
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
static int symtable_raise_if_comprehension_block(struct symtable *st, expr_ty);
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
gh-109390: add dump_symtable utility under #if 0 (#109391) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-09-20 16:55:56 +00:00
/* For debugging purposes only */
#if _PY_DUMP_SYMTABLE
static void _dump_symtable(PySTEntryObject* ste, PyObject* prefix)
{
const char *blocktype = "";
switch (ste->ste_type) {
case FunctionBlock: blocktype = "FunctionBlock"; break;
case ClassBlock: blocktype = "ClassBlock"; break;
case ModuleBlock: blocktype = "ModuleBlock"; break;
case AnnotationBlock: blocktype = "AnnotationBlock"; break;
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
case TypeVariableBlock: blocktype = "TypeVariableBlock"; break;
gh-109390: add dump_symtable utility under #if 0 (#109391) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-09-20 16:55:56 +00:00
case TypeAliasBlock: blocktype = "TypeAliasBlock"; break;
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
case TypeParametersBlock: blocktype = "TypeParametersBlock"; break;
gh-109390: add dump_symtable utility under #if 0 (#109391) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-09-20 16:55:56 +00:00
}
const char *comptype = "";
switch (ste->ste_comprehension) {
case ListComprehension: comptype = " ListComprehension"; break;
case DictComprehension: comptype = " DictComprehension"; break;
case SetComprehension: comptype = " SetComprehension"; break;
case GeneratorExpression: comptype = " GeneratorExpression"; break;
case NoComprehension: break;
}
PyObject* msg = PyUnicode_FromFormat(
(
"%U=== Symtable for %U ===\n"
"%U%s%s\n"
"%U%s%s%s%s%s%s%s%s%s%s%s%s%s\n"
"%Ulineno: %d col_offset: %d\n"
"%U--- Symbols ---\n"
),
prefix,
ste->ste_name,
prefix,
blocktype,
comptype,
prefix,
ste->ste_nested ? " nested" : "",
ste->ste_free ? " free" : "",
ste->ste_child_free ? " child_free" : "",
ste->ste_generator ? " generator" : "",
ste->ste_coroutine ? " coroutine" : "",
ste->ste_varargs ? " varargs" : "",
ste->ste_varkeywords ? " varkeywords" : "",
ste->ste_returns_value ? " returns_value" : "",
ste->ste_needs_class_closure ? " needs_class_closure" : "",
ste->ste_needs_classdict ? " needs_classdict" : "",
ste->ste_comp_inlined ? " comp_inlined" : "",
ste->ste_comp_iter_target ? " comp_iter_target" : "",
ste->ste_can_see_class_scope ? " can_see_class_scope" : "",
prefix,
ste->ste_lineno,
ste->ste_col_offset,
prefix
);
assert(msg != NULL);
printf("%s", PyUnicode_AsUTF8(msg));
Py_DECREF(msg);
PyObject *name, *value;
Py_ssize_t pos = 0;
while (PyDict_Next(ste->ste_symbols, &pos, &name, &value)) {
int scope = _PyST_GetScope(ste, name);
long flags = _PyST_GetSymbol(ste, name);
printf("%s %s: ", PyUnicode_AsUTF8(prefix), PyUnicode_AsUTF8(name));
if (flags & DEF_GLOBAL) printf(" DEF_GLOBAL");
if (flags & DEF_LOCAL) printf(" DEF_LOCAL");
if (flags & DEF_PARAM) printf(" DEF_PARAM");
if (flags & DEF_NONLOCAL) printf(" DEF_NONLOCAL");
if (flags & USE) printf(" USE");
if (flags & DEF_FREE_CLASS) printf(" DEF_FREE_CLASS");
if (flags & DEF_IMPORT) printf(" DEF_IMPORT");
if (flags & DEF_ANNOT) printf(" DEF_ANNOT");
if (flags & DEF_COMP_ITER) printf(" DEF_COMP_ITER");
if (flags & DEF_TYPE_PARAM) printf(" DEF_TYPE_PARAM");
if (flags & DEF_COMP_CELL) printf(" DEF_COMP_CELL");
switch (scope) {
case LOCAL: printf(" LOCAL"); break;
case GLOBAL_EXPLICIT: printf(" GLOBAL_EXPLICIT"); break;
case GLOBAL_IMPLICIT: printf(" GLOBAL_IMPLICIT"); break;
case FREE: printf(" FREE"); break;
case CELL: printf(" CELL"); break;
}
printf("\n");
}
printf("%s--- Children ---\n", PyUnicode_AsUTF8(prefix));
PyObject *new_prefix = PyUnicode_FromFormat(" %U", prefix);
assert(new_prefix != NULL);
for (Py_ssize_t i = 0; i < PyList_GET_SIZE(ste->ste_children); i++) {
PyObject *child = PyList_GetItem(ste->ste_children, i);
assert(child != NULL && PySTEntry_Check(child));
_dump_symtable((PySTEntryObject *)child, new_prefix);
}
Py_DECREF(new_prefix);
}
static void dump_symtable(PySTEntryObject* ste)
{
PyObject *empty = PyUnicode_FromString("");
assert(empty != NULL);
_dump_symtable(ste, empty);
Py_DECREF(empty);
}
#endif
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
#define DUPLICATE_ARGUMENT \
Use unicode for error messages of duplicate args
2007-08-24 22:53:58 +00:00
"duplicate argument '%U' in function definition"
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
static struct symtable *
symtable_new(void)
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
struct symtable *st;
st = (struct symtable *)PyMem_Malloc(sizeof(struct symtable));
Add a missed PyErr_NoMemory() in symtable_new(). (GH-10576) This missed PyErr_NoMemory() could cause a SystemError when calling _symtable.symtable().
2018-11-16 15:58:55 +00:00
if (st == NULL) {
PyErr_NoMemory();
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return NULL;
Add a missed PyErr_NoMemory() in symtable_new(). (GH-10576) This missed PyErr_NoMemory() could cause a SystemError when calling _symtable.symtable().
2018-11-16 15:58:55 +00:00
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
st->st_filename = NULL;
st->st_blocks = NULL;
if ((st->st_stack = PyList_New(0)) == NULL)
goto fail;
if ((st->st_blocks = PyDict_New()) == NULL)
goto fail;
st->st_cur = NULL;
st->st_private = NULL;
return st;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
fail:
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
_PySymtable_Free(st);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return NULL;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
struct symtable *
gh-117411: move PyFutureFeatures to pycore_symtable.h and make it private (#117412)
2024-04-02 10:34:49 +00:00
_PySymtable_Build(mod_ty mod, PyObject *filename, _PyFutureFeatures *future)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
Close #14857: fix regression in references to PEP 3135 implicit __class__ closure variable. Reopens issue #12370, but also updates unittest.mock to workaround that issue
2012-05-27 08:17:07 +00:00
struct symtable *st = symtable_new();
bpo-41746: Add type information to asdl_seq objects (GH-22223) * Add new capability to the PEG parser to type variable assignments. For instance: ``` | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a } ``` * Add new sequence types from the asdl definition (automatically generated) * Make `asdl_seq` type a generic aliasing pointer type. * Create a new `asdl_generic_seq` for the generic case using `void*`. * The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed. * New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences. * Changes all possible `asdl_seq` types to use specific versions everywhere.
2020-09-16 18:42:00 +00:00
asdl_stmt_seq *seq;
Fixes loop variables to be the same types as their limit (GH-120958)
2024-06-24 16:11:47 +00:00
Py_ssize_t i;
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
PyThreadState *tstate;
bpo-37947: Avoid double-decrement in symtable recursion counting (GH-15593) With `symtable_visit_expr` now correctly adjusting the recursion depth for named expressions, `symtable_handle_namedexpr` should be leaving it alone. Also adds a new check to `PySymtable_BuildObject` that raises `SystemError` if a successful first symbol analysis pass fails to keep the stack depth accounting clean.
2019-08-29 13:26:53 +00:00
int starting_recursion_depth;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (st == NULL)
Close #11619: The parser and the import machinery do not encode Unicode filenames anymore on Windows.
2013-08-26 20:28:21 +00:00
return NULL;
if (filename == NULL) {
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
_PySymtable_Free(st);
Close #11619: The parser and the import machinery do not encode Unicode filenames anymore on Windows.
2013-08-26 20:28:21 +00:00
return NULL;
}
gh-99300: Use Py_NewRef() in Python/ directory (#99302) Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and Py_XNewRef() in C files of the Python/ directory.
2022-11-10 08:03:39 +00:00
st->st_filename = Py_NewRef(filename);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
st->st_future = future;
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
/* Setup recursion depth check counters */
bpo-35081: Add _PyThreadState_GET() internal macro (GH-10266) If Py_BUILD_CORE is defined, the PyThreadState_GET() macro access _PyRuntime which comes from the internal pycore_state.h header. Public headers must not require internal headers. Move PyThreadState_GET() and _PyInterpreterState_GET_UNSAFE() from Include/pystate.h to Include/internal/pycore_state.h, and rename PyThreadState_GET() to _PyThreadState_GET() there. The PyThreadState_GET() macro of pystate.h is now redefined when pycore_state.h is included, to use the fast _PyThreadState_GET(). Changes: * Add _PyThreadState_GET() macro * Replace "PyThreadState_GET()->interp" with _PyInterpreterState_GET_UNSAFE() * Replace PyThreadState_GET() with _PyThreadState_GET() in internal C files (compiled with Py_BUILD_CORE defined), but keep PyThreadState_GET() in the public header files. * _testcapimodule.c: replace PyThreadState_GET() with PyThreadState_Get(); the module is not compiled with Py_BUILD_CORE defined. * pycore_state.h now requires Py_BUILD_CORE to be defined.
2018-11-01 00:51:40 +00:00
tstate = _PyThreadState_GET();
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
if (!tstate) {
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
_PySymtable_Free(st);
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
return NULL;
}
don't scale compiler stack frames if the recursion limit is huge (closes #19098)
2013-09-27 02:17:45 +00:00
/* Be careful here to prevent overflow. */
GH-91079: Rename C_RECURSION_LIMIT to Py_C_RECURSION_LIMIT (#108507) Symbols of the C API should be prefixed by "Py_" to avoid conflict with existing names in 3rd party C extensions on "#include <Python.h>". test.pythoninfo now logs Py_C_RECURSION_LIMIT constant and other _testcapi and _testinternalcapi constants.
2023-09-08 09:48:28 +00:00
int recursion_depth = Py_C_RECURSION_LIMIT - tstate->c_recursion_remaining;
GH-113655: Lower the C recursion limit on various platforms (GH-113944)
2024-01-16 09:32:01 +00:00
starting_recursion_depth = recursion_depth;
bpo-37947: Avoid double-decrement in symtable recursion counting (GH-15593) With `symtable_visit_expr` now correctly adjusting the recursion depth for named expressions, `symtable_handle_namedexpr` should be leaving it alone. Also adds a new check to `PySymtable_BuildObject` that raises `SystemError` if a successful first symbol analysis pass fails to keep the stack depth accounting clean.
2019-08-29 13:26:53 +00:00
st->recursion_depth = starting_recursion_depth;
GH-113655: Lower the C recursion limit on various platforms (GH-113944)
2024-01-16 09:32:01 +00:00
st->recursion_limit = Py_C_RECURSION_LIMIT;
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* Make the initial symbol information gathering pass */
bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366) https://bugs.python.org/issue46765
2022-02-23 00:23:51 +00:00
if (!symtable_enter_block(st, &_Py_ID(top), ModuleBlock, (void *)mod, 0, 0, 0, 0)) {
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
_PySymtable_Free(st);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return NULL;
}
st->st_top = st->st_cur;
switch (mod->kind) {
case Module_kind:
seq = mod->v.Module.body;
for (i = 0; i < asdl_seq_LEN(seq); i++)
if (!symtable_visit_stmt(st,
(stmt_ty)asdl_seq_GET(seq, i)))
goto error;
break;
case Expression_kind:
if (!symtable_visit_expr(st, mod->v.Expression.body))
goto error;
break;
case Interactive_kind:
seq = mod->v.Interactive.body;
for (i = 0; i < asdl_seq_LEN(seq); i++)
if (!symtable_visit_stmt(st,
(stmt_ty)asdl_seq_GET(seq, i)))
goto error;
break;
Complete switch cases in symtable.c; fixes bpo-35963 (GH-11821)
2019-02-11 19:34:50 +00:00
case FunctionType_kind:
PyErr_SetString(PyExc_RuntimeError,
"this compiler does not handle FunctionTypes");
goto error;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
closes bpo-39872: Remove unused args from symtable_exit_block and symtable_visit_annotations. (GH-18800)
2020-03-06 15:46:04 +00:00
if (!symtable_exit_block(st)) {
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
_PySymtable_Free(st);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return NULL;
}
bpo-37947: Avoid double-decrement in symtable recursion counting (GH-15593) With `symtable_visit_expr` now correctly adjusting the recursion depth for named expressions, `symtable_handle_namedexpr` should be leaving it alone. Also adds a new check to `PySymtable_BuildObject` that raises `SystemError` if a successful first symbol analysis pass fails to keep the stack depth accounting clean.
2019-08-29 13:26:53 +00:00
/* Check that the recursion depth counting balanced correctly */
if (st->recursion_depth != starting_recursion_depth) {
PyErr_Format(PyExc_SystemError,
"symtable analysis recursion depth mismatch (before=%d, after=%d)",
starting_recursion_depth, st->recursion_depth);
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
_PySymtable_Free(st);
bpo-37947: Avoid double-decrement in symtable recursion counting (GH-15593) With `symtable_visit_expr` now correctly adjusting the recursion depth for named expressions, `symtable_handle_namedexpr` should be leaving it alone. Also adds a new check to `PySymtable_BuildObject` that raises `SystemError` if a successful first symbol analysis pass fails to keep the stack depth accounting clean.
2019-08-29 13:26:53 +00:00
return NULL;
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* Make the second symbol analysis pass */
gh-109390: add dump_symtable utility under #if 0 (#109391) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-09-20 16:55:56 +00:00
if (symtable_analyze(st)) {
#if _PY_DUMP_SYMTABLE
dump_symtable(st->st_top);
#endif
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return st;
gh-109390: add dump_symtable utility under #if 0 (#109391) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-09-20 16:55:56 +00:00
}
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
_PySymtable_Free(st);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return NULL;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
error:
closes bpo-39872: Remove unused args from symtable_exit_block and symtable_visit_annotations. (GH-18800)
2020-03-06 15:46:04 +00:00
(void) symtable_exit_block(st);
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
_PySymtable_Free(st);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return NULL;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
Close #11619: The parser and the import machinery do not encode Unicode filenames anymore on Windows.
2013-08-26 20:28:21 +00:00
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
void
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
_PySymtable_Free(struct symtable *st)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
Close #11619: The parser and the import machinery do not encode Unicode filenames anymore on Windows.
2013-08-26 20:28:21 +00:00
Py_XDECREF(st->st_filename);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Py_XDECREF(st->st_blocks);
Py_XDECREF(st->st_stack);
PyMem_Free((void *)st);
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
PySTEntryObject *
gh-107211: Rename PySymtable_Lookup() to _PySymtable_Lookup() (#107212) Rename the internal PySymtable_Lookup() function to _PySymtable_Lookup() and no longer export it.
2023-07-25 00:54:09 +00:00
_PySymtable_Lookup(struct symtable *st, void *key)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PyObject *k, *v;
k = PyLong_FromVoidPtr(key);
if (k == NULL)
return NULL;
gh-111789: Use PyDict_GetItemRef() in Python/symtable.c (gh-112084)
2023-11-27 17:55:30 +00:00
if (PyDict_GetItemRef(st->st_blocks, k, &v) == 0) {
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PyErr_SetString(PyExc_KeyError,
"unknown symbol table entry");
}
gh-111789: Use PyDict_GetItemRef() in Python/symtable.c (gh-112084)
2023-11-27 17:55:30 +00:00
Py_DECREF(k);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
gh-111789: Use PyDict_GetItemRef() in Python/symtable.c (gh-112084)
2023-11-27 17:55:30 +00:00
assert(v == NULL || PySTEntry_Check(v));
return (PySTEntryObject *)v;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
int
_PySymtable_LookupOptional(struct symtable *st, void *key,
PySTEntryObject **out)
{
PyObject *k = PyLong_FromVoidPtr(key);
if (k == NULL) {
*out = NULL;
return -1;
}
int result = PyDict_GetItemRef(st->st_blocks, k, (PyObject **)out);
Py_DECREF(k);
assert(*out == NULL || PySTEntry_Check(*out));
return result;
}
bpo-44313: generate LOAD_ATTR/CALL_FUNCTION for top-level imported objects (GH-26677)
2021-06-30 22:53:36 +00:00
long
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
_PyST_GetSymbol(PySTEntryObject *ste, PyObject *name)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
bpo-42006: Stop using PyDict_GetItem, PyDict_GetItemString and _PyDict_GetItemId. (GH-22648) These functions are considered not safe because they suppress all internal errors and can return wrong result. PyDict_GetItemString and _PyDict_GetItemId can also silence current exception in rare cases. Remove no longer used _PyDict_GetItemId. Add _PyDict_ContainsId and rename _PyDict_Contains into _PyDict_Contains_KnownHash.
2020-10-26 06:43:39 +00:00
PyObject *v = PyDict_GetItemWithError(ste->ste_symbols, name);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (!v)
return 0;
assert(PyLong_Check(v));
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
return PyLong_AS_LONG(v);
}
int
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
_PyST_GetScope(PySTEntryObject *ste, PyObject *name)
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
{
long symbol = _PyST_GetSymbol(ste, name);
return (symbol >> SCOPE_OFFSET) & SCOPE_MASK;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
int
_PyST_IsFunctionLike(PySTEntryObject *ste)
{
return ste->ste_type == FunctionBlock
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
|| ste->ste_type == AnnotationBlock
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
|| ste->ste_type == TypeVariableBlock
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
|| ste->ste_type == TypeAliasBlock
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
|| ste->ste_type == TypeParametersBlock;
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
}
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
static int
error_at_directive(PySTEntryObject *ste, PyObject *name)
{
Py_ssize_t i;
PyObject *data;
assert(ste->ste_directives);
make recording and reporting errors and nonlocal and global directives more robust (closes #25973)
2015-12-29 16:08:34 +00:00
for (i = 0; i < PyList_GET_SIZE(ste->ste_directives); i++) {
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
data = PyList_GET_ITEM(ste->ste_directives, i);
assert(PyTuple_CheckExact(data));
make recording and reporting errors and nonlocal and global directives more robust (closes #25973)
2015-12-29 16:08:34 +00:00
assert(PyUnicode_CheckExact(PyTuple_GET_ITEM(data, 0)));
if (PyUnicode_Compare(PyTuple_GET_ITEM(data, 0), name) == 0) {
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
PyErr_RangedSyntaxLocationObject(ste->ste_table->st_filename,
PyLong_AsLong(PyTuple_GET_ITEM(data, 1)),
PyLong_AsLong(PyTuple_GET_ITEM(data, 2)) + 1,
PyLong_AsLong(PyTuple_GET_ITEM(data, 3)),
PyLong_AsLong(PyTuple_GET_ITEM(data, 4)) + 1);
make recording and reporting errors and nonlocal and global directives more robust (closes #25973)
2015-12-29 16:08:34 +00:00
return 0;
Issue #28003: Implement PEP 525 -- Asynchronous Generators.
2016-09-09 05:01:51 +00:00
}
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
}
make recording and reporting errors and nonlocal and global directives more robust (closes #25973)
2015-12-29 16:08:34 +00:00
PyErr_SetString(PyExc_RuntimeError,
"BUG: internal directive bookkeeping broken");
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
return 0;
}
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
/* Analyze raw symbol information to determine scope of each name.
Merged revisions 55817-55961 via svnmerge from svn+ssh://pythondev@svn.python.org/python/branches/p3yk ................ r55837 | guido.van.rossum | 2007-06-08 16:04:42 -0700 (Fri, 08 Jun 2007) | 2 lines PEP 3119 -- the abc module. ................ r55838 | guido.van.rossum | 2007-06-08 17:38:55 -0700 (Fri, 08 Jun 2007) | 2 lines Implement part of PEP 3119 -- One Trick Ponies. ................ r55847 | guido.van.rossum | 2007-06-09 08:28:06 -0700 (Sat, 09 Jun 2007) | 2 lines Different way to do one trick ponies, allowing registration (per PEP strawman). ................ r55849 | guido.van.rossum | 2007-06-09 18:06:38 -0700 (Sat, 09 Jun 2007) | 3 lines Make sure that the magic looking for __hash__ (etc.) doesn't apply to real subclasses of Hashable. ................ r55852 | guido.van.rossum | 2007-06-10 08:29:51 -0700 (Sun, 10 Jun 2007) | 2 lines Add some more examples, e.g. generators and dict views. ................ r55853 | guido.van.rossum | 2007-06-10 08:31:59 -0700 (Sun, 10 Jun 2007) | 2 lines keys() and items() *are* containers -- just values() isn't. ................ r55864 | georg.brandl | 2007-06-10 15:29:40 -0700 (Sun, 10 Jun 2007) | 2 lines PEP 3127: new octal literals, binary literals. ................ r55865 | georg.brandl | 2007-06-10 15:31:37 -0700 (Sun, 10 Jun 2007) | 2 lines Some octal literal fixes in Tools. ................ r55866 | georg.brandl | 2007-06-10 15:37:43 -0700 (Sun, 10 Jun 2007) | 2 lines Tokenizer changes for PEP 3127. ................ r55867 | georg.brandl | 2007-06-10 15:37:55 -0700 (Sun, 10 Jun 2007) | 2 lines Some docs for PEP 3127. ................ r55868 | georg.brandl | 2007-06-10 15:44:39 -0700 (Sun, 10 Jun 2007) | 2 lines Missed a place in intobject.c. Is that used anymore anyway? ................ r55871 | neal.norwitz | 2007-06-10 18:31:49 -0700 (Sun, 10 Jun 2007) | 182 lines Merged revisions 55729-55868 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r55731 | neal.norwitz | 2007-06-01 00:29:12 -0700 (Fri, 01 Jun 2007) | 7 lines SF 1668596/1720897: distutils now copies data files even if package_dir is empty. This needs to be backported. I'm too tired tonight. It would be great if someone backports this if the buildbots are ok with it. Otherwise, I will try to get to it tomorrow. ........ r55732 | georg.brandl | 2007-06-01 04:33:33 -0700 (Fri, 01 Jun 2007) | 2 lines Bug #1722484: remove docstrings again when running with -OO. ........ r55735 | georg.brandl | 2007-06-01 12:20:27 -0700 (Fri, 01 Jun 2007) | 2 lines Fix wrong issue number. ........ r55739 | brett.cannon | 2007-06-01 20:02:29 -0700 (Fri, 01 Jun 2007) | 3 lines Have configure raise an error when building on AtheOS. Code specific to AtheOS will be removed in Python 2.7. ........ r55746 | neal.norwitz | 2007-06-02 11:33:53 -0700 (Sat, 02 Jun 2007) | 1 line Update expected birthday of 2.6 ........ r55751 | neal.norwitz | 2007-06-03 13:32:50 -0700 (Sun, 03 Jun 2007) | 10 lines Backout the original 'fix' to 1721309 which had no effect. Different versions of Berkeley DB handle this differently. The comments and bug report should have the details. Memory is allocated in 4.4 (and presumably earlier), but not in 4.5. Thus 4.5 has the free error, but not earlier versions. Mostly update comments, plus make the free conditional. This fix was already applied to the 2.5 branch. ........ r55752 | brett.cannon | 2007-06-03 16:13:41 -0700 (Sun, 03 Jun 2007) | 6 lines Make _strptime.TimeRE().pattern() use ``\s+`` for matching whitespace instead of ``\s*``. This prevents patterns from "stealing" bits from other patterns in order to make a match work. Closes bug #1730389. Will be backported. ........ r55766 | hyeshik.chang | 2007-06-05 11:16:52 -0700 (Tue, 05 Jun 2007) | 4 lines Fix build on FreeBSD. Bluetooth HCI API in FreeBSD is quite different from Linux's. Just fix the build for now but the code doesn't support the complete capability of HCI on FreeBSD yet. ........ r55770 | hyeshik.chang | 2007-06-05 11:58:51 -0700 (Tue, 05 Jun 2007) | 4 lines Bug #1728403: Fix a bug that CJKCodecs StreamReader hangs when it reads a file that ends with incomplete sequence and sizehint argument for .read() is specified. ........ r55775 | hyeshik.chang | 2007-06-05 12:28:15 -0700 (Tue, 05 Jun 2007) | 2 lines Fix for Windows: close a temporary file before trying to delete it. ........ r55783 | guido.van.rossum | 2007-06-05 14:24:47 -0700 (Tue, 05 Jun 2007) | 2 lines Patch by Tim Delany (missing DECREF). SF #1731330. ........ r55785 | collin.winter | 2007-06-05 17:17:35 -0700 (Tue, 05 Jun 2007) | 3 lines Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). ........ r55786 | facundo.batista | 2007-06-06 08:13:37 -0700 (Wed, 06 Jun 2007) | 4 lines FTP.ntransfercmd method now uses create_connection when passive, using the timeout received in connection time. ........ r55792 | facundo.batista | 2007-06-06 10:15:23 -0700 (Wed, 06 Jun 2007) | 7 lines Added an optional timeout parameter to function urllib2.urlopen, with tests in test_urllib2net.py (must have network resource enabled to execute them). Also modified test_urllib2.py because testing mock classes must take it into acount. Docs are also updated. ........ r55793 | thomas.heller | 2007-06-06 13:19:19 -0700 (Wed, 06 Jun 2007) | 1 line Build _ctypes and _ctypes_test in the ReleaseAMD64 configuration. ........ r55802 | georg.brandl | 2007-06-07 06:23:24 -0700 (Thu, 07 Jun 2007) | 3 lines Disallow function calls like foo(None=1). Backport from py3k rev. 55708 by Guido. ........ r55804 | georg.brandl | 2007-06-07 06:30:24 -0700 (Thu, 07 Jun 2007) | 2 lines Make reindent.py executable. ........ r55805 | georg.brandl | 2007-06-07 06:34:10 -0700 (Thu, 07 Jun 2007) | 2 lines Patch #1667860: Fix UnboundLocalError in urllib2. ........ r55821 | kristjan.jonsson | 2007-06-07 16:53:49 -0700 (Thu, 07 Jun 2007) | 1 line Fixing changes to getbuildinfo.c that broke linux builds ........ r55828 | thomas.heller | 2007-06-08 09:10:27 -0700 (Fri, 08 Jun 2007) | 1 line Make this test work with older Python releases where struct has no 't' format character. ........ r55829 | martin.v.loewis | 2007-06-08 10:29:20 -0700 (Fri, 08 Jun 2007) | 3 lines Bug #1733488: Fix compilation of bufferobject.c on AIX. Will backport to 2.5. ........ r55831 | thomas.heller | 2007-06-08 11:20:09 -0700 (Fri, 08 Jun 2007) | 2 lines [ 1715718 ] x64 clean compile patch for _ctypes, by Kristj?n Valur with small modifications. ........ r55832 | thomas.heller | 2007-06-08 12:01:06 -0700 (Fri, 08 Jun 2007) | 1 line Fix gcc warnings intruduced by passing Py_ssize_t to PyErr_Format calls. ........ r55833 | thomas.heller | 2007-06-08 12:08:31 -0700 (Fri, 08 Jun 2007) | 2 lines Fix wrong documentation, and correct the punktuation. Closes [1700455]. ........ r55834 | thomas.heller | 2007-06-08 12:14:23 -0700 (Fri, 08 Jun 2007) | 1 line Fix warnings by using proper function prototype. ........ r55839 | neal.norwitz | 2007-06-08 20:36:34 -0700 (Fri, 08 Jun 2007) | 7 lines Prevent expandtabs() on string and unicode objects from causing a segfault when a large width is passed on 32-bit platforms. Found by Google. It would be good for people to review this especially carefully and verify I don't have an off by one error and there is no other way to cause overflow. ........ r55841 | neal.norwitz | 2007-06-08 21:48:22 -0700 (Fri, 08 Jun 2007) | 1 line Use macro version of GET_SIZE to avoid Coverity warning (#150) about a possible error. ........ r55842 | martin.v.loewis | 2007-06-09 00:42:52 -0700 (Sat, 09 Jun 2007) | 3 lines Patch #1733960: Allow T_LONGLONG to accept ints. Will backport to 2.5. ........ r55843 | martin.v.loewis | 2007-06-09 00:58:05 -0700 (Sat, 09 Jun 2007) | 2 lines Fix Windows build. ........ r55845 | martin.v.loewis | 2007-06-09 03:10:26 -0700 (Sat, 09 Jun 2007) | 2 lines Provide LLONG_MAX for S390. ........ r55854 | thomas.heller | 2007-06-10 08:59:17 -0700 (Sun, 10 Jun 2007) | 4 lines First version of build scripts for Windows/AMD64 (no external components are built yet, and 'kill_python' is disabled). ........ r55855 | thomas.heller | 2007-06-10 10:55:51 -0700 (Sun, 10 Jun 2007) | 3 lines For now, disable the _bsddb, _sqlite3, _ssl, _testcapi, _tkinter modules in the ReleaseAMD64 configuration because they do not compile. ........ r55856 | thomas.heller | 2007-06-10 11:27:54 -0700 (Sun, 10 Jun 2007) | 1 line Need to set the environment variables, otherwise devenv.com is not found. ........ r55860 | thomas.heller | 2007-06-10 14:01:17 -0700 (Sun, 10 Jun 2007) | 1 line Revert commit 55855. ........ ................ r55880 | neal.norwitz | 2007-06-10 22:07:36 -0700 (Sun, 10 Jun 2007) | 5 lines Fix the refleak counter on test_collections. The ABC metaclass creates a registry which must be cleared on each run. Otherwise, there *seem* to be refleaks when there really aren't any. (The class is held within the registry even though it's no longer needed.) ................ r55884 | neal.norwitz | 2007-06-10 22:46:33 -0700 (Sun, 10 Jun 2007) | 1 line These tests have been removed, so they are no longer needed here ................ r55886 | georg.brandl | 2007-06-11 00:26:37 -0700 (Mon, 11 Jun 2007) | 3 lines Optimize access to True and False in the compiler (if True) and the peepholer (LOAD_NAME True). ................ r55905 | georg.brandl | 2007-06-11 10:02:26 -0700 (Mon, 11 Jun 2007) | 5 lines Remove __oct__ and __hex__ and use __index__ for converting non-ints before formatting in a base. Add a bin() builtin. ................ r55906 | georg.brandl | 2007-06-11 10:04:44 -0700 (Mon, 11 Jun 2007) | 2 lines int(x, 0) does not "guess". ................ r55907 | georg.brandl | 2007-06-11 10:05:47 -0700 (Mon, 11 Jun 2007) | 2 lines Add a comment to explain that nb_oct and nb_hex are nonfunctional. ................ r55908 | guido.van.rossum | 2007-06-11 10:49:18 -0700 (Mon, 11 Jun 2007) | 2 lines Get rid of unused imports and comment. ................ r55910 | guido.van.rossum | 2007-06-11 13:05:17 -0700 (Mon, 11 Jun 2007) | 2 lines _Abstract.__new__ now requires either no arguments or __init__ overridden. ................ r55911 | guido.van.rossum | 2007-06-11 13:07:49 -0700 (Mon, 11 Jun 2007) | 7 lines Move the collections ABCs to a separate file, _abcoll.py, in order to avoid needing to import _collections.so during the bootstrap (this will become apparent in the next submit of os.py). Add (plain and mutable) ABCs for Set, Mapping, Sequence. ................ r55912 | guido.van.rossum | 2007-06-11 13:09:31 -0700 (Mon, 11 Jun 2007) | 2 lines Rewrite the _Environ class to use the new collections ABCs. ................ r55913 | guido.van.rossum | 2007-06-11 13:59:45 -0700 (Mon, 11 Jun 2007) | 72 lines Merged revisions 55869-55912 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r55869 | neal.norwitz | 2007-06-10 17:42:11 -0700 (Sun, 10 Jun 2007) | 1 line Add Atul Varma for patch # 1667860 ........ r55870 | neal.norwitz | 2007-06-10 18:22:03 -0700 (Sun, 10 Jun 2007) | 1 line Ignore valgrind problems on Ubuntu from ld ........ r55872 | neal.norwitz | 2007-06-10 18:48:46 -0700 (Sun, 10 Jun 2007) | 2 lines Ignore config.status.lineno which seems new (new autoconf?) ........ r55873 | neal.norwitz | 2007-06-10 19:14:39 -0700 (Sun, 10 Jun 2007) | 1 line Prevent these tests from running on Win64 since they don\'t apply there either ........ r55874 | neal.norwitz | 2007-06-10 19:16:10 -0700 (Sun, 10 Jun 2007) | 5 lines Fix a bug when there was a newline in the string expandtabs was called on. This also catches another condition that can overflow. Will backport. ........ r55879 | neal.norwitz | 2007-06-10 21:52:37 -0700 (Sun, 10 Jun 2007) | 1 line Prevent hang if the port cannot be opened. ........ r55881 | neal.norwitz | 2007-06-10 22:28:45 -0700 (Sun, 10 Jun 2007) | 4 lines Add all of the distuils modules that don't seem to have explicit tests. :-( Move an import in mworkscompiler so that this module can be imported on any platform. Hopefully this works on all platforms. ........ r55882 | neal.norwitz | 2007-06-10 22:35:10 -0700 (Sun, 10 Jun 2007) | 4 lines SF #1734732, lower case the module names per PEP 8. Will backport. ........ r55885 | neal.norwitz | 2007-06-10 23:16:48 -0700 (Sun, 10 Jun 2007) | 4 lines Not sure why this only fails sometimes on Unix machines. Better to disable it and only import msvccompiler on Windows since that's the only place it can work anyways. ........ r55887 | neal.norwitz | 2007-06-11 00:29:43 -0700 (Mon, 11 Jun 2007) | 4 lines Bug #1734723: Fix repr.Repr() so it doesn't ignore the maxtuple attribute. Will backport ........ r55889 | neal.norwitz | 2007-06-11 00:36:24 -0700 (Mon, 11 Jun 2007) | 1 line Reflow long line ........ r55896 | thomas.heller | 2007-06-11 08:58:33 -0700 (Mon, 11 Jun 2007) | 3 lines Use "O&" in calls to PyArg_Parse when we need a 'void*' instead of "k" or "K" codes. ........ r55901 | facundo.batista | 2007-06-11 09:27:08 -0700 (Mon, 11 Jun 2007) | 5 lines Added versionchanged flag to all the methods which received a new optional timeout parameter, and a versionadded flag to the socket.create_connection function. ........ ................ r55914 | guido.van.rossum | 2007-06-11 14:19:50 -0700 (Mon, 11 Jun 2007) | 3 lines New super() implementation, for PEP 3135 (though the PEP is not yet updated to this design, and small tweaks may still be made later). ................ r55923 | guido.van.rossum | 2007-06-11 21:15:24 -0700 (Mon, 11 Jun 2007) | 4 lines I'm guessing this module broke when Neal ripped out the types module -- it used 'list' both as a local variable and as the built-in list type. Renamed the local variable since that was easier. ................ r55924 | guido.van.rossum | 2007-06-11 21:20:05 -0700 (Mon, 11 Jun 2007) | 5 lines Change all occurrences of super(<thisclass>, <firstarg>) to super(). Seems to have worked, all the tests still pass. Exception: test_descr and test_descrtut, which have tons of these and are there to test the various usages. ................ r55939 | collin.winter | 2007-06-12 13:57:33 -0700 (Tue, 12 Jun 2007) | 1 line Patch #1735485: remove StandardError from the exception hierarchy. ................ r55954 | neal.norwitz | 2007-06-12 21:56:32 -0700 (Tue, 12 Jun 2007) | 51 lines Merged revisions 55913-55950 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r55926 | marc-andre.lemburg | 2007-06-12 02:09:58 -0700 (Tue, 12 Jun 2007) | 3 lines Apply patch #1734945 to support TurboLinux as distribution. ........ r55927 | marc-andre.lemburg | 2007-06-12 02:26:49 -0700 (Tue, 12 Jun 2007) | 3 lines Add patch #1726668: Windows Vista support. ........ r55929 | thomas.heller | 2007-06-12 08:36:22 -0700 (Tue, 12 Jun 2007) | 1 line Checkout, but do not yet try to build, exernal sources. ........ r55930 | thomas.heller | 2007-06-12 09:08:27 -0700 (Tue, 12 Jun 2007) | 6 lines Add bufferoverflowU.lib to the libraries needed by _ssl (is this the right thing to do?). Set the /XP64 /RETAIL build enviroment in the makefile when building ReleaseAMD64. ........ r55931 | thomas.heller | 2007-06-12 09:23:19 -0700 (Tue, 12 Jun 2007) | 5 lines Revert this change, since it breaks the win32 build: Add bufferoverflowU.lib to the libraries needed by _ssl (is this the right thing to do?). ........ r55934 | thomas.heller | 2007-06-12 10:28:31 -0700 (Tue, 12 Jun 2007) | 3 lines Specify the bufferoverflowU.lib to the makefile on the command line (for ReleaseAMD64 builds). ........ r55937 | thomas.heller | 2007-06-12 12:02:59 -0700 (Tue, 12 Jun 2007) | 3 lines Add bufferoverflowU.lib to PCBuild\_bsddb.vcproj. Build sqlite3.dll and bsddb. ........ r55938 | thomas.heller | 2007-06-12 12:56:12 -0700 (Tue, 12 Jun 2007) | 2 lines Don't rebuild Berkeley DB if not needed (this was committed by accident). ........ r55948 | martin.v.loewis | 2007-06-12 20:42:19 -0700 (Tue, 12 Jun 2007) | 3 lines Provide PY_LLONG_MAX on all systems having long long. Will backport to 2.5. ........ ................ r55959 | guido.van.rossum | 2007-06-13 09:22:41 -0700 (Wed, 13 Jun 2007) | 2 lines Fix a compilation warning. ................
2007-06-13 18:07:49 +00:00
The next several functions are helpers for symtable_analyze(),
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
which determines whether a name is local, global, or free. In addition,
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
it determines which local variables are cell variables; they provide
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
bindings that are used for free variables in enclosed blocks.
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
There are also two kinds of global variables, implicit and explicit. An
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
explicit global is declared with the global statement. An implicit
global is a free variable for which the compiler has found no binding
in an enclosing function scope. The implicit global is either a global
or a builtin. Python's module and class blocks use the xxx_NAME opcodes
to handle these names to implement slightly odd semantics. In such a
block, the name is treated as global until it is assigned to; then it
is treated as a local.
The symbol table requires two passes to determine the scope of each name.
Hide list comp variables and support set comprehensions
2007-04-15 12:05:43 +00:00
The first pass collects raw facts from the AST via the symtable_visit_*
functions: the name is a parameter here, the name is used but not defined
here, etc. The second pass analyzes these facts during a pass over the
PySTEntryObjects created during pass 1.
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
When a function is entered during the second pass, the parent passes
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
the set of all name bindings visible to its children. These bindings
Hide list comp variables and support set comprehensions
2007-04-15 12:05:43 +00:00
are used to determine if non-local variables are free or implicit globals.
Don't crash when nonlocal is used at module level (fixes SF#1705365)
2007-04-23 10:14:27 +00:00
Names which are explicitly declared nonlocal must exist in this set of
visible names - if they do not, a syntax error is raised. After doing
the local analysis, it analyzes each of its child blocks using an
updated set of name bindings.
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
Hide list comp variables and support set comprehensions
2007-04-15 12:05:43 +00:00
The children update the free variable set. If a local variable is added to
the free variable set by the child, the variable is marked as a cell. The
function object being defined must provide runtime storage for the variable
that may outlive the function's frame. Cell variables are removed from the
free set before the analyze function returns to its parent.
Don't crash when nonlocal is used at module level (fixes SF#1705365)
2007-04-23 10:14:27 +00:00
Hide list comp variables and support set comprehensions
2007-04-15 12:05:43 +00:00
During analysis, the names are:
symbols: dict mapping from symbol names to flag values (including offset scope values)
scopes: dict mapping from symbol names to scope values (no offset)
local: set of all symbol names local to the current scope
bound: set of all symbol names local to a containing function scope
free: set of all symbol names referenced but not bound in child scopes
global: set of all symbol names explicitly declared as global
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
*/
gh-119981: Use do while(0) in some symtable.c multi-line macros (#119982)
2024-06-03 15:48:02 +00:00
#define SET_SCOPE(DICT, NAME, I) \
do { \
PyObject *o = PyLong_FromLong(I); \
if (!o) \
return 0; \
if (PyDict_SetItem((DICT), (NAME), o) < 0) { \
Py_DECREF(o); \
return 0; \
} \
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Py_DECREF(o); \
gh-119981: Use do while(0) in some symtable.c multi-line macros (#119982)
2024-06-03 15:48:02 +00:00
} while(0)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
/* Decide on scope of name, given flags.
Merged revisions 70801,70809 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk The merge ran into a lot of conflicts because dicts were replaced with sets in the Python 3 version of the symbol table. ........ r70801 | jeremy.hylton | 2009-03-31 09:17:03 -0400 (Tue, 31 Mar 2009) | 3 lines Add is_declared_global() which distinguishes between implicit and explicit global variables. ........ r70809 | jeremy.hylton | 2009-03-31 09:48:15 -0400 (Tue, 31 Mar 2009) | 14 lines Global statements from one function leaked into parallel functions. Re http://bugs.python.org/issue4315 The symbol table used the same name dictionaries to recursively analyze each of its child blocks, even though the dictionaries are modified during analysis. The fix is to create new temporary dictionaries via the analyze_child_block(). The only information that needs to propagate back up is the names of the free variables. Add more comments and break out a helper function. This code doesn't get any easier to understand when you only look at it once a year. ........
2009-03-31 15:26:37 +00:00
The namespace dictionaries may be modified to record information
about the new name. For example, a new global will add an entry to
global. A name that was global can be changed to local.
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
*/
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
static int
Hide list comp variables and support set comprehensions
2007-04-15 12:05:43 +00:00
analyze_name(PySTEntryObject *ste, PyObject *scopes, PyObject *name, long flags,
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PyObject *bound, PyObject *local, PyObject *free,
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
PyObject *global, PyObject *type_params, PySTEntryObject *class_entry)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
Check the result of PySet_Contains() for error in Python/symtable.c (GH-109146)
2023-09-08 16:57:41 +00:00
int contains;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (flags & DEF_GLOBAL) {
Provisional implementation of PEP 3104. Add nonlocal_stmt to Grammar and Nonlocal node to AST. They both parallel the definitions for globals. The symbol table treats variables declared as nonlocal just like variables that are free implicitly. This change is missing the language spec changes, but makes some decisions about what the spec should say via the unittests. The PEP is silent on a number of decisions, so we should review those before claiming that nonlocal is complete. Thomas Wouters made the grammer and ast changes. Jeremy Hylton added the symbol table changes and the tests. Pete Shinners and Neal Norwitz helped review the code.
2007-02-27 06:50:52 +00:00
if (flags & DEF_NONLOCAL) {
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PyErr_Format(PyExc_SyntaxError,
"name '%U' is nonlocal and global",
name);
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
return error_at_directive(ste, name);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
SET_SCOPE(scopes, name, GLOBAL_EXPLICIT);
if (PySet_Add(global, name) < 0)
return 0;
if (bound && (PySet_Discard(bound, name) < 0))
return 0;
return 1;
}
if (flags & DEF_NONLOCAL) {
if (!bound) {
PyErr_Format(PyExc_SyntaxError,
"nonlocal declaration not allowed at module level");
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
return error_at_directive(ste, name);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
Check the result of PySet_Contains() for error in Python/symtable.c (GH-109146)
2023-09-08 16:57:41 +00:00
contains = PySet_Contains(bound, name);
if (contains < 0) {
return 0;
}
if (!contains) {
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PyErr_Format(PyExc_SyntaxError,
"no binding for nonlocal '%U' found",
name);
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
return error_at_directive(ste, name);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
Check the result of PySet_Contains() for error in Python/symtable.c (GH-109146)
2023-09-08 16:57:41 +00:00
contains = PySet_Contains(type_params, name);
if (contains < 0) {
return 0;
}
if (contains) {
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
PyErr_Format(PyExc_SyntaxError,
"nonlocal binding not allowed for type parameter '%U'",
name);
return error_at_directive(ste, name);
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
SET_SCOPE(scopes, name, FREE);
ste->ste_free = 1;
return PySet_Add(free, name) >= 0;
}
if (flags & DEF_BOUND) {
revert unintended changes
2010-07-20 22:39:34 +00:00
SET_SCOPE(scopes, name, LOCAL);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (PySet_Add(local, name) < 0)
return 0;
if (PySet_Discard(global, name) < 0)
return 0;
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
if (flags & DEF_TYPE_PARAM) {
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
if (PySet_Add(type_params, name) < 0)
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
return 0;
}
else {
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
if (PySet_Discard(type_params, name) < 0)
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
return 0;
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return 1;
}
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
// If we were passed class_entry (i.e., we're in an ste_can_see_class_scope scope)
// and the bound name is in that set, then the name is potentially bound both by
// the immediately enclosing class namespace, and also by an outer function namespace.
// In that case, we want the runtime name resolution to look at only the class
// namespace and the globals (not the namespace providing the bound).
// Similarly, if the name is explicitly global in the class namespace (through the
// global statement), we want to also treat it as a global in this scope.
if (class_entry != NULL) {
long class_flags = _PyST_GetSymbol(class_entry, name);
if (class_flags & DEF_GLOBAL) {
SET_SCOPE(scopes, name, GLOBAL_EXPLICIT);
return 1;
}
else if (class_flags & DEF_BOUND && !(class_flags & DEF_NONLOCAL)) {
SET_SCOPE(scopes, name, GLOBAL_IMPLICIT);
return 1;
}
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* If an enclosing block has a binding for this name, it
is a free variable rather than a global variable.
Note that having a non-NULL bound implies that the block
is nested.
*/
Check the result of PySet_Contains() for error in Python/symtable.c (GH-109146)
2023-09-08 16:57:41 +00:00
if (bound) {
contains = PySet_Contains(bound, name);
if (contains < 0) {
return 0;
}
if (contains) {
SET_SCOPE(scopes, name, FREE);
ste->ste_free = 1;
return PySet_Add(free, name) >= 0;
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
/* If a parent has a global statement, then call it global
explicit? It could also be global implicit.
*/
Check the result of PySet_Contains() for error in Python/symtable.c (GH-109146)
2023-09-08 16:57:41 +00:00
if (global) {
contains = PySet_Contains(global, name);
if (contains < 0) {
return 0;
}
if (contains) {
SET_SCOPE(scopes, name, GLOBAL_IMPLICIT);
return 1;
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
if (ste->ste_nested)
ste->ste_free = 1;
SET_SCOPE(scopes, name, GLOBAL_IMPLICIT);
return 1;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
static int
is_free_in_any_child(PySTEntryObject *entry, PyObject *key)
{
for (Py_ssize_t i = 0; i < PyList_GET_SIZE(entry->ste_children); i++) {
PySTEntryObject *child_ste = (PySTEntryObject *)PyList_GET_ITEM(
entry->ste_children, i);
long scope = _PyST_GetScope(child_ste, key);
if (scope == FREE) {
return 1;
}
}
return 0;
}
static int
inline_comprehension(PySTEntryObject *ste, PySTEntryObject *comp,
gh-104404: fix crasher with nested comprehensions plus lambdas (#104442)
2023-05-13 00:42:04 +00:00
PyObject *scopes, PyObject *comp_free,
gh-104602: ensure all cellvars are known up front (#104603) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-19 00:07:35 +00:00
PyObject *inlined_cells)
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
{
PyObject *k, *v;
Py_ssize_t pos = 0;
gh-114828: Fix __class__ in class-scope inlined comprehensions (#115139)
2024-02-07 16:56:16 +00:00
int remove_dunder_class = 0;
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
while (PyDict_Next(comp->ste_symbols, &pos, &k, &v)) {
// skip comprehension parameter
long comp_flags = PyLong_AS_LONG(v);
if (comp_flags & DEF_PARAM) {
assert(_PyUnicode_EqualToASCIIString(k, ".0"));
continue;
}
int scope = (comp_flags >> SCOPE_OFFSET) & SCOPE_MASK;
int only_flags = comp_flags & ((1 << SCOPE_OFFSET) - 1);
gh-104602: ensure all cellvars are known up front (#104603) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-19 00:07:35 +00:00
if (scope == CELL || only_flags & DEF_COMP_CELL) {
if (PySet_Add(inlined_cells, k) < 0) {
return 0;
}
}
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
PyObject *existing = PyDict_GetItemWithError(ste->ste_symbols, k);
if (existing == NULL && PyErr_Occurred()) {
return 0;
}
gh-119666: fix multiple class-scope comprehensions referencing __class__ (#120295)
2024-06-10 02:23:30 +00:00
// __class__ is never allowed to be free through a class scope (see
// drop_class_free)
if (scope == FREE && ste->ste_type == ClassBlock &&
_PyUnicode_EqualToASCIIString(k, "__class__")) {
scope = GLOBAL_IMPLICIT;
if (PySet_Discard(comp_free, k) < 0) {
return 0;
}
remove_dunder_class = 1;
}
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
if (!existing) {
// name does not exist in scope, copy from comprehension
assert(scope != FREE || PySet_Contains(comp_free, k) == 1);
PyObject *v_flags = PyLong_FromLong(only_flags);
if (v_flags == NULL) {
return 0;
}
int ok = PyDict_SetItem(ste->ste_symbols, k, v_flags);
Py_DECREF(v_flags);
if (ok < 0) {
return 0;
}
SET_SCOPE(scopes, k, scope);
}
else {
gh-104357: fix inlined comprehensions that close over iteration var (#104368)
2023-05-11 01:08:40 +00:00
if (PyLong_AsLong(existing) & DEF_BOUND) {
// free vars in comprehension that are locals in outer scope can
gh-104374: Remove access to class scopes for inlined comprehensions (#104528) Co-authored-by: Carl Meyer <carl@oddbird.net>
2023-05-18 05:22:17 +00:00
// now simply be locals, unless they are free in comp children,
// or if the outer scope is a class block
if (!is_free_in_any_child(comp, k) && ste->ste_type != ClassBlock) {
gh-104357: fix inlined comprehensions that close over iteration var (#104368)
2023-05-11 01:08:40 +00:00
if (PySet_Discard(comp_free, k) < 0) {
return 0;
}
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
}
}
}
}
gh-114828: Fix __class__ in class-scope inlined comprehensions (#115139)
2024-02-07 16:56:16 +00:00
comp->ste_free = PySet_Size(comp_free) > 0;
if (remove_dunder_class && PyDict_DelItemString(comp->ste_symbols, "__class__") < 0) {
return 0;
}
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
return 1;
}
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
#undef SET_SCOPE
/* If a name is defined in free and also in locals, then this block
provides the binding for the free variable. The name should be
marked CELL in this block and removed from the free list.
Note that the current block's free variables are included in free.
That's safe because no name can be free and local in the same scope.
*/
static int
gh-104602: ensure all cellvars are known up front (#104603) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-19 00:07:35 +00:00
analyze_cells(PyObject *scopes, PyObject *free, PyObject *inlined_cells)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PyObject *name, *v, *v_cell;
int success = 0;
Py_ssize_t pos = 0;
v_cell = PyLong_FromLong(CELL);
if (!v_cell)
return 0;
while (PyDict_Next(scopes, &pos, &name, &v)) {
long scope;
assert(PyLong_Check(v));
scope = PyLong_AS_LONG(v);
revert unintended changes
2010-07-20 22:39:34 +00:00
if (scope != LOCAL)
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
continue;
Check the result of PySet_Contains() for error in Python/symtable.c (GH-109146)
2023-09-08 16:57:41 +00:00
int contains = PySet_Contains(free, name);
if (contains < 0) {
goto error;
}
if (!contains) {
contains = PySet_Contains(inlined_cells, name);
if (contains < 0) {
goto error;
}
if (!contains) {
continue;
}
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* Replace LOCAL with CELL for this name, and remove
from free. It is safe to replace the value of name
in the dict, because it will not cause a resize.
*/
if (PyDict_SetItem(scopes, name, v_cell) < 0)
goto error;
if (PySet_Discard(free, name) < 0)
goto error;
}
success = 1;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
error:
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Py_DECREF(v_cell);
return success;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
hide the __class__ closure from the class body (#12370)
2013-05-15 20:26:42 +00:00
static int
drop_class_free(PySTEntryObject *ste, PyObject *free)
{
int res;
bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366) https://bugs.python.org/issue46765
2022-02-23 00:23:51 +00:00
res = PySet_Discard(free, &_Py_ID(__class__));
hide the __class__ closure from the class body (#12370)
2013-05-15 20:26:42 +00:00
if (res < 0)
return 0;
if (res)
ste->ste_needs_class_closure = 1;
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
res = PySet_Discard(free, &_Py_ID(__classdict__));
if (res < 0)
return 0;
if (res)
ste->ste_needs_classdict = 1;
hide the __class__ closure from the class body (#12370)
2013-05-15 20:26:42 +00:00
return 1;
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* Enter the final scope information into the ste_symbols dict.
*
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
* All arguments are dicts. Modifies symbols, others are read-only.
*/
static int
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
update_symbols(PyObject *symbols, PyObject *scopes,
gh-104602: ensure all cellvars are known up front (#104603) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-19 00:07:35 +00:00
PyObject *bound, PyObject *free,
PyObject *inlined_cells, int classflag)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PyObject *name = NULL, *itr = NULL;
PyObject *v = NULL, *v_scope = NULL, *v_new = NULL, *v_free = NULL;
Py_ssize_t pos = 0;
/* Update scope information for all symbols in this scope */
while (PyDict_Next(symbols, &pos, &name, &v)) {
long scope, flags;
assert(PyLong_Check(v));
flags = PyLong_AS_LONG(v);
Check the result of PySet_Contains() for error in Python/symtable.c (GH-109146)
2023-09-08 16:57:41 +00:00
int contains = PySet_Contains(inlined_cells, name);
if (contains < 0) {
return 0;
}
if (contains) {
gh-104602: ensure all cellvars are known up front (#104603) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-19 00:07:35 +00:00
flags |= DEF_COMP_CELL;
}
bpo-42006: Stop using PyDict_GetItem, PyDict_GetItemString and _PyDict_GetItemId. (GH-22648) These functions are considered not safe because they suppress all internal errors and can return wrong result. PyDict_GetItemString and _PyDict_GetItemId can also silence current exception in rare cases. Remove no longer used _PyDict_GetItemId. Add _PyDict_ContainsId and rename _PyDict_Contains into _PyDict_Contains_KnownHash.
2020-10-26 06:43:39 +00:00
v_scope = PyDict_GetItemWithError(scopes, name);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
assert(v_scope && PyLong_Check(v_scope));
scope = PyLong_AS_LONG(v_scope);
flags |= (scope << SCOPE_OFFSET);
v_new = PyLong_FromLong(flags);
if (!v_new)
return 0;
if (PyDict_SetItem(symbols, name, v_new) < 0) {
Py_DECREF(v_new);
return 0;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Py_DECREF(v_new);
}
/* Record not yet resolved free variables from children (if any) */
v_free = PyLong_FromLong(FREE << SCOPE_OFFSET);
if (!v_free)
return 0;
itr = PyObject_GetIter(free);
Fix a possible decref of a borrowed reference in symtable.c. (GH-9786)
2018-10-11 05:05:35 +00:00
if (itr == NULL) {
Py_DECREF(v_free);
return 0;
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
while ((name = PyIter_Next(itr))) {
bpo-35459: Use PyDict_GetItemWithError() instead of PyDict_GetItem(). (GH-11112)
2019-02-25 15:59:46 +00:00
v = PyDict_GetItemWithError(symbols, name);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* Handle symbol that already exists in this scope */
if (v) {
/* Handle a free variable in a method of
the class that has the same name as a local
or global in the class scope.
*/
gh-109219: propagate free vars through type param scopes (#109377) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-09-14 16:20:32 +00:00
if (classflag) {
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
long flags = PyLong_AS_LONG(v) | DEF_FREE_CLASS;
v_new = PyLong_FromLong(flags);
if (!v_new) {
goto error;
}
if (PyDict_SetItem(symbols, name, v_new) < 0) {
Py_DECREF(v_new);
goto error;
}
Py_DECREF(v_new);
}
/* It's a cell, or already free in this scope */
Py_DECREF(name);
continue;
}
bpo-35459: Use PyDict_GetItemWithError() instead of PyDict_GetItem(). (GH-11112)
2019-02-25 15:59:46 +00:00
else if (PyErr_Occurred()) {
goto error;
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* Handle global symbol */
Check the result of PySet_Contains() for error in Python/symtable.c (GH-109146)
2023-09-08 16:57:41 +00:00
if (bound) {
int contains = PySet_Contains(bound, name);
if (contains < 0) {
goto error;
}
if (!contains) {
Py_DECREF(name);
continue; /* it's a global */
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
/* Propagate new free symbol up the lexical stack */
if (PyDict_SetItem(symbols, name, v_free) < 0) {
goto error;
}
Py_DECREF(name);
}
gh-113842: Add missing error check for PyIter_Next() in Python/symtable.c (GH-113843)
2024-01-09 10:43:58 +00:00
/* Check if loop ended because of exception in PyIter_Next */
if (PyErr_Occurred()) {
goto error;
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Py_DECREF(itr);
Py_DECREF(v_free);
return 1;
Hide list comp variables and support set comprehensions
2007-04-15 12:05:43 +00:00
error:
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Py_XDECREF(v_free);
Py_XDECREF(itr);
Py_XDECREF(name);
return 0;
}
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
/* Make final symbol table decisions for block of ste.
Merged revisions 70801,70809 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk The merge ran into a lot of conflicts because dicts were replaced with sets in the Python 3 version of the symbol table. ........ r70801 | jeremy.hylton | 2009-03-31 09:17:03 -0400 (Tue, 31 Mar 2009) | 3 lines Add is_declared_global() which distinguishes between implicit and explicit global variables. ........ r70809 | jeremy.hylton | 2009-03-31 09:48:15 -0400 (Tue, 31 Mar 2009) | 14 lines Global statements from one function leaked into parallel functions. Re http://bugs.python.org/issue4315 The symbol table used the same name dictionaries to recursively analyze each of its child blocks, even though the dictionaries are modified during analysis. The fix is to create new temporary dictionaries via the analyze_child_block(). The only information that needs to propagate back up is the names of the free variables. Add more comments and break out a helper function. This code doesn't get any easier to understand when you only look at it once a year. ........
2009-03-31 15:26:37 +00:00
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
Arguments:
ste -- current symtable entry (input/output)
Merged revisions 70801,70809 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk The merge ran into a lot of conflicts because dicts were replaced with sets in the Python 3 version of the symbol table. ........ r70801 | jeremy.hylton | 2009-03-31 09:17:03 -0400 (Tue, 31 Mar 2009) | 3 lines Add is_declared_global() which distinguishes between implicit and explicit global variables. ........ r70809 | jeremy.hylton | 2009-03-31 09:48:15 -0400 (Tue, 31 Mar 2009) | 14 lines Global statements from one function leaked into parallel functions. Re http://bugs.python.org/issue4315 The symbol table used the same name dictionaries to recursively analyze each of its child blocks, even though the dictionaries are modified during analysis. The fix is to create new temporary dictionaries via the analyze_child_block(). The only information that needs to propagate back up is the names of the free variables. Add more comments and break out a helper function. This code doesn't get any easier to understand when you only look at it once a year. ........
2009-03-31 15:26:37 +00:00
bound -- set of variables bound in enclosing scopes (input). bound
is NULL for module blocks.
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
free -- set of free variables in enclosed scopes (output)
globals -- set of declared global variables in enclosing scopes (input)
Merged revisions 70801,70809 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk The merge ran into a lot of conflicts because dicts were replaced with sets in the Python 3 version of the symbol table. ........ r70801 | jeremy.hylton | 2009-03-31 09:17:03 -0400 (Tue, 31 Mar 2009) | 3 lines Add is_declared_global() which distinguishes between implicit and explicit global variables. ........ r70809 | jeremy.hylton | 2009-03-31 09:48:15 -0400 (Tue, 31 Mar 2009) | 14 lines Global statements from one function leaked into parallel functions. Re http://bugs.python.org/issue4315 The symbol table used the same name dictionaries to recursively analyze each of its child blocks, even though the dictionaries are modified during analysis. The fix is to create new temporary dictionaries via the analyze_child_block(). The only information that needs to propagate back up is the names of the free variables. Add more comments and break out a helper function. This code doesn't get any easier to understand when you only look at it once a year. ........
2009-03-31 15:26:37 +00:00
The implementation uses two mutually recursive functions,
analyze_block() and analyze_child_block(). analyze_block() is
responsible for analyzing the individual names defined in a block.
analyze_child_block() prepares temporary namespace dictionaries
used to evaluated nested blocks.
The two functions exist because a child block should see the name
bindings of its enclosing blocks, but those bindings should not
propagate back to a parent block.
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
*/
Merged revisions 70801,70809 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk The merge ran into a lot of conflicts because dicts were replaced with sets in the Python 3 version of the symbol table. ........ r70801 | jeremy.hylton | 2009-03-31 09:17:03 -0400 (Tue, 31 Mar 2009) | 3 lines Add is_declared_global() which distinguishes between implicit and explicit global variables. ........ r70809 | jeremy.hylton | 2009-03-31 09:48:15 -0400 (Tue, 31 Mar 2009) | 14 lines Global statements from one function leaked into parallel functions. Re http://bugs.python.org/issue4315 The symbol table used the same name dictionaries to recursively analyze each of its child blocks, even though the dictionaries are modified during analysis. The fix is to create new temporary dictionaries via the analyze_child_block(). The only information that needs to propagate back up is the names of the free variables. Add more comments and break out a helper function. This code doesn't get any easier to understand when you only look at it once a year. ........
2009-03-31 15:26:37 +00:00
static int
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
analyze_child_block(PySTEntryObject *entry, PyObject *bound, PyObject *free,
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
PyObject *global, PyObject *type_params,
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
PySTEntryObject *class_entry, PyObject **child_free);
Merged revisions 70801,70809 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk The merge ran into a lot of conflicts because dicts were replaced with sets in the Python 3 version of the symbol table. ........ r70801 | jeremy.hylton | 2009-03-31 09:17:03 -0400 (Tue, 31 Mar 2009) | 3 lines Add is_declared_global() which distinguishes between implicit and explicit global variables. ........ r70809 | jeremy.hylton | 2009-03-31 09:48:15 -0400 (Tue, 31 Mar 2009) | 14 lines Global statements from one function leaked into parallel functions. Re http://bugs.python.org/issue4315 The symbol table used the same name dictionaries to recursively analyze each of its child blocks, even though the dictionaries are modified during analysis. The fix is to create new temporary dictionaries via the analyze_child_block(). The only information that needs to propagate back up is the names of the free variables. Add more comments and break out a helper function. This code doesn't get any easier to understand when you only look at it once a year. ........
2009-03-31 15:26:37 +00:00
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
static int
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
analyze_block(PySTEntryObject *ste, PyObject *bound, PyObject *free,
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
PyObject *global, PyObject *type_params,
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
PySTEntryObject *class_entry)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PyObject *name, *v, *local = NULL, *scopes = NULL, *newbound = NULL;
gh-104602: ensure all cellvars are known up front (#104603) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-19 00:07:35 +00:00
PyObject *newglobal = NULL, *newfree = NULL, *inlined_cells = NULL;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PyObject *temp;
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
int success = 0;
Py_ssize_t i, pos = 0;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
local = PySet_New(NULL); /* collect new names bound in block */
if (!local)
goto error;
scopes = PyDict_New(); /* collect scopes defined for each name */
if (!scopes)
goto error;
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
/* Allocate new global, bound and free variable sets. These
sets hold the names visible in nested blocks. For
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
ClassBlocks, the bound and global names are initialized
before analyzing names, because class bindings aren't
visible in methods. For other blocks, they are initialized
after names are analyzed.
*/
/* TODO(jhylton): Package these dicts in a struct so that we
can write reasonable helper functions?
*/
newglobal = PySet_New(NULL);
if (!newglobal)
goto error;
newfree = PySet_New(NULL);
if (!newfree)
goto error;
newbound = PySet_New(NULL);
if (!newbound)
goto error;
gh-104602: ensure all cellvars are known up front (#104603) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-19 00:07:35 +00:00
inlined_cells = PySet_New(NULL);
if (!inlined_cells)
gh-104404: fix crasher with nested comprehensions plus lambdas (#104442)
2023-05-13 00:42:04 +00:00
goto error;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* Class namespace has no effect on names visible in
nested functions, so populate the global and bound
sets to be passed to child blocks before analyzing
this one.
*/
if (ste->ste_type == ClassBlock) {
/* Pass down known globals */
temp = PyNumber_InPlaceOr(newglobal, global);
if (!temp)
goto error;
Py_DECREF(temp);
/* Pass down previously bound symbols */
if (bound) {
temp = PyNumber_InPlaceOr(newbound, bound);
if (!temp)
goto error;
Py_DECREF(temp);
}
}
while (PyDict_Next(ste->ste_symbols, &pos, &name, &v)) {
long flags = PyLong_AS_LONG(v);
if (!analyze_name(ste, scopes, name, flags,
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
bound, local, free, global, type_params, class_entry))
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
goto error;
}
/* Populate global and bound sets to be passed to children. */
if (ste->ste_type != ClassBlock) {
/* Add function locals to bound set */
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
if (_PyST_IsFunctionLike(ste)) {
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
temp = PyNumber_InPlaceOr(newbound, local);
if (!temp)
goto error;
Py_DECREF(temp);
}
/* Pass down previously bound symbols */
if (bound) {
temp = PyNumber_InPlaceOr(newbound, bound);
if (!temp)
goto error;
Py_DECREF(temp);
}
/* Pass down known globals */
temp = PyNumber_InPlaceOr(newglobal, global);
if (!temp)
goto error;
Py_DECREF(temp);
}
else {
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
/* Special-case __class__ and __classdict__ */
bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366) https://bugs.python.org/issue46765
2022-02-23 00:23:51 +00:00
if (PySet_Add(newbound, &_Py_ID(__class__)) < 0)
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
goto error;
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
if (PySet_Add(newbound, &_Py_ID(__classdict__)) < 0)
goto error;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
Issue #9904: fix and clarify some comments + fix indentation in symtable code
2011-04-10 04:37:26 +00:00
/* Recursively call analyze_child_block() on each child block.
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
newbound, newglobal now contain the names visible in
nested blocks. The free variables in the children will
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
be added to newfree.
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
*/
for (i = 0; i < PyList_GET_SIZE(ste->ste_children); ++i) {
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
PyObject *child_free = NULL;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PyObject *c = PyList_GET_ITEM(ste->ste_children, i);
PySTEntryObject* entry;
assert(c && PySTEntry_Check(c));
entry = (PySTEntryObject*)c;
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
PySTEntryObject *new_class_entry = NULL;
if (entry->ste_can_see_class_scope) {
if (ste->ste_type == ClassBlock) {
new_class_entry = ste;
}
else if (class_entry) {
new_class_entry = class_entry;
}
}
gh-109118: Make comprehensions work within annotation scopes, but without inlining (#118160) Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-04-28 13:21:28 +00:00
// we inline all non-generator-expression comprehensions,
// except those in annotation scopes that are nested in classes
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
int inline_comp =
entry->ste_comprehension &&
gh-109118: Make comprehensions work within annotation scopes, but without inlining (#118160) Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-04-28 13:21:28 +00:00
!entry->ste_generator &&
!ste->ste_can_see_class_scope;
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (!analyze_child_block(entry, newbound, newfree, newglobal,
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
type_params, new_class_entry, &child_free))
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
goto error;
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
}
if (inline_comp) {
gh-104602: ensure all cellvars are known up front (#104603) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-19 00:07:35 +00:00
if (!inline_comprehension(ste, entry, scopes, child_free, inlined_cells)) {
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
Py_DECREF(child_free);
goto error;
}
entry->ste_comp_inlined = 1;
}
temp = PyNumber_InPlaceOr(newfree, child_free);
Py_DECREF(child_free);
if (!temp)
goto error;
Py_DECREF(temp);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* Check if any children have free variables */
if (entry->ste_free || entry->ste_child_free)
ste->ste_child_free = 1;
}
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
/* Splice children of inlined comprehensions into our children list */
for (i = PyList_GET_SIZE(ste->ste_children) - 1; i >= 0; --i) {
PyObject* c = PyList_GET_ITEM(ste->ste_children, i);
PySTEntryObject* entry;
assert(c && PySTEntry_Check(c));
entry = (PySTEntryObject*)c;
if (entry->ste_comp_inlined &&
PyList_SetSlice(ste->ste_children, i, i + 1,
entry->ste_children) < 0)
{
goto error;
}
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* Check if any local variables must be converted to cell variables */
gh-104602: ensure all cellvars are known up front (#104603) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-19 00:07:35 +00:00
if (_PyST_IsFunctionLike(ste) && !analyze_cells(scopes, newfree, inlined_cells))
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
goto error;
hide the __class__ closure from the class body (#12370)
2013-05-15 20:26:42 +00:00
else if (ste->ste_type == ClassBlock && !drop_class_free(ste, newfree))
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
goto error;
/* Records the results of the analysis in the symbol table entry */
gh-104602: ensure all cellvars are known up front (#104603) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-19 00:07:35 +00:00
if (!update_symbols(ste->ste_symbols, scopes, bound, newfree, inlined_cells,
gh-109219: propagate free vars through type param scopes (#109377) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-09-14 16:20:32 +00:00
(ste->ste_type == ClassBlock) || ste->ste_can_see_class_scope))
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
goto error;
temp = PyNumber_InPlaceOr(free, newfree);
if (!temp)
goto error;
Py_DECREF(temp);
success = 1;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
error:
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Py_XDECREF(scopes);
Py_XDECREF(local);
Py_XDECREF(newbound);
Py_XDECREF(newglobal);
Py_XDECREF(newfree);
gh-104602: ensure all cellvars are known up front (#104603) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-19 00:07:35 +00:00
Py_XDECREF(inlined_cells);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (!success)
assert(PyErr_Occurred());
return success;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
Merged revisions 70801,70809 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk The merge ran into a lot of conflicts because dicts were replaced with sets in the Python 3 version of the symbol table. ........ r70801 | jeremy.hylton | 2009-03-31 09:17:03 -0400 (Tue, 31 Mar 2009) | 3 lines Add is_declared_global() which distinguishes between implicit and explicit global variables. ........ r70809 | jeremy.hylton | 2009-03-31 09:48:15 -0400 (Tue, 31 Mar 2009) | 14 lines Global statements from one function leaked into parallel functions. Re http://bugs.python.org/issue4315 The symbol table used the same name dictionaries to recursively analyze each of its child blocks, even though the dictionaries are modified during analysis. The fix is to create new temporary dictionaries via the analyze_child_block(). The only information that needs to propagate back up is the names of the free variables. Add more comments and break out a helper function. This code doesn't get any easier to understand when you only look at it once a year. ........
2009-03-31 15:26:37 +00:00
static int
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
analyze_child_block(PySTEntryObject *entry, PyObject *bound, PyObject *free,
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
PyObject *global, PyObject *type_params,
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
PySTEntryObject *class_entry, PyObject** child_free)
Merged revisions 70801,70809 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk The merge ran into a lot of conflicts because dicts were replaced with sets in the Python 3 version of the symbol table. ........ r70801 | jeremy.hylton | 2009-03-31 09:17:03 -0400 (Tue, 31 Mar 2009) | 3 lines Add is_declared_global() which distinguishes between implicit and explicit global variables. ........ r70809 | jeremy.hylton | 2009-03-31 09:48:15 -0400 (Tue, 31 Mar 2009) | 14 lines Global statements from one function leaked into parallel functions. Re http://bugs.python.org/issue4315 The symbol table used the same name dictionaries to recursively analyze each of its child blocks, even though the dictionaries are modified during analysis. The fix is to create new temporary dictionaries via the analyze_child_block(). The only information that needs to propagate back up is the names of the free variables. Add more comments and break out a helper function. This code doesn't get any easier to understand when you only look at it once a year. ........
2009-03-31 15:26:37 +00:00
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PyObject *temp_bound = NULL, *temp_global = NULL, *temp_free = NULL;
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
PyObject *temp_type_params = NULL;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
/* Copy the bound/global/free sets.
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
These sets are used by all blocks enclosed by the
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
current block. The analyze_block() call modifies these
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
sets.
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
*/
temp_bound = PySet_New(bound);
if (!temp_bound)
goto error;
temp_free = PySet_New(free);
if (!temp_free)
goto error;
temp_global = PySet_New(global);
if (!temp_global)
goto error;
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
temp_type_params = PySet_New(type_params);
if (!temp_type_params)
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
goto error;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
if (!analyze_block(entry, temp_bound, temp_free, temp_global,
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
temp_type_params, class_entry))
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
goto error;
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 17:02:14 +00:00
*child_free = temp_free;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Py_DECREF(temp_bound);
Py_DECREF(temp_global);
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
Py_DECREF(temp_type_params);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return 1;
Merged revisions 70801,70809 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk The merge ran into a lot of conflicts because dicts were replaced with sets in the Python 3 version of the symbol table. ........ r70801 | jeremy.hylton | 2009-03-31 09:17:03 -0400 (Tue, 31 Mar 2009) | 3 lines Add is_declared_global() which distinguishes between implicit and explicit global variables. ........ r70809 | jeremy.hylton | 2009-03-31 09:48:15 -0400 (Tue, 31 Mar 2009) | 14 lines Global statements from one function leaked into parallel functions. Re http://bugs.python.org/issue4315 The symbol table used the same name dictionaries to recursively analyze each of its child blocks, even though the dictionaries are modified during analysis. The fix is to create new temporary dictionaries via the analyze_child_block(). The only information that needs to propagate back up is the names of the free variables. Add more comments and break out a helper function. This code doesn't get any easier to understand when you only look at it once a year. ........
2009-03-31 15:26:37 +00:00
error:
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Py_XDECREF(temp_bound);
Py_XDECREF(temp_free);
Py_XDECREF(temp_global);
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
Py_XDECREF(temp_type_params);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return 0;
Merged revisions 70801,70809 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk The merge ran into a lot of conflicts because dicts were replaced with sets in the Python 3 version of the symbol table. ........ r70801 | jeremy.hylton | 2009-03-31 09:17:03 -0400 (Tue, 31 Mar 2009) | 3 lines Add is_declared_global() which distinguishes between implicit and explicit global variables. ........ r70809 | jeremy.hylton | 2009-03-31 09:48:15 -0400 (Tue, 31 Mar 2009) | 14 lines Global statements from one function leaked into parallel functions. Re http://bugs.python.org/issue4315 The symbol table used the same name dictionaries to recursively analyze each of its child blocks, even though the dictionaries are modified during analysis. The fix is to create new temporary dictionaries via the analyze_child_block(). The only information that needs to propagate back up is the names of the free variables. Add more comments and break out a helper function. This code doesn't get any easier to understand when you only look at it once a year. ........
2009-03-31 15:26:37 +00:00
}
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
static int
symtable_analyze(struct symtable *st)
{
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
PyObject *free, *global, *type_params;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
int r;
free = PySet_New(NULL);
if (!free)
return 0;
global = PySet_New(NULL);
if (!global) {
Py_DECREF(free);
return 0;
}
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
type_params = PySet_New(NULL);
if (!type_params) {
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
Py_DECREF(free);
Py_DECREF(global);
return 0;
}
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
r = analyze_block(st->st_top, NULL, free, global, type_params, NULL);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Py_DECREF(free);
Py_DECREF(global);
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
Py_DECREF(type_params);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return r;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
Merged revisions 65715,65724,65726,65732,65736-65739,65775 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r65715 | benjamin.peterson | 2008-08-16 16:04:16 -0500 (Sat, 16 Aug 2008) | 1 line add some documentation for symtable ........ r65724 | benjamin.peterson | 2008-08-16 17:11:33 -0500 (Sat, 16 Aug 2008) | 2 lines include filename and line number in SyntaxError ........ r65726 | georg.brandl | 2008-08-16 17:37:05 -0500 (Sat, 16 Aug 2008) | 2 lines Review symtable docs. ........ r65732 | benjamin.peterson | 2008-08-16 18:29:40 -0500 (Sat, 16 Aug 2008) | 1 line PySTEntry's constructor is static; there's no point in a fancy API name ........ r65736 | benjamin.peterson | 2008-08-16 20:09:17 -0500 (Sat, 16 Aug 2008) | 1 line expose PySTEntry.nested so the symtable module will work ........ r65737 | benjamin.peterson | 2008-08-16 20:17:15 -0500 (Sat, 16 Aug 2008) | 1 line a few improvements ........ r65738 | benjamin.peterson | 2008-08-16 20:27:30 -0500 (Sat, 16 Aug 2008) | 1 line fix compile errors ........ r65739 | benjamin.peterson | 2008-08-16 21:23:43 -0500 (Sat, 16 Aug 2008) | 1 line uhh PySTEntry->ste_unoptimized has to be exposed too ........ r65775 | benjamin.peterson | 2008-08-17 12:13:26 -0500 (Sun, 17 Aug 2008) | 5 lines get the symtable module back in working order - Fix broken functions - Add (hopefully) extensive tests - Modernize a little ........
2008-08-17 18:02:44 +00:00
/* symtable_enter_block() gets a reference via ste_new.
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
This reference is released when the block is exited, via the DECREF
in symtable_exit_block().
*/
static int
closes bpo-39872: Remove unused args from symtable_exit_block and symtable_visit_annotations. (GH-18800)
2020-03-06 15:46:04 +00:00
symtable_exit_block(struct symtable *st)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
store the current scope on the stack right away
2011-06-30 03:52:39 +00:00
Py_ssize_t size;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
store the current scope on the stack right away
2011-06-30 03:52:39 +00:00
st->st_cur = NULL;
size = PyList_GET_SIZE(st->st_stack);
if (size) {
if (PyList_SetSlice(st->st_stack, size - 1, size, NULL) < 0)
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return 0;
fix possibily uninitialized memory usage (closes #12474)
2011-07-02 14:22:13 +00:00
if (--size)
st->st_cur = (PySTEntryObject *)PyList_GET_ITEM(st->st_stack, size - 1);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
return 1;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
static int
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
symtable_enter_existing_block(struct symtable *st, PySTEntryObject* ste)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
store the current scope on the stack right away
2011-06-30 03:52:39 +00:00
if (PyList_Append(st->st_stack, (PyObject *)ste) < 0) {
return 0;
}
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
PySTEntryObject *prev = st->st_cur;
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
/* bpo-37757: For now, disallow *all* assignment expressions in the
* outermost iterator expression of a comprehension, even those inside
* a nested comprehension or a lambda expression.
*/
if (prev) {
ste->ste_comp_iter_expr = prev->ste_comp_iter_expr;
}
gh-119311: Fix name mangling with PEP 695 generic classes (#119464) Fixes #119311. Fixes #119395.
2024-05-28 02:50:38 +00:00
/* No need to inherit ste_mangled_names in classes, where all names
* are mangled. */
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
if (prev && prev->ste_mangled_names != NULL && ste->ste_type != ClassBlock) {
gh-119311: Fix name mangling with PEP 695 generic classes (#119464) Fixes #119311. Fixes #119395.
2024-05-28 02:50:38 +00:00
ste->ste_mangled_names = Py_NewRef(prev->ste_mangled_names);
}
store the current scope on the stack right away
2011-06-30 03:52:39 +00:00
/* The entry is owned by the stack. Borrow it for st_cur. */
st->st_cur = ste;
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
/* If "from __future__ import annotations" is active,
* annotation blocks shouldn't have any affect on the symbol table since in
* the compilation stage, they will all be transformed to strings. */
if (st->st_future->ff_features & CO_FUTURE_ANNOTATIONS && ste->ste_type == AnnotationBlock) {
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
return 1;
}
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
if (ste->ste_type == ModuleBlock)
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
st->st_global = st->st_cur->ste_symbols;
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (prev) {
store the current scope on the stack right away
2011-06-30 03:52:39 +00:00
if (PyList_Append(prev->ste_children, (PyObject *)ste) < 0) {
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return 0;
}
}
return 1;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
static int
symtable_enter_block(struct symtable *st, identifier name, _Py_block_ty block,
void *ast, int lineno, int col_offset,
int end_lineno, int end_col_offset)
{
PySTEntryObject *ste = ste_new(st, name, block, ast,
lineno, col_offset, end_lineno, end_col_offset);
if (ste == NULL)
return 0;
int result = symtable_enter_existing_block(st, ste);
Py_DECREF(ste);
return result;
}
Fix icc warnings: conversion from "long" to "int" may lose significant bits
2006-01-08 01:06:06 +00:00
static long
gh-96497: Mangle name before symtable lookup in 'symtable_extend_namedexpr_scope' (GH-96561)
2024-02-17 12:06:31 +00:00
symtable_lookup_entry(struct symtable *st, PySTEntryObject *ste, PyObject *name)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
gh-119311: Fix name mangling with PEP 695 generic classes (#119464) Fixes #119311. Fixes #119395.
2024-05-28 02:50:38 +00:00
PyObject *mangled = _Py_MaybeMangle(st->st_private, ste, name);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (!mangled)
return 0;
gh-96497: Mangle name before symtable lookup in 'symtable_extend_namedexpr_scope' (GH-96561)
2024-02-17 12:06:31 +00:00
long ret = _PyST_GetSymbol(ste, mangled);
bpo-37954: Fix reference leak in the symtable (GH-15514)
2019-08-26 15:14:07 +00:00
Py_DECREF(mangled);
return ret;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
gh-96497: Mangle name before symtable lookup in 'symtable_extend_namedexpr_scope' (GH-96561)
2024-02-17 12:06:31 +00:00
static long
symtable_lookup(struct symtable *st, PyObject *name)
{
return symtable_lookup_entry(st, st->st_cur, name);
}
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
static int
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
symtable_add_def_helper(struct symtable *st, PyObject *name, int flag, struct _symtable_entry *ste,
int lineno, int col_offset, int end_lineno, int end_col_offset)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PyObject *o;
PyObject *dict;
long val;
gh-119311: Fix name mangling with PEP 695 generic classes (#119464) Fixes #119311. Fixes #119395.
2024-05-28 02:50:38 +00:00
PyObject *mangled = _Py_MaybeMangle(st->st_private, st->st_cur, name);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (!mangled)
return 0;
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
dict = ste->ste_symbols;
bpo-35459: Use PyDict_GetItemWithError() instead of PyDict_GetItem(). (GH-11112)
2019-02-25 15:59:46 +00:00
if ((o = PyDict_GetItemWithError(dict, mangled))) {
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
val = PyLong_AS_LONG(o);
if ((flag & DEF_PARAM) && (val & DEF_PARAM)) {
/* Is it better to use 'mangled' or 'name' here? */
PyErr_Format(PyExc_SyntaxError, DUPLICATE_ARGUMENT, name);
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
PyErr_RangedSyntaxLocationObject(st->st_filename,
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
lineno, col_offset + 1,
end_lineno, end_col_offset + 1);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
goto error;
}
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
if ((flag & DEF_TYPE_PARAM) && (val & DEF_TYPE_PARAM)) {
PyErr_Format(PyExc_SyntaxError, DUPLICATE_TYPE_PARAM, name);
PyErr_RangedSyntaxLocationObject(st->st_filename,
lineno, col_offset + 1,
end_lineno, end_col_offset + 1);
goto error;
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
val |= flag;
bpo-35459: Use PyDict_GetItemWithError() instead of PyDict_GetItem(). (GH-11112)
2019-02-25 15:59:46 +00:00
}
else if (PyErr_Occurred()) {
goto error;
}
else {
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
val = flag;
bpo-35459: Use PyDict_GetItemWithError() instead of PyDict_GetItem(). (GH-11112)
2019-02-25 15:59:46 +00:00
}
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
if (ste->ste_comp_iter_target) {
/* This name is an iteration variable in a comprehension,
* so check for a binding conflict with any named expressions.
* Otherwise, mark it as an iteration variable so subsequent
* named expressions can check for conflicts.
*/
if (val & (DEF_GLOBAL | DEF_NONLOCAL)) {
PyErr_Format(PyExc_SyntaxError,
NAMED_EXPR_COMP_INNER_LOOP_CONFLICT, name);
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
PyErr_RangedSyntaxLocationObject(st->st_filename,
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
lineno, col_offset + 1,
end_lineno, end_col_offset + 1);
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
goto error;
}
val |= DEF_COMP_ITER;
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
o = PyLong_FromLong(val);
if (o == NULL)
goto error;
if (PyDict_SetItem(dict, mangled, o) < 0) {
Py_DECREF(o);
goto error;
}
Py_DECREF(o);
if (flag & DEF_PARAM) {
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
if (PyList_Append(ste->ste_varnames, mangled) < 0)
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
goto error;
} else if (flag & DEF_GLOBAL) {
/* XXX need to update DEF_GLOBAL for other flags too;
perhaps only DEF_FREE_GLOBAL */
val = flag;
bpo-42006: Stop using PyDict_GetItem, PyDict_GetItemString and _PyDict_GetItemId. (GH-22648) These functions are considered not safe because they suppress all internal errors and can return wrong result. PyDict_GetItemString and _PyDict_GetItemId can also silence current exception in rare cases. Remove no longer used _PyDict_GetItemId. Add _PyDict_ContainsId and rename _PyDict_Contains into _PyDict_Contains_KnownHash.
2020-10-26 06:43:39 +00:00
if ((o = PyDict_GetItemWithError(st->st_global, mangled))) {
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
val |= PyLong_AS_LONG(o);
}
bpo-42006: Stop using PyDict_GetItem, PyDict_GetItemString and _PyDict_GetItemId. (GH-22648) These functions are considered not safe because they suppress all internal errors and can return wrong result. PyDict_GetItemString and _PyDict_GetItemId can also silence current exception in rare cases. Remove no longer used _PyDict_GetItemId. Add _PyDict_ContainsId and rename _PyDict_Contains into _PyDict_Contains_KnownHash.
2020-10-26 06:43:39 +00:00
else if (PyErr_Occurred()) {
goto error;
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
o = PyLong_FromLong(val);
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
if (o == NULL)
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
goto error;
if (PyDict_SetItem(st->st_global, mangled, o) < 0) {
Py_DECREF(o);
goto error;
}
Py_DECREF(o);
}
Py_DECREF(mangled);
return 1;
Fix private name mangling. The symtable also must do mangles so that the scope of names can be correctly determined.
2005-10-23 18:37:42 +00:00
error:
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Py_DECREF(mangled);
return 0;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
static int
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
symtable_add_def(struct symtable *st, PyObject *name, int flag,
int lineno, int col_offset, int end_lineno, int end_col_offset)
{
gh-119311: Fix name mangling with PEP 695 generic classes (#119464) Fixes #119311. Fixes #119395.
2024-05-28 02:50:38 +00:00
if ((flag & DEF_TYPE_PARAM) && st->st_cur->ste_mangled_names != NULL) {
if(PySet_Add(st->st_cur->ste_mangled_names, name) < 0) {
return 0;
}
}
bpo-46417: Clear symtable identifiers at exit (GH-30809) Add _PySymtable_Fini() function, called by finalize_interp_clear(). Update test_cmd_line.test_showrefcount() to tolerate negative reference count.
2022-01-22 23:06:56 +00:00
return symtable_add_def_helper(st, name, flag, st->st_cur,
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
lineno, col_offset, end_lineno, end_col_offset);
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
}
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
static int
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
symtable_enter_type_param_block(struct symtable *st, identifier name,
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
void *ast, int has_defaults, int has_kwdefaults,
enum _stmt_kind kind,
int lineno, int col_offset,
int end_lineno, int end_col_offset)
{
_Py_block_ty current_type = st->st_cur->ste_type;
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
if(!symtable_enter_block(st, name, TypeParametersBlock, ast, lineno,
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
col_offset, end_lineno, end_col_offset)) {
return 0;
}
if (current_type == ClassBlock) {
st->st_cur->ste_can_see_class_scope = 1;
if (!symtable_add_def(st, &_Py_ID(__classdict__), USE, lineno, col_offset, end_lineno, end_col_offset)) {
return 0;
}
}
if (kind == ClassDef_kind) {
_Py_DECLARE_STR(type_params, ".type_params");
// It gets "set" when we create the type params tuple and
// "used" when we build up the bases.
if (!symtable_add_def(st, &_Py_STR(type_params), DEF_LOCAL,
lineno, col_offset, end_lineno, end_col_offset)) {
return 0;
}
if (!symtable_add_def(st, &_Py_STR(type_params), USE,
lineno, col_offset, end_lineno, end_col_offset)) {
return 0;
}
// This is used for setting the generic base
_Py_DECLARE_STR(generic_base, ".generic_base");
if (!symtable_add_def(st, &_Py_STR(generic_base), DEF_LOCAL,
lineno, col_offset, end_lineno, end_col_offset)) {
return 0;
}
if (!symtable_add_def(st, &_Py_STR(generic_base), USE,
lineno, col_offset, end_lineno, end_col_offset)) {
return 0;
}
}
if (has_defaults) {
_Py_DECLARE_STR(defaults, ".defaults");
if (!symtable_add_def(st, &_Py_STR(defaults), DEF_PARAM,
lineno, col_offset, end_lineno, end_col_offset)) {
return 0;
}
}
if (has_kwdefaults) {
_Py_DECLARE_STR(kwdefaults, ".kwdefaults");
if (!symtable_add_def(st, &_Py_STR(kwdefaults), DEF_PARAM,
lineno, col_offset, end_lineno, end_col_offset)) {
return 0;
}
}
return 1;
}
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
/* VISIT, VISIT_SEQ and VIST_SEQ_TAIL take an ASDL type as their second argument.
They use the ASDL name to synthesize the name of the C type and the visit
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
function.
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
VISIT_SEQ_TAIL permits the start of an ASDL sequence to be skipped, which is
useful if the first node in the sequence requires special treatment.
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT macro returns the specified value exiting from the function but
first adjusts current recursion counter depth.
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
*/
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
#define VISIT_QUIT(ST, X) \
return --(ST)->recursion_depth,(X)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
#define VISIT(ST, TYPE, V) \
gh-119981: Use do while(0) in some symtable.c multi-line macros (#119982)
2024-06-03 15:48:02 +00:00
do { \
if (!symtable_visit_ ## TYPE((ST), (V))) { \
VISIT_QUIT((ST), 0); \
} \
} while(0)
#define VISIT_SEQ(ST, TYPE, SEQ) \
do { \
Fixes loop variables to be the same types as their limit (GH-120958)
2024-06-24 16:11:47 +00:00
Py_ssize_t i; \
gh-119981: Use do while(0) in some symtable.c multi-line macros (#119982)
2024-06-03 15:48:02 +00:00
asdl_ ## TYPE ## _seq *seq = (SEQ); /* avoid variable capture */ \
for (i = 0; i < asdl_seq_LEN(seq); i++) { \
TYPE ## _ty elt = (TYPE ## _ty)asdl_seq_GET(seq, i); \
if (!symtable_visit_ ## TYPE((ST), elt)) \
VISIT_QUIT((ST), 0); \
} \
} while(0)
#define VISIT_SEQ_TAIL(ST, TYPE, SEQ, START) \
do { \
Fixes loop variables to be the same types as their limit (GH-120958)
2024-06-24 16:11:47 +00:00
Py_ssize_t i; \
gh-119981: Use do while(0) in some symtable.c multi-line macros (#119982)
2024-06-03 15:48:02 +00:00
asdl_ ## TYPE ## _seq *seq = (SEQ); /* avoid variable capture */ \
for (i = (START); i < asdl_seq_LEN(seq); i++) { \
TYPE ## _ty elt = (TYPE ## _ty)asdl_seq_GET(seq, i); \
if (!symtable_visit_ ## TYPE((ST), elt)) \
VISIT_QUIT((ST), 0); \
} \
} while(0)
#define VISIT_SEQ_WITH_NULL(ST, TYPE, SEQ) \
do { \
int i = 0; \
asdl_ ## TYPE ## _seq *seq = (SEQ); /* avoid variable capture */ \
for (i = 0; i < asdl_seq_LEN(seq); i++) { \
TYPE ## _ty elt = (TYPE ## _ty)asdl_seq_GET(seq, i); \
if (!elt) continue; /* can be NULL */ \
if (!symtable_visit_ ## TYPE((ST), elt)) \
VISIT_QUIT((ST), 0); \
} \
} while(0)
Jiwon Seo's PEP 3102 implementation. See SF#1549670. The compiler package has not yet been updated.
2006-10-27 23:31:49 +00:00
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
static int
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
symtable_record_directive(struct symtable *st, identifier name, int lineno,
int col_offset, int end_lineno, int end_col_offset)
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
{
make recording and reporting errors and nonlocal and global directives more robust (closes #25973)
2015-12-29 16:08:34 +00:00
PyObject *data, *mangled;
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
int res;
if (!st->st_cur->ste_directives) {
st->st_cur->ste_directives = PyList_New(0);
if (!st->st_cur->ste_directives)
return 0;
}
gh-119311: Fix name mangling with PEP 695 generic classes (#119464) Fixes #119311. Fixes #119395.
2024-05-28 02:50:38 +00:00
mangled = _Py_MaybeMangle(st->st_private, st->st_cur, name);
make recording and reporting errors and nonlocal and global directives more robust (closes #25973)
2015-12-29 16:08:34 +00:00
if (!mangled)
return 0;
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
data = Py_BuildValue("(Niiii)", mangled, lineno, col_offset, end_lineno, end_col_offset);
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
if (!data)
return 0;
res = PyList_Append(st->st_cur->ste_directives, data);
Py_DECREF(data);
return res == 0;
}
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
static int
has_kwonlydefaults(asdl_arg_seq *kwonlyargs, asdl_expr_seq *kw_defaults)
{
for (int i = 0; i < asdl_seq_LEN(kwonlyargs); i++) {
expr_ty default_ = asdl_seq_GET(kw_defaults, i);
if (default_) {
return 1;
}
}
return 0;
}
point errors related to nonlocals and globals to the statement declaring them (closes #10189)
2012-11-01 00:26:20 +00:00
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
static int
symtable_visit_stmt(struct symtable *st, stmt_ty s)
{
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
if (++st->recursion_depth > st->recursion_limit) {
Issue #19235: Add new RecursionError exception. Patch by Georg Brandl.
2015-07-03 05:04:23 +00:00
PyErr_SetString(PyExc_RecursionError,
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
"maximum recursion depth exceeded during compilation");
VISIT_QUIT(st, 0);
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
switch (s->kind) {
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
case FunctionDef_kind: {
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_add_def(st, s->v.FunctionDef.name, DEF_LOCAL, LOCATION(s)))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (s->v.FunctionDef.args->defaults)
VISIT_SEQ(st, expr, s->v.FunctionDef.args->defaults);
if (s->v.FunctionDef.args->kw_defaults)
PEP 448: additional unpacking generalizations (closes #2292) Patch by Neil Girdhar.
2015-05-06 00:16:41 +00:00
VISIT_SEQ_WITH_NULL(st, expr, s->v.FunctionDef.args->kw_defaults);
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
if (s->v.FunctionDef.decorator_list)
VISIT_SEQ(st, expr, s->v.FunctionDef.decorator_list);
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
if (asdl_seq_LEN(s->v.FunctionDef.type_params) > 0) {
if (!symtable_enter_type_param_block(
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
st, s->v.FunctionDef.name,
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
(void *)s->v.FunctionDef.type_params,
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
s->v.FunctionDef.args->defaults != NULL,
has_kwonlydefaults(s->v.FunctionDef.args->kwonlyargs,
s->v.FunctionDef.args->kw_defaults),
s->kind,
LOCATION(s))) {
VISIT_QUIT(st, 0);
}
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
VISIT_SEQ(st, type_param, s->v.FunctionDef.type_params);
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
}
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
PySTEntryObject *new_ste = ste_new(st, s->v.FunctionDef.name, FunctionBlock, (void *)s,
LOCATION(s));
if (!new_ste) {
VISIT_QUIT(st, 0);
}
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
if (!symtable_visit_annotations(st, s, s->v.FunctionDef.args,
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
s->v.FunctionDef.returns, new_ste)) {
Py_DECREF(new_ste);
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
}
if (!symtable_enter_existing_block(st, new_ste)) {
Py_DECREF(new_ste);
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
}
Py_DECREF(new_ste);
remove VISIT_*_IN_BLOCK macros These are pointless because on error, all blocks will be finalized by symtable_dealloc.
2011-06-29 20:27:14 +00:00
VISIT(st, arguments, s->v.FunctionDef.args);
VISIT_SEQ(st, stmt, s->v.FunctionDef.body);
closes bpo-39872: Remove unused args from symtable_exit_block and symtable_visit_annotations. (GH-18800)
2020-03-06 15:46:04 +00:00
if (!symtable_exit_block(st))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
if (asdl_seq_LEN(s->v.FunctionDef.type_params) > 0) {
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
if (!symtable_exit_block(st))
VISIT_QUIT(st, 0);
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
break;
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
case ClassDef_kind: {
PyObject *tmp;
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_add_def(st, s->v.ClassDef.name, DEF_LOCAL, LOCATION(s)))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (s->v.ClassDef.decorator_list)
VISIT_SEQ(st, expr, s->v.ClassDef.decorator_list);
gh-119311: Fix name mangling with PEP 695 generic classes (#119464) Fixes #119311. Fixes #119395.
2024-05-28 02:50:38 +00:00
tmp = st->st_private;
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
if (asdl_seq_LEN(s->v.ClassDef.type_params) > 0) {
if (!symtable_enter_type_param_block(st, s->v.ClassDef.name,
(void *)s->v.ClassDef.type_params,
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
false, false, s->kind,
LOCATION(s))) {
VISIT_QUIT(st, 0);
}
gh-119311: Fix name mangling with PEP 695 generic classes (#119464) Fixes #119311. Fixes #119395.
2024-05-28 02:50:38 +00:00
st->st_private = s->v.ClassDef.name;
st->st_cur->ste_mangled_names = PySet_New(NULL);
if (!st->st_cur->ste_mangled_names) {
VISIT_QUIT(st, 0);
}
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
VISIT_SEQ(st, type_param, s->v.ClassDef.type_params);
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
}
VISIT_SEQ(st, expr, s->v.ClassDef.bases);
VISIT_SEQ(st, keyword, s->v.ClassDef.keywords);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (!symtable_enter_block(st, s->v.ClassDef.name, ClassBlock,
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
(void *)s, s->lineno, s->col_offset,
s->end_lineno, s->end_col_offset))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
st->st_private = s->v.ClassDef.name;
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
if (asdl_seq_LEN(s->v.ClassDef.type_params) > 0) {
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
if (!symtable_add_def(st, &_Py_ID(__type_params__),
DEF_LOCAL, LOCATION(s))) {
VISIT_QUIT(st, 0);
}
_Py_DECLARE_STR(type_params, ".type_params");
if (!symtable_add_def(st, &_Py_STR(type_params),
USE, LOCATION(s))) {
VISIT_QUIT(st, 0);
}
}
remove VISIT_*_IN_BLOCK macros These are pointless because on error, all blocks will be finalized by symtable_dealloc.
2011-06-29 20:27:14 +00:00
VISIT_SEQ(st, stmt, s->v.ClassDef.body);
closes bpo-39872: Remove unused args from symtable_exit_block and symtable_visit_annotations. (GH-18800)
2020-03-06 15:46:04 +00:00
if (!symtable_exit_block(st))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
if (asdl_seq_LEN(s->v.ClassDef.type_params) > 0) {
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
if (!symtable_exit_block(st))
VISIT_QUIT(st, 0);
}
gh-119311: Fix name mangling with PEP 695 generic classes (#119464) Fixes #119311. Fixes #119395.
2024-05-28 02:50:38 +00:00
st->st_private = tmp;
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
break;
}
case TypeAlias_kind: {
VISIT(st, expr, s->v.TypeAlias.name);
assert(s->v.TypeAlias.name->kind == Name_kind);
PyObject *name = s->v.TypeAlias.name->v.Name.id;
int is_in_class = st->st_cur->ste_type == ClassBlock;
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
int is_generic = asdl_seq_LEN(s->v.TypeAlias.type_params) > 0;
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
if (is_generic) {
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
if (!symtable_enter_type_param_block(
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
st, name,
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
(void *)s->v.TypeAlias.type_params,
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
false, false, s->kind,
LOCATION(s))) {
VISIT_QUIT(st, 0);
}
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
VISIT_SEQ(st, type_param, s->v.TypeAlias.type_params);
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
}
if (!symtable_enter_block(st, name, TypeAliasBlock,
(void *)s, LOCATION(s)))
VISIT_QUIT(st, 0);
st->st_cur->ste_can_see_class_scope = is_in_class;
if (is_in_class && !symtable_add_def(st, &_Py_ID(__classdict__), USE, LOCATION(s->v.TypeAlias.value))) {
VISIT_QUIT(st, 0);
}
VISIT(st, expr, s->v.TypeAlias.value);
if (!symtable_exit_block(st))
VISIT_QUIT(st, 0);
if (is_generic) {
if (!symtable_exit_block(st))
VISIT_QUIT(st, 0);
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
break;
}
case Return_kind:
if (s->v.Return.value) {
VISIT(st, expr, s->v.Return.value);
st->st_cur->ste_returns_value = 1;
}
break;
case Delete_kind:
VISIT_SEQ(st, expr, s->v.Delete.targets);
break;
case Assign_kind:
VISIT_SEQ(st, expr, s->v.Assign.targets);
VISIT(st, expr, s->v.Assign.value);
break;
Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. Patch by Ivan Levkivskyi.
2016-09-09 03:50:03 +00:00
case AnnAssign_kind:
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
st->st_cur->ste_annotations_used = 1;
Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. Patch by Ivan Levkivskyi.
2016-09-09 03:50:03 +00:00
if (s->v.AnnAssign.target->kind == Name_kind) {
expr_ty e_name = s->v.AnnAssign.target;
long cur = symtable_lookup(st, e_name->v.Name.id);
if (cur < 0) {
VISIT_QUIT(st, 0);
}
if ((cur & (DEF_GLOBAL | DEF_NONLOCAL))
bpo-34939: Allow annotated global names in module namespace (GH-9844) Allow annotated global names in the module namespace after the symbol is declared as global. Previously, only symbols annotated before they are declared as global (i.e. inside a function) were allowed. This change allows symbols to be declared as global before the annotation happens in the global scope.
2018-10-14 17:01:03 +00:00
&& (st->st_cur->ste_symbols != st->st_global)
Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. Patch by Ivan Levkivskyi.
2016-09-09 03:50:03 +00:00
&& s->v.AnnAssign.simple) {
PyErr_Format(PyExc_SyntaxError,
Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. Patch by Ivan Levkivskyi.
2016-09-09 16:36:26 +00:00
cur & DEF_GLOBAL ? GLOBAL_ANNOT : NONLOCAL_ANNOT,
e_name->v.Name.id);
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
PyErr_RangedSyntaxLocationObject(st->st_filename,
s->lineno,
s->col_offset + 1,
s->end_lineno,
s->end_col_offset + 1);
Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. Patch by Ivan Levkivskyi.
2016-09-09 03:50:03 +00:00
VISIT_QUIT(st, 0);
}
if (s->v.AnnAssign.simple &&
!symtable_add_def(st, e_name->v.Name.id,
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
DEF_ANNOT | DEF_LOCAL, LOCATION(e_name))) {
Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. Patch by Ivan Levkivskyi.
2016-09-09 03:50:03 +00:00
VISIT_QUIT(st, 0);
}
else {
if (s->v.AnnAssign.value
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
&& !symtable_add_def(st, e_name->v.Name.id, DEF_LOCAL, LOCATION(e_name))) {
Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. Patch by Ivan Levkivskyi.
2016-09-09 03:50:03 +00:00
VISIT_QUIT(st, 0);
}
}
}
else {
VISIT(st, expr, s->v.AnnAssign.target);
}
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
if (!symtable_visit_annotation(st, s->v.AnnAssign.annotation,
(void *)((uintptr_t)st->st_cur->ste_id + 1))) {
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
VISIT_QUIT(st, 0);
}
Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. Patch by Ivan Levkivskyi.
2016-09-09 03:50:03 +00:00
if (s->v.AnnAssign.value) {
VISIT(st, expr, s->v.AnnAssign.value);
}
break;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
case AugAssign_kind:
VISIT(st, expr, s->v.AugAssign.target);
VISIT(st, expr, s->v.AugAssign.value);
break;
case For_kind:
VISIT(st, expr, s->v.For.target);
VISIT(st, expr, s->v.For.iter);
VISIT_SEQ(st, stmt, s->v.For.body);
if (s->v.For.orelse)
VISIT_SEQ(st, stmt, s->v.For.orelse);
break;
case While_kind:
VISIT(st, expr, s->v.While.test);
VISIT_SEQ(st, stmt, s->v.While.body);
if (s->v.While.orelse)
VISIT_SEQ(st, stmt, s->v.While.orelse);
break;
case If_kind:
/* XXX if 0: and lookup_yield() hacks */
VISIT(st, expr, s->v.If.test);
VISIT_SEQ(st, stmt, s->v.If.body);
if (s->v.If.orelse)
VISIT_SEQ(st, stmt, s->v.If.orelse);
break;
bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917) Co-authored-by: Guido van Rossum <guido@python.org> Co-authored-by: Talin <viridia@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-02-26 22:51:55 +00:00
case Match_kind:
VISIT(st, expr, s->v.Match.subject);
VISIT_SEQ(st, match_case, s->v.Match.cases);
break;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
case Raise_kind:
if (s->v.Raise.exc) {
VISIT(st, expr, s->v.Raise.exc);
Issue #9904: fix and clarify some comments + fix indentation in symtable code
2011-04-10 04:37:26 +00:00
if (s->v.Raise.cause) {
VISIT(st, expr, s->v.Raise.cause);
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
break;
unify TryExcept and TryFinally (closes #12199)
2011-05-29 16:43:10 +00:00
case Try_kind:
VISIT_SEQ(st, stmt, s->v.Try.body);
VISIT_SEQ(st, excepthandler, s->v.Try.handlers);
gh-111123: symtable should visit exception handlers before the else block (#111142)
2023-10-21 12:38:29 +00:00
VISIT_SEQ(st, stmt, s->v.Try.orelse);
unify TryExcept and TryFinally (closes #12199)
2011-05-29 16:43:10 +00:00
VISIT_SEQ(st, stmt, s->v.Try.finalbody);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
break;
bpo-45292: [PEP-654] add except* (GH-29581)
2021-12-14 16:48:15 +00:00
case TryStar_kind:
VISIT_SEQ(st, stmt, s->v.TryStar.body);
VISIT_SEQ(st, excepthandler, s->v.TryStar.handlers);
gh-111123: symtable should visit exception handlers before the else block (#111142)
2023-10-21 12:38:29 +00:00
VISIT_SEQ(st, stmt, s->v.TryStar.orelse);
bpo-45292: [PEP-654] add except* (GH-29581)
2021-12-14 16:48:15 +00:00
VISIT_SEQ(st, stmt, s->v.TryStar.finalbody);
break;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
case Assert_kind:
VISIT(st, expr, s->v.Assert.test);
if (s->v.Assert.msg)
VISIT(st, expr, s->v.Assert.msg);
break;
case Import_kind:
VISIT_SEQ(st, alias, s->v.Import.names);
break;
case ImportFrom_kind:
VISIT_SEQ(st, alias, s->v.ImportFrom.names);
break;
case Global_kind: {
Fixes loop variables to be the same types as their limit (GH-120958)
2024-06-24 16:11:47 +00:00
Py_ssize_t i;
bpo-41746: Add type information to asdl_seq objects (GH-22223) * Add new capability to the PEG parser to type variable assignments. For instance: ``` | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a } ``` * Add new sequence types from the asdl definition (automatically generated) * Make `asdl_seq` type a generic aliasing pointer type. * Create a new `asdl_generic_seq` for the generic case using `void*`. * The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed. * New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences. * Changes all possible `asdl_seq` types to use specific versions everywhere.
2020-09-16 18:42:00 +00:00
asdl_identifier_seq *seq = s->v.Global.names;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
for (i = 0; i < asdl_seq_LEN(seq); i++) {
identifier name = (identifier)asdl_seq_GET(seq, i);
long cur = symtable_lookup(st, name);
if (cur < 0)
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
bpo-28936: Detect lexically first syntax error first (#4097) Lexically first global and nonlocal syntax errors at given scope should be detected first.
2017-10-26 21:28:35 +00:00
if (cur & (DEF_PARAM | DEF_LOCAL | USE | DEF_ANNOT)) {
const char* msg;
if (cur & DEF_PARAM) {
msg = GLOBAL_PARAM;
} else if (cur & USE) {
Issue #28100: Refactor error messages, patch by Ivan Levkivskyi
2016-09-23 18:26:30 +00:00
msg = GLOBAL_AFTER_USE;
} else if (cur & DEF_ANNOT) {
Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. Patch by Ivan Levkivskyi.
2016-09-09 16:36:26 +00:00
msg = GLOBAL_ANNOT;
Issue #28100: Refactor error messages, patch by Ivan Levkivskyi
2016-09-23 18:26:30 +00:00
} else { /* DEF_LOCAL */
Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. Patch by Ivan Levkivskyi.
2016-09-09 16:36:26 +00:00
msg = GLOBAL_AFTER_ASSIGN;
}
Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. Patch by Ivan Levkivskyi.
2016-09-09 03:50:03 +00:00
PyErr_Format(PyExc_SyntaxError,
Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. Patch by Ivan Levkivskyi.
2016-09-09 16:36:26 +00:00
msg, name);
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
PyErr_RangedSyntaxLocationObject(st->st_filename,
s->lineno,
s->col_offset + 1,
s->end_lineno,
s->end_col_offset + 1);
Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. Patch by Ivan Levkivskyi.
2016-09-09 03:50:03 +00:00
VISIT_QUIT(st, 0);
}
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_add_def(st, name, DEF_GLOBAL, LOCATION(s)))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
if (!symtable_record_directive(st, name, s->lineno, s->col_offset,
s->end_lineno, s->end_col_offset))
Issue #5765: Merge from 3.3
2012-11-04 13:53:15 +00:00
VISIT_QUIT(st, 0);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
break;
}
case Nonlocal_kind: {
Fixes loop variables to be the same types as their limit (GH-120958)
2024-06-24 16:11:47 +00:00
Py_ssize_t i;
bpo-41746: Add type information to asdl_seq objects (GH-22223) * Add new capability to the PEG parser to type variable assignments. For instance: ``` | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a } ``` * Add new sequence types from the asdl definition (automatically generated) * Make `asdl_seq` type a generic aliasing pointer type. * Create a new `asdl_generic_seq` for the generic case using `void*`. * The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed. * New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences. * Changes all possible `asdl_seq` types to use specific versions everywhere.
2020-09-16 18:42:00 +00:00
asdl_identifier_seq *seq = s->v.Nonlocal.names;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
for (i = 0; i < asdl_seq_LEN(seq); i++) {
identifier name = (identifier)asdl_seq_GET(seq, i);
long cur = symtable_lookup(st, name);
if (cur < 0)
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
bpo-28936: Detect lexically first syntax error first (#4097) Lexically first global and nonlocal syntax errors at given scope should be detected first.
2017-10-26 21:28:35 +00:00
if (cur & (DEF_PARAM | DEF_LOCAL | USE | DEF_ANNOT)) {
const char* msg;
if (cur & DEF_PARAM) {
msg = NONLOCAL_PARAM;
} else if (cur & USE) {
Issue #28100: Refactor error messages, patch by Ivan Levkivskyi
2016-09-23 18:26:30 +00:00
msg = NONLOCAL_AFTER_USE;
} else if (cur & DEF_ANNOT) {
Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. Patch by Ivan Levkivskyi.
2016-09-09 16:36:26 +00:00
msg = NONLOCAL_ANNOT;
Issue #28100: Refactor error messages, patch by Ivan Levkivskyi
2016-09-23 18:26:30 +00:00
} else { /* DEF_LOCAL */
Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal. Patch by Ivan Levkivskyi.
2016-09-09 16:36:26 +00:00
msg = NONLOCAL_AFTER_ASSIGN;
}
PyErr_Format(PyExc_SyntaxError, msg, name);
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
PyErr_RangedSyntaxLocationObject(st->st_filename,
s->lineno,
s->col_offset + 1,
s->end_lineno,
s->end_col_offset + 1);
Issue #27985: Implement PEP 526 -- Syntax for Variable Annotations. Patch by Ivan Levkivskyi.
2016-09-09 03:50:03 +00:00
VISIT_QUIT(st, 0);
}
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_add_def(st, name, DEF_NONLOCAL, LOCATION(s)))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
if (!symtable_record_directive(st, name, s->lineno, s->col_offset,
s->end_lineno, s->end_col_offset))
Issue #5765: Merge from 3.3
2012-11-04 13:53:15 +00:00
VISIT_QUIT(st, 0);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
break;
}
case Expr_kind:
VISIT(st, expr, s->v.Expr.value);
break;
case Pass_kind:
case Break_kind:
case Continue_kind:
/* nothing to do here */
break;
case With_kind:
reflect with statements with multiple items in the AST (closes #12106)
2011-05-27 18:58:08 +00:00
VISIT_SEQ(st, withitem, s->v.With.items);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
VISIT_SEQ(st, stmt, s->v.With.body);
break;
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
case AsyncFunctionDef_kind: {
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_add_def(st, s->v.AsyncFunctionDef.name, DEF_LOCAL, LOCATION(s)))
PEP 0492 -- Coroutines with async and await syntax. Issue #24017.
2015-05-12 02:57:16 +00:00
VISIT_QUIT(st, 0);
if (s->v.AsyncFunctionDef.args->defaults)
VISIT_SEQ(st, expr, s->v.AsyncFunctionDef.args->defaults);
if (s->v.AsyncFunctionDef.args->kw_defaults)
VISIT_SEQ_WITH_NULL(st, expr,
s->v.AsyncFunctionDef.args->kw_defaults);
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
if (s->v.AsyncFunctionDef.decorator_list)
VISIT_SEQ(st, expr, s->v.AsyncFunctionDef.decorator_list);
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
if (asdl_seq_LEN(s->v.AsyncFunctionDef.type_params) > 0) {
if (!symtable_enter_type_param_block(
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
st, s->v.AsyncFunctionDef.name,
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
(void *)s->v.AsyncFunctionDef.type_params,
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
s->v.AsyncFunctionDef.args->defaults != NULL,
has_kwonlydefaults(s->v.AsyncFunctionDef.args->kwonlyargs,
s->v.AsyncFunctionDef.args->kw_defaults),
s->kind,
LOCATION(s))) {
VISIT_QUIT(st, 0);
}
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
VISIT_SEQ(st, type_param, s->v.AsyncFunctionDef.type_params);
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
}
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
PySTEntryObject *new_ste = ste_new(st, s->v.FunctionDef.name, FunctionBlock, (void *)s,
LOCATION(s));
if (!new_ste) {
VISIT_QUIT(st, 0);
}
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
if (!symtable_visit_annotations(st, s, s->v.AsyncFunctionDef.args,
gh-120385: Fix reference leak in symtable (#120386) Decref 'new_ste' if symtable_visit_annotations() fails.
2024-06-12 07:37:14 +00:00
s->v.AsyncFunctionDef.returns, new_ste)) {
Py_DECREF(new_ste);
PEP 0492 -- Coroutines with async and await syntax. Issue #24017.
2015-05-12 02:57:16 +00:00
VISIT_QUIT(st, 0);
gh-120385: Fix reference leak in symtable (#120386) Decref 'new_ste' if symtable_visit_annotations() fails.
2024-06-12 07:37:14 +00:00
}
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
if (!symtable_enter_existing_block(st, new_ste)) {
Py_DECREF(new_ste);
PEP 0492 -- Coroutines with async and await syntax. Issue #24017.
2015-05-12 02:57:16 +00:00
VISIT_QUIT(st, 0);
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
}
Py_DECREF(new_ste);
Issue #28003: Implement PEP 525 -- Asynchronous Generators.
2016-09-09 05:01:51 +00:00
st->st_cur->ste_coroutine = 1;
PEP 0492 -- Coroutines with async and await syntax. Issue #24017.
2015-05-12 02:57:16 +00:00
VISIT(st, arguments, s->v.AsyncFunctionDef.args);
VISIT_SEQ(st, stmt, s->v.AsyncFunctionDef.body);
closes bpo-39872: Remove unused args from symtable_exit_block and symtable_visit_annotations. (GH-18800)
2020-03-06 15:46:04 +00:00
if (!symtable_exit_block(st))
PEP 0492 -- Coroutines with async and await syntax. Issue #24017.
2015-05-12 02:57:16 +00:00
VISIT_QUIT(st, 0);
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
if (asdl_seq_LEN(s->v.AsyncFunctionDef.type_params) > 0) {
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
if (!symtable_exit_block(st))
VISIT_QUIT(st, 0);
}
PEP 0492 -- Coroutines with async and await syntax. Issue #24017.
2015-05-12 02:57:16 +00:00
break;
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
}
PEP 0492 -- Coroutines with async and await syntax. Issue #24017.
2015-05-12 02:57:16 +00:00
case AsyncWith_kind:
VISIT_SEQ(st, withitem, s->v.AsyncWith.items);
VISIT_SEQ(st, stmt, s->v.AsyncWith.body);
break;
case AsyncFor_kind:
VISIT(st, expr, s->v.AsyncFor.target);
VISIT(st, expr, s->v.AsyncFor.iter);
VISIT_SEQ(st, stmt, s->v.AsyncFor.body);
if (s->v.AsyncFor.orelse)
VISIT_SEQ(st, stmt, s->v.AsyncFor.orelse);
break;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 1);
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
static int
symtable_extend_namedexpr_scope(struct symtable *st, expr_ty e)
{
assert(st->st_stack);
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
assert(e->kind == Name_kind);
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
PyObject *target_name = e->v.Name.id;
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
Py_ssize_t i, size;
struct _symtable_entry *ste;
size = PyList_GET_SIZE(st->st_stack);
assert(size);
/* Iterate over the stack in reverse and add to the nearest adequate scope */
for (i = size - 1; i >= 0; i--) {
ste = (struct _symtable_entry *) PyList_GET_ITEM(st->st_stack, i);
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
/* If we find a comprehension scope, check for a target
* binding conflict with iteration variables, otherwise skip it
*/
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
if (ste->ste_comprehension) {
gh-96497: Mangle name before symtable lookup in 'symtable_extend_namedexpr_scope' (GH-96561)
2024-02-17 12:06:31 +00:00
long target_in_scope = symtable_lookup_entry(st, ste, target_name);
gh-87447: Fix walrus comprehension rebind checking (#100581) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2023-01-08 22:51:29 +00:00
if ((target_in_scope & DEF_COMP_ITER) &&
(target_in_scope & DEF_LOCAL)) {
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
PyErr_Format(PyExc_SyntaxError, NAMED_EXPR_COMP_CONFLICT, target_name);
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
PyErr_RangedSyntaxLocationObject(st->st_filename,
e->lineno,
e->col_offset + 1,
e->end_lineno,
e->end_col_offset + 1);
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
VISIT_QUIT(st, 0);
}
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
continue;
}
bpo-38469: Handle named expression scope with global/nonlocal keywords (GH-16755) The symbol table handing of PEP572's assignment expressions is not resolving correctly the scope of some variables in presence of global/nonlocal keywords in conjunction with comprehensions.
2019-10-14 04:18:05 +00:00
/* If we find a FunctionBlock entry, add as GLOBAL/LOCAL or NONLOCAL/LOCAL */
gh-104640: Disallow walrus in comprehension within type scopes (#104641)
2023-05-19 14:31:09 +00:00
if (ste->ste_type == FunctionBlock) {
gh-96497: Mangle name before symtable lookup in 'symtable_extend_namedexpr_scope' (GH-96561)
2024-02-17 12:06:31 +00:00
long target_in_scope = symtable_lookup_entry(st, ste, target_name);
bpo-38469: Handle named expression scope with global/nonlocal keywords (GH-16755) The symbol table handing of PEP572's assignment expressions is not resolving correctly the scope of some variables in presence of global/nonlocal keywords in conjunction with comprehensions.
2019-10-14 04:18:05 +00:00
if (target_in_scope & DEF_GLOBAL) {
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_add_def(st, target_name, DEF_GLOBAL, LOCATION(e)))
bpo-38469: Handle named expression scope with global/nonlocal keywords (GH-16755) The symbol table handing of PEP572's assignment expressions is not resolving correctly the scope of some variables in presence of global/nonlocal keywords in conjunction with comprehensions.
2019-10-14 04:18:05 +00:00
VISIT_QUIT(st, 0);
} else {
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_add_def(st, target_name, DEF_NONLOCAL, LOCATION(e)))
bpo-38469: Handle named expression scope with global/nonlocal keywords (GH-16755) The symbol table handing of PEP572's assignment expressions is not resolving correctly the scope of some variables in presence of global/nonlocal keywords in conjunction with comprehensions.
2019-10-14 04:18:05 +00:00
VISIT_QUIT(st, 0);
}
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_record_directive(st, target_name, LOCATION(e)))
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
VISIT_QUIT(st, 0);
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
return symtable_add_def_helper(st, target_name, DEF_LOCAL, ste, LOCATION(e));
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
}
/* If we find a ModuleBlock entry, add as GLOBAL */
if (ste->ste_type == ModuleBlock) {
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_add_def(st, target_name, DEF_GLOBAL, LOCATION(e)))
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
VISIT_QUIT(st, 0);
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_record_directive(st, target_name, LOCATION(e)))
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
VISIT_QUIT(st, 0);
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
return symtable_add_def_helper(st, target_name, DEF_GLOBAL, ste, LOCATION(e));
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
}
gh-104640: Disallow walrus in comprehension within type scopes (#104641)
2023-05-19 14:31:09 +00:00
/* Disallow usage in ClassBlock and type scopes */
if (ste->ste_type == ClassBlock ||
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
ste->ste_type == TypeParametersBlock ||
gh-104640: Disallow walrus in comprehension within type scopes (#104641)
2023-05-19 14:31:09 +00:00
ste->ste_type == TypeAliasBlock ||
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
ste->ste_type == TypeVariableBlock) {
gh-104640: Disallow walrus in comprehension within type scopes (#104641)
2023-05-19 14:31:09 +00:00
switch (ste->ste_type) {
case ClassBlock:
PyErr_Format(PyExc_SyntaxError, NAMED_EXPR_COMP_IN_CLASS);
break;
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
case TypeParametersBlock:
gh-104640: Disallow walrus in comprehension within type scopes (#104641)
2023-05-19 14:31:09 +00:00
PyErr_Format(PyExc_SyntaxError, NAMED_EXPR_COMP_IN_TYPEPARAM);
break;
case TypeAliasBlock:
PyErr_Format(PyExc_SyntaxError, NAMED_EXPR_COMP_IN_TYPEALIAS);
break;
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
case TypeVariableBlock:
gh-104640: Disallow walrus in comprehension within type scopes (#104641)
2023-05-19 14:31:09 +00:00
PyErr_Format(PyExc_SyntaxError, NAMED_EXPR_COMP_IN_TYPEVAR_BOUND);
break;
default:
Py_UNREACHABLE();
}
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
PyErr_RangedSyntaxLocationObject(st->st_filename,
e->lineno,
e->col_offset + 1,
e->end_lineno,
e->end_col_offset + 1);
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
VISIT_QUIT(st, 0);
}
}
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
/* We should always find either a function-like block, ModuleBlock or ClassBlock
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
and should never fall to this case
*/
gh-100577: Replace `assert(0)` with `Py_UNREACHABLE` in `symtable.c` (#100579)
2022-12-28 15:31:53 +00:00
Py_UNREACHABLE();
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
return 0;
}
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
static int
symtable_handle_namedexpr(struct symtable *st, expr_ty e)
{
if (st->st_cur->ste_comp_iter_expr > 0) {
/* Assignment isn't allowed in a comprehension iterable expression */
PyErr_Format(PyExc_SyntaxError, NAMED_EXPR_COMP_ITER_EXPR);
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
PyErr_RangedSyntaxLocationObject(st->st_filename,
e->lineno,
e->col_offset + 1,
e->end_lineno,
e->end_col_offset + 1);
bpo-37947: Avoid double-decrement in symtable recursion counting (GH-15593) With `symtable_visit_expr` now correctly adjusting the recursion depth for named expressions, `symtable_handle_namedexpr` should be leaving it alone. Also adds a new check to `PySymtable_BuildObject` that raises `SystemError` if a successful first symbol analysis pass fails to keep the stack depth accounting clean.
2019-08-29 13:26:53 +00:00
return 0;
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
}
if (st->st_cur->ste_comprehension) {
/* Inside a comprehension body, so find the right target scope */
if (!symtable_extend_namedexpr_scope(st, e->v.NamedExpr.target))
bpo-37947: Avoid double-decrement in symtable recursion counting (GH-15593) With `symtable_visit_expr` now correctly adjusting the recursion depth for named expressions, `symtable_handle_namedexpr` should be leaving it alone. Also adds a new check to `PySymtable_BuildObject` that raises `SystemError` if a successful first symbol analysis pass fails to keep the stack depth accounting clean.
2019-08-29 13:26:53 +00:00
return 0;
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
}
VISIT(st, expr, e->v.NamedExpr.value);
VISIT(st, expr, e->v.NamedExpr.target);
bpo-37947: Avoid double-decrement in symtable recursion counting (GH-15593) With `symtable_visit_expr` now correctly adjusting the recursion depth for named expressions, `symtable_handle_namedexpr` should be leaving it alone. Also adds a new check to `PySymtable_BuildObject` that raises `SystemError` if a successful first symbol analysis pass fails to keep the stack depth accounting clean.
2019-08-29 13:26:53 +00:00
return 1;
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
static int
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
symtable_visit_expr(struct symtable *st, expr_ty e)
{
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
if (++st->recursion_depth > st->recursion_limit) {
Issue #19235: Add new RecursionError exception. Patch by Georg Brandl.
2015-07-03 05:04:23 +00:00
PyErr_SetString(PyExc_RecursionError,
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
"maximum recursion depth exceeded during compilation");
VISIT_QUIT(st, 0);
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
switch (e->kind) {
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
case NamedExpr_kind:
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
if (!symtable_raise_if_annotation_block(st, "named expression", e)) {
VISIT_QUIT(st, 0);
}
bpo-37947: Adjust correctly the recursion level in symtable for named expressions (GH-15499)
2019-08-26 14:52:25 +00:00
if(!symtable_handle_namedexpr(st, e))
VISIT_QUIT(st, 0);
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
break;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
case BoolOp_kind:
VISIT_SEQ(st, expr, e->v.BoolOp.values);
break;
case BinOp_kind:
VISIT(st, expr, e->v.BinOp.left);
VISIT(st, expr, e->v.BinOp.right);
break;
case UnaryOp_kind:
VISIT(st, expr, e->v.UnaryOp.operand);
break;
case Lambda_kind: {
if (e->v.Lambda.args->defaults)
VISIT_SEQ(st, expr, e->v.Lambda.args->defaults);
Issue #13343: Fix a SystemError when a lambda expression uses a global variable in the default value of a keyword-only argument: (lambda *, arg=GLOBAL_NAME: None)
2011-11-04 21:17:45 +00:00
if (e->v.Lambda.args->kw_defaults)
PEP 448: additional unpacking generalizations (closes #2292) Patch by Neil Girdhar.
2015-05-06 00:16:41 +00:00
VISIT_SEQ_WITH_NULL(st, expr, e->v.Lambda.args->kw_defaults);
bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366) https://bugs.python.org/issue46765
2022-02-23 00:23:51 +00:00
if (!symtable_enter_block(st, &_Py_ID(lambda),
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
FunctionBlock, (void *)e,
e->lineno, e->col_offset,
e->end_lineno, e->end_col_offset))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
remove VISIT_*_IN_BLOCK macros These are pointless because on error, all blocks will be finalized by symtable_dealloc.
2011-06-29 20:27:14 +00:00
VISIT(st, arguments, e->v.Lambda.args);
VISIT(st, expr, e->v.Lambda.body);
closes bpo-39872: Remove unused args from symtable_exit_block and symtable_visit_annotations. (GH-18800)
2020-03-06 15:46:04 +00:00
if (!symtable_exit_block(st))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
break;
}
case IfExp_kind:
VISIT(st, expr, e->v.IfExp.test);
VISIT(st, expr, e->v.IfExp.body);
VISIT(st, expr, e->v.IfExp.orelse);
break;
case Dict_kind:
PEP 448: additional unpacking generalizations (closes #2292) Patch by Neil Girdhar.
2015-05-06 00:16:41 +00:00
VISIT_SEQ_WITH_NULL(st, expr, e->v.Dict.keys);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
VISIT_SEQ(st, expr, e->v.Dict.values);
break;
case Set_kind:
VISIT_SEQ(st, expr, e->v.Set.elts);
break;
case GeneratorExp_kind:
if (!symtable_visit_genexp(st, e))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
break;
case ListComp_kind:
if (!symtable_visit_listcomp(st, e))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
break;
case SetComp_kind:
if (!symtable_visit_setcomp(st, e))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
break;
case DictComp_kind:
if (!symtable_visit_dictcomp(st, e))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
break;
case Yield_kind:
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
if (!symtable_raise_if_annotation_block(st, "yield expression", e)) {
VISIT_QUIT(st, 0);
}
Issue #16546: make ast.YieldFrom argument mandatory.
2012-11-25 14:36:26 +00:00
if (e->v.Yield.value)
VISIT(st, expr, e->v.Yield.value);
st->st_cur->ste_generator = 1;
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (st->st_cur->ste_comprehension) {
return symtable_raise_if_comprehension_block(st, e);
}
Issue #16546: make ast.YieldFrom argument mandatory.
2012-11-25 14:36:26 +00:00
break;
case YieldFrom_kind:
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
if (!symtable_raise_if_annotation_block(st, "yield expression", e)) {
VISIT_QUIT(st, 0);
}
Issue #16546: make ast.YieldFrom argument mandatory.
2012-11-25 14:36:26 +00:00
VISIT(st, expr, e->v.YieldFrom.value);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
st->st_cur->ste_generator = 1;
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (st->st_cur->ste_comprehension) {
return symtable_raise_if_comprehension_block(st, e);
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
break;
PEP 0492 -- Coroutines with async and await syntax. Issue #24017.
2015-05-12 02:57:16 +00:00
case Await_kind:
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
if (!symtable_raise_if_annotation_block(st, "await expression", e)) {
VISIT_QUIT(st, 0);
}
PEP 0492 -- Coroutines with async and await syntax. Issue #24017.
2015-05-12 02:57:16 +00:00
VISIT(st, expr, e->v.Await.value);
Issue #28003: Implement PEP 525 -- Asynchronous Generators.
2016-09-09 05:01:51 +00:00
st->st_cur->ste_coroutine = 1;
PEP 0492 -- Coroutines with async and await syntax. Issue #24017.
2015-05-12 02:57:16 +00:00
break;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
case Compare_kind:
VISIT(st, expr, e->v.Compare.left);
VISIT_SEQ(st, expr, e->v.Compare.comparators);
break;
case Call_kind:
VISIT(st, expr, e->v.Call.func);
VISIT_SEQ(st, expr, e->v.Call.args);
PEP 448: additional unpacking generalizations (closes #2292) Patch by Neil Girdhar.
2015-05-06 00:16:41 +00:00
VISIT_SEQ_WITH_NULL(st, keyword, e->v.Call.keywords);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
break;
Issue #24965: Implement PEP 498 "Literal String Interpolation". Documentation is still needed, I'll open an issue for that.
2015-09-19 18:51:32 +00:00
case FormattedValue_kind:
VISIT(st, expr, e->v.FormattedValue.value);
if (e->v.FormattedValue.format_spec)
VISIT(st, expr, e->v.FormattedValue.format_spec);
break;
case JoinedStr_kind:
VISIT_SEQ(st, expr, e->v.JoinedStr.values);
break;
Add ast.Constant Issue #26146: Add a new kind of AST node: ast.Constant. It can be used by external AST optimizers, but the compiler does not emit directly such node. An optimizer can replace the following AST nodes with ast.Constant: * ast.NameConstant: None, False, True * ast.Num: int, float, complex * ast.Str: str * ast.Bytes: bytes * ast.Tuple if items are constants too: tuple * frozenset Update code to accept ast.Constant instead of ast.Num and/or ast.Str: * compiler * docstrings * ast.literal_eval() * Tools/parser/unparse.py
2016-01-25 23:40:57 +00:00
case Constant_kind:
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* Nothing to do here. */
break;
/* The following exprs can be assignment targets. */
case Attribute_kind:
VISIT(st, expr, e->v.Attribute.value);
break;
case Subscript_kind:
VISIT(st, expr, e->v.Subscript.value);
bpo-34822: Simplify AST for subscription. (GH-9605) * Remove the slice type. * Make Slice a kind of the expr type instead of the slice type. * Replace ExtSlice(slices) with Tuple(slices, Load()). * Replace Index(value) with a value itself. All non-terminal nodes in AST for expressions are now of the expr type.
2020-03-10 16:52:34 +00:00
VISIT(st, expr, e->v.Subscript.slice);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
break;
case Starred_kind:
VISIT(st, expr, e->v.Starred.value);
break;
bpo-34822: Simplify AST for subscription. (GH-9605) * Remove the slice type. * Make Slice a kind of the expr type instead of the slice type. * Replace ExtSlice(slices) with Tuple(slices, Load()). * Replace Index(value) with a value itself. All non-terminal nodes in AST for expressions are now of the expr type.
2020-03-10 16:52:34 +00:00
case Slice_kind:
if (e->v.Slice.lower)
gh-119981: Use do while(0) in some symtable.c multi-line macros (#119982)
2024-06-03 15:48:02 +00:00
VISIT(st, expr, e->v.Slice.lower);
bpo-34822: Simplify AST for subscription. (GH-9605) * Remove the slice type. * Make Slice a kind of the expr type instead of the slice type. * Replace ExtSlice(slices) with Tuple(slices, Load()). * Replace Index(value) with a value itself. All non-terminal nodes in AST for expressions are now of the expr type.
2020-03-10 16:52:34 +00:00
if (e->v.Slice.upper)
gh-119981: Use do while(0) in some symtable.c multi-line macros (#119982)
2024-06-03 15:48:02 +00:00
VISIT(st, expr, e->v.Slice.upper);
bpo-34822: Simplify AST for subscription. (GH-9605) * Remove the slice type. * Make Slice a kind of the expr type instead of the slice type. * Replace ExtSlice(slices) with Tuple(slices, Load()). * Replace Index(value) with a value itself. All non-terminal nodes in AST for expressions are now of the expr type.
2020-03-10 16:52:34 +00:00
if (e->v.Slice.step)
gh-119981: Use do while(0) in some symtable.c multi-line macros (#119982)
2024-06-03 15:48:02 +00:00
VISIT(st, expr, e->v.Slice.step);
bpo-34822: Simplify AST for subscription. (GH-9605) * Remove the slice type. * Make Slice a kind of the expr type instead of the slice type. * Replace ExtSlice(slices) with Tuple(slices, Load()). * Replace Index(value) with a value itself. All non-terminal nodes in AST for expressions are now of the expr type.
2020-03-10 16:52:34 +00:00
break;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
case Name_kind:
if (!symtable_add_def(st, e->v.Name.id,
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
e->v.Name.ctx == Load ? USE : DEF_LOCAL, LOCATION(e)))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* Special-case super: it counts as a use of __class__ */
if (e->v.Name.ctx == Load &&
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
_PyST_IsFunctionLike(st->st_cur) &&
Issue #28701: Replace PyUnicode_CompareWithASCIIString with _PyUnicode_EqualToASCIIString. The latter function is more readable, faster and doesn't raise exceptions.
2016-11-16 08:17:58 +00:00
_PyUnicode_EqualToASCIIString(e->v.Name.id, "super")) {
bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366) https://bugs.python.org/issue46765
2022-02-23 00:23:51 +00:00
if (!symtable_add_def(st, &_Py_ID(__class__), USE, LOCATION(e)))
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 0);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
break;
/* child nodes of List and Tuple will have expr_context set */
case List_kind:
VISIT_SEQ(st, expr, e->v.List.elts);
break;
case Tuple_kind:
VISIT_SEQ(st, expr, e->v.Tuple.elts);
break;
bpo-43892: Make match patterns explicit in the AST (GH-25585) Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2021-04-29 05:58:44 +00:00
}
VISIT_QUIT(st, 1);
}
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
static int
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
symtable_visit_type_param_bound_or_default(
struct symtable *st, expr_ty e, identifier name,
void *key, const char *ste_scope_info)
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
{
if (e) {
int is_in_class = st->st_cur->ste_can_see_class_scope;
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
if (!symtable_enter_block(st, name, TypeVariableBlock, key, LOCATION(e)))
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
return 0;
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
st->st_cur->ste_can_see_class_scope = is_in_class;
if (is_in_class && !symtable_add_def(st, &_Py_ID(__classdict__), USE, LOCATION(e))) {
VISIT_QUIT(st, 0);
}
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
assert(ste_scope_info != NULL);
st->st_cur->ste_scope_info = ste_scope_info;
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
VISIT(st, expr, e);
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
if (!symtable_exit_block(st)) {
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
return 0;
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
}
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
}
return 1;
}
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
static int
gh-104656: Rename typeparams AST node to type_params (#104657)
2023-05-22 04:25:09 +00:00
symtable_visit_type_param(struct symtable *st, type_param_ty tp)
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
{
if (++st->recursion_depth > st->recursion_limit) {
PyErr_SetString(PyExc_RecursionError,
"maximum recursion depth exceeded during compilation");
VISIT_QUIT(st, 0);
}
switch(tp->kind) {
case TypeVar_kind:
if (!symtable_add_def(st, tp->v.TypeVar.name, DEF_TYPE_PARAM | DEF_LOCAL, LOCATION(tp)))
VISIT_QUIT(st, 0);
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
const char *ste_scope_info = NULL;
const expr_ty bound = tp->v.TypeVar.bound;
if (bound != NULL) {
ste_scope_info = bound->kind == Tuple_kind ? "a TypeVar constraint" : "a TypeVar bound";
}
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
// We must use a different key for the bound and default. The obvious choice would be to
// use the .bound and .default_value pointers, but that fails when the expression immediately
// inside the bound or default is a comprehension: we would reuse the same key for
// the comprehension scope. Therefore, use the address + 1 as the second key.
// The only requirement for the key is that it is unique and it matches the logic in
// compile.c where the scope is retrieved.
if (!symtable_visit_type_param_bound_or_default(st, tp->v.TypeVar.bound, tp->v.TypeVar.name,
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
(void *)tp, ste_scope_info)) {
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
VISIT_QUIT(st, 0);
}
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
if (!symtable_visit_type_param_bound_or_default(st, tp->v.TypeVar.default_value, tp->v.TypeVar.name,
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
(void *)((uintptr_t)tp + 1), "a TypeVar default")) {
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
VISIT_QUIT(st, 0);
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
}
break;
case TypeVarTuple_kind:
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
if (!symtable_add_def(st, tp->v.TypeVarTuple.name, DEF_TYPE_PARAM | DEF_LOCAL, LOCATION(tp))) {
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
VISIT_QUIT(st, 0);
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
}
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
if (!symtable_visit_type_param_bound_or_default(st, tp->v.TypeVarTuple.default_value, tp->v.TypeVarTuple.name,
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
(void *)tp, "a TypeVarTuple default")) {
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
VISIT_QUIT(st, 0);
}
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
break;
case ParamSpec_kind:
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
if (!symtable_add_def(st, tp->v.ParamSpec.name, DEF_TYPE_PARAM | DEF_LOCAL, LOCATION(tp))) {
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
VISIT_QUIT(st, 0);
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
}
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
if (!symtable_visit_type_param_bound_or_default(st, tp->v.ParamSpec.default_value, tp->v.ParamSpec.name,
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
(void *)tp, "a ParamSpec default")) {
gh-116126: Implement PEP 696 (#116129) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 13:17:32 +00:00
VISIT_QUIT(st, 0);
}
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
break;
}
VISIT_QUIT(st, 1);
}
bpo-43892: Make match patterns explicit in the AST (GH-25585) Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2021-04-29 05:58:44 +00:00
static int
symtable_visit_pattern(struct symtable *st, pattern_ty p)
{
if (++st->recursion_depth > st->recursion_limit) {
PyErr_SetString(PyExc_RecursionError,
"maximum recursion depth exceeded during compilation");
VISIT_QUIT(st, 0);
}
switch (p->kind) {
case MatchValue_kind:
VISIT(st, expr, p->v.MatchValue.value);
break;
case MatchSingleton_kind:
/* Nothing to do here. */
break;
case MatchSequence_kind:
VISIT_SEQ(st, pattern, p->v.MatchSequence.patterns);
break;
case MatchStar_kind:
if (p->v.MatchStar.name) {
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
symtable_add_def(st, p->v.MatchStar.name, DEF_LOCAL, LOCATION(p));
bpo-43892: Make match patterns explicit in the AST (GH-25585) Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2021-04-29 05:58:44 +00:00
}
break;
case MatchMapping_kind:
VISIT_SEQ(st, expr, p->v.MatchMapping.keys);
VISIT_SEQ(st, pattern, p->v.MatchMapping.patterns);
if (p->v.MatchMapping.rest) {
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
symtable_add_def(st, p->v.MatchMapping.rest, DEF_LOCAL, LOCATION(p));
bpo-43892: Make match patterns explicit in the AST (GH-25585) Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2021-04-29 05:58:44 +00:00
}
break;
case MatchClass_kind:
VISIT(st, expr, p->v.MatchClass.cls);
VISIT_SEQ(st, pattern, p->v.MatchClass.patterns);
VISIT_SEQ(st, pattern, p->v.MatchClass.kwd_patterns);
break;
bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917) Co-authored-by: Guido van Rossum <guido@python.org> Co-authored-by: Talin <viridia@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-02-26 22:51:55 +00:00
case MatchAs_kind:
bpo-43892: Make match patterns explicit in the AST (GH-25585) Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2021-04-29 05:58:44 +00:00
if (p->v.MatchAs.pattern) {
VISIT(st, pattern, p->v.MatchAs.pattern);
}
if (p->v.MatchAs.name) {
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
symtable_add_def(st, p->v.MatchAs.name, DEF_LOCAL, LOCATION(p));
bpo-43892: Make match patterns explicit in the AST (GH-25585) Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2021-04-29 05:58:44 +00:00
}
bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917) Co-authored-by: Guido van Rossum <guido@python.org> Co-authored-by: Talin <viridia@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-02-26 22:51:55 +00:00
break;
case MatchOr_kind:
bpo-43892: Make match patterns explicit in the AST (GH-25585) Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2021-04-29 05:58:44 +00:00
VISIT_SEQ(st, pattern, p->v.MatchOr.patterns);
bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917) Co-authored-by: Guido van Rossum <guido@python.org> Co-authored-by: Talin <viridia@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-02-26 22:51:55 +00:00
break;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
Issue #5765: Apply a hard recursion limit in the compiler Previously, excessive nesting in expressions would blow the stack and segfault the interpreter. Now, a hard limit based on the configured recursion limit and a hardcoded scaling factor is applied.
2012-11-04 13:14:34 +00:00
VISIT_QUIT(st, 1);
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
static int
symtable_implicit_arg(struct symtable *st, int pos)
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
PyObject *id = PyUnicode_FromFormat(".%d", pos);
if (id == NULL)
return 0;
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_add_def(st, id, DEF_PARAM, ST_LOCATION(st->st_cur))) {
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Py_DECREF(id);
return 0;
}
Py_DECREF(id);
return 1;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
static int
bpo-41746: Add type information to asdl_seq objects (GH-22223) * Add new capability to the PEG parser to type variable assignments. For instance: ``` | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a } ``` * Add new sequence types from the asdl definition (automatically generated) * Make `asdl_seq` type a generic aliasing pointer type. * Create a new `asdl_generic_seq` for the generic case using `void*`. * The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed. * New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences. * Changes all possible `asdl_seq` types to use specific versions everywhere.
2020-09-16 18:42:00 +00:00
symtable_visit_params(struct symtable *st, asdl_arg_seq *args)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
Fixes loop variables to be the same types as their limit (GH-120958)
2024-06-24 16:11:47 +00:00
Py_ssize_t i;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (!args)
return -1;
for (i = 0; i < asdl_seq_LEN(args); i++) {
arg_ty arg = (arg_ty)asdl_seq_GET(args, i);
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_add_def(st, arg->arg, DEF_PARAM, LOCATION(arg)))
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return 0;
}
return 1;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
static int
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
symtable_visit_annotation(struct symtable *st, expr_ty annotation, void *key)
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
{
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
struct _symtable_entry *parent_ste = st->st_cur;
if (parent_ste->ste_annotation_block == NULL) {
_Py_block_ty current_type = parent_ste->ste_type;
if (!symtable_enter_block(st, &_Py_ID(__annotate__), AnnotationBlock,
key, LOCATION(annotation))) {
VISIT_QUIT(st, 0);
}
parent_ste->ste_annotation_block =
(struct _symtable_entry *)Py_NewRef(st->st_cur);
int future_annotations = st->st_future->ff_features & CO_FUTURE_ANNOTATIONS;
if (current_type == ClassBlock && !future_annotations) {
st->st_cur->ste_can_see_class_scope = 1;
if (!symtable_add_def(st, &_Py_ID(__classdict__), USE, LOCATION(annotation))) {
return 0;
}
}
_Py_DECLARE_STR(format, ".format");
// The generated __annotate__ function takes a single parameter with the
// internal name ".format".
if (!symtable_add_def(st, &_Py_STR(format), DEF_PARAM,
LOCATION(annotation))) {
return 0;
}
if (!symtable_add_def(st, &_Py_STR(format), USE,
LOCATION(annotation))) {
return 0;
}
}
else {
if (!symtable_enter_existing_block(st, parent_ste->ste_annotation_block)) {
VISIT_QUIT(st, 0);
}
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
}
VISIT(st, expr, annotation);
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
if (!symtable_exit_block(st)) {
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
VISIT_QUIT(st, 0);
}
return 1;
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
static int
bpo-41746: Add type information to asdl_seq objects (GH-22223) * Add new capability to the PEG parser to type variable assignments. For instance: ``` | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a } ``` * Add new sequence types from the asdl definition (automatically generated) * Make `asdl_seq` type a generic aliasing pointer type. * Create a new `asdl_generic_seq` for the generic case using `void*`. * The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed. * New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences. * Changes all possible `asdl_seq` types to use specific versions everywhere.
2020-09-16 18:42:00 +00:00
symtable_visit_argannotations(struct symtable *st, asdl_arg_seq *args)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
Fixes loop variables to be the same types as their limit (GH-120958)
2024-06-24 16:11:47 +00:00
Py_ssize_t i;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (!args)
return -1;
for (i = 0; i < asdl_seq_LEN(args); i++) {
arg_ty arg = (arg_ty)asdl_seq_GET(args, i);
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
if (arg->annotation) {
st->st_cur->ste_annotations_used = 1;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
VISIT(st, expr, arg->annotation);
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
return 1;
PEP 3107 - Function Annotations thanks to Tony Lownds
2006-12-28 06:47:50 +00:00
}
static int
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
symtable_visit_annotations(struct symtable *st, stmt_ty o, arguments_ty a, expr_ty returns,
struct _symtable_entry *function_ste)
PEP 3107 - Function Annotations thanks to Tony Lownds
2006-12-28 06:47:50 +00:00
{
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
int is_in_class = st->st_cur->ste_can_see_class_scope;
_Py_block_ty current_type = st->st_cur->ste_type;
if (!symtable_enter_block(st, &_Py_ID(__annotate__), AnnotationBlock,
(void *)a, LOCATION(o))) {
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
VISIT_QUIT(st, 0);
}
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
if (is_in_class || current_type == ClassBlock) {
st->st_cur->ste_can_see_class_scope = 1;
if (!symtable_add_def(st, &_Py_ID(__classdict__), USE, LOCATION(o))) {
return 0;
}
}
_Py_DECLARE_STR(format, ".format");
// We need to insert code that reads this "parameter" to the function.
if (!symtable_add_def(st, &_Py_STR(format), DEF_PARAM, LOCATION(o))) {
return 0;
}
if (!symtable_add_def(st, &_Py_STR(format), USE, LOCATION(o))) {
return 0;
}
Fix SystemError when nested function has annotation on positional-only argument (GH-17826)
2020-01-05 01:57:21 +00:00
if (a->posonlyargs && !symtable_visit_argannotations(st, a->posonlyargs))
return 0;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (a->args && !symtable_visit_argannotations(st, a->args))
return 0;
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
if (a->vararg && a->vararg->annotation) {
st->st_cur->ste_annotations_used = 1;
unify some ast.argument's attrs; change Attribute column offset (closes #16795) Patch from Sven Brauch.
2013-03-18 17:48:58 +00:00
VISIT(st, expr, a->vararg->annotation);
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
}
if (a->kwarg && a->kwarg->annotation) {
st->st_cur->ste_annotations_used = 1;
unify some ast.argument's attrs; change Attribute column offset (closes #16795) Patch from Sven Brauch.
2013-03-18 17:48:58 +00:00
VISIT(st, expr, a->kwarg->annotation);
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (a->kwonlyargs && !symtable_visit_argannotations(st, a->kwonlyargs))
return 0;
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
if (returns) {
st->st_cur->ste_annotations_used = 1;
VISIT(st, expr, returns);
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
}
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
if (!symtable_exit_block(st)) {
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
VISIT_QUIT(st, 0);
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return 1;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
static int
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
symtable_visit_arguments(struct symtable *st, arguments_ty a)
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* skip default arguments inside function block
XXX should ast be different?
*/
bpo-36540: PEP 570 -- Implementation (GH-12701) This commit contains the implementation of PEP570: Python positional-only parameters. * Update Grammar/Grammar with new typedarglist and varargslist * Regenerate grammar files * Update and regenerate AST related files * Update code object * Update marshal.c * Update compiler and symtable * Regenerate importlib files * Update callable objects * Implement positional-only args logic in ceval.c * Regenerate frozen data * Update standard library to account for positional-only args * Add test file for positional-only args * Update other test files to account for positional-only args * Add News entry * Update inspect module and related tests
2019-04-29 12:36:57 +00:00
if (a->posonlyargs && !symtable_visit_params(st, a->posonlyargs))
return 0;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (a->args && !symtable_visit_params(st, a->args))
return 0;
if (a->kwonlyargs && !symtable_visit_params(st, a->kwonlyargs))
return 0;
if (a->vararg) {
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_add_def(st, a->vararg->arg, DEF_PARAM, LOCATION(a->vararg)))
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return 0;
st->st_cur->ste_varargs = 1;
}
if (a->kwarg) {
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_add_def(st, a->kwarg->arg, DEF_PARAM, LOCATION(a->kwarg)))
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return 0;
st->st_cur->ste_varkeywords = 1;
}
return 1;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
static int
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
symtable_visit_excepthandler(struct symtable *st, excepthandler_ty eh)
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (eh->v.ExceptHandler.type)
VISIT(st, expr, eh->v.ExceptHandler.type);
if (eh->v.ExceptHandler.name)
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
if (!symtable_add_def(st, eh->v.ExceptHandler.name, DEF_LOCAL, LOCATION(eh)))
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return 0;
VISIT_SEQ(st, stmt, eh->v.ExceptHandler.body);
return 1;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
reflect with statements with multiple items in the AST (closes #12106)
2011-05-27 18:58:08 +00:00
static int
symtable_visit_withitem(struct symtable *st, withitem_ty item)
{
VISIT(st, expr, item->context_expr);
if (item->optional_vars) {
VISIT(st, expr, item->optional_vars);
}
return 1;
}
bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917) Co-authored-by: Guido van Rossum <guido@python.org> Co-authored-by: Talin <viridia@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-02-26 22:51:55 +00:00
static int
symtable_visit_match_case(struct symtable *st, match_case_ty m)
{
bpo-43892: Make match patterns explicit in the AST (GH-25585) Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2021-04-29 05:58:44 +00:00
VISIT(st, pattern, m->pattern);
bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917) Co-authored-by: Guido van Rossum <guido@python.org> Co-authored-by: Talin <viridia@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-02-26 22:51:55 +00:00
if (m->guard) {
VISIT(st, expr, m->guard);
}
VISIT_SEQ(st, stmt, m->body);
return 1;
}
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
static int
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
symtable_visit_alias(struct symtable *st, alias_ty a)
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* Compute store_name, the name actually bound by the import
Merged revisions 81906 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81906 | benjamin.peterson | 2010-06-11 16:40:37 -0500 (Fri, 11 Jun 2010) | 1 line different spellings are just unacceptable ........
2010-06-11 21:53:07 +00:00
operation. It is different than a->name when a->name is a
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
dotted package name (e.g. spam.eggs)
*/
PyObject *store_name;
PyObject *name = (a->asname == NULL) ? a->name : a->asname;
Implement PEP 393.
2011-09-28 05:41:54 +00:00
Py_ssize_t dot = PyUnicode_FindChar(name, '.', 0,
PyUnicode_GET_LENGTH(name), 1);
if (dot != -1) {
store_name = PyUnicode_Substring(name, 0, dot);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (!store_name)
return 0;
}
else {
gh-99300: Use Py_NewRef() in Python/ directory (#99302) Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and Py_XNewRef() in C files of the Python/ directory.
2022-11-10 08:03:39 +00:00
store_name = Py_NewRef(name);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
Issue #28701: Replace PyUnicode_CompareWithASCIIString with _PyUnicode_EqualToASCIIString. The latter function is more readable, faster and doesn't raise exceptions.
2016-11-16 08:17:58 +00:00
if (!_PyUnicode_EqualToASCIIString(name, "*")) {
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
int r = symtable_add_def(st, store_name, DEF_IMPORT, LOCATION(a));
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
Py_DECREF(store_name);
return r;
}
else {
if (st->st_cur->ste_type != ModuleBlock) {
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
int lineno = a->lineno;
int col_offset = a->col_offset;
int end_lineno = a->end_lineno;
int end_col_offset = a->end_col_offset;
fix indentation
2011-06-21 03:09:13 +00:00
PyErr_SetString(PyExc_SyntaxError, IMPORT_STAR_WARNING);
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
PyErr_RangedSyntaxLocationObject(st->st_filename,
lineno, col_offset + 1,
end_lineno, end_col_offset + 1);
fix indentation
2011-06-21 03:09:13 +00:00
Py_DECREF(store_name);
return 0;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
}
Py_DECREF(store_name);
return 1;
}
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
static int
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
symtable_visit_comprehension(struct symtable *st, comprehension_ty lc)
{
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
st->st_cur->ste_comp_iter_target = 1;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
VISIT(st, expr, lc->target);
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
st->st_cur->ste_comp_iter_target = 0;
st->st_cur->ste_comp_iter_expr++;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
VISIT(st, expr, lc->iter);
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
st->st_cur->ste_comp_iter_expr--;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
VISIT_SEQ(st, expr, lc->ifs);
Issue #28008: Implement PEP 530 -- asynchronous comprehensions.
2016-09-09 17:36:01 +00:00
if (lc->is_async) {
st->st_cur->ste_coroutine = 1;
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return 1;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
static int
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
symtable_visit_keyword(struct symtable *st, keyword_ty k)
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
VISIT(st, expr, k->value);
return 1;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
static int
Hide list comp variables and support set comprehensions
2007-04-15 12:05:43 +00:00
symtable_handle_comprehension(struct symtable *st, expr_ty e,
bpo-41746: Add type information to asdl_seq objects (GH-22223) * Add new capability to the PEG parser to type variable assignments. For instance: ``` | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a } ``` * Add new sequence types from the asdl definition (automatically generated) * Make `asdl_seq` type a generic aliasing pointer type. * Create a new `asdl_generic_seq` for the generic case using `void*`. * The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed. * New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences. * Changes all possible `asdl_seq` types to use specific versions everywhere.
2020-09-16 18:42:00 +00:00
identifier scope_name, asdl_comprehension_seq *generators,
Merged revisions 56154-56264 via svnmerge from svn+ssh://pythondev@svn.python.org/python/branches/p3yk ................ r56155 | neal.norwitz | 2007-07-03 08:59:08 +0300 (Tue, 03 Jul 2007) | 1 line Get this test working after converting map to return an iterator ................ r56202 | neal.norwitz | 2007-07-09 04:30:09 +0300 (Mon, 09 Jul 2007) | 37 lines Merged revisions 56124-56201 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r56129 | georg.brandl | 2007-06-30 04:01:01 -0700 (Sat, 30 Jun 2007) | 2 lines Document smtp.SMTPAuthenticationError. ........ r56137 | georg.brandl | 2007-07-01 01:11:35 -0700 (Sun, 01 Jul 2007) | 2 lines Fix a few webbrowser.py problems. ........ r56143 | georg.brandl | 2007-07-02 04:54:28 -0700 (Mon, 02 Jul 2007) | 2 lines Remove duplicate sentence from alarm() doc. ........ r56170 | mark.hammond | 2007-07-03 19:03:10 -0700 (Tue, 03 Jul 2007) | 3 lines copy built files to the PCBuild directory, where tools like distutils or external build processes can find them. ........ r56176 | kurt.kaiser | 2007-07-05 15:03:39 -0700 (Thu, 05 Jul 2007) | 10 lines Many calls to tk.call involve an arglist containing a single tuple. Calls using METH_OLDARGS unpack this tuple; calls using METH_VARARG don't. Tcl's concatenation of args was affected; IDLE doesn't start. Modify Tkapp_Call() to unpack single tuple arglists. Bug 1733943 Ref http://mail.python.org/pipermail/python-checkins/2007-May/060454.html ........ r56177 | neal.norwitz | 2007-07-05 21:13:39 -0700 (Thu, 05 Jul 2007) | 1 line Fix typo in comment ........ ................ r56251 | neal.norwitz | 2007-07-11 10:01:01 +0300 (Wed, 11 Jul 2007) | 1 line Get working with map returning an iterator (had to fix whitespace too) ................ r56255 | thomas.wouters | 2007-07-11 13:41:37 +0300 (Wed, 11 Jul 2007) | 6 lines Clean up merge glitch or copy-paste error (the entire module was duplicated, except the first half even had some more copy-paste errors, referring to listcomps and genexps instead of setcomps) ................ r56256 | thomas.wouters | 2007-07-11 15:16:01 +0300 (Wed, 11 Jul 2007) | 14 lines Dict comprehensions. Still needs doc changes (like many python-3000 features ;-). It generates bytecode similar to: x = {} for k, v in (generator here): x[k] = v except there is no tuple-packing and -unpacking involved. Trivial measurement suggests it's significantly faster than dict(generator here) (in the order of 2 to 3 times as fast) but I have not done extensive measurements. ................ r56263 | guido.van.rossum | 2007-07-11 15:36:26 +0300 (Wed, 11 Jul 2007) | 3 lines Patch 1724999 by Ali Gholami Rudi -- avoid complaints about dict size change during iter in destroy call. ................
2007-07-11 13:09:30 +00:00
expr_ty elt, expr_ty value)
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
{
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
int is_generator = (e->kind == GeneratorExp_kind);
comprehension_ty outermost = ((comprehension_ty)
asdl_seq_GET(generators, 0));
/* Outermost iterator is evaluated in current scope */
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
st->st_cur->ste_comp_iter_expr++;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
VISIT(st, expr, outermost->iter);
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
st->st_cur->ste_comp_iter_expr--;
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* Create comprehension scope for the rest */
if (!scope_name ||
add column offset to all syntax errors
2010-09-20 23:02:10 +00:00
!symtable_enter_block(st, scope_name, FunctionBlock, (void *)e,
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525) To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way: >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^ SyntaxError: Generator expression must be parenthesized becomes >>> foo(x, z for z in range(10), t, w) File "<stdin>", line 1 foo(x, z for z in range(10), t, w) ^^^^^^^^^^^^^^^^^^^^ SyntaxError: Generator expression must be parenthesized
2021-04-23 13:27:05 +00:00
e->lineno, e->col_offset,
e->end_lineno, e->end_col_offset)) {
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return 0;
}
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
switch(e->kind) {
case ListComp_kind:
st->st_cur->ste_comprehension = ListComprehension;
break;
case SetComp_kind:
st->st_cur->ste_comprehension = SetComprehension;
break;
case DictComp_kind:
st->st_cur->ste_comprehension = DictComprehension;
break;
default:
st->st_cur->ste_comprehension = GeneratorExpression;
break;
}
Issue #28008: Implement PEP 530 -- asynchronous comprehensions.
2016-09-09 17:36:01 +00:00
if (outermost->is_async) {
st->st_cur->ste_coroutine = 1;
}
bpo-35224: PEP 572 Implementation (#10497) * Add tokenization of := - Add token to Include/token.h. Add token to documentation in Doc/library/token.rst. - Run `./python Lib/token.py` to regenerate Lib/token.py. - Update Parser/tokenizer.c: add case to handle `:=`. * Add initial usage of := in grammar. * Update Python.asdl to match the grammar updates. Regenerated Include/Python-ast.h and Python/Python-ast.c * Update AST and compiler files in Python/ast.c and Python/compile.c. Basic functionality, this isn't scoped properly * Regenerate Lib/symbol.py using `./python Lib/symbol.py` * Tests - Fix failing tests in test_parser.py due to changes in token numbers for internal representation * Tests - Add simple test for := token * Tests - Add simple tests for named expressions using expr and suite * Tests - Update number of levels for nested expressions to prevent stack overflow * Update symbol table to handle NamedExpr * Update Grammar to allow assignment expressions in if statements. Regenerate Python/graminit.c accordingly using `make regen-grammar` * Tests - Add additional tests for named expressions in RoundtripLegalSyntaxTestCase, based on examples and information directly from PEP 572 Note: failing tests are currently commented out (4 out of 24 tests currently fail) * Tests - Add temporary syntax test failure tests in test_parser.py Note: There is an outstanding TODO for this -- syntax tests need to be moved to a different file (presumably test_syntax.py), but this is covering what needs to be tested at the moment, and it's more convenient to run a single test for the time being * Add support for allowing assignment expressions as function argument annotations. Uncomment tests for these cases because they all pass now! * Tests - Move existing syntax tests out of test_parser.py and into test_named_expressions.py. Refactor syntax tests to use unittest * Add TargetScopeError exception to extend SyntaxError Note: This simply creates the TargetScopeError exception, it is not yet used anywhere * Tests - Update tests per PEP 572 Continue refactoring test suite: The named expression test suite now checks for any invalid cases that throw exceptions (no longer limited to SyntaxErrors), assignment tests to ensure that variables are properly assigned, and scope tests to ensure that variable availability and values are correct Note: - There are still tests that are marked to skip, as they are not yet implemented - There are approximately 300 lines of the PEP that have not yet been addressed, though these may be deferred * Documentation - Small updates to XXX/todo comments - Remove XXX from child description in ast.c - Add comment with number of previously supported nested expressions for 3.7.X in test_parser.py * Fix assert in seq_for_testlist() * Cleanup - Denote "Not implemented -- No keyword args" on failing test case. Fix PEP8 error for blank lines at beginning of test classes in test_parser.py * Tests - Wrap all file opens in `with...as` to ensure files are closed * WIP: handle f(a := 1) * Tests and Cleanup - No longer skips keyword arg test. Keyword arg test now uses a simpler test case and does not rely on an external file. Remove print statements from ast.c * Tests - Refactor last remaining test case that relied on on external file to use a simpler test case without the dependency * Tests - Add better description of remaning skipped tests. Add test checking scope when using assignment expression in a function argument * Tests - Add test for nested comprehension, testing value and scope. Fix variable name in skipped comprehension scope test * Handle restriction of LHS for named expressions - can only assign to LHS of type NAME. Specifically, restrict assignment to tuples This adds an alternative set_context specifically for named expressions, set_namedexpr_context. Thus, context is now set differently for standard assignment versus assignment for named expressions in order to handle restrictions. * Tests - Update negative test case for assigning to lambda to match new error message. Add negative test case for assigning to tuple * Tests - Reorder test cases to group invalid syntax cases and named assignment target errors * Tests - Update test case for named expression in function argument - check that result and variable are set correctly * Todo - Add todo for TargetScopeError based on Guido's comment (https://github.com/python/cpython/commit/2b3acd37bdfc2d35e5094228c6684050d2aa8b0a#r30472562) * Tests - Add named expression tests for assignment operator in function arguments Note: One of two tests are skipped, as function arguments are currently treating an assignment expression inside of parenthesis as one child, which does not properly catch the named expression, nor does it count arguments properly * Add NamedStore to expr_context. Regenerate related code with `make regen-ast` * Add usage of NamedStore to ast_for_named_expr in ast.c. Update occurances of checking for Store to also handle NamedStore where appropriate * Add ste_comprehension to _symtable_entry to track if the namespace is a comprehension. Initialize ste_comprehension to 0. Set set_comprehension to 1 in symtable_handle_comprehension * s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to handle grabbing st->st_cur and passing it to symtable_add_def_helper. This now allows us to call the original code from symtable_add_def by instead calling symtable_add_def_helper with a different ste. * Refactor symtable_record_directive to take lineno and col_offset as arguments instead of stmt_ty. This allows symtable_record_directive to be used for stmt_ty and expr_ty * Handle elevating scope for named expressions in comprehensions. * Handle error for usage of named expression inside a class block * Tests - No longer skip scope tests. Add additional scope tests * Cleanup - Update error message for named expression within a comprehension within a class. Update comments. Add assert for symtable_extend_namedexpr_scope to validate that we always find at least a ModuleScope if we don't find a Class or FunctionScope * Cleanup - Add missing case for NamedStore in expr_context_name. Remove unused var in set_namedexpr_content * Refactor - Consolidate set_context and set_namedexpr_context to reduce duplicated code. Special cases for named expressions are handled by checking if ctx is NamedStore * Cleanup - Add additional use cases for ast_for_namedexpr in usage comment. Fix multiple blank lines in test_named_expressions * Tests - Remove unnecessary test case. Renumber test case function names * Remove TargetScopeError for now. Will add back if needed * Cleanup - Small comment nit for consistency * Handle positional argument check with named expression * Add TargetScopeError exception definition. Add documentation for TargetScopeError in c-api docs. Throw TargetScopeError instead of SyntaxError when using a named expression in a comprehension within a class scope * Increase stack size for parser by 200. This is a minimal change (approx. 5kb) and should not have an impact on any systems. Update parser test to allow 99 nested levels again * Add TargetScopeError to exception_hierarchy.txt for test_baseexception.py_ * Tests - Major update for named expression tests, both in test_named_expressions and test_parser - Add test for TargetScopeError - Add tests for named expressions in comprehension scope and edge cases - Add tests for named expressions in function arguments (declarations and call sites) - Reorganize tests to group them more logically * Cleanup - Remove unnecessary comment * Cleanup - Comment nitpicks * Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0) - Add check for LHS types to detect a parenthesis then a name (see note) - Add test for this scenario - Update tests for changed error message for named assignment to a tuple (also, see note) Note: This caused issues with the previous error handling for named assignment to a LHS that contained an expression, such as a tuple. Thus, the check for the LHS of a named expression must be changed to be more specific if we wish to maintain the previous error messages * Cleanup - Wrap lines more strictly in test file * Revert "Explicitly disallow assignment expressions to a name inside parentheses, e.g.: ((x) := 0)" This reverts commit f1531400ca7d7a2d148830c8ac703f041740896d. * Add NEWS.d entry * Tests - Fix error in test_pickle.test_exceptions by adding TargetScopeError to list of exceptions * Tests - Update error message tests to reflect improved messaging convention (s/can't/cannot) * Remove cases that cannot be reached in compile.c. Small linting update. * Update Grammar/Tokens to add COLONEQUAL. Regenerate all files * Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefully left out when fixing rebase conflicts * Add NamedStore back and regenerate files * Pass along line number and end col info for named expression * Simplify News entry * Fix compiler warning and explicity mark fallthrough
2019-01-24 23:49:56 +00:00
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
/* Outermost iter is received as an argument */
if (!symtable_implicit_arg(st, 0)) {
closes bpo-39872: Remove unused args from symtable_exit_block and symtable_visit_annotations. (GH-18800)
2020-03-06 15:46:04 +00:00
symtable_exit_block(st);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
return 0;
}
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
/* Visit iteration variable target, and mark them as such */
st->st_cur->ste_comp_iter_target = 1;
remove VISIT_*_IN_BLOCK macros These are pointless because on error, all blocks will be finalized by symtable_dealloc.
2011-06-29 20:27:14 +00:00
VISIT(st, expr, outermost->target);
bpo-37757: Disallow PEP 572 cases that expose implementation details (GH-15131) - drop TargetScopeError in favour of raising SyntaxError directly as per the updated PEP 572 - comprehension iteration variables are explicitly local, but named expression targets in comprehensions are nonlocal or global. Raise SyntaxError as specified in PEP 572 - named expression targets in the outermost iterable of a comprehension have an ambiguous target scope. Avoid resolving that question now by raising SyntaxError. PEP 572 originally required this only for cases where the bound name conflicts with the iteration variable in the comprehension, but CPython can't easily restrict the exception to that case (as it doesn't know the target variable names when visiting the outermost iterator expression)
2019-08-25 13:45:40 +00:00
st->st_cur->ste_comp_iter_target = 0;
/* Visit the rest of the comprehension body */
remove VISIT_*_IN_BLOCK macros These are pointless because on error, all blocks will be finalized by symtable_dealloc.
2011-06-29 20:27:14 +00:00
VISIT_SEQ(st, expr, outermost->ifs);
VISIT_SEQ_TAIL(st, comprehension, generators, 1);
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
if (value)
remove VISIT_*_IN_BLOCK macros These are pointless because on error, all blocks will be finalized by symtable_dealloc.
2011-06-29 20:27:14 +00:00
VISIT(st, expr, value);
VISIT(st, expr, elt);
bpo-10544: Disallow "yield" in comprehensions and generator expressions. (GH-4564)
2018-02-04 08:53:48 +00:00
st->st_cur->ste_generator = is_generator;
bpo-33346: Allow async comprehensions inside implicit async comprehensions (GH-6766) Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-07-13 21:27:50 +00:00
int is_async = st->st_cur->ste_coroutine && !is_generator;
if (!symtable_exit_block(st)) {
return 0;
}
if (is_async) {
st->st_cur->ste_coroutine = 1;
}
return 1;
Merge ast-branch to head This change implements a new bytecode compiler, based on a transformation of the parse tree to an abstract syntax defined in Parser/Python.asdl. The compiler implementation is not complete, but it is in stable enough shape to run the entire test suite excepting two disabled tests.
2005-10-20 19:59:25 +00:00
}
Hide list comp variables and support set comprehensions
2007-04-15 12:05:43 +00:00
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
static int
Hide list comp variables and support set comprehensions
2007-04-15 12:05:43 +00:00
symtable_visit_genexp(struct symtable *st, expr_ty e)
{
bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366) https://bugs.python.org/issue46765
2022-02-23 00:23:51 +00:00
return symtable_handle_comprehension(st, e, &_Py_ID(genexpr),
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
e->v.GeneratorExp.generators,
e->v.GeneratorExp.elt, NULL);
Hide list comp variables and support set comprehensions
2007-04-15 12:05:43 +00:00
}
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
static int
Hide list comp variables and support set comprehensions
2007-04-15 12:05:43 +00:00
symtable_visit_listcomp(struct symtable *st, expr_ty e)
{
bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366) https://bugs.python.org/issue46765
2022-02-23 00:23:51 +00:00
return symtable_handle_comprehension(st, e, &_Py_ID(listcomp),
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
e->v.ListComp.generators,
e->v.ListComp.elt, NULL);
Hide list comp variables and support set comprehensions
2007-04-15 12:05:43 +00:00
}
static int
symtable_visit_setcomp(struct symtable *st, expr_ty e)
{
bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366) https://bugs.python.org/issue46765
2022-02-23 00:23:51 +00:00
return symtable_handle_comprehension(st, e, &_Py_ID(setcomp),
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
e->v.SetComp.generators,
e->v.SetComp.elt, NULL);
Merged revisions 56154-56264 via svnmerge from svn+ssh://pythondev@svn.python.org/python/branches/p3yk ................ r56155 | neal.norwitz | 2007-07-03 08:59:08 +0300 (Tue, 03 Jul 2007) | 1 line Get this test working after converting map to return an iterator ................ r56202 | neal.norwitz | 2007-07-09 04:30:09 +0300 (Mon, 09 Jul 2007) | 37 lines Merged revisions 56124-56201 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r56129 | georg.brandl | 2007-06-30 04:01:01 -0700 (Sat, 30 Jun 2007) | 2 lines Document smtp.SMTPAuthenticationError. ........ r56137 | georg.brandl | 2007-07-01 01:11:35 -0700 (Sun, 01 Jul 2007) | 2 lines Fix a few webbrowser.py problems. ........ r56143 | georg.brandl | 2007-07-02 04:54:28 -0700 (Mon, 02 Jul 2007) | 2 lines Remove duplicate sentence from alarm() doc. ........ r56170 | mark.hammond | 2007-07-03 19:03:10 -0700 (Tue, 03 Jul 2007) | 3 lines copy built files to the PCBuild directory, where tools like distutils or external build processes can find them. ........ r56176 | kurt.kaiser | 2007-07-05 15:03:39 -0700 (Thu, 05 Jul 2007) | 10 lines Many calls to tk.call involve an arglist containing a single tuple. Calls using METH_OLDARGS unpack this tuple; calls using METH_VARARG don't. Tcl's concatenation of args was affected; IDLE doesn't start. Modify Tkapp_Call() to unpack single tuple arglists. Bug 1733943 Ref http://mail.python.org/pipermail/python-checkins/2007-May/060454.html ........ r56177 | neal.norwitz | 2007-07-05 21:13:39 -0700 (Thu, 05 Jul 2007) | 1 line Fix typo in comment ........ ................ r56251 | neal.norwitz | 2007-07-11 10:01:01 +0300 (Wed, 11 Jul 2007) | 1 line Get working with map returning an iterator (had to fix whitespace too) ................ r56255 | thomas.wouters | 2007-07-11 13:41:37 +0300 (Wed, 11 Jul 2007) | 6 lines Clean up merge glitch or copy-paste error (the entire module was duplicated, except the first half even had some more copy-paste errors, referring to listcomps and genexps instead of setcomps) ................ r56256 | thomas.wouters | 2007-07-11 15:16:01 +0300 (Wed, 11 Jul 2007) | 14 lines Dict comprehensions. Still needs doc changes (like many python-3000 features ;-). It generates bytecode similar to: x = {} for k, v in (generator here): x[k] = v except there is no tuple-packing and -unpacking involved. Trivial measurement suggests it's significantly faster than dict(generator here) (in the order of 2 to 3 times as fast) but I have not done extensive measurements. ................ r56263 | guido.van.rossum | 2007-07-11 15:36:26 +0300 (Wed, 11 Jul 2007) | 3 lines Patch 1724999 by Ali Gholami Rudi -- avoid complaints about dict size change during iter in destroy call. ................
2007-07-11 13:09:30 +00:00
}
static int
symtable_visit_dictcomp(struct symtable *st, expr_ty e)
{
bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366) https://bugs.python.org/issue46765
2022-02-23 00:23:51 +00:00
return symtable_handle_comprehension(st, e, &_Py_ID(dictcomp),
Recorded merge of revisions 81029 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
2010-05-09 15:52:27 +00:00
e->v.DictComp.generators,
e->v.DictComp.key,
e->v.DictComp.value);
Hide list comp variables and support set comprehensions
2007-04-15 12:05:43 +00:00
}
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
static int
symtable_raise_if_annotation_block(struct symtable *st, const char *name, expr_ty e)
{
gh-119180: PEP 649 compiler changes (#119361)
2024-06-11 13:06:49 +00:00
_Py_block_ty type = st->st_cur->ste_type;
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
if (type == AnnotationBlock)
PyErr_Format(PyExc_SyntaxError, ANNOTATION_NOT_ALLOWED, name);
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 13:51:03 +00:00
else if (type == TypeVariableBlock) {
const char *info = st->st_cur->ste_scope_info;
assert(info != NULL); // e.g., info == "a ParamSpec default"
PyErr_Format(PyExc_SyntaxError, EXPR_NOT_ALLOWED_IN_TYPE_VARIABLE, name, info);
}
else if (type == TypeAliasBlock) {
// for now, we do not have any extra information
assert(st->st_cur->ste_scope_info == NULL);
PyErr_Format(PyExc_SyntaxError, EXPR_NOT_ALLOWED_IN_TYPE_ALIAS, name);
}
else if (type == TypeParametersBlock) {
// for now, we do not have any extra information
assert(st->st_cur->ste_scope_info == NULL);
PyErr_Format(PyExc_SyntaxError, EXPR_NOT_ALLOWED_IN_TYPE_PARAMETERS, name);
}
gh-103763: Implement PEP 695 (#103764) This implements PEP 695, Type Parameter Syntax. It adds support for: - Generic functions (def func[T](): ...) - Generic classes (class X[T](): ...) - Type aliases (type X = ...) - New scoping when the new syntax is used within a class body - Compiler and interpreter changes to support the new syntax and scoping rules Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Co-authored-by: Eric Traut <eric@traut.com> Co-authored-by: Larry Hastings <larry@hastings.org> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-16 03:36:23 +00:00
else
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
return 1;
PyErr_RangedSyntaxLocationObject(st->st_filename,
e->lineno,
e->col_offset + 1,
e->end_lineno,
e->end_col_offset + 1);
return 0;
}
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
static int
symtable_raise_if_comprehension_block(struct symtable *st, expr_ty e) {
_Py_comprehension_ty type = st->st_cur->ste_comprehension;
bpo-46417: Clear symtable identifiers at exit (GH-30809) Add _PySymtable_Fini() function, called by finalize_interp_clear(). Update test_cmd_line.test_showrefcount() to tolerate negative reference count.
2022-01-22 23:06:56 +00:00
PyErr_SetString(PyExc_SyntaxError,
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059)
2021-12-11 21:28:24 +00:00
(type == ListComprehension) ? "'yield' inside list comprehension" :
(type == SetComprehension) ? "'yield' inside set comprehension" :
(type == DictComprehension) ? "'yield' inside dict comprehension" :
"'yield' inside generator expression");
PyErr_RangedSyntaxLocationObject(st->st_filename,
e->lineno, e->col_offset + 1,
e->end_lineno, e->end_col_offset + 1);
VISIT_QUIT(st, 0);
}
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
struct symtable *
_Py_SymtableStringObjectFlags(const char *str, PyObject *filename,
int start, PyCompilerFlags *flags)
{
struct symtable *st;
mod_ty mod;
PyArena *arena;
bpo-43244: Remove the pyarena.h header (GH-25007) Remove the pyarena.h header file with functions: * PyArena_New() * PyArena_Free() * PyArena_Malloc() * PyArena_AddPyObject() These functions were undocumented, excluded from the limited C API, and were only used internally by the compiler. Add pycore_pyarena.h header. Rename functions: * PyArena_New() => _PyArena_New() * PyArena_Free() => _PyArena_Free() * PyArena_Malloc() => _PyArena_Malloc() * PyArena_AddPyObject() => _PyArena_AddPyObject()
2021-03-24 01:23:01 +00:00
arena = _PyArena_New();
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
if (arena == NULL)
return NULL;
bpo-43244: Remove parser_interface.h header file (GH-25001) Remove parser functions using the "struct _mod" type, because the AST C API was removed: * PyParser_ASTFromFile() * PyParser_ASTFromFileObject() * PyParser_ASTFromFilename() * PyParser_ASTFromString() * PyParser_ASTFromStringObject() These functions were undocumented and excluded from the limited C API. Add pycore_parser.h internal header file. Rename functions: * PyParser_ASTFromFileObject() => _PyParser_ASTFromFile() * PyParser_ASTFromStringObject() => _PyParser_ASTFromString() These functions are no longer exported (replace PyAPI_FUNC() with extern). Remove also _PyPegen_run_parser_from_file() function. Update test_peg_generator to use _PyPegen_run_parser_from_file_pointer() instead.
2021-03-24 00:29:09 +00:00
mod = _PyParser_ASTFromString(str, filename, start, flags, arena);
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
if (mod == NULL) {
bpo-43244: Remove the pyarena.h header (GH-25007) Remove the pyarena.h header file with functions: * PyArena_New() * PyArena_Free() * PyArena_Malloc() * PyArena_AddPyObject() These functions were undocumented, excluded from the limited C API, and were only used internally by the compiler. Add pycore_pyarena.h header. Rename functions: * PyArena_New() => _PyArena_New() * PyArena_Free() => _PyArena_Free() * PyArena_Malloc() => _PyArena_Malloc() * PyArena_AddPyObject() => _PyArena_AddPyObject()
2021-03-24 01:23:01 +00:00
_PyArena_Free(arena);
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
return NULL;
}
gh-117411: move PyFutureFeatures to pycore_symtable.h and make it private (#117412)
2024-04-02 10:34:49 +00:00
_PyFutureFeatures future;
gh-87092: do not allocate PyFutureFeatures dynamically (GH-98913)
2022-11-02 15:13:07 +00:00
if (!_PyFuture_FromAST(mod, filename, &future)) {
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583)
2021-05-03 07:43:00 +00:00
_PyArena_Free(arena);
return NULL;
}
gh-87092: do not allocate PyFutureFeatures dynamically (GH-98913)
2022-11-02 15:13:07 +00:00
future.ff_features |= flags->cf_flags;
st = _PySymtable_Build(mod, filename, &future);
bpo-43244: Remove the pyarena.h header (GH-25007) Remove the pyarena.h header file with functions: * PyArena_New() * PyArena_Free() * PyArena_Malloc() * PyArena_AddPyObject() These functions were undocumented, excluded from the limited C API, and were only used internally by the compiler. Add pycore_pyarena.h header. Rename functions: * PyArena_New() => _PyArena_New() * PyArena_Free() => _PyArena_Free() * PyArena_Malloc() => _PyArena_Malloc() * PyArena_AddPyObject() => _PyArena_AddPyObject()
2021-03-24 01:23:01 +00:00
_PyArena_Free(arena);
bpo-43244: Remove symtable.h header file (GH-24910) Rename Include/symtable.h to to Include/internal/pycore_symtable.h, don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with extern) and rename functions: * PyST_GetScope() to _PyST_GetScope() * PySymtable_BuildObject() to _PySymtable_Build() * PySymtable_Free() to _PySymtable_Free() Remove PySymtable_Build(), Py_SymtableString() and Py_SymtableStringObject() functions. The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, since the symtable.h header file was excluded from the limited C API. The Python symtable module remains available and is unchanged.
2021-03-19 11:41:49 +00:00
return st;
}
gh-102371: move _Py_Mangle from compile.c to symtable.c (#102372)
2023-03-02 18:38:22 +00:00
gh-119311: Fix name mangling with PEP 695 generic classes (#119464) Fixes #119311. Fixes #119395.
2024-05-28 02:50:38 +00:00
PyObject *
_Py_MaybeMangle(PyObject *privateobj, PySTEntryObject *ste, PyObject *name)
{
/* Special case for type parameter blocks around generic classes:
* we want to mangle type parameter names (so a type param with a private
* name can be used inside the class body), but we don't want to mangle
* any other names that appear within the type parameter scope.
*/
if (ste->ste_mangled_names != NULL) {
int result = PySet_Contains(ste->ste_mangled_names, name);
if (result < 0) {
return NULL;
}
if (result == 0) {
return Py_NewRef(name);
}
}
return _Py_Mangle(privateobj, name);
}
gh-102371: move _Py_Mangle from compile.c to symtable.c (#102372)
2023-03-02 18:38:22 +00:00
PyObject *
_Py_Mangle(PyObject *privateobj, PyObject *ident)
{
/* Name mangling: __private becomes _classname__private.
This is independent from how the name is used. */
if (privateobj == NULL || !PyUnicode_Check(privateobj) ||
PyUnicode_READ_CHAR(ident, 0) != '_' ||
PyUnicode_READ_CHAR(ident, 1) != '_') {
return Py_NewRef(ident);
}
size_t nlen = PyUnicode_GET_LENGTH(ident);
size_t plen = PyUnicode_GET_LENGTH(privateobj);
/* Don't mangle __id__ or names with dots.
The only time a name with a dot can occur is when
we are compiling an import statement that has a
package name.
TODO(jhylton): Decide whether we want to support
mangling of the module name, e.g. __M.X.
*/
if ((PyUnicode_READ_CHAR(ident, nlen-1) == '_' &&
PyUnicode_READ_CHAR(ident, nlen-2) == '_') ||
PyUnicode_FindChar(ident, '.', 0, nlen, 1) != -1) {
return Py_NewRef(ident); /* Don't mangle __whatever__ */
}
/* Strip leading underscores from class name */
size_t ipriv = 0;
while (PyUnicode_READ_CHAR(privateobj, ipriv) == '_') {
ipriv++;
}
if (ipriv == plen) {
return Py_NewRef(ident); /* Don't mangle if class is just underscores */
}
plen -= ipriv;
if (plen + nlen >= PY_SSIZE_T_MAX - 1) {
PyErr_SetString(PyExc_OverflowError,
"private identifier too large to be mangled");
return NULL;
}
Py_UCS4 maxchar = PyUnicode_MAX_CHAR_VALUE(ident);
if (PyUnicode_MAX_CHAR_VALUE(privateobj) > maxchar) {
maxchar = PyUnicode_MAX_CHAR_VALUE(privateobj);
}
PyObject *result = PyUnicode_New(1 + nlen + plen, maxchar);
if (!result) {
return NULL;
}
/* ident = "_" + priv[ipriv:] + ident # i.e. 1+plen+nlen bytes */
PyUnicode_WRITE(PyUnicode_KIND(result), PyUnicode_DATA(result), 0, '_');
if (PyUnicode_CopyCharacters(result, 1, privateobj, ipriv, plen) < 0) {
Py_DECREF(result);
return NULL;
}
if (PyUnicode_CopyCharacters(result, plen+1, ident, 0, nlen) < 0) {
Py_DECREF(result);
return NULL;
}
assert(_PyUnicode_CheckConsistency(result, 1));
return result;
}
Reference in a new issue Copy permalink