Commit graph

693 commits

Author SHA1 Message Date
Zackery Spytz 5061a74a4c Remove unneeded PyUnicode_READY() in tokenizer.c (GH-9114) 2018-09-10 09:27:31 +03:00
Zackery Spytz 3e26e42c90 bpo-34400: Fix more undefined behavior in parsetok.c (GH-8833) 2018-08-20 20:11:40 -07:00
Zackery Spytz 7c4ab2afb1 closes bpo-34400: Fix undefined behavior in parsetok(). (GH-4439)
Avoid undefined pointer arithmetic with NULL.
2018-08-14 23:27:26 -07:00
Victor Stinner 53b7d4e402
bpo-34170: Add _PyCoreConfig.bytes_warning (GH-8447)
Add more fields to _PyCoreConfig:

* _check_hash_pycs_mode
* bytes_warning
* debug
* inspect
* interactive
* legacy_windows_fs_encoding
* legacy_windows_stdio
* optimization_level
* quiet
* unbuffered_stdio
* user_site_directory
* verbose
* write_bytecode

Changes:

* Remove pymain_get_global_config() and pymain_set_global_config()
  which became useless. These functions have been replaced by
  _PyCoreConfig_GetGlobalConfig() and
  _PyCoreConfig_SetGlobalConfig().
* sys.flags.dont_write_bytecode value is now restricted to 1 even if
  -B option is specified multiple times on the command line.
* PyThreadState_Clear() now uses the config from the current
  interpreter rather than using global Py_VerboseFlag
2018-07-25 01:37:05 +02:00
Serhiy Storchaka aba24ff360
bpo-34084: Fix setting an error message for the "Barry as BDFL" easter egg. (GH-8262) 2018-07-23 23:41:11 +03:00
Victor Stinner c884616390
Fix Windows compiler warning in tokenize.c (GH-8359)
Fix the following warning on Windows:

parser\tokenizer.c(1297): warning C4244: 'function': conversion from
'__int64' to 'int', possible loss of data.
2018-07-21 03:36:06 +02:00
ValeriyaSinevich ce75df3031 bpo-30237: Output error when ReadConsole is canceled by CancelSynchronousIo. (GH-7911) 2018-07-19 15:34:03 -07:00
Serhiy Storchaka cf7303ed2a
bpo-33305: Improve SyntaxError for invalid numerical literals. (GH-6517) 2018-07-09 15:09:35 +03:00
Thomas A Caswell 9b9d58f0d8 bpo-31546: Fix input hook integration (GH-7978) 2018-06-28 09:29:44 -07:00
Serhiy Storchaka a5c42284e6
bpo-33677: Fix signatures of tp_clear handlers for AST and deque. (GH-7196) 2018-05-31 07:34:34 +03:00
Serhiy Storchaka 73cbe7a01a
bpo-32911: Revert bpo-29463. (GH-7121) (GH-7197)
Remove the docstring attribute of AST types and restore docstring
expression as a first stmt in their body.

Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
2018-05-29 12:04:55 +03:00
Serhiy Storchaka f320be77ff bpo-32571: Avoid raising unneeded AttributeError and silencing it in C code (GH-5222)
Add two new private APIs: _PyObject_LookupAttr() and _PyObject_LookupAttrId()
2018-01-25 17:49:40 +09:00
Benjamin Peterson 0a37a30037
closes bpo-32460: ensure all non-static globals have initializers (#5061) 2017-12-31 10:04:13 -08:00
Serhiy Storchaka 598ceae876
bpo-32150: Expand tabs to spaces in C files. (#4583) 2017-11-28 17:56:10 +02:00
Victor Stinner 9e87e7776f
bpo-32096: Remove obj and mem from _PyRuntime (#4532)
bpo-32096, bpo-30860:  Partially revert the commit
2ebc5ce42a:

* Move structures back from Include/internal/mem.h to
  Objects/obmalloc.c
* Remove _PyObject_Initialize() and _PyMem_Initialize()
* Remove Include/internal/pymalloc.h
* Add test_capi.test_pre_initialization_api():
   Make sure that it's possible to call Py_DecodeLocale(), and then call
   Py_SetProgramName() with the decoded string, before Py_Initialize().

PyMem_RawMalloc() and Py_DecodeLocale() can be called again before
_PyRuntimeState_Init().

Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
2017-11-24 12:09:24 +01:00
Victor Stinner f2ddc6ac93
tokenizer: Remove unused tabs options (#4422)
Remove the following fields from tok_state structure which are now
used unused:

* altwarning: "Issue warning if alternate tabs don't match"
* alterror: "Issue error if alternate tabs don't match"
* alttabsize: "Alternate tab spacing"

Replace alttabsize variable with ALTTABSIZE define.
2017-11-17 01:25:47 -08:00
Victor Stinner f7e5b56c37
bpo-32030: Split Py_Main() into subfunctions (#4399)
* Don't use "Python runtime" anymore to parse command line options or
  to get environment variables: pymain_init() is now a strict
  separation.
* Use an error message rather than "crashing" directly with
  Py_FatalError(). Limit the number of calls to Py_FatalError(). It
  prepares the code to handle errors more nicely later.
* Warnings options (-W, PYTHONWARNINGS) and "XOptions" (-X) are now
  only added to the sys module once Python core is properly
  initialized.
* _PyMain is now the well identified owner of some important strings
  like: warnings options, XOptions, and the "program name". The
  program name string is now properly freed at exit.
  pymain_free() is now responsible to free the "command" string.
* Rename most methods in Modules/main.c to use a "pymain_" prefix to
  avoid conflits and ease debug.
* Replace _Py_CommandLineDetails_INIT with memset(0)
* Reorder a lot of code to fix the initialization ordering. For
  example, initializing standard streams now comes before parsing
  PYTHONWARNINGS.
* Py_Main() now handles errors when adding warnings options and
  XOptions.
* Add _PyMem_GetDefaultRawAllocator() private function.
* Cleanup _PyMem_Initialize(): remove useless global constants: move
  them into _PyMem_Initialize().
* Call _PyRuntime_Initialize() as soon as possible:
  _PyRuntime_Initialize() now returns an error message on failure.
* Add _PyInitError structure and following macros:

  * _Py_INIT_OK()
  * _Py_INIT_ERR(msg)
  * _Py_INIT_USER_ERR(msg): "user" error, don't abort() in that case
  * _Py_INIT_FAILED(err)
2017-11-15 15:48:08 -08:00
Serhiy Storchaka bba2239c17
bpo-31572: Get rid of _PyObject_HasAttrId() in the ASDL parser. (#3725)
Silence only expected AttributeError.
2017-11-11 16:41:32 +02:00
Jelle Zijlstra ac317700ce bpo-30406: Make async and await proper keywords (#1669)
Per PEP 492, 'async' and 'await' should become proper keywords in 3.7.
2017-10-05 23:24:46 -04:00
Antoine Pitrou b091bec824 bpo-31536: Avoid wholesale rebuild after make regen-all (#3678)
* bpo-31536: Avoid wholesale rebuild after `make regen-all`

* Add NEWS
2017-09-20 14:57:56 -07:00
Serhiy Storchaka 5d84cb368c bpo-31464: asdl_c.py no longer emits trailing spaces in Python-ast.h. (#3568) 2017-09-14 20:28:22 -07:00
Barry Warsaw b2e5794870 bpo-31338 (#3374)
* Add Py_UNREACHABLE() as an alias to abort().
* Use Py_UNREACHABLE() instead of assert(0)
* Convert more unreachable code to use Py_UNREACHABLE()
* Document Py_UNREACHABLE() and a few other macros.
2017-09-14 18:13:16 -07:00
Eric Snow 2ebc5ce42a bpo-30860: Consolidate stateful runtime globals. (#3397)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-07 23:51:28 -06:00
Antoine Pitrou a6a4dc816d bpo-31370: Remove support for threads-less builds (#3385)
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
2017-09-07 18:56:24 +02:00
Eric Snow 05351c1bd8 Revert "bpo-30860: Consolidate stateful runtime globals." (#3379)
Windows buildbots started failing due to include-related errors.
2017-09-05 21:43:08 -07:00
Eric Snow 76d5abc868 bpo-30860: Consolidate stateful runtime globals. (#2594)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-05 18:26:16 -07:00
INADA Naoki a6296d34a4 bpo-31095: fix potential crash during GC (GH-2974) 2017-08-24 14:55:17 +09:00
Yuan Chao Chou 2af565baf4 Fix a shadow-compatible-local warning (#2180)
Change the shadowing naming, 'value' (Python-ast.c:4652), to 'val'
to prevent the variables from being misused.
2017-08-04 10:53:12 -07:00
Albert-Jan Nijburg c9ccacea3f bpo-25324: add missing comma in Parser/tokenizer.c (GH-1910) 2017-06-01 13:51:27 -07:00
Albert-Jan Nijburg fc354f0785 bpo-25324: copy tok_name before changing it (#1608)
* add test to check if were modifying token

* copy list so import tokenize doesnt have side effects on token

* shorten line

* add tokenize tokens to token.h to get them to show up in token

* move ERRORTOKEN back to its previous location, and fix nitpick

* copy comments from token.h automatically

* fix whitespace and make more pythonic

* change to fix comments from @haypo

* update token.rst and Misc/NEWS

* change wording

* some more wording changes
2017-05-31 16:00:21 +02:00
Jim Fasarakis-Hilliard cf1958af4c Remove obsolete declaration in tokenizer.h (#962) 2017-04-03 19:18:32 +03:00
Serhiy Storchaka 0b3ec19225 Use NULL rather than 0. (#778)
There was few cases of using literal 0 instead of NULL in the context of
pointers.  While this was a legitimate C code, using NULL rather than 0 makes
the code clearer.
2017-03-23 17:53:47 +02:00
INADA Naoki 4c78c527d2 bpo-29622: Make AST constructor to accept less than enough number of positional arguments (GH-249)
bpo-29463 added optional "docstring" field to 4 AST types.
While it is optional, it breaks backward compatibility because AST constructor
requires number of positional argument is same to number of fields.

AST types accepts empty arguments, and incomplete keyword arguments.
But it's not big problem because field can be filled after creation, and checked when compiling.
So stop requiring complete set of fields for positional arguments too.
2017-02-24 02:48:17 +09:00
INADA Naoki cb41b2766d bpo-29463: Add docstring field to some AST nodes. (#46)
* bpo-29463: Add docstring field to some AST nodes.

ClassDef, ModuleDef, FunctionDef, and AsyncFunctionDef has docstring
field for now.  It was first statement of there body.

* fix document.  thanks travis!

* doc fixes
2017-02-22 16:31:59 +01:00
Berker Peksag d2f4404bbb Issue #28489: Merge from 3.6 2017-02-05 04:33:11 +03:00
Berker Peksag 6f80562862 Issue #28489: Fix comment in tokenizer.c
Patch by Ryan Gonzalez.
2017-02-05 04:32:39 +03:00
INADA Naoki fc489082c8 Issue #29369: Use Py_IDENTIFIER in Python-ast.c 2017-01-25 22:33:43 +09:00
Victor Stinner a5ed5f000a Use _PyObject_CallNoArg()
Replace:
    PyObject_CallObject(callable, NULL)
with:
    _PyObject_CallNoArg(callable)
2016-12-06 18:45:50 +01:00
Serhiy Storchaka 06515833fe Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.
2016-11-20 09:13:07 +02:00
Steve Dower 6c2b9d3479 Issue #28333: Fixes off-by-one error that was adding an extra space. 2016-10-25 11:51:54 -07:00
Steve Dower 59bd34fa8a Issue #28333: Remove unnecessary increment. 2016-10-08 12:20:45 -07:00
Steve Dower 3cd187b9f5 Issue #28333: Enables Unicode for ps1/ps2 and input() prompts. (Patch by Eryk Sun) 2016-10-08 12:18:16 -07:00
Serhiy Storchaka 5e80855af3 Issue #24098: Fixed possible crash when AST is changed in process of
compiling it.
2016-10-07 21:55:49 +03:00
Serhiy Storchaka cf3806026b Issue #24098: Fixed possible crash when AST is changed in process of
compiling it.
2016-10-07 21:51:28 +03:00
Benjamin Peterson e2e792d98f merge 3.5 (#28184) 2016-09-19 22:17:16 -07:00
Benjamin Peterson f5e8e8fc2b merge 3.5 (#24022) 2016-09-18 23:44:02 -07:00
Benjamin Peterson 57bda335e1 merge 3.4 2016-09-18 23:43:18 -07:00
Benjamin Peterson 26d998cfdd properly handle the single null-byte file (closes #24022) 2016-09-18 23:41:11 -07:00
Benjamin Peterson 9ac11a752a properly free memory in pgen 2016-09-18 18:00:25 -07:00
Benjamin Peterson 5a715cfc57 merge 3.5 (#27981) 2016-09-12 22:07:14 -07:00