Commit graph

621 commits

Author SHA1 Message Date
Victor Stinner 7b3c252dc7
bpo-39877: _PyRuntimeState.finalizing becomes atomic (GH-18816)
Convert _PyRuntimeState.finalizing field to an atomic variable:

* Rename it to _finalizing
* Change its type to _Py_atomic_address
* Add _PyRuntimeState_GetFinalizing() and _PyRuntimeState_SetFinalizing()
  functions
* Remove _Py_CURRENTLY_FINALIZING() function: replace it with testing
  directly _PyRuntimeState_GetFinalizing() value

Convert _PyRuntimeState_GetThreadState() to static inline function.
2020-03-07 00:24:23 +01:00
Andy Lester da4d656e95
closes bpo-39870: Remove unused arg from sys_displayhook_unencodable. (GH-18796)
Also move int err to its innermost scope.
2020-03-05 20:34:36 -08:00
Andy Lester e6be9b59a9
closes bpo-39605: Fix some casts to not cast away const. (GH-18453)
gcc -Wcast-qual turns up a number of instances of casting away constness of pointers. Some of these can be safely modified, by either:

Adding the const to the type cast, as in:

-    return _PyUnicode_FromUCS1((unsigned char*)s, size);
+    return _PyUnicode_FromUCS1((const unsigned char*)s, size);

or, Removing the cast entirely, because it's not necessary (but probably was at one time), as in:

-    PyDTrace_FUNCTION_ENTRY((char *)filename, (char *)funcname, lineno);
+    PyDTrace_FUNCTION_ENTRY(filename, funcname, lineno);

These changes will not change code, but they will make it much easier to check for errors in consts
2020-02-11 18:28:35 -08:00
Petr Viktorin ffd9753a94
bpo-39245: Switch to public API for Vectorcall (GH-18460)
The bulk of this patch was generated automatically with:

    for name in \
        PyObject_Vectorcall \
        Py_TPFLAGS_HAVE_VECTORCALL \
        PyObject_VectorcallMethod \
        PyVectorcall_Function \
        PyObject_CallOneArg \
        PyObject_CallMethodNoArgs \
        PyObject_CallMethodOneArg \
    ;
    do
        echo $name
        git grep -lwz _$name | xargs -0 sed -i "s/\b_$name\b/$name/g"
    done

    old=_PyObject_FastCallDict
    new=PyObject_VectorcallDict
    git grep -lwz $old | xargs -0 sed -i "s/\b$old\b/$new/g"

and then cleaned up:

- Revert changes to in docs & news
- Revert changes to backcompat defines in headers
- Nudge misaligned comments
2020-02-11 17:46:57 +01:00
Victor Stinner a102ed7d2f
bpo-39573: Use Py_TYPE() macro in Python and Include directories (GH-18391)
Replace direct access to PyObject.ob_type with Py_TYPE().
2020-02-07 02:24:48 +01:00
Victor Stinner a93c51e3a8
bpo-39573: Use Py_REFCNT() macro (GH-18388)
Replace direct acccess to PyObject.ob_refcnt with usage of the
Py_REFCNT() macro.
2020-02-07 00:38:59 +01:00
Victor Stinner c6e5c1123b
bpo-39489: Remove COUNT_ALLOCS special build (GH-18259)
Remove:

* COUNT_ALLOCS macro
* sys.getcounts() function
* SHOW_ALLOC_COUNT code in listobject.c
* SHOW_TRACK_COUNT code in tupleobject.c
* PyConfig.show_alloc_count field
* -X showalloccount command line option
* @test.support.requires_type_collecting decorator
2020-02-03 15:17:15 +01:00
David Carlier aabdeb766b bpo-38960: DTrace build fix for FreeBSD. (GH-17451)
DTrace build fix for FreeBSD.

- allowing passing an extra flag as it need to define the arch size.
- casting some probe's arguments.
2020-01-28 13:53:32 +01:00
Steve Dower b8cbe74c34
bpo-39008: Require Py_ssize_t for PySys_Audit formats rather than raise a deprecation warning (GH-17540) 2019-12-09 11:05:39 -08:00
Victor Stinner 81fe5bd3d7
bpo-38858: new_interpreter() reuses _PySys_Create() (GH-17481)
new_interpreter() now calls _PySys_Create() to create a new sys
module isolated from the main interpreter. It now calls
_PySys_InitCore() and _PyImport_FixupBuiltin().

init_interp_main() now calls _PySys_InitMain().
2019-12-06 02:43:30 +01:00
Steve Dower bea33f5e1d
bpo-38920: Add audit hooks for when sys.excepthook and sys.unraisable hooks are invoked (GH-17392)
Also fixes some potential segfaults in unraisable hook handling.
2019-11-28 08:46:11 -08:00
Victor Stinner 01b1cc12e7
bpo-36710: Add PyInterpreterState.runtime field (GH-17270)
Add PyInterpreterState.runtime field: reference to the _PyRuntime
global variable. This field exists to not have to pass runtime in
addition to tstate to a function.  Get runtime from tstate:
tstate->interp->runtime.

Remove "_PyRuntimeState *runtime" parameter from functions already
taking a "PyThreadState *tstate" parameter.

_PyGC_Init() first parameter becomes "PyThreadState *tstate".
2019-11-20 02:27:56 +01:00
Victor Stinner fb4ae152a9
bpo-38317: Fix PyConfig.warnoptions priority (GH-16478)
Fix warnings options priority: PyConfig.warnoptions has the highest
priority, as stated in the PEP 587.

* Document options order in PyConfig.warnoptions documentation.
* Make PyWideStringList_INIT macro private: replace "Py" prefix
  with "_Py".
* test_embed: add test_init_warnoptions().
2019-09-30 01:40:17 +02:00
Serhiy Storchaka 279f44678c
bpo-37206: Unrepresentable default values no longer represented as None. (GH-13933)
In ArgumentClinic, value "NULL" should now be used only for unrepresentable default values
(like in the optional third parameter of getattr). "None" should be used if None is accepted
as argument and passing None has the same effect as not passing the argument at all.
2019-09-14 12:24:05 +03:00
Raymond Hettinger 7117074410 bpo-38096: Clean up the "struct sequence" / "named tuple" docs (GH-15895)
* bpo-38096: Clean up the "struct sequence" / "named tuple" docs

* Fix remaining occurrences of "struct sequence"

* Repair a user visible docstring
2019-09-11 15:17:32 +01:00
Serhiy Storchaka 41c57b3353
bpo-37994: Fix silencing all errors if an attribute lookup fails. (GH-15630)
Only AttributeError should be silenced.
2019-09-01 12:03:39 +03:00
Victor Stinner 120b707a6d
bpo-36763: PyConfig_Read() handles PySys_AddXOption() (GH-15431)
PyConfig_Read() is now responsible to handle early calls to
PySys_AddXOption() and PySys_AddWarnOption().

Options added by PySys_AddXOption() are now handled the same way than
PyConfig.xoptions and command line -X options.

For example, PySys_AddXOption(L"faulthandler") enables faulthandler
as expected.
2019-08-23 18:03:08 +01:00
Victor Stinner c48682509d
bpo-37926: Fix PySys_SetArgvEx(0, NULL, 0) crash (GH-15415)
empty_argv is no longer static in Python 3.8, but it is declared in
a temporary scope, whereas argv keeps a reference to it.
empty_argv memory (allocated on the stack) is reused by
make_sys_argv() code which is inlined when using gcc -O3.

Define empty_argv in PySys_SetArgvEx() body, to ensure
that it remains valid for the whole lifetime of
the PySys_SetArgvEx() call.
2019-08-23 11:04:16 +01:00
Min ho Kim c4cacc8c5e Fix typos in comments, docs and test names (#15018)
* Fix typos in comments, docs and test names

* Update test_pyparse.py

account for change in string length

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: Dealloccte -> Deallocate

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Update posixmodule checksum.

* Reverse idlelib changes.
2019-07-30 18:16:13 -04:00
Jeroen Demeyer 59ad110d7a bpo-37547: add _PyObject_CallMethodOneArg (GH-14685) 2019-07-11 17:59:05 +09:00
Steve Dower 9048c49322
bpo-37369: Fix initialization of sys members when launched via an app container (GH-14428)
sys._base_executable is now always defined on all platforms, and can be overridden through configuration.
Also adds test.support.PythonSymlink to encapsulate platform-specific logic for symlinking sys.executable
2019-06-29 10:34:11 -07:00
Jeroen Demeyer b1263d5a60 bpo-37337: Add _PyObject_VectorcallMethod() (GH-14228) 2019-06-28 18:49:00 +09:00
Victor Stinner 69150669f2
bpo-37414: Remove sys.callstats() (GH-14398)
Remove the undocumented sys.callstats() function. Since Python 3.7,
it was deprecated and always returned None. It required a special
build option CALL_PROFILE which was already removed in Python 3.7.
2019-06-26 18:01:10 +02:00
Victor Stinner 36456df138
bpo-37392: Remove sys.setcheckinterval() (GH-14355)
Remove sys.getcheckinterval() and sys.setcheckinterval() functions.
They were deprecated since Python 3.2. Use sys.getswitchinterval()
and sys.setswitchinterval() instead.

Remove also check_interval field of the PyInterpreterState structure.
2019-06-25 03:01:08 +02:00
Zackery Spytz 08286d52b2 bpo-37316: mmap.mmap() passes the wrong variable to PySys_Audit() (GH-14152)
Also, add a missing call to va_end() in PySys_Audit().
2019-06-21 08:31:59 -07:00
Victor Stinner b45d259bdd
bpo-36710: Use tstate in pylifecycle.c (GH-14249)
In pylifecycle.c: pass tstate argument, rather than interp argument,
to functions.
2019-06-20 00:05:23 +02:00
Victor Stinner 838f26402d
bpo-36710: Pass explicitly tstate in sysmodule.c (GH-14060)
* Replace global var Py_VerboseFlag with interp->config.verbose.
* Add _PyErr_NoMemory(tstate) function.
* Add tstate parameter to _PyEval_SetCoroutineOriginTrackingDepth()
  and move the function to the internal API.
* Replace _PySys_InitMain(runtime, interp)
  with _PySys_InitMain(runtime, tstate).
2019-06-13 22:41:23 +02:00
Victor Stinner 0fd2c300c2
Revert "bpo-36818: Add PyInterpreterState.runtime field. (gh-13129)" (GH-13795)
This reverts commit 396e0a8d9d.
2019-06-04 03:15:09 +02:00
Eric Snow 396e0a8d9d
bpo-36818: Add PyInterpreterState.runtime field. (gh-13129)
https://bugs.python.org/issue36818
2019-05-31 21:16:47 -06:00
Jeroen Demeyer aacc77fbd7 bpo-36974: implement PEP 590 (GH-13185)
Co-authored-by: Jeroen Demeyer <J.Demeyer@UGent.be>
Co-authored-by: Mark Shannon <mark@hotpy.org>
2019-05-29 20:31:52 +02:00
Matthias Bussonnier 3880f263d2 bpo-36933: Remove sys.set_coroutine_wrapper (marked for removal in 3.8) (GH-13577)
It has been documented as deprecated and to be removed in 3.8; 

From a comment on another thread – which I can't find ; leave get_coro_wrapper() for now, but always return `None`.


https://bugs.python.org/issue36933
2019-05-28 00:10:59 -07:00
Victor Stinner 331a6a56e9
bpo-36763: Implement the PEP 587 (GH-13592)
* Add a whole new documentation page:
  "Python Initialization Configuration"
* PyWideStringList_Append() return type is now PyStatus,
  instead of int
* PyInterpreterState_New() now calls PyConfig_Clear() if
  PyConfig_InitPythonConfig() fails.
* Rename files:

  * Python/coreconfig.c => Python/initconfig.c
  * Include/cpython/coreconfig.h => Include/cpython/initconfig.h
  * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h

* Rename structures

  * _PyCoreConfig => PyConfig
  * _PyPreConfig => PyPreConfig
  * _PyInitError => PyStatus
  * _PyWstrList => PyWideStringList

* Rename PyConfig fields:

  * use_module_search_paths => module_search_paths_set
  * module_search_path_env => pythonpath_env

* Rename PyStatus field: _func => func
* PyInterpreterState: rename core_config field to config
* Rename macros and functions:

  * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv()
  * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv()
  * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString()
  * _PyInitError_Failed() => PyStatus_Exception()
  * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx
  * _Py_UnixMain() => Py_BytesMain()
  * _Py_ExitInitError() => Py_ExitStatusException()
  * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs()
  * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs()
  * _Py_PreInitialize() => Py_PreInitialize()
  * _Py_RunMain() => Py_RunMain()
  * _Py_InitializeFromConfig() => Py_InitializeFromConfig()
  * _Py_INIT_XXX() => _PyStatus_XXX()
  * _Py_INIT_FAILED() => _PyStatus_EXCEPTION()

* Rename 'err' PyStatus variables to 'status'
* Convert RUN_CODE() macro to config_run_code() static inline function
* Remove functions:

  * _Py_InitializeFromArgs()
  * _Py_InitializeFromWideArgs()
  * _PyInterpreterState_GetCoreConfig()
2019-05-27 16:39:22 +02:00
Victor Stinner 71c52e3048
bpo-36829: Add _PyErr_WriteUnraisableMsg() (GH-13488)
* sys.unraisablehook: add 'err_msg' field to UnraisableHookArgs.
* Use _PyErr_WriteUnraisableMsg() in _ctypes _DictRemover_call()
  and gc delete_garbage().
2019-05-27 08:57:14 +02:00
Steve Dower b82e17e626
bpo-36842: Implement PEP 578 (GH-12613)
Adds sys.audit, sys.addaudithook, io.open_code, and associated C APIs.
2019-05-23 08:45:22 -07:00
Victor Stinner ef9d9b6312
bpo-36829: Add sys.unraisablehook() (GH-13187)
Add new sys.unraisablehook() function which can be overridden to
control how "unraisable exceptions" are handled. It is called when an
exception has occurred but there is no way for Python to handle it.
For example, when a destructor raises an exception or during garbage
collection (gc.collect()).

Changes:

* Add an internal UnraisableHookArgs type used to pass arguments to
  sys.unraisablehook.
* Add _PyErr_WriteUnraisableDefaultHook().
* The default hook now ignores exception on writing the traceback.
* test_sys now uses unittest.main() to automatically discover tests:
  remove test_main().
* Add _PyErr_Init().
* Fix PyErr_WriteUnraisable(): hold a strong reference to sys.stderr
  while using it
2019-05-22 11:28:22 +02:00
Zackery Spytz 1a2252ed39 bpo-36594: Fix incorrect use of %p in format strings (GH-12769)
In addition, fix some other minor violations of C99.
2019-05-06 12:56:50 -04:00
Victor Stinner 709d23dee6
bpo-36775: _PyCoreConfig only uses wchar_t* (GH-13062)
_PyCoreConfig: Change filesystem_encoding, filesystem_errors,
stdio_encoding and stdio_errors fields type from char* to wchar_t*.

Changes:

* PyInterpreterState: replace fscodec_initialized (int) with fs_codec
  structure.
* Add get_error_handler_wide() and unicode_encode_utf8() helper
  functions.
* Add error_handler parameter to unicode_encode_locale()
  and unicode_decode_locale().
* Remove _PyCoreConfig_SetString().
* Rename _PyCoreConfig_SetWideString() to _PyCoreConfig_SetString().
* Rename _PyCoreConfig_SetWideStringFromString()
  to _PyCoreConfig_DecodeLocale().
2019-05-02 14:56:30 -04:00
Victor Stinner 43125224d6
bpo-36710: Add runtime variable to Py_InitializeEx() (GH-12939)
Py_InitializeEx() now uses a runtime variable passed to subfunctions,
rather than working directly on the global variable _PyRuntime.

Add 'runtime' parameter to _PyCoreConfig_Write(), _PySys_Create(),
_PySys_InitMain(), _PyGILState_Init(),
emit_stderr_warning_for_legacy_locale() and other subfunctions.
2019-04-24 18:23:53 +02:00
Pablo Galindo 34ef64fe59 bpo-36447, bpo-36447: Fix refleak in _PySys_InitMain() (GH-12586)
Fix refleak in sysmodule.c when calling SET_SYS_FROM_STRING_BORROW.
2019-03-27 13:43:47 +01:00
Victor Stinner 8b9dbc017a
bpo-36444: Remove _PyMainInterpreterConfig (GH-12571) 2019-03-27 01:36:16 +01:00
Victor Stinner 20004959d2
bpo-36301: Remove _PyCoreConfig.preconfig (GH-12546)
* Replace _PyCoreConfig.preconfig with 3 new fields in _PyCoreConfig:
  isolated, use_environment, dev_mode.
* Add _PyPreCmdline.dev_mode.
* Add _Py_PreInitializeFromPreConfigInPlace().
2019-03-26 02:31:11 +01:00
Stefan Krah 027b09c5a1
bpo-36370: Check for PyErr_Occurred() after PyImport_GetModule() (GH-12504) 2019-03-25 21:50:58 +01:00
Victor Stinner dcf617152e
bpo-36236: Handle removed cwd at Python init (GH-12424)
At Python initialization, the current directory is no longer
prepended to sys.path if it has been removed.

Rename _PyPathConfig_ComputeArgv0() to
_PyPathConfig_ComputeSysPath0() to avoid confusion between argv[0]
and sys.path[0].
2019-03-19 16:09:27 +01:00
Victor Stinner 74f6568bbd
bpo-36301: Add _PyWstrList structure (GH-12343)
Replace messy _Py_wstrlist_xxx() functions with a new clean
_PyWstrList structure and new _PyWstrList_xxx() functions.

Changes:

* Add _PyCoreConfig.use_module_search_paths to decide if
  _PyCoreConfig.module_search_paths should be computed or not, to
  support empty search path list.
* _PyWstrList_Clear() sets length to 0 and items to NULL, whereas
  _Py_wstrlist_clear() only freed memory.
* _PyWstrList_Append() returns an int, whereas _Py_wstrlist_append()
  returned _PyInitError.
* _PyWstrList uses Py_ssize_t for the length, instead of int.
* Replace (int, wchar_t**) with _PyWstrList in:

  * _PyPreConfig
  * _PyCoreConfig
  * _PyPreCmdline
  * _PyCmdline

* Replace "int orig_argv; wchar_t **orig_argv;"
  with "_PyWstrList orig_argv".
* _PyCmdline and _PyPreCmdline now also copy wchar_argv.
* Rename _PyArgv_Decode() to _PyArgv_AsWstrList().
* PySys_SetArgvEx() now pass the fixed (argc, argv) to
  _PyPathConfig_ComputeArgv0() (don't pass negative argc or NULL
  argv).
* _PyOS_GetOpt() uses Py_ssize_t
2019-03-15 15:08:05 +01:00
Victor Stinner b35be4b333
bpo-36142: Add _PyPreConfig.allocator (GH-12181)
* Move 'allocator' and 'dev_mode' fields from _PyCoreConfig
  to _PyPreConfig.
* Fix InitConfigTests of test_embed: dev_mode sets allocator to
  "debug", add a new tests for env vars with dev mode enabled.
2019-03-05 17:37:44 +01:00
Victor Stinner 5a02e0d1c8
bpo-36142: Add _PyPreConfig.utf8_mode (GH-12174)
* Move following fields from _PyCoreConfig to _PyPreConfig:

  * coerce_c_locale
  * coerce_c_locale_warn
  * legacy_windows_stdio
  * utf8_mode

* _PyPreConfig_ReadFromArgv() is now responsible to choose the
  filesystem encoding
* _PyPreConfig_Write() now sets the LC_CTYPE locale
2019-03-05 12:32:09 +01:00
Victor Stinner cad1f747da
bpo-36142: Add _PyPreConfig structure (GH-12172)
* Add _PyPreConfig structure
* Move 'ignored' and 'use_environment' fields from _PyCoreConfig
  to _PyPreConfig
* Add a new "_PyPreConfig preconfig;" field to _PyCoreConfig
2019-03-05 02:01:27 +01:00
Minmin Gong 8ebc6451f3 bpo-35890 : Fix some API calling consistency (GH-11742)
Unicode version of Windows APIs are used in places, but not for GetVersionEx in Python/sysmodule.c
The wcstok_s is called on Windows in Modules/main.c and PC/launcher.c, but not in Python/pathconfig.c
2019-02-02 20:26:55 -08:00
Tony Roberts 4860f01ac0 bpo-33895: Relase GIL while calling functions that acquire Windows loader lock (GH-7789)
LoadLibrary, GetProcAddress, FreeLibrary and GetModuleHandle acquire the system loader lock. Calling these while holding the GIL will cause a deadlock on the rare occasion that another thread is detaching and needs to destroy its thread state at the same time.
2019-02-02 09:16:42 -08:00
Victor Stinner ab67281e95
bpo-35713: Reorganize sys module initialization (GH-11658)
* Rename _PySys_BeginInit() to _PySys_InitCore().
* Rename _PySys_EndInit() to _PySys_InitMain().
* Add _PySys_Create(). It calls _PySys_InitCore() which becomes
  private.
* Add _PySys_SetPreliminaryStderr().
* Rename _Py_ReadyTypes() to _PyTypes_Init().
* Misc code cleanup.
2019-01-23 15:04:40 +01:00
Victor Stinner bf4ac2d2fd
bpo-35713: Rework Python initialization (GH-11647)
* The PyByteArray_Init() and PyByteArray_Fini() functions have been
  removed. They did nothing since Python 2.7.4 and Python 3.2.0, were
  excluded from the limited API (stable ABI), and were not
  documented.
* Move "_PyXXX_Init()" and "_PyXXX_Fini()" declarations from
  Include/cpython/pylifecycle.h to
  Include/internal/pycore_pylifecycle.h. Replace
  "PyAPI_FUNC(TYPE)" with "extern TYPE".
* _PyExc_Init() now returns an error on failure rather than calling
  Py_FatalError(). Move macros inside _PyExc_Init() and undefine them
  when done. Rewrite macros to make them look more like statement:
  add ";" when using them, add "do { ... } while (0)".
* _PyUnicode_Init() now returns a _PyInitError error rather than call
  Py_FatalError().
* Move stdin check from _PySys_BeginInit() to init_sys_streams().
* _Py_ReadyTypes() now returns a _PyInitError error rather than
  calling Py_FatalError().
2019-01-22 17:39:03 +01:00
Serhiy Storchaka 3607ef43c4
bpo-35742: Fix test_envar_unimportable in test_builtin. (GH-11561)
Handle the case of an empty module name in PYTHONBREAKPOINT.

Fixes a regression introduced in bpo-34756.
2019-01-15 13:26:38 +02:00
Serhiy Storchaka 6fe9c446f8
bpo-34756: Silence only ImportError and AttributeError in sys.breakpointhook(). (GH-9457) 2019-01-14 12:58:37 +02:00
Tal Einat a5b76167de
remove doc-string declaration no longer used after AC conversion (GH-11444) 2019-01-06 10:10:34 +02:00
Tal Einat ede0b6fae2
bpo-20182: AC convert Python/sysmodule.c (GH-11328) 2018-12-31 17:12:08 +02:00
Serhiy Storchaka dffccc6b59
bpo-35452: Make PySys_HasWarnOptions() never raising an exception. (GH-11075) 2018-12-10 13:50:22 +02:00
Serhiy Storchaka 72ff7b4c00
bpo-35451: Fix reference counting for sys.warnoptions and sys._xoptions. (GH-11063) 2018-12-10 12:08:54 +02:00
Zackery Spytz 99d56b5356 bpo-35441: Remove dead and buggy code related to PyList_SetItem(). (GH-11033)
In _localemodule.c and selectmodule.c, remove dead code that would
cause double decrefs if run.

In addition, replace PyList_SetItem() with PyList_SET_ITEM() in cases
where a new list is populated and there is no possibility of an error.

In addition, check if the list changed size in the loop in array_array_fromlist().
2018-12-08 16:16:55 +02:00
Serhiy Storchaka 62be74290a
bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749)
Fix invalid function cast warnings with gcc 8
for method conventions different from METH_NOARGS, METH_O and
METH_VARARGS excluding Argument Clinic generated code.
2018-11-27 13:27:31 +02:00
Victor Stinner 37cd982df0
bpo-35239: _PySys_EndInit() copies module_search_path (GH-10532)
* The _PySys_EndInit() function now copies the
  config->module_search_path list, so config is longer modified when
  sys.path is updated.
* config->warnoptions list and config->xoptions dict are also copied
* test_embed: InitConfigTests now also tests
  main_config['module_search_path']
* Fix _Py_InitializeMainInterpreter(): don't use config->warnoptions
   but sys.warnoptions to decide if the warnings module should
   be imported at startup.
2018-11-16 11:55:35 +01:00
Victor Stinner 621cebe81b
bpo-35081: Rename internal headers (GH-10275)
Rename Include/internal/ headers:

* pycore_hash.h -> pycore_pyhash.h
* pycore_lifecycle.h -> pycore_pylifecycle.h
* pycore_mem.h -> pycore_pymem.h
* pycore_state.h -> pycore_pystate.h

Add missing headers to Makefile.pre.in and PCbuild:

* pycore_condvar.h.
* pycore_hamt.h
* pycore_pyhash.h
2018-11-12 16:53:38 +01:00
Anthony Sottile dce345c51a Simplify sys.breakpointhook implementation (#9519) 2018-11-01 10:25:05 -07:00
Victor Stinner a1c249c405
bpo-35081: And pycore_lifecycle.h and pycore_pathconfig.h (GH-10273)
* And pycore_lifecycle.h and pycore_pathconfig.h headers to
  Include/internal/
* Move Py_BUILD_CORE specific code from coreconfig.h and
  pylifecycle.h to pycore_pathconfig.h and pycore_lifecycle.h
* Move _Py_wstrlist_XXX() definitions and _PyPathConfig code
  from pycore_state.h to pycore_pathconfig.h
* Move "Init" and "Fini" function definitions from pylifecycle.c to
  pycore_lifecycle.h.
2018-11-01 03:15:58 +01:00
Victor Stinner 50b48572d9
bpo-35081: Add _PyThreadState_GET() internal macro (GH-10266)
If Py_BUILD_CORE is defined, the PyThreadState_GET() macro access
_PyRuntime which comes from the internal pycore_state.h header.
Public headers must not require internal headers.

Move PyThreadState_GET() and _PyInterpreterState_GET_UNSAFE() from
Include/pystate.h to Include/internal/pycore_state.h, and rename
PyThreadState_GET() to _PyThreadState_GET() there.

The PyThreadState_GET() macro of pystate.h is now redefined when
pycore_state.h is included, to use the fast _PyThreadState_GET().

Changes:

* Add _PyThreadState_GET() macro
* Replace "PyThreadState_GET()->interp" with
  _PyInterpreterState_GET_UNSAFE()
* Replace PyThreadState_GET() with _PyThreadState_GET() in internal C
  files (compiled with Py_BUILD_CORE defined), but keep
  PyThreadState_GET() in the public header files.
* _testcapimodule.c: replace PyThreadState_GET() with
  PyThreadState_Get(); the module is not compiled with Py_BUILD_CORE
  defined.
* pycore_state.h now requires Py_BUILD_CORE to be defined.
2018-11-01 01:51:40 +01:00
Victor Stinner 27e2d1f219
bpo-35081: Add pycore_ prefix to internal header files (GH-10263)
* Rename Include/internal/ header files:

  * pyatomic.h -> pycore_atomic.h
  * ceval.h -> pycore_ceval.h
  * condvar.h -> pycore_condvar.h
  * context.h -> pycore_context.h
  * pygetopt.h -> pycore_getopt.h
  * gil.h -> pycore_gil.h
  * hamt.h -> pycore_hamt.h
  * hash.h -> pycore_hash.h
  * mem.h -> pycore_mem.h
  * pystate.h -> pycore_state.h
  * warnings.h -> pycore_warnings.h

* PCbuild project, Makefile.pre.in, Modules/Setup: add the
  Include/internal/ directory to the search paths of header files.
* Update includes. For example, replace #include "internal/mem.h"
  with #include "pycore_mem.h".
2018-11-01 00:52:28 +01:00
Victor Stinner 2be00d987d
bpo-35081: Move Py_BUILD_CORE code to internal/mem.h (GH-10249)
* Add #include "internal/mem.h" to C files using
  _PyMem_SetDefaultAllocator().
* Include/internal/mem.h now requires Py_BUILD_CORE to be defined.
2018-10-31 20:19:24 +01:00
Victor Stinner e1b29950bf
bpo-32030: Make _PySys_AddXOptionWithError() private (GH-10236)
Make _PySys_AddXOptionWithError() and _PySys_AddWarnOptionWithError()
functions private again. They are no longer needed to initialize Python:
_PySys_EndInit() is now responsible to add these options instead.

Moreover, PySys_AddWarnOptionUnicode() now clears the exception on
failure if possible.
2018-10-30 14:31:42 +01:00
Pablo Galindo 49c75a8086
bpo-35064 prefix smelly symbols that appear with COUNT_ALLOCS with _Py_ (GH-10152)
Configuring python with ./configure --with-pydebug CFLAGS="-D COUNT_ALLOCS -O0"
makes "make smelly" fail as some symbols were being exported without the "Py_" or
"_Py" prefixes.
2018-10-28 15:02:17 +00:00
Victor Stinner fbca90856d
bpo-34523: Use _PyCoreConfig instead of globals (GH-9005)
Use the core configuration of the interpreter, rather
than using global configuration variables. For example, replace
Py_QuietFlag with core_config->quiet.
2018-08-30 00:50:45 +02:00
Victor Stinner b2457efc78
bpo-34523: Add _PyCoreConfig.filesystem_encoding (GH-8963)
_PyCoreConfig_Read() is now responsible to choose the filesystem
encoding and error handler. Using Py_Main(), the encoding is now
chosen even before calling Py_Initialize().

_PyCoreConfig.filesystem_encoding is now the reference, instead of
Py_FileSystemDefaultEncoding, for the Python filesystem encoding.

Changes:

* Add filesystem_encoding and filesystem_errors to _PyCoreConfig
* _PyCoreConfig_Read() now reads the locale encoding for the file
  system encoding.
* PyUnicode_EncodeFSDefault() and PyUnicode_DecodeFSDefaultAndSize()
  now use the interpreter configuration rather than
  Py_FileSystemDefaultEncoding and Py_FileSystemDefaultEncodeErrors
  global configuration variables.
* Add _Py_SetFileSystemEncoding() and _Py_ClearFileSystemEncoding()
  private functions to only modify Py_FileSystemDefaultEncoding and
  Py_FileSystemDefaultEncodeErrors in coreconfig.c.
* _Py_CoerceLegacyLocale() now takes an int rather than
  _PyCoreConfig for the warning.
2018-08-29 13:25:36 +02:00
Victor Stinner caba55b3b7
bpo-34301: Add _PyInterpreterState_Get() helper function (GH-8592)
sys_setcheckinterval() now uses a local variable to parse arguments,
before writing into interp->check_interval.
2018-08-03 15:33:52 +02:00
Serhiy Storchaka f60bf0e168
bpo-22689: Copy the result of getenv() in sys_breakpointhook(). (GH-8194) 2018-07-09 21:46:51 +03:00
Carl Meyer b193fa996a bpo-33499: Add PYTHONPYCACHEPREFIX env var for alt bytecode cache location. (GH-6834)
In some development setups it is inconvenient or impossible to write bytecode
caches to the code tree, but the bytecode caches are still useful. The
PYTHONPYCACHEPREFIX environment variable allows specifying an alternate
location for cached bytecode files, within which a directory tree mirroring the code
tree will be created. This cache tree is then used (for both reading and writing)
instead of the local `__pycache__` subdirectory within each source directory.

Exposed at runtime as sys.pycache_prefix (defaulting to None), and can
be set from the CLI as "-X pycache_prefix=path".

Patch by Carl Meyer.
2018-06-16 14:40:56 +10:00
Siddhesh Poyarekar 55edd0c185 bpo-33012: Fix invalid function cast warnings with gcc 8 for METH_NOARGS. (GH-6030)
METH_NOARGS functions need only a single argument but they are cast
into a PyCFunction, which takes two arguments.  This triggers an
invalid function cast warning in gcc8 due to the argument mismatch.
Fix this by adding a dummy unused argument.
2018-04-29 21:59:33 +03:00
Nick Coghlan bc77eff8b9
bpo-33042: Fix pre-initialization sys module configuration (GH-6157)
- new test case for pre-initialization of sys.warnoptions and sys._xoptions
- restored ability to call these APIs prior to Py_Initialize
- updated the docs for the affected APIs to make it clear they can be
  called before Py_Initialize
- also enhanced the existing embedding test cases
  to check for expected settings in the sys module
2018-03-25 20:44:30 +10:00
oldk aa0735f597 bpo-32747: Remove trailing spaces in docstrings. (GH-5491) 2018-02-02 10:52:55 +02:00
Nathaniel J. Smith fc2f407829 bpo-32591: Add native coroutine origin tracking (#5250)
* Add coro.cr_origin and sys.set_coroutine_origin_tracking_depth
* Use coroutine origin information in the unawaited coroutine warning
* Stop using set_coroutine_wrapper in asyncio debug mode
* In BaseEventLoop.set_debug, enable debugging in the correct thread
2018-01-21 09:44:07 -05:00
Serhiy Storchaka a5552f023e
bpo-32240: Add the const qualifier to declarations of PyObject* array arguments. (#4746) 2017-12-15 13:11:11 +02:00
Victor Stinner 41264f1cd4
bpo-32030: Add _PyMainInterpreterConfig.executable (#4876)
* Add new fields to _PyMainInterpreterConfig:

  * executable
  * prefix
  * base_prefix
  * exec_prefix
  * base_exec_prefix

* _PySys_EndInit() now sets sys attributes from
  _PyMainInterpreterConfig
2017-12-15 02:05:29 +01:00
Victor Stinner 11a247df88
bpo-32030: Add _PyPathConfig_ComputeArgv0() (#4845)
Changes:

* Split _PySys_SetArgvWithError() into subfunctions for Py_Main():

  * Create the Python list object
  * Set sys.argv to the list
  * Compute argv0
  * Prepend argv0 to sys.path

* Add _PyPathConfig_ComputeArgv0()
* Remove _PySys_SetArgvWithError()
* Py_Main() now splits the code to compute sys.argv/path0 and the
  code to update the sys module: add pymain_compute_argv()
  subfunction.
2017-12-13 21:05:57 +01:00
Victor Stinner d5dda98fa8
pymain_set_sys_argv() now copies argv (#4838)
bpo-29240, bpo-32030:

* Rename pymain_set_argv() to pymain_set_sys_argv()
* pymain_set_sys_argv() now creates of copy of argv and modify the
  copy, rather than modifying pymain->argv
* Call pymain_set_sys_argv() earlier: before pymain_run_python(), but
  after pymain_get_importer().
* Add _PySys_SetArgvWithError() to handle errors
2017-12-13 17:31:16 +01:00
Victor Stinner 91106cd9ff
bpo-29240: PEP 540: Add a new UTF-8 Mode (#855)
* Add -X utf8 command line option, PYTHONUTF8 environment variable
  and a new sys.flags.utf8_mode flag.
* If the LC_CTYPE locale is "C" at startup: enable automatically the
  UTF-8 mode.
* Add _winapi.GetACP(). encodings._alias_mbcs() now calls
  _winapi.GetACP() to get the ANSI code page
* locale.getpreferredencoding() now returns 'UTF-8' in the UTF-8
  mode. As a side effect, open() now uses the UTF-8 encoding by
  default in this mode.
* Py_DecodeLocale() and Py_EncodeLocale() now use the UTF-8 encoding
  in the UTF-8 Mode.
* Update subprocess._args_from_interpreter_flags() to handle -X utf8
* Skip some tests relying on the current locale if the UTF-8 mode is
  enabled.
* Add test_utf8mode.py.
* _Py_DecodeUTF8_surrogateescape() gets a new optional parameter to
  return also the length (number of wide characters).
* pymain_get_global_config() and pymain_set_global_config() now
  always copy flag values, rather than only copying if the new value
  is greater than the old value.
2017-12-13 12:29:09 +01:00
Serhiy Storchaka 4ae06c5337
bpo-32241: Add the const qualifire to declarations of umodifiable strings. (#4748) 2017-12-12 13:55:04 +02:00
Victor Stinner 6bf992a1ac
bpo-32030: Add pymain_get_global_config() (#4735)
* Py_Main() now starts by reading Py_xxx configuration variables to
  only work on its own private structure, and then later writes back
  the configuration into these variables.
* Replace Py_GETENV() with pymain_get_env_var() which ignores empty
  variables.
* Add _PyCoreConfig.dump_refs
* Add _PyCoreConfig.malloc_stats
* _PyObject_DebugMallocStats() is now responsible to check if debug
  hooks are installed. The function returns 1 if stats were written,
  or 0 if the hooks are disabled. Mark _PyMem_PymallocEnabled() as
  static.
2017-12-06 17:26:10 +01:00
Victor Stinner 5e3806f8cf
bpo-32101: Add PYTHONDEVMODE environment variable (#4624)
* bpo-32101: Add sys.flags.dev_mode flag
  Rename also the "Developer mode" to the "Development mode".
* bpo-32101: Add PYTHONDEVMODE environment variable
  Mention it in the development chapiter.
2017-11-30 11:40:24 +01:00
Victor Stinner f7e5b56c37
bpo-32030: Split Py_Main() into subfunctions (#4399)
* Don't use "Python runtime" anymore to parse command line options or
  to get environment variables: pymain_init() is now a strict
  separation.
* Use an error message rather than "crashing" directly with
  Py_FatalError(). Limit the number of calls to Py_FatalError(). It
  prepares the code to handle errors more nicely later.
* Warnings options (-W, PYTHONWARNINGS) and "XOptions" (-X) are now
  only added to the sys module once Python core is properly
  initialized.
* _PyMain is now the well identified owner of some important strings
  like: warnings options, XOptions, and the "program name". The
  program name string is now properly freed at exit.
  pymain_free() is now responsible to free the "command" string.
* Rename most methods in Modules/main.c to use a "pymain_" prefix to
  avoid conflits and ease debug.
* Replace _Py_CommandLineDetails_INIT with memset(0)
* Reorder a lot of code to fix the initialization ordering. For
  example, initializing standard streams now comes before parsing
  PYTHONWARNINGS.
* Py_Main() now handles errors when adding warnings options and
  XOptions.
* Add _PyMem_GetDefaultRawAllocator() private function.
* Cleanup _PyMem_Initialize(): remove useless global constants: move
  them into _PyMem_Initialize().
* Call _PyRuntime_Initialize() as soon as possible:
  _PyRuntime_Initialize() now returns an error message on failure.
* Add _PyInitError structure and following macros:

  * _Py_INIT_OK()
  * _Py_INIT_ERR(msg)
  * _Py_INIT_USER_ERR(msg): "user" error, don't abort() in that case
  * _Py_INIT_FAILED(err)
2017-11-15 15:48:08 -08:00
Mark Shannon ae3087c638 Move exc state to generator. Fixes bpo-25612 (#1773)
Move exception state information from frame objects to coroutine (generator/thread) object where it belongs.
2017-10-22 23:41:51 +02:00
Barry Warsaw 36c1d1f1e5 PEP 553 built-in breakpoint() function (bpo-31353) (#3355)
Implement PEP 553, built-in breakpoint() with support from sys.breakpointhook(), along with documentation and tests.  Closes bpo-31353
2017-10-05 12:11:18 -04:00
Eric Snow 3f9eee6eb4 bpo-28411: Support other mappings in PyInterpreterState.modules. (#3593)
The concrete PyDict_* API is used to interact with PyInterpreterState.modules in a number of places. This isn't compatible with all dict subclasses, nor with other Mapping implementations. This patch switches the concrete API usage to the corresponding abstract API calls.

We also add a PyImport_GetModule() function (and some other helpers) to reduce a bunch of code duplication.
2017-09-15 16:35:20 -06:00
Eric Snow d393c1b227 bpo-28411: Isolate PyInterpreterState.modules (#3575)
A bunch of code currently uses PyInterpreterState.modules directly instead of PyImport_GetModuleDict(). This complicates efforts to make changes relative to sys.modules. This patch switches to using PyImport_GetModuleDict() uniformly. Also, a number of related uses of sys.modules are updated for uniformity for the same reason.

Note that this code was already reviewed and merged as part of #1638. I reverted that and am now splitting it up into more focused parts.
2017-09-14 12:18:12 -06:00
Eric Snow dae0276bb6 bpo-30860: Fix a refleak. (#3567)
Resolves bpo-31420.

(This was accidentally reverted when in #3565.)
2017-09-14 00:35:58 -07:00
Eric Snow 93c92f7d1d bpo-31404: Revert "remove modules from Py_InterpreterState (#1638)" (#3565)
PR #1638, for bpo-28411, causes problems in some (very) edge cases. Until that gets sorted out, we're reverting the merge. PR #3506, a fix on top of #1638, is also getting reverted.
2017-09-13 23:46:04 -07:00
Eric Snow 8728018624 bpo-30860: Fix a refleak. (#3506)
* Drop warnoptions from PyInterpreterState.

* Drop xoptions from PyInterpreterState.

* Don't set warnoptions and _xoptions again.

* Decref after adding to sys.__dict__.

* Drop an unused macro.

* Check sys.xoptions *before* we delete it.
2017-09-11 17:59:22 -07:00
Eric Snow 2ebc5ce42a bpo-30860: Consolidate stateful runtime globals. (#3397)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-07 23:51:28 -06:00
Nick Coghlan 5a8516701f bpo-31344: Per-frame control of trace events (GH-3417)
f_trace_lines: enable/disable line trace events
f_trace_opcodes: enable/disable opcode trace events

These are intended primarily for testing of the interpreter
itself, as they make it much easier to emulate signals
arriving at unfortunate times.
2017-09-08 10:14:16 +10:00
Antoine Pitrou a6a4dc816d bpo-31370: Remove support for threads-less builds (#3385)
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
2017-09-07 18:56:24 +02:00
Eric Snow 05351c1bd8 Revert "bpo-30860: Consolidate stateful runtime globals." (#3379)
Windows buildbots started failing due to include-related errors.
2017-09-05 21:43:08 -07:00
Eric Snow 76d5abc868 bpo-30860: Consolidate stateful runtime globals. (#2594)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-05 18:26:16 -07:00
Eric Snow 86b7afdfee bpo-28411: Remove "modules" field from Py_InterpreterState. (#1638)
sys.modules is the one true source.
2017-09-04 17:54:09 -06:00
INADA Naoki 6b42eb1764 bpo-29585: Fix sysconfig.get_config_var("PYTHONFRAMEWORK") (GH-2483)
`PYTHONFRAMEWORK` is defined in `Makefile` and it shoulnd't be used
in `pyconfig.h`.

`sysconfig.py --generate-posix-vars` reads config vars from Makefile
and `pyconfig.h`.  Conflicting variables should be avoided.

Especially, string config variables in Makefile are unquoted, but
in `pyconfig.h` are keep quoted.  So it should be private (starts with
underscore).
2017-06-29 15:31:38 +09:00
INADA Naoki a8f8d5b4bd bpo-29585: optimize site.py startup time (GH-136)
Avoid importing `sysconfig` from `site` by copying minimum code.
Python startup is 5% faster on Linux and 30% faster on macOS
2017-06-29 00:31:53 +09:00
Victor Stinner 865de27dd7 bpo-30598: _PySys_EndInit() now duplicates warnoptions (#1998)
Fix a reference in subinterpreters, like test_callbacks_leak() of
test_atexit.

warnoptions is a list used to pass options from the command line to
the sys module constructor. Before this change, the list was shared
by multiple interpreter which is not the expected behaviour. Each
interpreter should have their own independent mutable world.

This change duplicates the list in each interpreter. So each
interpreter owns its own list, so each interpreter can clear its own
list.
2017-06-08 13:27:47 +02:00
Segev Finer 48fb766f70 bpo-30567: Fix refleak in sys.getwindowsversion (#1940) 2017-06-04 20:52:27 +03:00
Eric Snow 6b4be195cd bpo-22257: Small changes for PEP 432. (#1728)
PEP 432 specifies a number of large changes to interpreter startup code, including exposing a cleaner C-API. The major changes depend on a number of smaller changes. This patch includes all those smaller changes.
2017-05-22 21:36:03 -07:00
Ned Deily 5c4b0d063a bpo-27593: Get SCM build info from git instead of hg. (#446)
sys.version and the platform module python_build(),
python_branch(), and python_revision() functions now use
git information rather than hg when building from a repo.

Based on original patches by Brett Cannon and Steve Dower.
2017-03-04 00:19:55 -05:00
Yen Chi Hsuan 72e81d00ee bpo-29556: Remove unused #include <langinfo.h> (#98)
bltinmodule.c: Added in b744ba1 and no longer necessary since d64e8a7
posixmodule.c: Added in d1cd4d4 and no longer necessary since efb00c0
pythonrun.c:   Added in 73d538b and no longer necessary since d600951
sysmodule.c:   Added in 5467d4c and no longer necessary since a2c17c5
2017-02-16 00:34:30 +01:00
Serhiy Storchaka 228b12edcc Issue #28999: Use Py_RETURN_NONE, Py_RETURN_TRUE and Py_RETURN_FALSE wherever
possible.  Patch is writen with Coccinelle.
2017-01-23 09:47:21 +02:00
Victor Stinner 7e42541d08 Use _PyObject_CallMethodIdObjArgs()
Issue #28915: Replace _PyObject_CallMethodId() with
_PyObject_CallMethodIdObjArgs() when the format string only use the format 'O'
for objects, like "(O)".

_PyObject_CallMethodIdObjArgs() avoids the code to parse a format string and
avoids the creation of a temporary tuple.
2016-12-09 00:36:19 +01:00
Victor Stinner f17c3de263 Use _PyObject_CallNoArg()
Replace:
    PyObject_CallFunctionObjArgs(callable, NULL)
with:
    _PyObject_CallNoArg(callable)
2016-12-06 18:46:19 +01:00
Victor Stinner 7bfb42d5b7 Issue #28858: Remove _PyObject_CallArg1() macro
Replace
   _PyObject_CallArg1(func, arg)
with
   PyObject_CallFunctionObjArgs(func, arg, NULL)

Using the _PyObject_CallArg1() macro increases the usage of the C stack, which
was unexpected and unwanted. PyObject_CallFunctionObjArgs() doesn't have this
issue.
2016-12-05 17:04:32 +01:00
Victor Stinner de4ae3d486 Backed out changeset b9c9691c72c5
Issue #28858: The change b9c9691c72c5 introduced a regression. It seems like
_PyObject_CallArg1() uses more stack memory than
PyObject_CallFunctionObjArgs().
2016-12-04 22:59:09 +01:00
Victor Stinner d6958ac6c0 Add sys.getandroidapilevel()
Issue #28740: Add sys.getandroidapilevel(): return the build time
API version of Android as an integer.

Function only available on Android.
2016-12-02 01:13:46 +01:00
Victor Stinner 27580c1fb5 Replace PyObject_CallFunctionObjArgs() with fastcall
* PyObject_CallFunctionObjArgs(func, NULL) => _PyObject_CallNoArg(func)
* PyObject_CallFunctionObjArgs(func, arg, NULL) => _PyObject_CallArg1(func, arg)

PyObject_CallFunctionObjArgs() allocates 40 bytes on the C stack and requires
extra work to "parse" C arguments to build a C array of PyObject*.

_PyObject_CallNoArg() and _PyObject_CallArg1() are simpler and don't allocate
memory on the C stack.

This change is part of the fastcall project. The change on listsort() is
related to the issue #23507.
2016-12-01 14:43:22 +01:00
Victor Stinner 048afd98b3 Remove CALL_PROFILE special build
Issue #28799:

* Remove the PyEval_GetCallStats() function.
* Deprecate the untested and undocumented sys.callstats() function.
* Remove the CALL_PROFILE special build

Use the sys.setprofile() function, cProfile or profile module to profile
function calls.
2016-11-28 11:59:04 +01:00
Serhiy Storchaka 85b0f5beb1 Added the const qualifier to char* variables that refer to readonly internal
UTF-8 represenatation of Unicode objects.
2016-11-20 10:16:47 +02:00
Serhiy Storchaka a98c4a984b Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.
2016-11-20 09:13:40 +02:00
Serhiy Storchaka 06515833fe Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.
2016-11-20 09:13:07 +02:00
Victor Stinner 0cae609847 Use PyThreadState_GET() in performance critical code
It seems like _PyThreadState_UncheckedGet() is not inlined as expected, even
when using gcc -O3.
2016-11-11 01:43:56 +01:00
Ned Deily 7d76c906f7 Issue #28616: merge from 3.5 2016-11-04 17:07:06 -04:00
Ned Deily da4887a88d Issue #28616: Correct help for sys.version_info releaselevel component.
Patch by Anish Tambe.
2016-11-04 17:03:34 -04:00
Steve Dower 74f4af7ac3 Issue #27932: Prevent memory leak in win32_ver(). 2016-09-17 17:27:48 -07:00
Steve Dower 1ec262be80 Issue #27932: Prevent memory leak in win32_ver(). 2016-09-17 17:25:42 -07:00
Benjamin Peterson 4fd64b9a6a remove ceval timestamp support 2016-09-09 14:57:58 -07:00
Yury Selivanov 87672d777a Issue #28003: Fix a compiler warning 2016-09-09 00:05:42 -07:00
Yury Selivanov eb6364557f Issue #28003: Implement PEP 525 -- Asynchronous Generators. 2016-09-08 22:01:51 -07:00
Steve Dower cc16be85c0 Issue #27781: Change file system encoding on Windows to UTF-8 (PEP 529) 2016-09-08 10:35:16 -07:00
Larry Hastings 10108a7b9a Issue #27355: Removed support for Windows CE. It was never finished,
and Windows CE is no longer a relevant platform for Python.
2016-09-05 15:11:23 -07:00
Victor Stinner 559bb6a713 Rename _PyObject_FastCall() to _PyObject_FastCallDict()
Issue #27809:

* Rename _PyObject_FastCall() function to _PyObject_FastCallDict()
* Add _PyObject_FastCall(), _PyObject_CallNoArg() and _PyObject_CallArg1()
  macros calling _PyObject_FastCallDict()
2016-08-22 22:48:54 +02:00
Victor Stinner c3ccaae6f3 sys_pyfile_write_unicode() now uses fast call
Issue #27128.
2016-08-20 01:24:22 +02:00
Victor Stinner 78da82bf3e call_trampoline() now uses fast call
Issue #27128.
2016-08-20 01:22:57 +02:00
doko@ubuntu.com 5553231b91 - Issue #23968: Rename the platform directory from plat-$(MACHDEP) to
plat-$(PLATFORM_TRIPLET).
  Rename the config directory (LIBPL) from config-$(LDVERSION) to
  config-$(LDVERSION)-$(PLATFORM_TRIPLET).
  Install the platform specifc _sysconfigdata module into the platform
  directory and rename it to include the ABIFLAGS.
2016-06-14 08:55:19 +02:00
Stefan Krah 1845d144bc Issue #17905: Do not guard locale include with HAVE_LANGINFO_H. 2016-04-25 21:38:53 +02:00
Serhiy Storchaka ec39756960 Issue #22570: Renamed Py_SETREF to Py_XSETREF. 2016-04-06 09:50:03 +03:00
Victor Stinner 34be807ca4 Add PYTHONMALLOC env var
Issue #26516:

* Add PYTHONMALLOC environment variable to set the Python memory
  allocators and/or install debug hooks.
* PyMem_SetupDebugHooks() can now also be used on Python compiled in release
  mode.
* The PYTHONMALLOCSTATS environment variable can now also be used on Python
  compiled in release mode. It now has no effect if set to an empty string.
* In debug mode, debug hooks are now also installed on Python memory allocators
  when Python is configured without pymalloc.
2016-03-14 12:04:26 +01:00
Victor Stinner b56837a033 Merge 3.5
Issue #26154: Add a new private _PyThreadState_UncheckedGet() function.
2016-01-20 11:19:46 +01:00
Victor Stinner bfd316e750 Add _PyThreadState_UncheckedGet()
Issue #26154: Add a new private _PyThreadState_UncheckedGet() function which
gets the current thread state, but don't call Py_FatalError() if it is NULL.

Python 3.5.1 removed the _PyThreadState_Current symbol from the Python C API to
no more expose complex and private atomic types. Atomic types depends on the
compiler or can even depend on compiler options. The new function
_PyThreadState_UncheckedGet() allows to get the variable value without having
to care of the exact implementation of atomic types.

Changes:

* Replace direct usage of the _PyThreadState_Current variable with a call to
  _PyThreadState_UncheckedGet().
* In pystate.c, replace direct usage of the _PyThreadState_Current variable
  with the PyThreadState_GET() macro for readability.
* Document also PyThreadState_Get() in pystate.h
2016-01-20 11:12:38 +01:00
Serhiy Storchaka 576f132b98 Issue #20440: Cleaning up the code by using Py_SETREF. 2016-01-05 21:27:54 +02:00
Serhiy Storchaka 2d06e84455 Issue #25923: Added the const qualifier to static constant arrays. 2015-12-25 19:53:18 +02:00
Victor Stinner cf01b68b88 sysmodule.c: reuse Py_STRINGIFY() macro 2015-11-05 11:21:38 +01:00
Victor Stinner 50856d5ae7 sys.setrecursionlimit() now raises RecursionError
Issue #25274: sys.setrecursionlimit() now raises a RecursionError if the new
recursion limit is too low depending at the current recursion depth. Modify
also the "lower-water mark" formula to make it monotonic. This mark is used to
decide when the overflowed flag of the thread state is reset.
2015-10-13 00:11:21 +02:00
Yury Selivanov d8cf382ee7 Issue 24017: Make PyEval_(Set|Get)CoroutineWrapper private 2015-06-01 12:15:23 -04:00
Benjamin Peterson baa2e562ce use our normal bracing style 2015-05-12 11:32:41 -04:00
Yury Selivanov 1dde177f82 Issue #24017: Plug ref leak. 2015-05-12 11:30:14 -04:00
Yury Selivanov 7544508f02 PEP 0492 -- Coroutines with async and await syntax. Issue #24017. 2015-05-11 22:57:16 -04:00
Victor Stinner e134a7fe36 Issue #23752: _Py_fstat() is now responsible to raise the Python exception
Add _Py_fstat_noraise() function when a Python exception is not welcome.
2015-03-30 10:09:31 +02:00
Steve Dower 3e96f324dc Issue #23451: Update pyconfig.h for Windows to require Vista headers and remove unnecessary version checks. 2015-03-02 08:01:10 -08:00
Steve Dower f2f373f593 Issue #23152: Implement _Py_fstat() to support files larger than 2 GB on Windows.
fstat() may fail with EOVERFLOW on files larger than 2 GB because the file size type is an signed 32-bit integer.
2015-02-21 08:44:05 -08:00
Serhiy Storchaka 82e07b92b3 Issue #23181: More "codepoint" -> "code point". 2015-01-18 11:33:31 +02:00
Serhiy Storchaka d3faf43f9b Issue #23181: More "codepoint" -> "code point". 2015-01-18 11:28:37 +02:00
Antoine Pitrou 5db1bb81ff Issue #22696: Add function :func:sys.is_finalizing to know about interpreter shutdown. 2014-12-07 01:28:27 +01:00
Nick Coghlan d600951748 Issue #22869: Split pythonrun into two modules
- interpreter startup and shutdown code moved to a new
  pylifecycle.c module
- Py_OptimizeFlag moved into the new module with the other
  global flags
2014-11-20 21:39:37 +10:00
Serhiy Storchaka 030e92d1a5 Issue #22193: Fixed integer overflow error in sys.getsizeof().
Fixed an error in _PySys_GetSizeOf declaration.
2014-11-15 13:21:37 +02:00
Serhiy Storchaka 547d3bc3a6 Issue #22193: Added private function _PySys_GetSizeOf() needed to implement
some __sizeof__() methods.
2014-08-14 22:21:18 +03:00
Ned Deily 529ea5d184 Issue #21891: remove extraneous semicolon. 2014-06-30 23:31:14 -07:00
Antoine Pitrou 871dfc41d3 Issue #13204: Calling sys.flags.__new__ would crash the interpreter, now it raises a TypeError. 2014-04-28 13:07:06 +02:00
Benjamin Peterson 9381343948 undefine SET_SYS_FROM_STRING_BORROW after its done being used (closes #21089) 2014-03-28 18:52:45 -04:00
Serhiy Storchaka dfe98a102e Issue #20437: Fixed 22 potential bugs when deleting objects references. 2014-02-09 13:46:20 +02:00
Serhiy Storchaka 505ff755d7 Issue #20437: Fixed 21 potential bugs when deleting objects references. 2014-02-09 13:33:53 +02:00
Christian Heimes af01f66817 Issue #16136: Remove VMS support and VMS-related code 2013-12-21 16:19:10 +01:00
Victor Stinner ed0b87d73c Fix the C definition of the sys._debugmallocstats() function: the function has
no parameter
2013-12-19 17:16:42 +01:00
Serhiy Storchaka 85c2497950 Issue #16404: Add checks for return value of PyLong_FromLong() in
sys.getwindowsversion() and ossaudiodev.setparameters().
Reported by Ned Batchelder.
2013-12-17 15:12:46 +02:00
Serhiy Storchaka 48d761e2b4 Issue #16404: Add checks for return value of PyLong_FromLong() in
sys.getwindowsversion() and ossaudiodev.setparameters().
Reported by Ned Batchelder.
2013-12-17 15:11:24 +02:00
Serhiy Storchaka 11ee080ea7 Fixed leak in sys.flags initialization. 2013-12-17 15:00:53 +02:00
Serhiy Storchaka 87a854dc73 Fixed leak in sys.flags initialization. 2013-12-17 14:59:42 +02:00
Victor Stinner fdeb6ec45a Issue #14432: Remove the thread state field from the frame structure. Fix a
crash when a generator is created in a C thread that is destroyed while the
generator is still used. The issue was that a generator contains a frame, and
the frame kept a reference to the Python state of the destroyed C thread. The
crash occurs when a trace function is setup.
2013-12-13 02:01:38 +01:00
Christian Heimes 985ecdcfc2 ssue #19183: Implement PEP 456 'secure and interchangeable hash algorithm'.
Python now uses SipHash24 on all major platforms.
2013-11-20 11:46:18 +01:00
Victor Stinner 2384714819 sysmodule.c: fix sys_update_path(), use Py_ARRAY_LENGTH() to get the size of
the fullpath buffer, not PATH_MAX. fullpath is declared using MAXPATHLEN or
MAX_PATH depending on the OS, and PATH_MAX is not declared on IRIX.
2013-11-15 17:33:43 +01:00
Victor Stinner 50e0157013 (Merge 3.3) sysmodule.c: fix sys_update_path(), use Py_ARRAY_LENGTH() to get
the size of the fullpath buffer, not PATH_MAX. fullpath is declared using
MAXPATHLEN or MAX_PATH depending on the OS, and PATH_MAX is not declared on
IRIX.
2013-11-15 17:35:31 +01:00
Victor Stinner bd303c165b Issue #19512, #19515: remove shared identifiers, move identifiers where they
are used.

Move also _Py_IDENTIFIER() defintions to the top in modified files to remove
identifiers duplicated in the same file.
2013-11-07 23:07:29 +01:00
Victor Stinner 090543736f Issue #19512: add some common identifiers to only create common strings once,
instead of creating temporary Unicode string objects

Add also more identifiers in pythonrun.c to avoid temporary Unicode string
objets for the interactive interpreter.
2013-11-06 22:41:44 +01:00
Victor Stinner d67bd45537 Issue #19512: Add _PySys_GetObjectId() and _PySys_SetObjectId() functions 2013-11-06 22:36:40 +01:00
Victor Stinner d02fbb8f71 Issue #19512: sys_displayhook() now uses an identifier for "builtins"
dictionary key and only decodes "\n" string once to write a newline.

So "builtins" and "\n" are only decoded once from UTF-8, at the first call.
2013-11-06 18:27:13 +01:00
Victor Stinner 41bb43a71e Issue #18408: Add a new PyFrame_FastToLocalsWithError() function to handle
exceptions when merging fast locals into f_locals of a frame.
PyEval_GetLocals() now raises an exception and return NULL on failure.
2013-10-29 01:19:37 +01:00
Victor Stinner 8fea252a50 Issue #18520: fix reference leak in _PySys_Init() 2013-10-27 17:15:42 +01:00
Ezio Melotti 0e1e04301b #18839: merge with 3.3. 2013-08-26 14:01:29 +03:00
Ezio Melotti 4af4d273bd #18839: document that sys.exit() will not accept a non-integer numeric value as exit status. 2013-08-26 14:00:39 +03:00
Christian Heimes ad73a9cf97 Issue #16400: Add command line option for isolated mode.
-I

    Run Python in isolated mode. This also implies -E and -s. In isolated mode
    sys.path contains neither the script’s directory nor the user’s
    site-packages directory. All PYTHON* environment variables are ignored,
    too. Further restrictions may be imposed to prevent the user from
    injecting malicious code.
2013-08-10 16:36:18 +02:00
Victor Stinner 580496005d Issue #18520: Fix _PySys_Init(), handle PyDict_SetItemString() errors 2013-07-22 22:40:00 +02:00
Victor Stinner 1c8f059019 Issue #18520: Add a new PyStructSequence_InitType2() function, same than
PyStructSequence_InitType() except that it has a return value (0 on success,
-1 on error).

 * PyStructSequence_InitType2() now raises MemoryError on memory allocation failure
 * Fix also some calls to PyDict_SetItemString(): handle error
2013-07-22 22:24:54 +02:00
Christian Heimes de0e63bd9c Issue #15905: Fix theoretical buffer overflow in handling of sys.argv[0],
prefix and exec_prefix if the operation system does not obey MAXPATHLEN.
2013-07-22 12:54:21 +02:00
Christian Heimes 60a6067709 Issue #15905: Fix theoretical buffer overflow in handling of sys.argv[0],
prefix and exec_prefix if the operation system does not obey MAXPATHLEN.
2013-07-22 12:53:32 +02:00
Andrew Kuchling c61b913078 #13226: update references from ctypes/DLFCN modules to os module 2013-06-21 10:58:41 -04:00
Antoine Pitrou f9d0b1256f Issue #13390: New function :func:sys.getallocatedblocks() returns the number of memory blocks currently allocated.
Also, the ``-R`` option to regrtest uses this function to guard against memory allocation leaks.
2012-12-09 14:28:26 +01:00
Christian Heimes 5c1c831211 RISCOS support has been removed a long time ago. Remove last remains in sys.flags code. #16501 can be closed, too. 2012-11-19 00:44:37 +01:00
Christian Heimes 743e0cd6b5 Issue #16166: Add PY_LITTLE_ENDIAN and PY_BIG_ENDIAN macros and unified
endianess detection and handling.
2012-10-17 23:52:17 +02:00
Brett Cannon 3adc7b75a5 Issue #15242: Have PyImport_GetMagicTag() return a const char *
defined in sysmodule.c instead of straight out of a Unicode object.

Thanks to Amaury Forgeot d'Arc for noticing the bug and Eric Snow for
writing the patch.
2012-07-09 14:22:12 -04:00
David Malcolm 49526f48fc Issue #14785: Add sys._debugmallocstats() to help debug low-level memory allocation issues 2012-06-22 14:55:41 -04:00
Barry Warsaw 409da157d7 Eric Snow's implementation of PEP 421.
Issue 14673: Add sys.implementation
2012-06-03 16:18:47 -04:00
Vinay Sajip 7ded1f0f69 Implemented PEP 405 (Python virtual environments). 2012-05-26 03:45:29 +01:00
Georg Brandl 2fb477c0f0 Merge 3.2: Issue #13703 plus some related test suite fixes. 2012-02-21 00:33:36 +01:00
Georg Brandl 09a7c72cad Merge from 3.1: Issue #13703: add a way to randomize the hash values of basic types (str, bytes, datetime)
in order to make algorithmic complexity attacks on (e.g.) web apps much more complicated.

The environment variable PYTHONHASHSEED and the new command line flag -R control this
behavior.
2012-02-20 21:31:46 +01:00
Georg Brandl 2daf6ae249 Issue #13703: add a way to randomize the hash values of basic types (str, bytes, datetime)
in order to make algorithmic complexity attacks on (e.g.) web apps much more complicated.

The environment variable PYTHONHASHSEED and the new command line flag -R control this
behavior.
2012-02-20 19:54:16 +01:00
Petri Lehtinen 4b0eab62f0 Merge branch 3.2
Closes #13402.
2012-02-02 21:23:15 +02:00
Petri Lehtinen 9713321f46 Document absoluteness of sys.executable
Closes #13402.
2012-02-02 20:59:50 +02:00
Benjamin Peterson ce79852077 use the static identifier api for looking up special methods
I had to move the static identifier code from unicodeobject.h to object.h in
order for this to work.
2012-01-22 11:24:29 -05:00
Victor Stinner f4afa43fd4 Issue #13226: Update sys.setdlopenflags() docstring
Refer to os.RTLD_xxx constants instead of ctypes and DLFCN modules.
2011-10-31 11:48:09 +01:00
Martin v. Löwis 1c67dd9b15 Port SetAttrString/HasAttrString to SetAttrId/GetAttrId. 2011-10-14 15:16:45 +02:00
Martin v. Löwis bd928fef42 Rename _Py_identifier to _Py_IDENTIFIER. 2011-10-14 10:20:37 +02:00
Martin v. Löwis 1ee1b6fe0d Use identifier API for PyObject_GetAttrString. 2011-10-10 18:11:30 +02:00
Martin v. Löwis afe55bba33 Add API for static strings, primarily good for identifiers.
Thanks to Konrad Schöbel and Jasper Schulz for helping with the mass-editing.
2011-10-09 10:38:36 +02:00