Commit graph

99 commits

Author SHA1 Message Date
Victor Stinner 331a6a56e9
bpo-36763: Implement the PEP 587 (GH-13592)
* Add a whole new documentation page:
  "Python Initialization Configuration"
* PyWideStringList_Append() return type is now PyStatus,
  instead of int
* PyInterpreterState_New() now calls PyConfig_Clear() if
  PyConfig_InitPythonConfig() fails.
* Rename files:

  * Python/coreconfig.c => Python/initconfig.c
  * Include/cpython/coreconfig.h => Include/cpython/initconfig.h
  * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h

* Rename structures

  * _PyCoreConfig => PyConfig
  * _PyPreConfig => PyPreConfig
  * _PyInitError => PyStatus
  * _PyWstrList => PyWideStringList

* Rename PyConfig fields:

  * use_module_search_paths => module_search_paths_set
  * module_search_path_env => pythonpath_env

* Rename PyStatus field: _func => func
* PyInterpreterState: rename core_config field to config
* Rename macros and functions:

  * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv()
  * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv()
  * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString()
  * _PyInitError_Failed() => PyStatus_Exception()
  * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx
  * _Py_UnixMain() => Py_BytesMain()
  * _Py_ExitInitError() => Py_ExitStatusException()
  * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs()
  * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs()
  * _Py_PreInitialize() => Py_PreInitialize()
  * _Py_RunMain() => Py_RunMain()
  * _Py_InitializeFromConfig() => Py_InitializeFromConfig()
  * _Py_INIT_XXX() => _PyStatus_XXX()
  * _Py_INIT_FAILED() => _PyStatus_EXCEPTION()

* Rename 'err' PyStatus variables to 'status'
* Convert RUN_CODE() macro to config_run_code() static inline function
* Remove functions:

  * _Py_InitializeFromArgs()
  * _Py_InitializeFromWideArgs()
  * _PyInterpreterState_GetCoreConfig()
2019-05-27 16:39:22 +02:00
Victor Stinner 410759fba8
bpo-36763: Remove _PyCoreConfig.dll_path (GH-13402) 2019-05-18 04:17:01 +02:00
Victor Stinner 871ff77c1c
bpo-36763: Add _PyInitError functions (GH-13395)
* Add _PyInitError functions:

  * _PyInitError_Ok()
  * _PyInitError_Error()
  * _PyInitError_NoMemory()
  * _PyInitError_Exit()
  * _PyInitError_IsError()
  * _PyInitError_IsExit()
  * _PyInitError_Failed()

* frozenmain.c and _testembed.c now use functions rather than macros.
* Move _Py_INIT_xxx() macros to the internal API.
* Move _PyWstrList_INIT macro to the internal API.
2019-05-17 23:54:00 +02:00
Steve Dower 1c3de541e6
bpo-34977: Use venv redirector instead of original python.exe on Windows (GH-11029) 2018-12-10 08:11:21 -08:00
Victor Stinner cb0b78a070
Revert "bpo-34977: Add Windows App Store package (GH-10245)" (GH-11019)
This reverts commit 468a15aaf9.
2018-12-07 12:57:43 +01:00
Zackery Spytz 4c49da0cb7 bpo-35436: Add missing PyErr_NoMemory() calls and other minor bug fixes. (GH-11015)
Set MemoryError when appropriate, add missing failure checks,
and fix some potential leaks.
2018-12-07 12:11:30 +02:00
Steve Dower 468a15aaf9
bpo-34977: Add Windows App Store package (GH-10245) 2018-12-06 21:09:20 -08:00
Steve Dower 177a41a07b
bpo-34725: Adds _Py_SetProgramFullPath so embedders may override sys.executable (GH-9860) 2018-11-17 20:41:48 -08:00
Victor Stinner 621cebe81b
bpo-35081: Rename internal headers (GH-10275)
Rename Include/internal/ headers:

* pycore_hash.h -> pycore_pyhash.h
* pycore_lifecycle.h -> pycore_pylifecycle.h
* pycore_mem.h -> pycore_pymem.h
* pycore_state.h -> pycore_pystate.h

Add missing headers to Makefile.pre.in and PCbuild:

* pycore_condvar.h.
* pycore_hamt.h
* pycore_pyhash.h
2018-11-12 16:53:38 +01:00
Victor Stinner 27e2d1f219
bpo-35081: Add pycore_ prefix to internal header files (GH-10263)
* Rename Include/internal/ header files:

  * pyatomic.h -> pycore_atomic.h
  * ceval.h -> pycore_ceval.h
  * condvar.h -> pycore_condvar.h
  * context.h -> pycore_context.h
  * pygetopt.h -> pycore_getopt.h
  * gil.h -> pycore_gil.h
  * hamt.h -> pycore_hamt.h
  * hash.h -> pycore_hash.h
  * mem.h -> pycore_mem.h
  * pystate.h -> pycore_state.h
  * warnings.h -> pycore_warnings.h

* PCbuild project, Makefile.pre.in, Modules/Setup: add the
  Include/internal/ directory to the search paths of header files.
* Update includes. For example, replace #include "internal/mem.h"
  with #include "pycore_mem.h".
2018-11-01 00:52:28 +01:00
erikjanss 6cf8255912 bpo-34217: Use lowercase header for Windows (GH-8453) 2018-07-25 09:41:46 +09:00
Victor Stinner d19d8d5279
bpo-34170: Add _PyCoreConfig.isolated (GH-8417)
* _PyCoreConfig: add isolated and site_import attributes
* Replace Py_IgnoreEnvironment with config->ignore_environment when
  reading the current configuration
* _PyCoreConfig_Read() now sets ignore_environment, utf8_mode,
  isolated and site_import from Py_IgnoreEnvironment, Py_UTF8Mode,
  Py_IsolatedFlag and Py_NoSiteFlag
* _Py_InitializeCore() now sets Py_xxx flags from the configuration
* pymain_read_conf() now uses _PyCoreConfig_Copy() to save/restore
  the configuration.
* Rename _disable_importlib of _PyCoreConfig to _install_importlib
* _PyCoreConfig_SetGlobalConfig() now also set
  Py_HashRandomizationFlag
* Replace !Py_NoSiteFlag with core_config->site_import
2018-07-24 13:55:48 +02:00
Victor Stinner f2626ce6d4
bpo-34170: _PyCoreConfig_Read() leaves Py_IsolatedFlag unchanged (GH-8361)
* _PyCoreConfig_Read() no longer directly modifies Py_IsolatedFlag
  and Py_NoSiteFlag global configuration flags. The function now
  requires two pointers to integer, so these flags can be set later,
  to avoid side effets in _PyCoreConfig_Read().
* pathconfig_global_init() now leaves Py_IsolatedFlag and
  Py_NoSiteFlag unchanged.
* Fix pathconfig_global_init(): avoid computing the path
  configuration twice, use _PyCoreConfig_SetPathConfig().
2018-07-21 03:54:20 +02:00
Victor Stinner b1147e43da
bpo-34170: Rework _PyCoreConfig_Read() to avoid side effect (GH-8353)
Rework _PyCoreConfig_Read() function which *reads* core configuration
to not *modify* the path configuration.

A new _PyCoreConfig_SetPathConfig() function now recreates the path
configuration from the core configuration. This function is now
called very late in _Py_InitializeCore(), just before calling
initimport().

Changes:

* Add _PyCoreConfig.dll_path
* Py_SetPath() now fails with a fatal python error on memory
   allocation failure.
* Rename _PyPathConfig_Calculate() to _PyPathConfig_Calculate_impl()
* Replace _PyPathConfig_Init() with _PyPathConfig_Calculate(): the
  function now requires a _PyPathConfig
* Add _PyPathConfig_SetGlobal() to set the _Py_path_config global
  variable.
* Add _PyCoreConfig_InitPathConfig(): compute the path configuration
* Add _PyCoreConfig_SetPathConfig(): set path configuration from core
  configuration
* Rename wstrlist_append() to _Py_wstrlist_append()
* _Py_wstrlist_append() now handles integer overflow.
2018-07-21 02:06:16 +02:00
Steve Dower 48e8c82fc6
bpo-32457: Improves handling of denormalized executable path when launching Python (GH-5756) 2018-02-22 10:39:26 -08:00
Victor Stinner 8ded5b8037
bpo-32030: Add _PyCoreConfig.module_search_paths (#4954)
_PyCoreConfig_Read() is now responsible to compute sys.path.
So sys.path is now computed before calling _Py_InitializeCore().

Changes:

* Add module_search_path, module_search_paths, executable, prefix,
  base_prefix, exec_prefix and base_exec_prefix to _PyCoreConfig.
* _PyMainInterpreterConfig_Read() now only converts wchar_t** lists
  into a Python list, it doesn't compute sys.path anymore.
2018-01-24 17:03:28 +01:00
Victor Stinner 9bee329130
bpo-32030: Add _Py_FindEnvConfigValue() (#4963)
Add a new _Py_FindEnvConfigValue() function: code shared between
Windows and Unix implementations of _PyPathConfig_Calculate() to read
the pyenv.cfg file.

_Py_FindEnvConfigValue() now uses _Py_DecodeUTF8_surrogateescape()
instead of using a Python Unicode string, the Python API must not be
used early during Python initialization. Same change in Unix
search_for_exec_prefix(): use _Py_DecodeUTF8_surrogateescape().

Cleanup also encode_current_locale(): PyMem_RawFree/PyMem_Free can be
called with NULL.

Fix also "NUL byte" => "NULL byte" typo.
2017-12-21 16:49:13 +01:00
Victor Stinner b5fd9ad05e
bpo-32030: Rewrite _PyMainInterpreterConfig (#4854)
_PyMainInterpreterConfig now contains Python objects, whereas
_PyCoreConfig contains wchar_t* strings.

Core config:

* Rename _PyMainInterpreterConfig_ReadEnv() to _PyCoreConfig_ReadEnv()
* Move 3 strings from _PyMainInterpreterConfig to _PyCoreConfig:
  module_search_path_env, home, program_name.
* Add _PyCoreConfig_Clear()
* _PyPathConfig_Calculate() now takes core config rather than main
  config
* _PyMainInterpreterConfig_Read() now requires also a core config

Main config:

* Add _PyMainInterpreterConfig.module_search_path: sys.path list
* Add _PyMainInterpreterConfig.argv: sys.argv list
* _PyMainInterpreterConfig_Read() now computes module_search_path
2017-12-14 02:20:52 +01:00
Serhiy Storchaka 4ae06c5337
bpo-32241: Add the const qualifire to declarations of umodifiable strings. (#4748) 2017-12-12 13:55:04 +02:00
Victor Stinner 31a8393cf6
Revert "bpo-32197: Try to fix a compiler error on OS X introduced in bpo-32030. (#4681)" (#4694)
* Revert "bpo-32197: Try to fix a compiler error on OS X introduced in bpo-32030. (#4681)"

This reverts commit 13badcbc60.

Re-apply commits:

* "bpo-32030: _PyPathConfig_Init() sets home and program_name (#4673)"
  commit af5a895073.
* "bpo-32030: Fix config_get_program_name() on macOS (#4669)"
  commit e23c06e2b0.
* "bpo-32030: Add Python/pathconfig.c (#4668)"
  commit 0ea395ae96.
* "bpo-32030: Don't call _PyPathConfig_Fini() in Py_FinalizeEx() (#4667)"
  commit ebac19dad6.
* "bpo-32030: Fix Py_GetPath(): init program_name (#4665)"
  commit 9ac3d88827.

* Fix compilation error on macOS
2017-12-04 13:39:15 +01:00
Serhiy Storchaka 13badcbc60
bpo-32197: Try to fix a compiler error on OS X introduced in bpo-32030. (#4681)
* Revert "bpo-32030: _PyPathConfig_Init() sets home and program_name (#4673)"

This reverts commit af5a895073.

* Revert "bpo-32030: Fix config_get_program_name() on macOS (#4669)"

This reverts commit e23c06e2b0.

* Revert "bpo-32030: Add Python/pathconfig.c (#4668)"

This reverts commit 0ea395ae96.

* Revert "bpo-32030: Don't call _PyPathConfig_Fini() in Py_FinalizeEx() (#4667)"

This reverts commit ebac19dad6.

* Revert "bpo-32030: Fix Py_GetPath(): init program_name (#4665)"

This reverts commit 9ac3d88827.
2017-12-02 21:36:00 +02:00
Victor Stinner af5a895073
bpo-32030: _PyPathConfig_Init() sets home and program_name (#4673)
_PyPathConfig_Init() now also initialize home and program_name:

* Rename existing _PyPathConfig_Init() to _PyPathConfig_Calculate().
  Add a new _PyPathConfig_Init() function in pathconfig.c which
  handles the _Py_path_config variable and call
  _PyPathConfig_Calculate().
* Add home and program_name fields to _PyPathConfig.home
* _PyPathConfig_Init() now initialize home and program_name
  from main_config
* Py_SetProgramName(), Py_SetPythonHome() and Py_GetPythonHome() now
  calls Py_FatalError() on failure, instead of silently ignoring
  failures.
* config_init_home() now gets directly _Py_path_config.home to only
  get the value set by Py_SetPythonHome(), or NULL if
  Py_SetPythonHome() was not called.
* config_get_program_name() now gets directly
  _Py_path_config.program_name to only get the value set by
  Py_SetProgramName(), or NULL if Py_SetProgramName() was not called.
* pymain_init_python() doesn't call Py_SetProgramName() anymore,
  _PyPathConfig_Init() now always sets the program name
* Call _PyMainInterpreterConfig_Read() in
  pymain_parse_cmdline_envvars_impl() to control the memory allocator
* C API documentation: it's no more safe to call Py_GetProgramName()
  before Py_Initialize().
2017-12-02 10:11:32 +01:00
Victor Stinner 0ea395ae96
bpo-32030: Add Python/pathconfig.c (#4668)
* Factorize code from PC/getpathp.c and Modules/getpath.c to remove
  duplicated code
* rename pathconfig_clear() to _PyPathConfig_Clear()
* Inline _PyPathConfig_Fini() in pymain_impl() and then remove it,
  since it's a oneliner
2017-12-01 20:50:58 +01:00
Victor Stinner ebac19dad6
bpo-32030: Don't call _PyPathConfig_Fini() in Py_FinalizeEx() (#4667)
Changes:

* _PyPathConfig_Fini() cannot be called in Py_FinalizeEx().
  Py_Initialize() and Py_Finalize() can be called multiple times, but
  it must not "forget" parameters set by Py_SetProgramName(),
  Py_SetPath() or Py_SetPythonHome(), whereas _PyPathConfig_Fini()
  clear all these parameters.
* config_get_program_name() and calculate_program_full_path() now
  also decode paths using Py_DecodeLocale() to use the
  surrogateescape error handler, rather than decoding using
  mbstowcs() which is strict.
* Change _Py_CheckPython3() prototype: () => (void)
* Truncate a few lines which were too long
2017-12-01 20:09:52 +01:00
Victor Stinner 9ac3d88827
bpo-32030: Fix Py_GetPath(): init program_name (#4665)
* _PyMainInterpreterConfig_ReadEnv() now sets program_name from
  environment variables and pymain_parse_envvars() implements the
  falls back on argv[0].
* Remove _PyMain.program_name: use the program_name from
  _PyMainInterpreterConfig
* Move the Py_SetProgramName() call back to pymain_init_python(),
  just before _Py_InitializeCore().
* pathconfig_global_init() now also calls
  _PyMainInterpreterConfig_Read() to set program_name if it isn't set
  yet
* Cleanup PyCalculatePath: pass main_config to subfunctions to get
  directly fields from main_config (home, module_search_path_env and
  program_name)
2017-12-01 19:30:41 +01:00
Victor Stinner b64de46aae
bpo-32030: Cleanup "path config" code (#4663)
* Rename PyPathConfig structure to _PyPathConfig and move it to
  Include/internal/pystate.h
* Rename path_config to _Py_path_config
* _PyPathConfig: Rename program_name field to program_full_path
* Add assert(str != NULL); to _PyMem_RawWcsdup(), _PyMem_RawStrdup()
  and _PyMem_Strdup().
* Rename calculate_path() to pathconfig_global_init(). The function
  now does nothing if it's already initiallized.
2017-12-01 18:27:09 +01:00
Victor Stinner 9316ee4da2
bpo-32030: Add _PyPathConfig_Init() (#4551)
* Add _PyPathConfig_Init() and _PyPathConfig_Fini()
* Remove _Py_GetPathWithConfig()
* _PyPathConfig_Init() returns _PyInitError to allow to handle errors
  properly
* Add pathconfig_clear()
* Windows calculate_path_impl(): replace Py_FatalError() with
  _PyInitError
* Py_FinalizeEx() now calls _PyPathConfig_Fini() to release memory
* Fix _Py_InitializeMainInterpreter() regression: don't initialize
  path config if _disable_importlib is false
* PyPathConfig now uses dynamically allocated memory
2017-11-25 03:17:57 +01:00
Victor Stinner f04ebe2a4d
bpo-32030: Add _PyMainInterpreterConfig.program_name (#4548)
* Py_Main() now calls Py_SetProgramName() earlier to be able to get
  the program name in _PyMainInterpreterConfig_ReadEnv().
* Rename prog to program_name
* Rename progpath to program_name
2017-11-25 00:01:23 +01:00
Victor Stinner 46972b7bc3
bpo-32030: Add _PyMainInterpreterConfig_ReadEnv() (#4542)
Py_GetPath() and Py_Main() now call
_PyMainInterpreterConfig_ReadEnv() to share the same code to get
environment variables.

Changes:

* Add _PyMainInterpreterConfig_ReadEnv()
* Add _PyMainInterpreterConfig_Clear()
* Add _PyMem_RawWcsdup()
* _PyMainInterpreterConfig: rename pythonhome to home
* Rename _Py_ReadMainInterpreterConfig() to
  _PyMainInterpreterConfig_Read()
* Use _Py_INIT_USER_ERR(), instead of _Py_INIT_ERR(), for decoding
  errors: the user is able to fix the issue, it's not a bug in
  Python. Same change was made in _Py_INIT_NO_MEMORY().
* Remove _Py_GetPythonHomeWithConfig()
2017-11-24 22:55:40 +01:00
Victor Stinner 0327bde9da
bpo-32030: Rewrite calculate_path() (#4521)
* calculate_path() rewritten in Modules/getpath.c and PC/getpathp.c
* Move global variables into a new PyPathConfig structure.
* calculate_path():

  * Split the huge calculate_path() function into subfunctions.
  * Add PyCalculatePath structure to pass data between subfunctions.
  * Document PyCalculatePath fields.
  * Move cleanup code into a new calculate_free() subfunction
  * calculate_init() now handles Py_DecodeLocale() failures properly
  * calculate_path() is now atomic: only replace PyPathConfig
    (path_config) at once on success.

* _Py_GetPythonHomeWithConfig() now returns an error on failure
* Add _Py_INIT_NO_MEMORY() helper: report a memory allocation failure
* Coding style fixes (PEP 7)
2017-11-23 17:03:20 +01:00
Victor Stinner 1f15111a6e
bpo-32030: Add _PyMainInterpreterConfig.pythonhome (#4513)
* Py_Main() now reads the PYTHONHOME environment variable
* Add _Py_GetPythonHomeWithConfig() private function
* Add _PyWarnings_InitWithConfig()
* init_filters() doesn't get the current core configuration from the
  current interpreter or Python thread anymore. Pass explicitly the
  configuration to _PyWarnings_InitWithConfig().
* _Py_InitializeCore() now fails on _PyWarnings_InitWithConfig()
  failure.
* Pass configuration as constant
2017-11-23 10:43:14 +01:00
Victor Stinner e32e79f7d8
bpo-32030: Move PYTHONPATH to _PyMainInterpreterConfig (#4511)
Move _PyCoreConfig.module_search_path_env to _PyMainInterpreterConfig
structure.
2017-11-23 01:49:45 +01:00
Victor Stinner d434110974
bpo-32030: Add _PyCoreConfig.module_search_path_env (#4504)
Changes:

* Py_Main() initializes _PyCoreConfig.module_search_path_env from
  the PYTHONPATH environment variable.
* PyInterpreterState_New() now initializes core_config and config
  fields
* Compute sys.path a little bit ealier in
  _Py_InitializeMainInterpreter() and new_interpreter()
* Add _Py_GetPathWithConfig() private function.
2017-11-23 00:12:09 +01:00
luzpaz a5293b4ff2 Fix miscellaneous typos (#4275) 2017-11-05 15:37:50 +02:00
Xiang Zhang 0710d75425 bpo-29770: remove outdated PYO related info (GH-590) 2017-03-11 13:02:52 +08:00
Steve Dower 5f9193a6ed Issue #29326: Ignores blank lines in ._pth files (Patch by Alexey Izbyshev) 2017-02-04 15:19:29 -08:00
Steve Dower c6dd415252 Issue #28522: Fixes mishandled buffer reallocation in getpathp.c 2016-10-27 14:28:07 -07:00
Steve Dower ed51b26858 Issue #28137: Renames Windows path file to ._pth
Issue #28138: Windows ._pth file should allow import site
2016-09-17 12:54:06 -07:00
Martin Panter fd13c0fa01 One more spelling fix 2016-09-10 10:45:28 +00:00
Zachary Ware c4b53afce4 Issue #28046: Remove platform-specific directories from sys.path 2016-09-09 17:59:49 -07:00
Steve Dower 3ceb573a72 Fix call to PathCombineW. 2016-09-09 15:53:58 -07:00
Steve Dower 502893896a Issue #27874: Allows use of pythonXX.zip file as landmark on Windows 2016-09-09 14:22:43 -07:00
Steve Dower 4db86bc1b4 Changes pyvenv.cfg trick into an actual sys.path file. 2016-09-09 09:17:35 -07:00
Victor Stinner ccb1f8cb1a getpathp.c: fix compiler warning
wcsnlen_s() result type is size_t.
2016-03-23 11:31:58 +01:00
Serhiy Storchaka e0cb9da9d6 Issue #25893: Removed unused variable reqdSize.
Added test for return code for the last RegQueryValueExW.
2015-12-18 09:54:19 +02:00
Steve Dower f64b9d5c71 Fixes handling of long values of PYTHONPATH on Windows. 2015-05-23 17:34:50 -07:00
Steve Dower 4a7fe7e397 Issue #23955: Add pyvenv.cfg option to suppress registry/environment lookup for generating sys.path.
Also cleans up and secures getpathp.c
2015-05-22 15:10:10 -07:00
Steve Dower f63dab5a84 Back-out wcstok deprecation suppression and updates calls to use wcstok_s. 2015-02-25 20:48:01 -08:00
Victor Stinner 1a7425f67a Issue #18203: Replace malloc() with PyMem_RawMalloc() at Python initialization
* Replace malloc() with PyMem_RawMalloc()
* Replace PyMem_Malloc() with PyMem_RawMalloc() where the GIL is not held.
* _Py_char2wchar() now returns a buffer allocated by PyMem_RawMalloc(), instead
  of PyMem_Malloc()
2013-07-07 16:25:15 +02:00
Victor Stinner 8bda465cae Issue #9566: Fix compiler warning on Windows 64-bit 2013-06-05 00:22:34 +02:00