From 18b6ca9660370c7fb5fd50c4036be078c3267f4c Mon Sep 17 00:00:00 2001 From: Kumar Aditya Date: Sun, 23 Jun 2024 18:44:12 +0530 Subject: [PATCH] GH-120804: add docs for removal for asyncio child watchers (#120895) Co-authored-by: Alex Waygood --- Doc/library/asyncio-platforms.rst | 5 - Doc/library/asyncio-policy.rst | 185 ----------------------------- Doc/library/asyncio-subprocess.rst | 12 -- Doc/tools/.nitignore | 1 - Doc/whatsnew/3.12.rst | 20 ++-- Doc/whatsnew/3.14.rst | 20 ++++ Doc/whatsnew/3.9.rst | 2 +- 7 files changed, 31 insertions(+), 214 deletions(-) diff --git a/Doc/library/asyncio-platforms.rst b/Doc/library/asyncio-platforms.rst index 19ec726c1be0..a2a3114ad6e4 100644 --- a/Doc/library/asyncio-platforms.rst +++ b/Doc/library/asyncio-platforms.rst @@ -77,11 +77,6 @@ Subprocess Support on Windows On Windows, the default event loop :class:`ProactorEventLoop` supports subprocesses, whereas :class:`SelectorEventLoop` does not. -The :meth:`policy.set_child_watcher() -` function is also -not supported, as :class:`ProactorEventLoop` has a different mechanism -to watch child processes. - macOS ===== diff --git a/Doc/library/asyncio-policy.rst b/Doc/library/asyncio-policy.rst index 346b740a8f75..837ccc660678 100644 --- a/Doc/library/asyncio-policy.rst +++ b/Doc/library/asyncio-policy.rst @@ -79,25 +79,6 @@ The abstract event loop policy base class is defined as follows: This method should never return ``None``. - .. method:: get_child_watcher() - - Get a child process watcher object. - - Return a watcher object implementing the - :class:`AbstractChildWatcher` interface. - - This function is Unix specific. - - .. deprecated:: 3.12 - - .. method:: set_child_watcher(watcher) - - Set the current child process watcher to *watcher*. - - This function is Unix specific. - - .. deprecated:: 3.12 - .. _asyncio-policy-builtin: @@ -139,172 +120,6 @@ asyncio ships with the following built-in policies: .. availability:: Windows. -.. _asyncio-watchers: - -Process Watchers -================ - -A process watcher allows customization of how an event loop monitors -child processes on Unix. Specifically, the event loop needs to know -when a child process has exited. - -In asyncio, child processes are created with -:func:`create_subprocess_exec` and :meth:`loop.subprocess_exec` -functions. - -asyncio defines the :class:`AbstractChildWatcher` abstract base class, which child -watchers should implement, and has four different implementations: -:class:`ThreadedChildWatcher` (configured to be used by default), -:class:`MultiLoopChildWatcher`, :class:`SafeChildWatcher`, and -:class:`FastChildWatcher`. - -See also the :ref:`Subprocess and Threads ` -section. - -The following two functions can be used to customize the child process watcher -implementation used by the asyncio event loop: - -.. function:: get_child_watcher() - - Return the current child watcher for the current policy. - - .. deprecated:: 3.12 - -.. function:: set_child_watcher(watcher) - - Set the current child watcher to *watcher* for the current - policy. *watcher* must implement methods defined in the - :class:`AbstractChildWatcher` base class. - - .. deprecated:: 3.12 - -.. note:: - Third-party event loops implementations might not support - custom child watchers. For such event loops, using - :func:`set_child_watcher` might be prohibited or have no effect. - -.. class:: AbstractChildWatcher - - .. method:: add_child_handler(pid, callback, *args) - - Register a new child handler. - - Arrange for ``callback(pid, returncode, *args)`` to be called - when a process with PID equal to *pid* terminates. Specifying - another callback for the same process replaces the previous - handler. - - The *callback* callable must be thread-safe. - - .. method:: remove_child_handler(pid) - - Removes the handler for process with PID equal to *pid*. - - The function returns ``True`` if the handler was successfully - removed, ``False`` if there was nothing to remove. - - .. method:: attach_loop(loop) - - Attach the watcher to an event loop. - - If the watcher was previously attached to an event loop, then - it is first detached before attaching to the new loop. - - Note: loop may be ``None``. - - .. method:: is_active() - - Return ``True`` if the watcher is ready to use. - - Spawning a subprocess with *inactive* current child watcher raises - :exc:`RuntimeError`. - - .. versionadded:: 3.8 - - .. method:: close() - - Close the watcher. - - This method has to be called to ensure that underlying - resources are cleaned-up. - - .. deprecated:: 3.12 - - -.. class:: ThreadedChildWatcher - - This implementation starts a new waiting thread for every subprocess spawn. - - It works reliably even when the asyncio event loop is run in a non-main OS thread. - - There is no noticeable overhead when handling a big number of children (*O*\ (1) each - time a child terminates), but starting a thread per process requires extra memory. - - This watcher is used by default. - - .. versionadded:: 3.8 - -.. class:: MultiLoopChildWatcher - - This implementation registers a :py:data:`SIGCHLD` signal handler on - instantiation. That can break third-party code that installs a custom handler for - :py:data:`SIGCHLD` signal. - - The watcher avoids disrupting other code spawning processes - by polling every process explicitly on a :py:data:`SIGCHLD` signal. - - There is no limitation for running subprocesses from different threads once the - watcher is installed. - - The solution is safe but it has a significant overhead when - handling a big number of processes (*O*\ (*n*) each time a - :py:data:`SIGCHLD` is received). - - .. versionadded:: 3.8 - - .. deprecated:: 3.12 - -.. class:: SafeChildWatcher - - This implementation uses active event loop from the main thread to handle - :py:data:`SIGCHLD` signal. If the main thread has no running event loop another - thread cannot spawn a subprocess (:exc:`RuntimeError` is raised). - - The watcher avoids disrupting other code spawning processes - by polling every process explicitly on a :py:data:`SIGCHLD` signal. - - This solution is as safe as :class:`MultiLoopChildWatcher` and has the same *O*\ (*n*) - complexity but requires a running event loop in the main thread to work. - - .. deprecated:: 3.12 - -.. class:: FastChildWatcher - - This implementation reaps every terminated processes by calling - ``os.waitpid(-1)`` directly, possibly breaking other code spawning - processes and waiting for their termination. - - There is no noticeable overhead when handling a big number of - children (*O*\ (1) each time a child terminates). - - This solution requires a running event loop in the main thread to work, as - :class:`SafeChildWatcher`. - - .. deprecated:: 3.12 - -.. class:: PidfdChildWatcher - - This implementation polls process file descriptors (pidfds) to await child - process termination. In some respects, :class:`PidfdChildWatcher` is a - "Goldilocks" child watcher implementation. It doesn't require signals or - threads, doesn't interfere with any processes launched outside the event - loop, and scales linearly with the number of subprocesses launched by the - event loop. The main disadvantage is that pidfds are specific to Linux, and - only work on recent (5.3+) kernels. - - .. versionadded:: 3.9 - - .. _asyncio-custom-policies: Custom Policies diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index 817a6ff3052f..b477a7fa2d37 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -316,18 +316,6 @@ default. On Windows subprocesses are provided by :class:`ProactorEventLoop` only (default), :class:`SelectorEventLoop` has no subprocess support. -On UNIX *child watchers* are used for subprocess finish waiting, see -:ref:`asyncio-watchers` for more info. - - -.. versionchanged:: 3.8 - - UNIX switched to use :class:`ThreadedChildWatcher` for spawning subprocesses from - different threads without any limitation. - - Spawning a subprocess with *inactive* current child watcher raises - :exc:`RuntimeError`. - Note that alternative event loop implementations might have own limitations; please refer to their documentation. diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 4790136a75cb..66914f79f3d4 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -17,7 +17,6 @@ Doc/howto/descriptor.rst Doc/howto/enum.rst Doc/library/ast.rst Doc/library/asyncio-extending.rst -Doc/library/asyncio-policy.rst Doc/library/asyncio-subprocess.rst Doc/library/collections.rst Doc/library/dbm.rst diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 93d18ffc76d0..cfc87cb9089c 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -652,14 +652,14 @@ asyncio making some use-cases 2x to 5x faster. (Contributed by Jacob Bower & Itamar Oren in :gh:`102853`, :gh:`104140`, and :gh:`104138`) -* On Linux, :mod:`asyncio` uses :class:`asyncio.PidfdChildWatcher` by default +* On Linux, :mod:`asyncio` uses :class:`!asyncio.PidfdChildWatcher` by default if :func:`os.pidfd_open` is available and functional instead of - :class:`asyncio.ThreadedChildWatcher`. + :class:`!asyncio.ThreadedChildWatcher`. (Contributed by Kumar Aditya in :gh:`98024`.) * The event loop now uses the best available child watcher for each platform - (:class:`asyncio.PidfdChildWatcher` if supported and - :class:`asyncio.ThreadedChildWatcher` otherwise), so manually + (:class:`!asyncio.PidfdChildWatcher` if supported and + :class:`!asyncio.ThreadedChildWatcher` otherwise), so manually configuring a child watcher is not recommended. (Contributed by Kumar Aditya in :gh:`94597`.) @@ -1162,15 +1162,15 @@ Deprecated * :mod:`asyncio`: - * The child watcher classes :class:`asyncio.MultiLoopChildWatcher`, - :class:`asyncio.FastChildWatcher`, :class:`asyncio.AbstractChildWatcher` - and :class:`asyncio.SafeChildWatcher` are deprecated and + * The child watcher classes :class:`!asyncio.MultiLoopChildWatcher`, + :class:`!asyncio.FastChildWatcher`, :class:`!asyncio.AbstractChildWatcher` + and :class:`!asyncio.SafeChildWatcher` are deprecated and will be removed in Python 3.14. (Contributed by Kumar Aditya in :gh:`94597`.) - * :func:`asyncio.set_child_watcher`, :func:`asyncio.get_child_watcher`, - :meth:`asyncio.AbstractEventLoopPolicy.set_child_watcher` and - :meth:`asyncio.AbstractEventLoopPolicy.get_child_watcher` are deprecated + * :func:`!asyncio.set_child_watcher`, :func:`!asyncio.get_child_watcher`, + :meth:`!asyncio.AbstractEventLoopPolicy.set_child_watcher` and + :meth:`!asyncio.AbstractEventLoopPolicy.get_child_watcher` are deprecated and will be removed in Python 3.14. (Contributed by Kumar Aditya in :gh:`94597`.) diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index 2eefa232cdcd..8806bbff9c86 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -173,6 +173,26 @@ ast (Contributed by Alex Waygood in :gh:`119562`.) +asyncio +------- + +* Remove the following classes and functions. They were all deprecated and + emitted deprecation warnings since Python 3.12: + + * :class:`!asyncio.AbstractChildWatcher` + * :class:`!asyncio.SafeChildWatcher` + * :class:`!asyncio.MultiLoopChildWatcher` + * :class:`!asyncio.FastChildWatcher` + * :class:`!asyncio.ThreadedChildWatcher` + * :class:`!asyncio.PidfdChildWatcher` + * :meth:`!asyncio.AbstractEventLoopPolicy.get_child_watcher` + * :meth:`!asyncio.AbstractEventLoopPolicy.set_child_watcher` + * :func:`!asyncio.get_child_watcher` + * :func:`!asyncio.set_child_watcher` + + (Contributed by Kumar Aditya in :gh:`120804`.) + + collections.abc --------------- diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 90bdcf954161..1bce90d248d6 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -352,7 +352,7 @@ that schedules a shutdown for the default executor that waits on the :func:`asyncio.run` has been updated to use the new :term:`coroutine`. (Contributed by Kyle Stanley in :issue:`34037`.) -Added :class:`asyncio.PidfdChildWatcher`, a Linux-specific child watcher +Added :class:`!asyncio.PidfdChildWatcher`, a Linux-specific child watcher implementation that polls process file descriptors. (:issue:`38692`) Added a new :term:`coroutine` :func:`asyncio.to_thread`. It is mainly used for