Commit graph

698 commits

Author SHA1 Message Date
Fantix King f683f46425 bpo-34745: Fix asyncio sslproto memory issues (GH-12386)
* Fix handshake timeout leak in asyncio/sslproto

Refs MagicStack/uvloop#222

* Break circular ref _SSLPipe <-> SSLProtocol

* bpo-34745: Fix asyncio ssl memory leak

* Break circular ref SSLProtocol <-> UserProtocol

* Add NEWS entry
2019-03-17 18:51:10 -04:00
Inada Naoki f345170940
asyncio: use dict instead of OrderedDict (GH-11710) 2019-02-05 17:04:40 +09:00
Victor Stinner c9f872b0bd
bpo-23846: Fix ProactorEventLoop._write_to_self() (GH-11566)
asyncio.ProactorEventLoop now catchs and logs send errors when the
self-pipe is full: BaseProactorEventLoop._write_to_self() now catchs
and logs OSError exceptions, as done by
BaseSelectorEventLoop._write_to_self().
2019-01-15 13:58:38 +01:00
Victor Stinner b91140fdb1
bpo-11555: Enhance IocpProactor.close() log again (GH-11563)
Add repr(self) to the log to display the number of pending overlapped
in the log.
2019-01-15 12:13:48 +01:00
Victor Stinner b1e45739d8
bpo-34323: Enhance IocpProactor.close() log (GH-11555)
IocpProactor.close() now uses time to decide when to log: wait 1
second before the first log, then log every second. Log also the
number of seconds since close() was called.
2019-01-15 11:48:00 +01:00
Victor Stinner fb2c3465f0
asyncio: __del__() keep reference to warnings.warn (GH-11491)
* asyncio: __del__() keep reference to warnings.warn

The __del__() methods of asyncio classes now keep a strong reference
to the warnings.warn() to be able to display the ResourceWarning
warning in more cases. Ensure that the function remains available if
instances are destroyed late during Python shutdown (while module
symbols are cleared).

* Rename warn parameter to _warn

"_warn" name is a hint that it's not the regular warnings.warn()
function.
2019-01-10 11:24:40 +01:00
Victor Stinner 9b07681c09
IocpProactor: prevent modification if closed (GH-11494)
* _wait_for_handle(), _register() and _unregister() methods of
  IocpProactor now raise an exception if closed
* Add "closed" to IocpProactor.__repr__()
* Simplify IocpProactor.close()
2019-01-10 11:23:26 +01:00
Victor Stinner 80fda712c8
bpo-35682: Fix _ProactorBasePipeTransport._force_close() (GH-11462)
bpo-32622, bpo-35682: Fix asyncio.ProactorEventLoop.sendfile(): don't
attempt to set the result of an internal future if it's already done.

Fix asyncio _ProactorBasePipeTransport._force_close(): don't set the
result of _empty_waiter if it's already done.
2019-01-08 02:46:59 +01:00
Vladimir Matveev 67ba547cf0 bpo-23057: Use 'raise' to emulate ctrl-c in proactor tests (#11274) 2019-01-05 22:44:59 +02:00
Vladimir Matveev b5c8cfa1da bpo-23057: add loop self socket as wakeup fd for signals (#11135) 2018-12-18 23:56:17 +02:00
Andrew Svetlov 5344501ad1
bpo-35394: Add empty slots to abstract asyncio protocols (#10889)
* bpo-35394: Add empty slots to abstract asyncio protocols

* Add missing test file
2018-12-11 19:07:05 +02:00
Andrew Svetlov 3bc0ebab17
bpo-35380: Enable TCP_NODELAY for proactor event loop (#10867) 2018-12-03 21:08:13 +02:00
Srinivas Thatiparthy (శ్రీనివాస్ తాటిపర్తి) 70f61b933f bpo-35202: Remove unused imports in Lib directory (GH-10450) 2018-11-15 21:59:19 +09:00
Andrew Svetlov 7438792607
bpo-30064: Refactor sock_* asyncio API (#10419) 2018-11-12 19:00:22 +02:00
Vincent Michel fd512d7645 bpo-35065: Remove StreamReaderProtocol._untrack_reader (#10212)
The call to `_untrack_reader` is performed too soon, causing the protocol
to forget about the reader before `connection_lost` can run and feed the
EOF to the reader. See bpo-35065.
2018-11-08 14:21:47 +02:00
Andrew Svetlov 97cf082872 bpo-34970: Protect tasks weak set manipulation in asyncio.all_tasks() (GH-9837)
https://bugs.python.org/issue34970
2018-10-13 11:12:40 -07:00
twisteroid ambassador c880ffe7d2 bpo-34769: Thread safety for _asyncgen_finalizer_hook(). (GH-9716) 2018-10-09 11:30:21 -04:00
Andrew Svetlov d5bd036138 bpo-34849: Don't log wating for selector.select in asyncio loop iteration (GH-9641)
The waiting is pretty normal for any asyncio program, logging its time just adds
a noise to logs without any useful information provided.



https://bugs.python.org/issue34849
2018-09-29 22:28:40 -07:00
Yury Selivanov fad6af2744
asyncio/docs: Replace Python 4.0 -> 3.10 (GH-9579) 2018-09-25 17:44:52 -04:00
Victor Stinner 6ea29c5e90 bpo-34687: Make asynico use ProactorEventLoop by default (GH-9538) 2018-09-25 11:27:08 -04:00
João Júnior 558c49bcf3 bpo-34728: Remove deprecate *loop* argument in asyncio.sleep (GH-9415)
* Insert the warn in the asyncio.sleep when the loop argument is used

* Insert the warn in the asyncio.wait and asyncio.wait_for when the loop argument is used

* Better format of the code

* Add news file

* change calls for get_event_loop() to calls for get_running_loop()

* Change message to be more clear in News

* Improve the comments in test_tasks
2018-09-24 05:51:22 -04:00
Andrew Svetlov 11194c877c
bpo-34666: Implement stream.awrite() and stream.aclose() (GH-9274) 2018-09-13 16:53:49 -07:00
Andrew Svetlov 0dd71807a9
bpo-34630: Skip logging SSL certificate errors by asyncio code (GH-9169) 2018-09-12 14:03:54 -07:00
Andrew Svetlov a5d1eb8d8b
bpo-34638: Store a weak reference to stream reader to break strong references loop (GH-9201)
Store a weak reference to stream readerfor breaking strong references

It breaks the strong reference loop between reader and protocol and allows to detect and close the socket if the stream is deleted (garbage collected)
2018-09-12 11:43:04 -07:00
Andrew Svetlov 0baa72f4b2
bpo-34622: Extract asyncio exceptions into a separate module (GH-9141) 2018-09-11 10:13:04 -07:00
Alex Grönholm cca4eec3c0 bpo-34270: Make it possible to name asyncio tasks (GH-8547)
Co-authored-by: Antti Haapala <antti.haapala@anttipatterns.com>
2018-08-08 17:06:47 -04:00
MartinAltmayer 944451cd8d bpo-34263 Cap timeout submitted to epoll/select etc. to one day. (GH-8532) 2018-07-31 10:06:12 -04:00
twisteroid ambassador 9045199c5a bpo-33833: Fix ProactorSocketTransport AssertionError (#7893) 2018-07-30 21:58:50 +03:00
Elvis Pranskevichus 22d25085db bpo-34075: Deprecate non-ThreadPoolExecutor in loop.set_default_executor() (GH-8533)
Various asyncio internals expect that the default executor is a
`ThreadPoolExecutor`, so deprecate passing anything else to
`loop.set_default_executor()`.
2018-07-30 12:42:43 +02:00
Yury Selivanov d904c238ca
bpo-27500: Fix static version of getaddrinfo to resolve IPv6 (GH-7993) 2018-06-28 21:59:32 -04:00
Yury Selivanov 12f482e0ae
bpo-30805: Avoid race condition with debug logging (GH-7545)
Supersedes https://github.com/python/cpython/pull/2490
2018-06-08 18:24:37 -04:00
Victor Stinner ff6c077292
bpo-33694: Fix typo in helper function name (GH-7522)
_feed_data_to_bufferred_proto() renamed to
_feed_data_to_buffered_proto() ("bufferred" => "buffered").

Typo spotted by Nathaniel J. Smith.
2018-06-08 10:32:06 +02:00
Yury Selivanov 8f4042964d
bpo-33792: Add selector and proactor windows policies (GH-7487) 2018-06-07 20:44:57 -04:00
Victor Stinner 79790bc35f
bpo-33694: Fix race condition in asyncio proactor (GH-7498)
The cancellation of an overlapped WSARecv() has a race condition
which causes data loss because of the current implementation of
proactor in asyncio.

No longer cancel overlapped WSARecv() in _ProactorReadPipeTransport
to work around the race condition.

Remove the optimized recv_into() implementation to get simple
implementation of pause_reading() using the single _pending_data
attribute.

Move _feed_data_to_bufferred_proto() to protocols.py.

Remove set_protocol() method which became useless.
2018-06-08 00:25:52 +02:00
Yury Selivanov 415bc46a78
bpo-33769: start_tls: Fix error message; cancel callbacks on error (GH-7403)
In addition to that, mark SSLTransport as "closed" in its "abort()" method to prevent bogus warnings.
2018-06-05 08:59:58 -04:00
Yury Selivanov 9602643120
bpo-33734: asyncio/ssl: a bunch of bugfixes (#7321)
* Fix AttributeError (not all SSL exceptions have 'errno' attribute)

* Increase default handshake timeout from 10 to 60 seconds
* Make sure start_tls can be cancelled correctly
* Make sure any error in SSLProtocol gets propagated (instead of just being logged)
2018-06-04 11:32:35 -04:00
jhaydaman 0a28c0d12e bpo-33238: Add InvalidStateError to concurrent.futures. (GH-7056)
Future.set_result and Future.set_exception now raise InvalidStateError
if the futures are not pending or running. This mirrors the behavior
of asyncio.Future, and prevents AssertionErrors in asyncio.wrap_future
when set_result is called multiple times.
2018-05-30 10:15:06 +03:00
Yury Selivanov 99279ad823
Revert "bpo-22087: Fix Policy.get_event_loop() to detect fork (GH-7208)" (#7232)
This reverts commit 5d97b7bcc1.
2018-05-29 20:47:56 -04:00
Elvis Pranskevichus e2b340ab41 bpo-32751: Wait for task cancellation in asyncio.wait_for() (GH-7216)
Currently, asyncio.wait_for(fut), upon reaching the timeout deadline,
cancels the future and returns immediately.  This is problematic for
when *fut* is a Task, because it will be left running for an arbitrary
amount of time.  This behavior is iself surprising and may lead to
related bugs such as the one described in bpo-33638:

    condition = asyncio.Condition()
    async with condition:
        await asyncio.wait_for(condition.wait(), timeout=0.5)

Currently, instead of raising a TimeoutError, the above code will fail
with `RuntimeError: cannot wait on un-acquired lock`, because
`__aexit__` is reached _before_ `condition.wait()` finishes its
cancellation and re-acquires the condition lock.

To resolve this, make `wait_for` await for the task cancellation.
The tradeoff here is that the `timeout` promise may be broken if the
task decides to handle its cancellation in a slow way.  This represents
a behavior change and should probably not be back-patched to 3.6 and
earlier.
2018-05-29 17:31:01 -04:00
Yury Selivanov 863b674909
bpo-32684: Fix gather to propagate cancel of itself with return_exceptions (GH-7209) 2018-05-29 17:20:02 -04:00
Yury Selivanov 5d97b7bcc1
bpo-22087: Fix Policy.get_event_loop() to detect fork (GH-7208)
Original patch by Dan O'Reilly.
2018-05-29 15:38:07 -04:00
Victor Stinner 9551f77192
bpo-33353: test_asyncio use set_write_buffer_limits() (GH-7200)
Use transport.set_write_buffer_limits() in sendfile tests of
test_asyncio to make sure that the protocol is paused after sending
4 KiB. Previously,
test_sendfile_fallback_close_peer_in_the_middle_of_receiving() failed
on FreeBSD if the DATA was smaller than the default limit of 64 KiB.
2018-05-29 16:02:07 +02:00
Yury Selivanov 2179022d94 bpo-33654: Support protocol type switching in SSLTransport.set_protocol() (#7194) 2018-05-29 12:02:40 +03:00
Yury Selivanov f295587c45
bpo-33674: Pause the transport as early as possible (#7192) 2018-05-29 01:00:12 -04:00
Victor Stinner be00a5583a
bpo-33674: asyncio: Fix SSLProtocol race (GH-7175)
Fix a race condition in SSLProtocol.connection_made() of
asyncio.sslproto: start immediately the handshake instead of using
call_soon(). Previously, data_received() could be called before the
handshake started, causing the handshake to hang or fail.
2018-05-29 01:33:35 +02:00
Yury Selivanov 7165754b6b
bpo-32410: Avoid blocking on file IO in sendfile fallback code (GH-7172) 2018-05-28 18:31:55 -04:00
Yury Selivanov 416c1ebd98
bpo-32610: Fix asyncio.all_tasks() to return only pending tasks. (GH-7174) 2018-05-28 17:54:02 -04:00
Yury Selivanov fdccfe09f0
bpo-33469: RuntimeError after closing loop that used run_in_executor (GH-7171) 2018-05-28 17:10:20 -04:00
Yury Selivanov 989b9e0e6d
bpo-33672: Fix Task.__repr__ crash with Cython's bogus coroutines (GH-7161) 2018-05-28 16:27:34 -04:00
Yury Selivanov dbf102271f
bpo-33654: Support BufferedProtocol in set_protocol() and start_tls() (GH-7130)
In this commit:

* Support BufferedProtocol in set_protocol() and start_tls()
* Fix proactor to cancel readers reliably
* Update tests to be compatible with OpenSSL 1.1.1
* Clarify BufferedProtocol docs
* Bump TLS tests timeouts to 60 seconds; eliminate possible race from start_serving
* Rewrite test_start_tls_server_1
2018-05-28 14:31:28 -04:00