* establish basic event loop for workers
* make "self.close()" inside worker
* remove "runWorkerMessageLoop() - instead manually call global function
in Rust when message arrives. This is done in preparation for structured clone
* refactor "WorkerChannel" and use distinct structs for internal
and external channels; "WorkerChannelsInternal" and "WorkerHandle"
* move "State.worker_channels_internal" to "Worker.internal_channels"
* add "WorkerEvent" enum for child->host communication;
currently "Message(Buf)" and "Error(ErrBox)" variants are supported
* add tests for nested workers
* add tests for worker throwing error on startup
This change simplifies how we execute V8. Previously V8 Isolates jumped
around threads every time they were woken up. This was overly complex and
potentially hurting performance in a myriad ways. Now isolates run on
their own dedicated thread and never move.
- blocking_json spawns a thread and does not use a thread pool
- op_host_poll_worker and op_host_resume_worker are non-operational
- removes Worker::get_message and Worker::post_message
- ThreadSafeState::workers table contains WorkerChannel entries instead
of actual Worker instances.
- MainWorker and CompilerWorker are no longer Futures.
- The multi-threaded version of deno_core_http_bench was removed.
- AyncOps no longer need to be Send + Sync
This PR is very large and several tests were disabled to speed
integration:
- installer_test_local_module_run
- installer_test_remote_module_run
- _015_duplicate_parallel_import
- _026_workers
This flag was added to evaluate performance relative to tokio's threaded
runtime. Although it's faster in the HTTP benchmark, it's clear the runtime
is not the only perf problem.
Removing this flag will simplify further refactors, in particular
adopting the #[tokio::main] macro. This will be done in a follow up.
Ultimately we expect to move to the current thread runtime with Isolates
pinned to specific threads, but that will be a much larger refactor. The
--current-thread just complicates that effort.
tokio_util::run and tokio::run_on_current_thread should accept Future<Output=()> instead of Future<Output=Result<(), ()>>. Currently, all the passed futures have to add Ok(()) or futures::future::ok(()) unnecessarily to call this method.
This PR removes tokio_util::block_on - refactored compiler and file
fetcher slightly so that we can safely block there - that's because
only blocking path consist of only synchronous operations.
Additionally I removed excessive use of tokio_util::panic_on_error
and tokio_util::run_in_task and moved both functions to cli/worker.rs,
to tests module.
Closes#2960
op_fetch_module_meta_data is an op that is used by the TypeScript
compiler. TypeScript requires this op to be sync. However the
implementation of the op does things on the event loop (like fetching
HTTP resources).
In certain situations this can lead to deadlocks. The runtime's thread
pool can be filled with ops waiting on the result of
op_fetch_module_meta_data. The runtime has a maximum number of
threads it can use (the number of logical CPUs on the system).
This patch changes tokio_util::block_on to launch a new Tokio runtime
for evaluating the future, thus bipassing the max-thread problem.
This is only an issue in op_fetch_module_meta_data. Other synchronous
ops are truly synchornous, not interacting with the event loop. TODO
comments are added to direct future development.
