This commit completely overhauls how module analysis is
performed in TS compiler by moving the logic to Rust.
In the current setup module analysis is performed using
"ts.preProcessFile" API in a special TS compiler worker
running on a separate thread.
"ts.preProcessFile" allowed us to build a lot of functionality
in CLI including X-TypeScript-Types header support
and @deno-types directive support. Unfortunately at the
same time complexity of the ops required to perform
supporting tasks exploded and caused some hidden
permission escapes.
This PR introduces "ModuleGraphLoader" which can parse
source and load recursively all dependent source files; as
well as declaration files. All dependencies used in TS
compiler and now fetched and collected upfront in Rust
before spinning up TS compiler.
To achieve feature parity with existing APIs this commit
includes a lot of changes:
* add "ModuleGraphLoader"
- can fetch local and remote sources
- parses source code using SWC and extracts imports, exports, file references, special
headers
- this struct inherited all of the hidden complexity and cruft from TS version and requires
several follow up PRs
* rewrite cli/tsc.rs to perform module analysis upfront and send all required source code to
TS worker in one message
* remove op_resolve_modules and op_fetch_source_files from cli/ops/compiler.rs
* run TS worker on the same thread
Importing .wasm files is non-standardized therefore deciding to
support current functionality past 1.0 release is risky.
Besides that .wasm import posed many challenges in our codebase
due to complex interactions with TS compiler which spawned
thread for each encountered .wasm import.
This commit removes:
- cli/compilers/wasm.rs
- cli/compilers/wasm_wrap.js
- two integration tests related to .wasm imports
This PR removes op_cache and refactors how Deno interacts with TS compiler.
Ultimate goal is to completely sandbox TS compiler worker; it should operate on
simple request -> response basis. With this commit TS compiler no longer
caches compiled sources as they are generated but rather collects all sources
and sends them back to Rust when compilation is done.
Additionally "Diagnostic" and its children got refactored to use "Deserialize" trait
instead of manually implementing JSON deserialization.
- moves compiler implementation to "cli/js/compiler/" directory
- moves more APIs to "cli/js/web":
* "console.ts"
* "console_table.ts"
* "performance.ts"
* "timers.ts"
* "workers.ts"
- removes some dead code from "cli/js/"
- Cleanup "tsCompilerOnMessage" by factoring out separate methods for each
request type:
* "compile"
* "runtimeCompile"
* "runtimeTranspile"
- Simplify control flow of compiler workers by a) no longer calling "close()" in worker runtime after a
single message; b) explicitly shutting down worker from host after a single message
Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
Fixes#4602
We turned off `allowJs` by default, to keep the compiler from grabbing
a bunch of files that it wouldn't actually do anything useful with. On
the other hand, this caused problems with bundles, where the compiler
needs to gather all the dependencies, including JavaScript ones. This
fixes this so that when we are bundling, we analyse JavaScript imports
in the compiler.
This PR fixes an issue where we recursively analysed imports on plain JS files
in the compiler irrespective of "checkJs" being true. This caused problems
where when analysing the imports of those files, we would mistake some
import like structures (AMD/CommonJS) as dependencies and try to resolve
the "modules" even though the compiler would not actually look at those files.
* establish basic event loop for workers
* make "self.close()" inside worker
* remove "runWorkerMessageLoop() - instead manually call global function
in Rust when message arrives. This is done in preparation for structured clone
* refactor "WorkerChannel" and use distinct structs for internal
and external channels; "WorkerChannelsInternal" and "WorkerHandle"
* move "State.worker_channels_internal" to "Worker.internal_channels"
* add "WorkerEvent" enum for child->host communication;
currently "Message(Buf)" and "Error(ErrBox)" variants are supported
* add tests for nested workers
* add tests for worker throwing error on startup
- Exports diagnostic items from `diagnostics.ts` which are missing at
runtime.
- Returns an array of diagnostics, instead of an object with a property
of `items`. This is because of the way Rust deals with certain
structures, and shouldn't be exposed in the APIs.
Ref #3712. This change allowed the deno_typescript crate to reference
cli/js/lib.deno_runtime.d.ts which breaks "cargo package". We intend to
reintroduce a revised version of this patch later once "cargo
package" is working and tested.
This reverts commit 737ab94ea1.
* split ops/worker.rs into ops/worker_host.rs and ops/web_worker.rs
* refactor js/workers.ts and factor out js/worker_main.ts - entry point for WebWorker runtime
* BREAKING CHANGE: remove support for blob: URL in Worker
* BREAKING CHANGE: remove Deno namespace support and noDenoNamespace option in Worker constructor
* introduce WebWorker struct which is a stripped down version of cli::Worker
- Bundles are fully standalone. They now include the shared loader with
`deno_typescript`.
- Refactor of the loader in `deno_typescript` to perform module
instantiation in a more
- Change of behaviour when an output file is not specified on the CLI.
Previously a default name was determined and the bundle written to that
file, now the bundle will be sent to `stdout`.
- Refactors in the TypeScript compiler to be able to support the concept
of a request type. This provides a cleaner abstraction and makes it
easier to support things like single module transpiles to the userland.
- Remove a "dangerous" circular dependency between `os.ts` and `deno.ts`,
and define `pid` and `noColor` in a better way.
- Don't bind early to `console` in `repl.ts`.
- Add an integration test for generating a bundle.
