This PR addresses many problems with module graph loading
introduced in #5029, as well as many long standing issues.
"ModuleGraphLoader" has been wired to "ModuleLoader" implemented
on "State" - that means that dependency analysis and fetching is done
before spinning up TS compiler worker.
Basic dependency tracking for TS compilation has been implemented.
Errors caused by import statements are now annotated with import
location.
Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
Since everything that Deno loads is treated as an ES Module,
it means that all code is treated as "use strict" except for
when using the REPL. This PR changes that so code in the
REPL is also always evaluated with "use strict". There are
also a couple other places where we load code as scripts
which should also use "use strict" just in case.
This commit changes how error occurring in SWC are handled.
Changed lexer settings to properly handle TS decorators.
Changed output of SWC error to annotate with position in file.
This commit fixes a bug introduced in #5029 that caused bad
handling of redirects during module analysis.
Also ensured that duplicate modules are not downloaded.
This commit disabled flaky test in cli/module_graph.rs.
The test uses 019_media_types.ts which was known to be flaky
for some time. Test should be reenabled once test HTTP server
is rewritten to Rust.
This commit completely overhauls how module analysis is
performed in TS compiler by moving the logic to Rust.
In the current setup module analysis is performed using
"ts.preProcessFile" API in a special TS compiler worker
running on a separate thread.
"ts.preProcessFile" allowed us to build a lot of functionality
in CLI including X-TypeScript-Types header support
and @deno-types directive support. Unfortunately at the
same time complexity of the ops required to perform
supporting tasks exploded and caused some hidden
permission escapes.
This PR introduces "ModuleGraphLoader" which can parse
source and load recursively all dependent source files; as
well as declaration files. All dependencies used in TS
compiler and now fetched and collected upfront in Rust
before spinning up TS compiler.
To achieve feature parity with existing APIs this commit
includes a lot of changes:
* add "ModuleGraphLoader"
- can fetch local and remote sources
- parses source code using SWC and extracts imports, exports, file references, special
headers
- this struct inherited all of the hidden complexity and cruft from TS version and requires
several follow up PRs
* rewrite cli/tsc.rs to perform module analysis upfront and send all required source code to
TS worker in one message
* remove op_resolve_modules and op_fetch_source_files from cli/ops/compiler.rs
* run TS worker on the same thread
This commit removes "check_stderr" setting from itest! macro used
to generate integration tests. Without this setting on tests discarded
output of stderr making it very hard to debug the problem in test.
Numerous tests were changed by adding "--quiet" flag to not display
"Compile"/"Download" prompts.
This commit fixes panic occurring if $DENO_DIR is set to a relative
path, eg. "DENO_DIR=denodir deno run main.ts".
Before creating DenoDir instance given path is checked and if necessary
resolved against current working directory.
Additional sanity checks were put in place to ensure all caches
receive absolute path for the location.
This commit fixes problems with source maps in Chrome Devtools
by substituting source map URL generated by TS compiler with
actual file URL pointing to DENO_DIR.
Dummy value of "source_map_url" has been removed from
"ScriptOrigin".
Also fixes lock file which used compiled source code to generate
lock hash; it now uses source code of the file that is
being compiled.
This PR hot-fixes permission escapes in dynamic imports, workers
and runtime compiler APIs.
"permissions" parameter was added to public APIs of SourceFileFetcher
and appropriate permission checks are performed during loading of
local and remote files.
* Prepend underscores to private modules
* Remove collectUint8Arrays() It would be a misuse of Deno.iter()'s result.
* Move std/_util/async.ts to std/async
* Move std/util/sha*.ts to std/hash
Importing .wasm files is non-standardized therefore deciding to
support current functionality past 1.0 release is risky.
Besides that .wasm import posed many challenges in our codebase
due to complex interactions with TS compiler which spawned
thread for each encountered .wasm import.
This commit removes:
- cli/compilers/wasm.rs
- cli/compilers/wasm_wrap.js
- two integration tests related to .wasm imports
This commit updates "deno_typescript" crate to properly map
bundle entrypoint file to internal specifier.
All import specifiers were remapped from "file:///a/b/c.ts" to
"$deno$/a/b/c.ts", but that was not the case for entrypoint file
"main.ts" and "compiler.ts".
Because of that internal stack traces were inconsistent; showing
"file:///some/random/path/on/ci/machine.ts" URL in frames that
originate from "main.ts" or "compiler.ts" and "$deno$/file.ts"
for all other imports.
This PR removes op_cache and refactors how Deno interacts with TS compiler.
Ultimate goal is to completely sandbox TS compiler worker; it should operate on
simple request -> response basis. With this commit TS compiler no longer
caches compiled sources as they are generated but rather collects all sources
and sends them back to Rust when compilation is done.
Additionally "Diagnostic" and its children got refactored to use "Deserialize" trait
instead of manually implementing JSON deserialization.
The issue is solved by proxying websocket messages over a pair of
`futures::mpsc::unbounded` channels. As these are are implemented in
the 'futures' crate, they can't participate in Tokio's cooperative
task yielding.
refactor: Parse URLs more sequentially. This makes it easier to change matching behaviour depending on the protocol.
fix: Fail when a host isn't given for certain protocols.
fix: Convert back-slashes info forward-slashes.
This PR removes the hack in CLI that allows to run scripts with shorthand: deno script.ts.
Removing this functionality because it hacks around short-comings of clap our CLI parser. We agree that this shorthand syntax is desirable, but it needs to be rethinked and reimplemented. For 1.0 we should go with conservative approach that is correct.
