Commit graph

9804 commits

Author SHA1 Message Date
Yusuke Tanaka f2e30a6f79
refactor(cli): move snapshot_from_lockfile function to deno_npm (#20024)
This commit moves `snapshot_from_lockfile` function to [deno_npm
crate](https://github.com/denoland/deno_npm). This allows this function
to be called outside Deno CLI (in particular, Deno Deploy).
2023-08-08 10:07:29 -07:00
David Sherret 05f838a57c
refactor: use deno_cache_dir crate (#20092)
Uses https://github.com/denoland/deno_cache/pull/26
2023-08-08 14:23:02 +00:00
David Sherret a037ed77a2
fix(fmt): do not insert expr stmt leading semi-colon in do while stmt body (#20093)
This is for when semiColons: false

Closes #20089
2023-08-08 09:15:19 -04:00
Marcos Casagrande 557b11b765
fix(ext/abort): trigger AbortSignal events in correct order (#20095)
This PR ensures that the original signal event is fired before any
dependent signal events.

---
The enabled tests fail on `main`:

```
assert_array_equals: Abort events fired in correct order expected property 0 to be 
"original-aborted" but got "clone-aborted" (expected array ["original-aborted", "clone-aborted"] 
got ["clone-aborted", "original-aborted"])
```
2023-08-08 12:05:42 +02:00
Nayeem Rahman 7ce8c2a1f1
build: remove redundant rerun-if-changed for compiler snapshot (#20094)
Not sure why `40_testing.js` is there. The other two `00_typescript.js`
and `99_main_compiler.js` should be covered by
`files_loaded_during_snapshot` at the end.

This helps with `__runtime_js_sources` wrt changing `40_testing.js`.
2023-08-08 09:21:54 +02:00
David Sherret b9b0386948
feat(unstable): rename deno_modules to vendor (#20065)
Renames the unstable `deno_modules` directory and corresponding settings
to `vendor` after feedback. Also causes the vendoring of the
`node_modules` directory which can be disabled via
`--node-modules-dir=false` or `"nodeModulesDir": false`.
2023-08-06 21:56:56 -04:00
David Sherret 7b5bc87f29
fix(unstable): vendor cache should support adding files to hashed directories (#20070)
This changes the design of the manifest.json file to have a separate
"folders" map for mapping hashed directories. This allows, for example,
to add files in a folder like `http_localhost_8000/#testing_5de71/` and
have them be resolved automatically as long as their remaining
components are identity-mappable to the file system (not hashed). It
also saves space in the manifest.json file by only including the hashed
directory instead of each descendant file.

```
// manifest.json
{
  "folders": {
    "https://localhost/NOT_MAPPABLE/": "localhost/#not_mappable_5cefgh"
  },
  "modules": {
    "https://localhost/folder/file": {
      "headers": {
        "content-type": "application/javascript"
      }
    },
  }
}

// folder structure
localhost
  - folder
    - #file_2defn (note: I've made up the hashes in these examples)
  - #not_mappable_5cefgh
    - mod.ts
    - etc.ts
    - more_files.ts
```
2023-08-06 12:25:48 -04:00
Nayeem Rahman c1c8eb3d55
build: allow disabling snapshots for dev (#20048)
Closes #19399 (running without snapshots at all was suggested as an
alternative solution).

Adds a `__runtime_js_sources` pseudo-private feature to load extension
JS sources at runtime for faster development, instead of building and
loading snapshots or embedding sources in the binary. Will only work in
a development environment obviously.

Try running `cargo test --features __runtime_js_sources
integration::node_unit_tests::os_test`. Then break some behaviour in
`ext/node/polyfills/os.ts` e.g. make `function cpus() {}` return an
empty array, and run it again. Fix and then run again. No more build
time in between.
2023-08-06 01:47:15 +02:00
Nayeem Rahman b96f283064
refactor: remove snapshot_module_load_cb (#20043) 2023-08-05 23:00:38 +00:00
Matt Mastracci 85a2b281f5
chore: use zlib-ng for flate2 (#20059)
Extracted from PR #16011
2023-08-05 15:43:16 -06:00
Marcos Casagrande a1280c36c2
chore(wpt): expose gc (#20060)
This PR exposes garbage collector for WPT

see:
3d80f7e879/common/gc.js (L34-L36)


```
/streams/readable-streams/garbage-collection.any.html


test stderr:
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.


file result: ok. 4 passed; 0 failed; 0 expected failure; total 4 (255ms)

----------------------------------------
/streams/readable-streams/garbage-collection.any.worker.html


test stderr:
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.
Tests are running without the ability to do manual garbage collection. They will still work, but coverage will be suboptimal.


file result: ok. 4 passed; 0 failed; 0 expected failure; total 4 (277ms)
```

This PR removes that warning and improves coverage.
2023-08-05 23:34:14 +02:00
David Sherret 6b0f75d1f5
fix(unstable): vendor cache override should handle forbidden windows directory names (#20069)
Meant to do this earlier.
2023-08-05 17:34:07 -04:00
Matt Mastracci 8ae7062931
fix(ext/http): serveHttp brotli compression level should be fastest (#20058)
Use brotli's fastest mode rather than default mode
2023-08-04 12:39:39 -06:00
Luca Bruno 72d9f06090
chore(cargo): update async-compression/flate2/miniz to latest (#20049)
This bumps `async-compression` dependency in `deno_http` to latest, in
order to avoid having multiple duplicate versions.
Related, it also unpin a stale `flate2` dependency so that the whole
chain of `async-compression` -> `flate2` -> `miniz_oxide` can surface up
to current versions.
The lockfile entries for all of the above crates have been update
accordingly; the new tree of dependencies looks like this:
```
$ cargo tree -i -p miniz_oxide

miniz_oxide v0.7.1
└── flate2 v1.0.26
    └── async-compression v0.4.1
```
2023-08-04 18:30:14 +02:00
Luca Bruno 5abf4cd951
fix(ext/http): unify default gzip compression level (#20050)
This tweaks the HTTP response-writer in order to align the two possible
execution flows into using the same gzip default compression level, that
is `1` (otherwise the implicit default level is `6`).
2023-08-04 17:28:32 +02:00
Yoshiya Hinosawa 8a175a780a
fix(test): make test runner work when global setTimeout is replaced (#20052) 2023-08-04 22:29:04 +09:00
Bartek Iwańczuk 6405b5f454
fix(node): polyfill process.title (#20044)
Closes https://github.com/denoland/deno/issues/19777
2023-08-04 14:31:13 +02:00
Bartek Iwańczuk 8d8a89ceea
fix(node): repl._builtinLibs (#20046)
Ref https://github.com/denoland/deno/issues/19733
2023-08-04 14:30:48 +02:00
Marcos Casagrande 5311f69bbb
fix(ext/file): resolve unresolved Promise in Blob.stream (#20039)
This PR fixes some crashing WPT tests due to an unresolved promise.

---
This could be a [stream spec](https://streams.spec.whatwg.org) bug

When `controller.close` is called on a byob stream, there's no cleanup
of pending `readIntoRequests`. The only cleanup of pending
`readIntoRequests` happen when `.byobRequest.respond(0)` is called, it
happens
here:6ba245fe25/ext/web/06_streams.js (L2026)
which ends up calling `readIntoRequest.closeSteps(chunk);` in
6ba245fe25/ext/web/06_streams.js (L2070)



To reproduce:

```js
async function byobRead() {
  const input = [new Uint8Array([8, 241, 48, 123, 151])];
  const stream = new ReadableStream({
    type: "bytes",
    async pull(controller) {
      if(input.length === 0) {
        controller.close();
        // controller.byobRequest.respond(0); // uncomment for fix 
        return 
      }
      controller.enqueue(input.shift())
    },
  });

  const reader = stream.getReader({ mode: 'byob' });
  const r1 = await reader.read(new Uint8Array(64));
  console.log(r1);
  const r2 = await reader.read(new Uint8Array(64));
  console.log(r2);
}

await byobRead();
```

Running the script triggers:
```
error: Top-level await promise never resolved
```
2023-08-04 13:57:54 +02:00
Bartek Iwańczuk 53ab6c004a
fix(test): use only a single timeout for op sanitizers (#20042)
Chipping away at making tests faster. Appears we don't need double
timeout before sanitizing ops. This should cut baseline cost of running a test
by half.
2023-08-04 10:05:15 +02:00
Bartek Iwańczuk 433ecc9047
refactor: rewrite http_next ops to use op2 macro (#19934)
Ref #19915

---------

Co-authored-by: Matt Mastracci <matthew@mastracci.com>
2023-08-03 20:36:32 +00:00
Matt Mastracci 7f8bf2537d
refactor(ext/fetch): refactor fetch to use new write_error method (#20029)
This is a prerequisite for fast streams work -- this particular resource
used a custom `mpsc`-style stream, and this work will allow us to unify
it with the streams in `ext/http` in time.

Instead of using Option as an internal semaphore for "correctly
completed EOF", we allow code to propagate errors into the channel which
can be picked up by downstream sinks like Hyper. EOF is signalled using
a more standard sender drop.
2023-08-03 14:27:25 -06:00
Matt Mastracci 0f07dc95f1
chore: fix pty support on Macs (#20037)
Many of the CI tests have been failing on my M2 Pro mac (Ventura 13.4)
when running inside of a vscode terminal (a strange `ENOTTY` error).
This modifies the pty-handling code to use libc directly rather than the
older pty library that appears mostly unmaintained (outside of
@littledivy's fork).

As a bonus, this should allow us to run pty tests on the mac CI runner.

After this PR, the tests now complete with 100% success on my local
machine. Before this PR, I needed to pass `CI=true` to get my local test
suite to pass.
2023-08-03 14:04:37 -06:00
denobot 6ba245fe25
1.36.0 (#20036)
Co-authored-by: bartlomieju <bartlomieju@users.noreply.github.com>
2023-08-03 18:26:25 +02:00
Asher Gomez 6fb7e8d93b
feat(permissions): add "--deny-*" flags (#19070)
This commit adds new "--deny-*" permission flags. These are complimentary to
"--allow-*" flags.

These flags can be used to restrict access to certain resources, even if they
were granted using "--allow-*" flags or the "--allow-all" ("-A") flag.

Eg. specifying "--allow-read --deny-read" will result in a permission error,
while "--allow-read --deny-read=/etc" will allow read access to all FS but the
"/etc" directory.

Runtime permissions APIs ("Deno.permissions") were adjusted as well, mainly
by adding, a new "PermissionStatus.partial" field. This field denotes that
while permission might be granted to requested resource, it's only partial (ie.
a "--deny-*" flag was specified that excludes some of the requested resources).
Eg. specifying "--allow-read=foo/ --deny-read=foo/bar" and then querying for
permissions like "Deno.permissions.query({ name: "read", path: "foo/" })"
will return "PermissionStatus { state: "granted", onchange: null, partial: true }",
denoting that some of the subpaths don't have read access.

Closes #18804.

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Nayeem Rahman <nayeemrmn99@gmail.com>
2023-08-03 13:19:19 +02:00
Bartek Iwańczuk db287e216d
refactor: use '--reporter' and '--junit-path' flags for 'deno test' (#20031)
This commit adds "--reporter" and "--junit-path" flags to "deno test"
subcommand instead of using "--dot" and "--junit" flags.
2023-08-02 22:05:34 -04:00
David Sherret 480894e5c8
feat(unstable/lsp): support navigating to deno_modules folder (#20030)
Closes #20015
Closes https://github.com/denoland/vscode_deno/issues/850 (only for
deno_modules, but I don't think this will be possible for the global
cache)
2023-08-02 16:57:25 -04:00
await-ovo e8d03119a0
fix(repl): highlight from ident in import from or export from (#20023) 2023-08-02 16:27:03 -04:00
Bartek Iwańczuk 029bdf0cd5
feat(cli): Add dot test reporter (#19804)
This commit adds a "dot" reporter to "deno test" subcommand,
that can be activated using "--dot" flag.

It provides a concise output using:
- "." for passing test
- "," for ignored test
- "!" for failing test

User output is silenced and not printed to the console.

In non-TTY environments each result is printed on a separate line.
2023-08-02 18:38:10 +02:00
Bartek Iwańczuk d9c85e016f
fix(node): node:test reports correct location (#20025)
Also removed some noisy output that caused test flakiness.
2023-08-02 17:11:04 +02:00
Bartek Iwańczuk 1cb16683bc
fix(bench): iter/s calculation (#20016)
Ref https://github.com/denoland/deno/pull/19994#discussion_r1281267606
Ref https://github.com/denoland/deno/pull/19994#discussion_r1281243602
2023-08-02 10:01:49 -04:00
Kyoh 9fa328f656
fix: make "suggest.autoImports" to switch completions from external modules (#19845) 2023-08-02 09:22:00 -04:00
Bartek Iwańczuk 9d1c32945c
fix: regression in workers using dynamic imports (#20006)
Closes https://github.com/denoland/deno/issues/19903
2023-08-02 01:19:44 +00:00
David Sherret 1cefa831fd
feat(unstable): optional deno_modules directory (#19977)
Closes #15633
2023-08-02 00:49:09 +00:00
Bartek Iwańczuk 36ae37604a
feat(bench): print iter/s in the report (#19994)
This commit adds "iter/s" column to the console report
generated by "deno bench" subcommand.
2023-08-01 23:55:23 +00:00
await-ovo fec34d8069
fix(ext/node): fix import json using npm specifier (#19723) 2023-08-01 23:20:08 +00:00
Bartek Iwańczuk 21f1b2f62b
feat(node): add polyfill for node:test module (#20002)
This commit provides basic polyfill for "node:test" module. Currently
only top-level "test" function is polyfilled, all remaining functions from
that module throw not implemented errors.
2023-08-02 01:17:38 +02:00
David Sherret 00b5fe8ba3
feat(npm): support running non-bin scripts in npm pkgs via deno run (#19975)
Closes https://github.com/denoland/deno/issues/19967
2023-08-01 18:47:57 -04:00
David Sherret 51ceed860b
chore: fix windows clippy errors (#20014) 2023-08-01 23:30:19 +02:00
David Sherret 5e89d1a0ab
ci: lint on all operating systems (#20012) 2023-08-01 16:08:41 -04:00
Matt Mastracci 45572e329a
refactor(runtime): use new fd methods from resource table (#20010)
Prereq for fast streams work. No longer need `#[cfg]` around
`backing_fd`.
2023-08-01 14:48:39 -04:00
Felipe Baltor ab2627a014
fix(test): request cloning should throw if body stream is locked (#19990)
Follow-up from #19869.
2023-08-01 18:24:42 +00:00
David Sherret b563746885
fix: deno diagnostic - clarify where to put triple-slash directive (#20009)
Some people might not know what "entrypoint" means or where to put the
triple-slash directive.
2023-08-01 15:27:25 +00:00
Bartek Iwańczuk 5df2b0b4dc
fix: retry module download once if server errored (#17252)
Closes https://github.com/denoland/deno/issues/17251
Closes #19970

This commits adds logic to retry failed module downloads once.
Both request and server errors are handled and the retry is done after
50 ms wait time.
2023-08-01 10:52:28 +02:00
Bartek Iwańczuk a05e890d56
chore: make LSP test less prone to changes (#20003)
This test was getting changed often, I modified it a bit
so it's less sensitive to changes in declaration files.
2023-07-31 21:31:30 -04:00
Ricardo Iván Vieitez Parra 98403691d1
fix: call setIsTrusted for generated events (MessageEvent) (#19919)
This addresses issue #19918.

## Issue description

Event messages have the wrong isTrusted value when they are not
triggered by user interaction, which differs from the browser. In
particular, all MessageEvents created by Deno have isTrusted set to
false, even though it should be true.

This is my first ever contribution to Deno, so I might be missing
something.
2023-07-31 23:22:07 +02:00
Leo Kettmeir aa8078b688
feat(node/os): implement getPriority, setPriority & userInfo (#19370)
Takes #4202 over
Closes #17850 

---------

Co-authored-by: ecyrbe <ecyrbe@gmail.com>
2023-07-31 22:29:09 +02:00
Luca Casonato 78ceeec6be
perf: faster node globals access in cjs (#19997)
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-07-31 19:45:32 +00:00
Nayeem Rahman d5efdeeff1
refactor: update core extension api usage (#19952) 2023-07-31 18:19:15 +00:00
David Sherret 99daad0541
refactor: NodeCodeTranslator - optional source to translate_cjs_to_esm (#20000) 2023-07-31 17:46:58 +00:00