Commit graph

6261 commits

Author SHA1 Message Date
snek e3b2ee183b
fix: Rewrite Node-API (#24101)
Phase 1 node-api rewrite
2024-06-10 16:20:44 +00:00
Nayeem Rahman 7c5dbd5d54
feat(lsp): workspace jsr resolution (#24121) 2024-06-10 17:03:17 +01:00
David Sherret 31154ff958
fix(check): attempt to resolve types from pkg before @types pkg (#24152)
I've been meaning to fix this for ages, but I finally ran into it here:


https://github.com/dsherret/ts-ast-viewer/actions/runs/9432038675/job/25981325408

We need to resolve the `@types` package as a fallback instead of eagerly
resolving it.
2024-06-08 20:05:28 -04:00
Luca Casonato c1f23c5788
fix(ext/node): lossy UTF-8 read node_modules files (#24140)
Previously various reads of files in `node_modules` would error on
invalid UTF-8. These were cases involving:

- reading package.json from Rust
- reading package.json from JS
- reading CommonJS files from JS
- reading CommonJS files from Rust (for ESM translation)
- reading ESM files from Rust
2024-06-08 18:36:13 +02:00
David Sherret ed20102713
fix: make writing to the deps cache more reliable (#24135)
I was able to reproduce this locally.

```
[error] Failed to execute snippet: 
import { validate } from "@std/uuid";
import { assert, assertFalse } from "@std/assert";

assert(validate("6ec0bd7f-11c0-43da-975e-2a8ad9ebae0b"));
assertFalse(validate("not a UUID"));
Download https://jsr.io/@std/uuid/meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1_meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/common.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/constants.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v1.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v3.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v4.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v5.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/_common.ts
error: Access is denied. (os error 5) (for 'V:\.cache\deno\deps\https\jsr.io\2ae5bb614c7526d0876be0b76da1372fd51304ae27d6202ee94df720b3523d08')
 at file:///V:/deno_std/uuid/common.ts:43
[error] Failed to execute snippet:
import { v5, NAMESPACE_DNS, NIL_UUID } from "@std/uuid";
import { assert, assertFalse } from "@std/assert";

const data = new TextEncoder().encode("deno.land");
const uuid = await v5.generate(NAMESPACE_DNS, data);

assert(v5.validate(uuid));
assertFalse(v5.validate(NIL_UUID));
Download https://jsr.io/@std/uuid/meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1_meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/common.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/constants.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v1.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v3.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v4.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v5.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/_common.ts
error: Access is denied. (os error 5) (for 'V:\.cache\deno\deps\https\jsr.io\63dd818c5fc1ac39c04df9b42bd9dd4bbc07f7d1b174e405d003731125778da1')
    at https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts:30:15
 at file:///V:/deno_std/uuid/mod.ts:4
[error] Failed to execute snippet:
import { isNil } from "@std/uuid";
import { assert, assertFalse } from "@std/assert";

assert(isNil("00000000-0000-0000-0000-000000000000"));
assertFalse(isNil(crypto.randomUUID()));
Download https://jsr.io/@std/uuid/meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1_meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/common.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/constants.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v1.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v3.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v4.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v5.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/_common.ts
error: Access is denied. (os error 5) (for 'V:\.cache\deno\deps\https\jsr.io\fd3a12fc091d16ee29f10fa7a05eeeb8bd6c3cc014642e72478c757f00e7261e')
    at https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts:34:40
 at file:///V:/deno_std/uuid/common.ts:23
[error] Failed to execute snippet:
import { version } from "@std/uuid";
import { assertEquals } from "@std/assert/assert-equals";

assertEquals(version("d9428888-122b-11e1-b85c-61cd3cbb3210"), 1);
assertEquals(version("6ec0bd7f-11c0-43da-975e-2a8ad9ebae0b"), 4);
Download https://jsr.io/@std/uuid/meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1_meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/common.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/constants.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v1.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v3.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v4.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v5.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/_common.ts
error: Access is denied. (os error 5) (for 'V:\.cache\deno\deps\https\jsr.io\2ae5bb614c7526d0876be0b76da1372fd51304ae27d6202ee94df720b3523d08')
 at file:///V:/deno_std/uuid/common.ts:66
4 errors found
```

It occurs when many Deno processes are writing to the deps cache at the
same time. Fix is to use `atomic_write_with_retries` which is much more
reliable (and the function that helped make the ecosystem tests more
reliable too). After this change I no longer have this issue.

Closes https://github.com/denoland/deno/issues/24073
2024-06-07 13:06:17 -04:00
David Sherret 78774dd664
fix: do not panic linting files with UTF-8 BOM (#24136)
Closes #24122
2024-06-07 17:02:47 +00:00
David Sherret 386d5c8310
refactor: remove PermissionsContainer in deno_runtime (#24119)
Also removes permissions being passed in for node resolution. It was
completely useless because we only checked it for reading package.json
files, but Deno reading package.json files for resolution is perfectly
fine.

My guess is this is also a perf improvement because Deno is doing less
work.
2024-06-06 23:37:53 -04:00
David Sherret 7a17eba5a7
refactor: NpmRegistryApi - #[async_trait(?Send)] (#24126) 2024-06-06 22:37:41 +00:00
Nathan Whitaker 47d19461a5
fix(cli): Overwrite existing bin entries in node_modules (#24123)
Previously we warned on unix and didn't touch them on windows, now we
unconditionally overwrite them. This matches what npm does.
2024-06-06 21:21:25 +00:00
denobot 4d531bf229
chore: forward v1.44.1 release commit to main (#24115)
This is the release commit being forwarded back to main for 1.44.1

Co-authored-by: devsnek <devsnek@users.noreply.github.com>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
2024-06-06 03:23:10 +00:00
David Sherret 566adb7c0a
fix(npm): use configured auth for tarball urls instead of scope auth (#24111)
Deno was using the scope auth for the tarball urls, which is not always
correct. We are going to do a release immediately for this issue.
2024-06-05 21:24:52 +00:00
Bartek Iwańczuk 0db73f6333
fix: update deno_npm (#24065) 2024-06-05 19:22:40 +00:00
David Sherret a013b9113d
fix: support importing statically unanalyzable npm specifiers (#24107)
Closes https://github.com/denoland/deno/issues/20479
Closes https://github.com/denoland/deno/issues/18744
2024-06-05 15:18:06 -04:00
David Sherret 1b355d8a87
refactor(npm): improve locking around updating npm resolution (#24104)
Introduces a `SyncReadAsyncWriteLock` to make it harder to write to the
npm resolution without first waiting async in a queue. For the npm
resolution, reading synchronously is fine, but when updating, someone
should wait async, clone the data, then write the data at the end back.
2024-06-05 15:17:35 -04:00
David Sherret 7ed90a20d0
fix: better handling of npm resolution occurring on workers (#24094)
Closes https://github.com/denoland/deno/issues/24063
2024-06-05 17:04:16 +02:00
Bartek Iwańczuk eb218c0f33
chore: upgrade dlint to 0.60.0 (#24041)
Factoring out `dlint` upgrade from
https://github.com/denoland/deno/pull/24034 as it
requires us to change the lint step on mac to use ARM runners.

---------

Co-authored-by: Luca Casonato <hello@lcas.dev>
Co-authored-by: David Sherret <dsherret@users.noreply.github.com>
2024-06-05 01:09:29 +02:00
Igor Zinkovsky 32f655db54
fix(docs): update Deno.Command docs (#24097) 2024-06-04 12:19:42 -07:00
Andreas Kohn 92c881faa7
docs(console): More clearly indicate the contents of DENO_CERT (#24091)
The PEM file specified by DENO_CERT can contain multiple certificates
13924fdb1b/cli/args/mod.rs (L722-L742)

Signed-off-by: Andreas Kohn <andreas.kohn@gmail.com>
2024-06-04 11:50:50 -04:00
David Sherret 3341c50b6a
refactor: don't share reqwest::HttpClient across tokio runtimes (#24092)
This also fixes several issues where we weren't properly creating http
clients with the user's settings.
2024-06-03 21:17:08 +00:00
Nayeem Rahman 72088f2f52
fix(lsp): complete exports for import mapped jsr specifiers (#24054) 2024-06-03 21:32:28 +01:00
David Sherret b1f776adef
refactor: extract structs for downloading tarballs and npm registry packuments (#24067) 2024-06-03 01:39:13 +00:00
Kenta Moriuchi eda43c46de
fix: validate integer values in Deno.exitCode setter (#24068) 2024-06-02 21:29:01 -04:00
David Sherret 38ff9faff6
fix: retry writing lockfile on failure (#24052)
Ran into this running the deno_graph ecosystem tests where many
processes writing to the same path at the same time would cause an
error.
2024-05-31 23:25:08 -04:00
denobot 9e2971d40f
1.44.0 (#24045)
Bumped versions for 1.44.0

Co-authored-by: littledivy <littledivy@users.noreply.github.com>
2024-05-30 04:49:23 +00:00
David Sherret 2d3b596cf5
feat: do not require DENO_FUTURE=1 for npmrc support (#24043) 2024-05-30 04:07:57 +00:00
Nayeem Rahman 556012320e
feat(lsp): support .npmrc (#24042)
Closes #24040
2024-05-30 03:16:15 +00:00
Aapo Alasuutari d67ee9a08b
BREAKING(ffi/unstable): use BigInt representation in turbocall (#23983)
Built ontop of #23981, this sets FFI
turbocalls (Fast Call API) to use the BigInt representation.
2024-05-30 08:00:11 +05:30
Nathan Whitaker a379009bfd
fix(cli): Prefer npm bin entries provided by packages closer to the root (#24024)
Fixes #24012.

In the case of multiple packages providing a binary with a same name, we
were basically leaving the results undefined (since we set up things in
parallel, and whichever got set up first won). In addition, we were
warning about these cases, even though it's a situation that's expected
to occur.

Instead, in the case of a collision in the binary names, we prefer the
binary provided by the package with the least depth in the dependency
tree.

While I was at it, I also took moved more code to `bin_entries.rs` since
it was starting to get a bit cluttered.
2024-05-29 17:45:22 -07:00
Luca Casonato e084fe10a9
feat(lint): add no-boolean-literal-for-arguments rule and enable no-unused-vars for jsx files (#24034)
* https://github.com/denoland/deno_lint/pull/1271
* https://github.com/denoland/deno_lint/pull/1277

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-05-30 00:09:16 +00:00
Luke Edwards 13723f267e
feat: Add Deno.exitCode API (#23609)
This commits adds the ability to set a would-be exit code 
for the Deno process without forcing an immediate exit, 
through the new `Deno.exitCode` API.

- **Implements `Deno.exitCode` getter and setter**: Adds support for
setting
and retrieving a would-be exit code via `Deno.exitCode`.
This allows for asynchronous cleanup before process termination 
without immediately exiting.
- **Ensures type safety**: The setter for `Deno.exitCode` validates that
the provided value is a number, throwing a TypeError if not, to ensure
that
only valid exit codes are set.

Closes to #23605

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-05-29 23:16:27 +00:00
Kyle Kelley cf611fbf54
chore: upgrade jupyter runtimelib to 0.11.0 (#24036)
Brings in:

* More fully typed structures (for when we get to implementing more)
* `with_metadata`, `with_buffers`, etc. from
https://github.com/runtimed/runtimed/pull/99

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-05-29 22:46:47 +00:00
Nayeem Rahman 3d3722507e
perf(repl): don't walk workspace in repl language server (#24037) 2024-05-29 21:31:09 +01:00
Marvin Hagemeister 3c3076a84c
fix(task): always use npm for npm run with flags (#24028) 2024-05-29 20:16:35 +00:00
David Sherret 814ac9a75d
perf: avoid building module graph if dynamic specifier already in graph (#24035) 2024-05-29 14:59:35 -04:00
David Sherret 94f040ac28
fix: bump cache sqlite dbs to v2 for WAL journal mode change (#24030)
In https://github.com/denoland/deno/pull/23955 we changed the sqlite db
journal mode to WAL. This causes issues when someone is running an old
version of Deno using TRUNCATE and a new version because the two fight
against each other.
2024-05-29 18:38:18 +00:00
Nayeem Rahman fada25b0dd
fix(lsp): don't discover deno.json in vendor dir (#24032) 2024-05-29 18:08:54 +01:00
David Sherret a8923534ed
chore: set lockfile as having no content changes after write (#24023)
Slight perf regression when updating deno_lockfile in
https://github.com/denoland/deno/pull/23979
2024-05-28 23:40:40 -04:00
Nayeem Rahman 14a74600de
perf(lsp): lock out requests until init is complete (#23998) 2024-05-29 01:26:43 +01:00
Nathan Whitaker 2024c974b6
perf(cli): Improve concurrency when setting up node_modules and loading cached npm package info (#24018)
The same issue in two different places - doing blocking FS work in an
async task, limiting the amount of work that happens concurrently.

- When setting up node_modules, where we try to set up entries
concurrently but were blocking other tasks from actually running.
- When loading package info from the npm registry file cache, loading
and deserializing is expensive and prevents concurrency. This was
especially noticeable when loading an npm resolution snapshot from a
lockfile (`snapshot_from_lockfile` in `deno_npm`).

Installing deps in `deno-docs`:

```
❯ hyperfine -i -p 'rm -rf node_modules/' '../d7/deno-main i' '../d7/target/release/deno i'
Benchmark 1: ../d7/deno-main i
  Time (mean ± σ):      2.193 s ±  0.027 s    [User: 0.589 s, System: 1.033 s]
  Range (min … max):    2.151 s …  2.242 s    10 runs

Benchmark 2: ../d7/target/release/deno i
  Time (mean ± σ):      1.597 s ±  0.021 s    [User: 0.977 s, System: 1.337 s]
  Range (min … max):    1.550 s …  1.627 s    10 runs

Summary
  ../d7/target/release/deno i ran
    1.37 ± 0.02 times faster than ../d7/deno-main i
```

Caching `npm:@11ty/eleventy`:
```
❯ hyperfine -i -p 'rm -rf node_modules/' --warmup 5 '../../d7/deno-main cache npm:@11ty/eleventy' '../../d7/target/release/deno cache npm:@11ty/eleventy'
Benchmark 1: ../../d7/deno-main cache npm:@11ty/eleventy
  Time (mean ± σ):     129.9 ms ±   2.2 ms    [User: 27.5 ms, System: 101.3 ms]
  Range (min … max):   127.5 ms … 135.8 ms    10 runs

Benchmark 2: ../../d7/target/release/deno cache npm:@11ty/eleventy
  Time (mean ± σ):     100.6 ms ±   1.3 ms    [User: 38.8 ms, System: 233.8 ms]
  Range (min … max):    99.3 ms … 103.2 ms    10 runs

Summary
  ../../d7/target/release/deno cache npm:@11ty/eleventy ran
    1.29 ± 0.03 times faster than ../../d7/deno-main cache npm:@11ty/eleventy
```

---------

Co-authored-by: David Sherret <dsherret@gmail.com>
2024-05-28 21:17:36 +00:00
David Sherret 69da5d8290
perf: skip npm install if graph has no new packages (#24017) 2024-05-28 16:24:07 -04:00
David Sherret 57617af16a
perf: parse source files in parallel (#23858) 2024-05-28 16:23:11 -04:00
Nathan Whitaker 3e8f29ae41
perf(cli): Optimize setting up node_modules on macOS (#23980)
Hard linking (`linkat`) is ridiculously slow on mac. `copyfile` is
better, but what's even faster is `clonefile`. It doesn't have the space
savings that comes with hardlinking, but the performance difference is
worth it imo.

```
❯ hyperfine -i -p 'rm -rf node_modules/' '../../d7/target/release/deno cache npm:@11ty/eleventy' 'deno cache npm:@11ty/eleventy'
Benchmark 1: ../../d7/target/release/deno cache npm:@11ty/eleventy
  Time (mean ± σ):     115.4 ms ±   1.2 ms    [User: 27.2 ms, System: 87.3 ms]
  Range (min … max):   113.7 ms … 117.5 ms    10 runs

Benchmark 2: deno cache npm:@11ty/eleventy
  Time (mean ± σ):     619.3 ms ±   6.4 ms    [User: 34.3 ms, System: 575.6 ms]
  Range (min … max):   612.2 ms … 633.3 ms    10 runs

Summary
  ../../d7/target/release/deno cache npm:@11ty/eleventy ran
    5.37 ± 0.08 times faster than deno cache npm:@11ty/eleventy
```
2024-05-28 11:59:17 -07:00
David Sherret 448fe67b7a
feat(vendor): support modifying remote files in vendor folder without checksum errors (#23979)
Includes:

* https://github.com/denoland/deno_graph/pull/486
* https://github.com/denoland/deno_graph/pull/488
* https://github.com/denoland/deno_lockfile/pull/25
* https://github.com/denoland/deno_lockfile/pull/22
* https://github.com/denoland/deno_graph/pull/483
* https://github.com/denoland/deno_graph/pull/470
2024-05-28 14:58:43 -04:00
Yoshiya Hinosawa 8b5089e41f
fix(coverage): do not generate script coverage with empty url (#24007)
closes #24004
2024-05-28 23:01:32 +09:00
Hajime-san 9aa593cd5d
fix(cli/test): decoding percent-encoding(non-ASCII) file path correctly (#23200)
# Summary

This PR resolves about the issue.
fixes #10810

And the formerly context is in the PR. 
#22582

Here is an expected behaviour example with this change.

- 🦕.test.ts
```ts
import { assertEquals } from "https://deno.land/std@0.215.0/assert/mod.ts";

Deno.test("example test", () => {
  assertEquals("🍋", "🦕");
});
```
2024-05-28 13:34:57 +00:00
Mike Mulchrone dc5c799c39
fix(deno_task): more descriptive error message (#24001)
Signed-off-by: Mike Mulchrone <mikemulchrone987@gmail.com>
Co-authored-by: Satya Rohith <me@satyarohith.com>
2024-05-28 13:33:08 +00:00
Asher Gomez adbd564758
FUTURE(ext/ffi,ext/webgpu): stabilize FFI and WebGPU APIs (#24011)
Closes #23906
2024-05-28 11:37:43 +00:00
Yoshiya Hinosawa e9cc8a2b53
fix(coverage): skip generating coverage json for http(s) scripts (#24008)
closes #21784
2024-05-28 20:25:46 +09:00
Divy Srivastava 53606de634
BREAKING(ffi/unstable): always return u64 as bigint (#23981)
The mixed `number | bigint` representation was useful optimization for
pointers. Now, pointers are represented as V8 externals. As part of the
FFI stabilization effort we want to make `bigint` the only
representation for `u64` and `i64`.

BigInt representation performance is almost on par with mixed
representation with the added benefit that its less confusing and users
don't need manual checks and conversions for doing operations on the
value.

```
cpu: AMD Ryzen 5 7530U with Radeon Graphics
runtime: deno 1.43.6+92a8d09 (x86_64-unknown-linux-gnu)

file:///home/divy/gh/ffi/main.ts
benchmark                 time (avg)        iter/s             (min … max)       p75       p99      p995
-------------------------------------------------------------------------- -----------------------------
nop                        4.01 ns/iter 249,533,690.5     (3.97 ns … 10.8 ns) 3.97 ns 4.36 ns 9.03 ns
ret bigint                 7.74 ns/iter 129,127,186.8    (7.72 ns … 10.46 ns) 7.72 ns 8.11 ns 8.82 ns
ret i32                    7.81 ns/iter 128,087,100.5    (7.77 ns … 12.72 ns) 7.78 ns 8.57 ns 9.75 ns
ret bigint (add op)       15.02 ns/iter  66,588,253.2   (14.64 ns … 24.99 ns) 14.76 ns 19.13 ns 19.44 ns
ret i32    (add op)       12.02 ns/iter  83,209,131.8   (11.95 ns … 18.18 ns) 11.98 ns 13.11 ns 14.5 ns
```
2024-05-28 09:31:09 +05:30
Yoshiya Hinosawa d99c6c1ea4
fix(coverage): handle ignore patterns (#23974)
closes #23972
2024-05-28 12:37:30 +09:00