deno/cli/bench
Nathan Whitaker 422cff1f24
chore(lsp): Add benchmark for performance on a large real-world repo (#23395)
This PR adds a benchmark intended to measure how the LSP handles larger
repos, as well as its performance on a more realistic workload.

The repo being benchmarked is
[deco-cx/apps](https://github.com/deco-cx/apps) which has been vendored
along with its dependencies. It's included as a git submodule as its
fairly large. The LSP requests used in the benchmark are the actual
requests sent by VSCode as I opened, modified, and navigated around a
file (to simulate an actual user interaction).

The main motivation is to have a more realistic benchmark that measures
how we do with a large number of files and dependencies. The
improvements made from 1.42 to 1.42.3 mostly improved performance with
larger repos, so none of our existing benchmarks showed an improvement.

Here are the results for the changes made from 1.42 to 1.42.3 (the new
benchmark is the last one listed):

**1.42.0**

```test
Starting Deno benchmark
-> Start benchmarking lsp
   - Simple Startup/Shutdown 
      (10 runs, mean: 379ms)
   - Big Document/Several Edits 
      (5 runs, mean: 1142ms)
   - Find/Replace
      (10 runs, mean: 51ms)
   - Code Lens
      (10 runs, mean: 443ms)
   - deco-cx/apps Multiple Edits + Navigation
      (5 runs, mean: 25121ms)
<- End benchmarking lsp
```

**1.42.3**

```text
Starting Deno benchmark
-> Start benchmarking lsp
   - Simple Startup/Shutdown 
      (10 runs, mean: 383ms)
   - Big Document/Several Edits 
      (5 runs, mean: 1135ms)
   - Find/Replace
      (10 runs, mean: 55ms)
   - Code Lens
      (10 runs, mean: 440ms)
   - deco-cx/apps Multiple Edits + Navigation
      (5 runs, mean: 11675ms)
<- End benchmarking lsp
```
2024-04-16 12:26:51 -07:00
..
fs chore: upgrade to dprint 0.39 (#19768) 2023-07-08 18:34:08 +00:00
http refactor: set removal version for Deno.RequestEvent, Deno.HttpConn and Deno.serveHttp() (#22034) 2024-01-23 03:24:03 +00:00
napi chore: move test_ffi and test_nap to tests/ [WIP] (#22394) 2024-02-12 13:46:50 -07:00
stdio chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
testdata chore(lsp): Add benchmark for performance on a large real-world repo (#23395) 2024-04-16 12:26:51 -07:00
cache_api.js chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
command.js chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
console.js chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
deno_common.js chore(ext/io): remove use of deprecated Deno.writeSync() (#22872) 2024-03-20 10:39:25 -07:00
encode_into.js chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
getrandom.js chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
http.rs chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
lsp_bench_standalone.rs chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
lsp.rs chore(lsp): Add benchmark for performance on a large real-world repo (#23395) 2024-04-16 12:26:51 -07:00
main.rs chore(lsp): Add benchmark for performance on a large real-world repo (#23395) 2024-04-16 12:26:51 -07:00
op_now.js chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
README.md refactor(bench): continuous benchmarking improvements (#14821) 2022-06-28 17:51:05 +05:30
secure_curves.js chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
tcp.js chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
tty.js chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
url_parse.js chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
webstorage.js chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
write_file.js chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00

benchmark filtering:

cargo bench --bench deno_bench -- bundle

benchmark plots:

new: https://denoland.grafana.net/d/vErC9VCnz/benchmarks?orgId=1 old: deno.land/benchmarks