Commit graph

255 commits

Author SHA1 Message Date
Jens Johansen f0ca213d60 [CFE et al] Optimize presubmit scripts
This CL optimizes how CFE et al presubmits are run.

In the examples below we'll that it takes the presubmit time from 31+
to ~13 seconds, from 31+ to ~20 seconds and from 30+ to ~19 seconds on
a few simple cases and from 76+ to ~27 seconds in a case where files in
both _fe_analyzer_shared, front_end, frontend_server and kernel are
changed.

Before this CL, if there was changes in both front_end and
frontend_server for instance it would run one smoke-test for each.
They would each technically only test things in their own directory,
but they would do a lot of overlapping work, e.g. compiling
frontend_server also compiles front_end; the startup cost of a script
is done several times etc.

The bulk of the change in this CL is thus to only run things once.
Now, if there is a change in both front_end and frontend_server the
python presubmit will still launch a script for each, but it's just a
light-weight script that will take ~400 ms to run (on my machine) if it
decides to not do anything. What it does is that it looks at the
changed files, from that it will know which presubmits will be run and
decide which of them will actually do the work - the rest will just
exit and say "it will be tested by this other one".

Furthermore it then tries to run only the smoke tests necessary.
For instance, if you have only changed a test in front_end it will only
run the spell checker (and only for that file).
Note that this is not perfect and there can be cases where you should
get a presubmit error but wont. For instance if you remove all content
from the spellchecking dictionary file it should give you lots of
spelling mistake errors, but it won't because it won't actually run the
spell checker (as no files it should spell check was changed).
Probably you have to actively try to cheat it though, so I don't see it
as a big problem. Things will still be checked fully on the CI.

Additionally
* the generated messages will have trailing commas which speeds up
  formatting of the generated files (in the cases where the
  generated files will have to be checked).
* the explicit creation testing tool will do the outline of everything,
  but only do the bodies of the changed files.
* building the "ast model" only compiles the outline.

Left to do:
* If only changing a single test, for instance, it will only run the
  spell checker on that file, but launching the isolate its run in
  still takes ~7 seconds because it loads up other stuff too. Maybe we
  could have special entry points for cases where it only should run an
  otherwise simple test.
* The presubmit in the sdk dir (not CFE related) doesn't do well with
  many (big) changed files and testing them for formatting errors can
  easily take 10+ seconds (see example below where it contributes ~5
  seconds for instance). Maybe `dart format` could be made faster, or
  maybe the script should test more than one file at once.


*Example runs before and after*:

Change in a single test file in front_end
=========================================

Now:

```
$ time git cl presubmit -v -f
[I2024-01-25 09:46:08,391 187077 140400494405504 presubmit_support.py] Found 1 file(s).
Running Python 3 presubmit commit checks ...
Running [...]/sdk/PRESUBMIT.py
Running [...]/sdk/pkg/front_end/PRESUBMIT.py
Presubmit checks took 11.5s to calculate.
Python 3 presubmit checks passed.


real    0m12.772s
user    0m16.093s
sys     0m2.146s
```

Before:

```
$ time git cl presubmit -v -f
[I2024-01-25 10:07:08,519 200015 140338735470464 presubmit_support.py] Found 1 file(s).
Running Python 3 presubmit commit checks ...
Running [...]/sdk/PRESUBMIT.py
Running [...]/sdk/pkg/front_end/PRESUBMIT.py
  28.3s to run CheckChangeOnCommit from [...]/sdk/pkg/front_end/PRESUBMIT.py.
Presubmit checks took 30.0s to calculate.
Python 3 presubmit checks passed.


real    0m31.396s
user    2m9.500s
sys     0m11.559s
```

So from 31+ to ~13 seconds.



---------------------------------------------------------------------

Change in a single test file and a single lib file in front_end
===============================================================

Now:

```
$ time git cl presubmit -v -f
Running Python 3 presubmit commit checks ...
Running [...]/sdk/PRESUBMIT.py
Running [...]/sdk/pkg/front_end/PRESUBMIT.py
  15.9s to run CheckChangeOnCommit from [...]/sdk/pkg/front_end/PRESUBMIT.py.
Presubmit checks took 18.0s to calculate.
Python 3 presubmit checks passed.


real    0m19.365s
user    0m33.157s
sys     0m5.049s
```

Before:

```
$ time git cl presubmit -v -f
[I2024-01-25 10:08:36,277 200953 140133274818432 presubmit_support.py] Found 2 file(s).
Running Python 3 presubmit commit checks ...
Running [...]/sdk/PRESUBMIT.py
Running [...]/sdk/pkg/front_end/PRESUBMIT.py
  27.9s to run CheckChangeOnCommit from [...]/sdk/pkg/front_end/PRESUBMIT.py.
Presubmit checks took 30.0s to calculate.
Python 3 presubmit checks passed.


real    0m31.311s
user    2m9.854s
sys     0m11.898s
```

So from 31+ to ~20 seconds.

---------------------------------------------------------------------

Change only the messages file in front_end (but with generated files not changing)
==================================================================================

Now:

```
$ time git cl presubmit -v -f
[I2024-01-25 09:53:02,823 190466 140548397250432 presubmit_support.py] Found 1 file(s).
Running Python 3 presubmit commit checks ...
Running [...]/sdk/PRESUBMIT.py
Running [...]/sdk/pkg/front_end/PRESUBMIT.py
  15.6s to run CheckChangeOnCommit from [...]/sdk/pkg/front_end/PRESUBMIT.py.
Presubmit checks took 17.0s to calculate.
Python 3 presubmit checks passed.


real    0m18.326s
user    0m38.999s
sys     0m4.530s
```

Before:

```
$ time git cl presubmit -v -f
[I2024-01-25 10:10:04,431 201892 140717686302592 presubmit_support.py] Found 1 file(s).
Running Python 3 presubmit commit checks ...
Running [...]/sdk/PRESUBMIT.py
Running [...]/sdk/pkg/front_end/PRESUBMIT.py
  28.0s to run CheckChangeOnCommit from [...]/sdk/pkg/front_end/PRESUBMIT.py.
Presubmit checks took 29.2s to calculate.
Python 3 presubmit checks passed.


real    0m30.550s
user    2m9.488s
sys     0m11.689s
```

So from 30+ to ~19 seconds.

---------------------------------------------------------------------

Change several files:
```
$ git diff --stat
 pkg/_fe_analyzer_shared/lib/src/messages/codes_generated.dart         | 4 ++--
 pkg/_fe_analyzer_shared/lib/src/parser/listener.dart                  | 2 ++
 pkg/front_end/lib/src/api_prototype/incremental_kernel_generator.dart | 2 ++
 pkg/front_end/lib/src/base/processed_options.dart                     | 2 ++
 pkg/front_end/messages.yaml                                           | 2 +-
 pkg/front_end/tool/dart_doctest_impl.dart                             | 2 ++
 pkg/frontend_server/lib/compute_kernel.dart                           | 2 ++
 pkg/kernel/lib/ast.dart                                               | 2 ++
 8 files changed, 15 insertions(+), 3 deletions(-)
```
====================

Now:

```
[I2024-01-25 09:57:53,270 193911 140320429016960 presubmit_support.py] Found 8 file(s).
Running Python 3 presubmit commit checks ...
Running [...]/sdk/PRESUBMIT.py
Running [...]/sdk/pkg/_fe_analyzer_shared/PRESUBMIT.py
  17.8s to run CheckChangeOnCommit from [...]/sdk/pkg/_fe_analyzer_shared/PRESUBMIT.py.
Running [...]/sdk/pkg/front_end/PRESUBMIT.py
Running [...]/sdk/pkg/frontend_server/PRESUBMIT.py
Running [...]/sdk/pkg/kernel/PRESUBMIT.py
Presubmit checks took 25.3s to calculate.
Python 3 presubmit checks passed.


real    0m26.585s
user    1m8.997s
sys     0m8.742s
```

Worth noting here is that "sdk/PRESUBMIT.py" takes 5+ seconds here

Before:

```
[I2024-01-25 10:11:39,863 203026 140202046494592 presubmit_support.py] Found 8 file(s).
Running Python 3 presubmit commit checks ...
Running [...]/sdk/PRESUBMIT.py
Running [...]/sdk/pkg/_fe_analyzer_shared/PRESUBMIT.py
  14.6s to run CheckChangeOnCommit from [...]/sdk/pkg/_fe_analyzer_shared/PRESUBMIT.py.
Running [...]/sdk/pkg/front_end/PRESUBMIT.py
  28.0s to run CheckChangeOnCommit from [...]/sdk/pkg/front_end/PRESUBMIT.py.
Running [...]/sdk/pkg/frontend_server/PRESUBMIT.py
  20.9s to run CheckChangeOnCommit from [...]/sdk/pkg/frontend_server/PRESUBMIT.py.
Running [...]/sdk/pkg/kernel/PRESUBMIT.py
Presubmit checks took 75.6s to calculate.
Python 3 presubmit checks passed.


real    1m16.870s
user    3m48.784s
sys     0m23.689s
```

So from 76+ to ~27 seconds.

In response to https://github.com/dart-lang/sdk/issues/54665

Change-Id: I59a43f5009bba8c2fdcb5d3a843b4cb408499214
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/348301
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2024-01-31 10:41:20 +00:00
David Morgan 3c77c86929 [macros] Improvements to exceptions.
Introduce `MacroException` base type that is serializable as a `RemoteInstance`. This means that if a macro implementation does not catch the exception then the exact same instance will be recovered on deserialization.

Add `UnexpectedMacroException`, `MacroImplementationException` and `MacroIntrospectionCycleInspection`.

Update API docs, analyzer and CFE to throw MacroImplementationException instead of ArgumentError.

To do after this PR: update spec, analyzer and tests `StateError`->`MacroIntrospectionCycleInspection`.

R=jakemac@google.com

Change-Id: I76be4ef7687a5f5611c963be40fb0918ac3fb7ea
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/347680
Reviewed-by: Jake Macdonald <jakemac@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Morgan :) <davidmorgan@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2024-01-29 15:53:27 +00:00
Jens Johansen cdad90dfb8 [frontend_server] frontend_server_flutter_suite
Generally the test loads a big dill thats 90+% the same content as the
previous load, then verifies it.
This CL loads smarter and verifies less.

Before this CL, locally, running
pkg/frontend_server/test/frontend_server_flutter.dart took
real    24m56.080s
user    48m42.422s
sys     1m2.360s

and the suite edition (using 4 shards in isolates) took
real    15m9.196s
user    53m41.118s
sys     1m30.045s

With this CL, locally running
pkg/frontend_server/test/frontend_server_flutter.dart takes
real    5m0.206s
user    9m23.933s
sys     0m20.984s

and the suite edition takes
real    3m24.243s
user    12m0.069s
sys     0m28.131s

On the try-bot the runtime seems to have gone from ~40 minutes
to ~20 minutes, the "compile flutter tests" step from ~30 minutes to
~10 minutes and the portion of time actually running the
dart-code that compiles, loads and verifies, from ~26 minutes
to ~7 minutes.

Change-Id: I6db225c33e1c0ee817f3880327e720446150ad7d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/347282
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2024-01-26 14:28:40 +00:00
Jens Johansen 39d68052a9 [frontend_server/DDC] Expression compilation for JavaScript can pass scriptUri
This allows using the new scope finder and facilitates expression
compilation with and in extension types.

For now the script uri is optional (and only passable in the new json
input via package:frontend_server), and only if the script uri is passed
we'll use the new scope finder.

Flutter etc should be updated to pass the new data.

Change-Id: I36eed1ea76a825e63e4c5b9ea60daf18aee39f3d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/342400
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Anna Gringauze <annagrin@google.com>
2024-01-10 10:12:22 +00:00
Jens Johansen c152cce35c [frontend server] Add test for expression compilation on/in extension types
This is done via the new json communication channel, which initially
also added support for sending "scriptUri" and "offset".

The js version of expression compilation still needs updating.
Flutter etc will also still have to be updated.

Change-Id: I428bda68aaa13aa947b57e87f7a5bfb28802cf95
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/341921
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2023-12-20 11:14:57 +00:00
Jens Johansen 69a8834d15 [frontend_server] Take json when wanting to compile expressions
This adds an option to compile expressions (vm ones and javascript ones)
by communicating via json. This allows for future expansion without
destroying the communication channel.

For now the old versions will continue to exist, again to not destroy
alreadyy used communication channels.

Flutter etc. should be updated to use this new json communication,
but it can wait until plumbing in passing offset and scriptUri anyway.

Change-Id: Ifbff752527fea35e09a73e0bb210072d8ab81cdc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/342340
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2023-12-20 10:39:01 +00:00
David Morgan 0c4db1ea87 [macros] Deduplicate RemoteException.
It's not clear that we need two; I kept the one with the better
`toString`.

R=jakemac@google.com

Change-Id: I80934c0f1b05b0c486105654b03b628e7acf439b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/341920
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Morgan :) <davidmorgan@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Auto-Submit: Morgan :) <davidmorgan@google.com>
2023-12-20 09:49:38 +00:00
Jens Johansen bf96a722bc [frontend_server] CFE team takes ownership of package:frontend_server
First of we apply the wanted lints etc we prefer.
This includes but isn't limited to using types (i.e. no "var") and
being explicit about creation (i.e. no missing "new").

Change-Id: I516bccdac9760221ea5311af4567466bb4a65c77
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/341960
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2023-12-19 10:38:28 +00:00
Nicholas Shahan 3838338bc4 [frontend_server] Add dart:_rti to test libraries
Compiling with the new type system in DDC requires a few more classes
from the dart:_rti library.

Change-Id: I984e0cdbe3937e4ef70c40a9a6eabc714738e88a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/341780
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
2023-12-15 22:20:18 +00:00
David Morgan 11e29a7d9e [ddc] Add macro flags, add test coverage for basic compile.
Support and test coverage for other modes to follow.

R=cstefantsova@google.com, jakemac@google.com

Change-Id: Ic715f43317364955b3b2341134cf3cf576cebd00
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/339321
Reviewed-by: Mark Zhou <markzipan@google.com>
Commit-Queue: Morgan :) <davidmorgan@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Auto-Submit: Morgan :) <davidmorgan@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
2023-12-12 10:07:42 +00:00
David Morgan 296a7f642e [macros] Remove Introspectable* types.
Per https://github.com/dart-lang/language/issues/3442 inrospection will
be allowed to succeed if possible, failing if there are cycles.

Change-Id: I341cc4ae87d1399df82cebb1d5c9df7e5070e777
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/339620
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Morgan :) <davidmorgan@google.com>
2023-12-07 09:44:59 +00:00
David Morgan c7df717d3e [CFE] Simplify path handling in macros_test.dart.
R=jensj@google.com

Change-Id: Ib94ef362d18016502f34e5dac15f0ec4ae996397
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/340081
Auto-Submit: Morgan :) <davidmorgan@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Morgan :) <davidmorgan@google.com>
2023-12-06 17:28:39 +00:00
David Morgan 590b5434f5 [cfe] Remove unused precompiled-macro-format option.
The format is deduced from the file extension: dill for kernel, anything
else is treated as an executable.

R=jakemac@google.com

Change-Id: Ic1202dea232806ede15929383caa0b1cb484e0f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/339580
Auto-Submit: Morgan :) <davidmorgan@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Commit-Queue: Jake Macdonald <jakemac@google.com>
2023-12-04 15:52:35 +00:00
David Morgan 74b851667f [CFE] Really fix macros_test.dart for Windows.
R=jensj@google.com

Change-Id: Ied86939d8f6f312b156639305eaab65053d8e90d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/338661
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Morgan :) <davidmorgan@google.com>
Auto-Submit: Morgan :) <davidmorgan@google.com>
2023-11-30 13:28:28 +00:00
David Morgan 26107a319a [CFE] Refactor macros setup.
Add `dispose` method to `ProcessedOptions` that does macro executor cleanup; add `dispose` method to MultiMacroExecutor for it to call.

R=asiva@google.com, jakemac@google.com

Change-Id: I90d9af133e84655ec14870400988fdee9adb219e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/338440
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Morgan :) <davidmorgan@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
2023-11-30 08:08:11 +00:00
Ian Hickson 86231e2d70 Remove the widget cache experiment.
Bug: https://github.com/flutter/flutter/issues/132157
Change-Id: Ie87ba74b8ed99477cc5032824286adb07f1157d6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319871
Reviewed-by: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2023-11-29 08:47:59 +00:00
David Morgan 7f80ebc6b9 [CFE] Extend macros_test to cover incremental mode too.
R=jensj@google.com

Change-Id: Ic122983a6fcab8fac6584083f16de1d954596c2d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/338123
Reviewed-by: Jens Johansen <jensj@google.com>
Auto-Submit: Morgan :) <davidmorgan@google.com>
Commit-Queue: Morgan :) <davidmorgan@google.com>
2023-11-28 08:52:25 +00:00
David Morgan cd4c378f82 [CFE] Fix macros_test.dart for Windows.
R=jensj@google.com

Change-Id: I8fce79c0726878caab1d8db37588f97af1bc1670
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/338340
Auto-Submit: Morgan :) <davidmorgan@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Morgan :) <davidmorgan@google.com>
2023-11-27 15:50:07 +00:00
David Morgan 1ff56282ce Add end to end test for basic macro compilation and application to frontend_server.
R=jakemac@google.com, jensj@google.com

Change-Id: Icdaf64f55c9a71c04b2984c12fbed8248131475a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/337802
Auto-Submit: Morgan :) <davidmorgan@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Morgan :) <davidmorgan@google.com>
2023-11-24 11:54:19 +00:00
Devon Carew 6917656134 [pkg] remove duplicate config from the analysis options files
Change-Id: I149a97bbe260600a3de664b28c4d212dd9adb889
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335862
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Phil Quitslund <pquitslund@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2023-11-14 00:37:53 +00:00
Alexander Markov b15e1bf19d [vm] Rename nullSafety to soundNullSafety in pkg/vm
Also:
* Clean up obsolete code for returning null safety from kernel
service.
* Enable sound null safety in TFA unit tests.

TEST=ci

Change-Id: I1e008ed661823e9cb8004670d6b66bef35089a19
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/334081
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-11-07 14:32:20 +00:00
Alexander Thomas b2a158cc43 [frontend_server] Cleanup license headers
Bug: b/286184681
Change-Id: Ia203b87fecf17903b46bf929b22d3667b6ec2d59
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330785
Auto-Submit: Alexander Thomas <athom@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2023-10-18 06:59:38 +00:00
Jake Macdonald 990e595bf0 Update precompiled-macro-flag behavior to match the new consensus format from https://github.com/dart-lang/sdk/issues/53710, and support multiple libraries registered from one binary.
Also shuts down all macro processes after each execution.

Bug: https://github.com/dart-lang/sdk/issues/53710
Change-Id: I40851418d970789e048798f615fcca32c95496f8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330501
Auto-Submit: Jake Macdonald <jakemac@google.com>
Commit-Queue: Jake Macdonald <jakemac@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2023-10-16 14:55:52 +00:00
Jens Johansen 73466729f3 [vm] Pass offset and script uri for expression compilation
This CL passes the offset and uri of the file (here called a script uri
as opposed to a library uri, the two will be different if we're in a
part) when doing expression compilation.

This CL only passes the data, but doesn't actually use it.
Future CL(s) will use this data to calculate the static type of
available variables which is needed for an upcomming feature.

TEST=Existing tests.
CoreLibraryReviewExempt: Not changing SDK APIs.
Change-Id: I67ead461ab4bb9341424e693946f3e4afe35ce92
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/329322
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2023-10-12 10:22:38 +00:00
Devon Carew e2fe203adc [deps] roll package:lints to the latest
Change-Id: I582f956cd4b712203c2f6dd630b4e1384040446d
Tested: analysis clean (this is a lint only related change)
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/329400
Reviewed-by: Phil Quitslund <pquitslund@google.com>
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2023-10-05 03:36:59 +00:00
Johnni Winther f0933ab69e [cfe] Copy const_finder visitor from Flutter SDK
This copies the const_finder visitor from the Flutter SDK to the
Dart SDK to avoid having source code dependency on package:kernel
in the Flutter repository.

Change-Id: I76f98453c1650e63623708a9f4860d80ab4b25aa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/326645
Reviewed-by: Christopher Fujino <fujino@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2023-09-19 12:19:26 +00:00
Alexander Thomas 9394b30b77 Fix incorrect file reference in license headers
Bug: b/286184681
Change-Id: I903528c4adfbc576644aec7541903df6b9633e26
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325523
Reviewed-by: Jonas Termansen <sortie@google.com>
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2023-09-18 06:44:08 +00:00
Jackson Gardner 8001257ed0 [dart2wasm] Support for kernel_worker and unevaluated constants.
Change-Id: Icd92cb870ffd57635962e1e9831d7d777a7b00d2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/323000
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Commit-Queue: Jackson Gardner <jacksongardner@google.com>
2023-09-06 18:02:58 +00:00
Ryan Macnak 4038b4f9df [frontend_server] Forward --keep-class-names-implementing.
Cf. 708f098c4f

TEST=ci
Change-Id: I65b51d5aa93eeb0e41ee276b4288dd231d6dbc87
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/323424
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2023-08-31 22:27:45 +00:00
Jens Johansen 77d21b8339 [frontend-server] flutter frontend try bot don't save to actual file
The test compiles a file, serializes it to disk, loads it from disk and
verifies it. There's more than 1500 compiles and they (at least some)
are up to something like 40 MB each meaning why might write something
like 60 GB (!) running this test.
As the data is only used in-process we can use `IOOverrides` to bybass
it actually saving to a file.

Change-Id: I4130d02c227ba6769d63482a499846a485aa5a3a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322702
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2023-08-25 16:56:29 +00:00
Jens Johansen d00706a8eb [frontend-server] flutter frontend try bot don't verify platform
Change-Id: I4c5140d81773d2261044dbac00f4c7f029fa8804
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322701
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2023-08-25 13:17:11 +00:00
Jens Johansen 9f9da91b84 [kernel] Facilitate faster sorting of additional exports
The "additionalExports" are sorted in production code to give stable
output.
Before this CL it did `toString()` on the canonical names which is
O(n^2) because of continued string concatenation.
This CL changes it to sort on the name, going to the parent if it's the
same. This _does_ change the sorting, but should still be stable.

When compiling via the flutter frontend try bot the time spent on
sorting does from ~45 seconds to ~9 seconds (arguable still a lot, but
much better).

This will also speed up flutter testing because it will spend less time
waiting for the frontend.

TEST=Assuming existing tests.

Change-Id: If350b7c2ce49f00b2924732a7fa7eb51c38c8172
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322280
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2023-08-24 13:21:38 +00:00
Anna Gringauze 789b1d1fcd [frontend_server] Add --canary flag
Closes: https://github.com/dart-lang/sdk/issues/52774
Change-Id: Ie42a803c5b63a41c37984a790ab0406c8939872d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311149
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Mark Zhou <markzipan@google.com>
2023-06-27 22:21:04 +00:00
Jake Macdonald d1ecf01f71 Cache remote objects by ID, only send IDs for already sent objects
This is done through a synchronized cache between the server and client. When serializing a remote instance, if the server has already serialized that object then it will only send the ID in the future.

These caches currently only live as long as a single macro application in a given phase, but could live longer in the future. They do need to get reliably cleared out to avoid memory leaks though, and the shorter lifetime is easier to manage consistently.

This also allowed me to remove the specialized server/client modes (clients would always only send back IDs previously).

Change-Id: I4e8a102403153829d66b0ac379636f5a95a70cea
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311420
Commit-Queue: Jake Macdonald <jakemac@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
2023-06-26 22:36:37 +00:00
Johnni Winther 82e3a751a9 [cfe] Remove unnecessary_null_comparison code
The frontend is now run with sound null safety so these are no longer needed.

TEST=existing

Change-Id: I6c1776845854695ff34e310a3bb5bc9d86715f06
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/307901
Reviewed-by: Jens Johansen <jensj@google.com>
Reviewed-by: Srujan Gaddam <srujzs@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2023-06-08 08:46:47 +00:00
Alexander Aprelev 17781dbc36 [frontend/dds] Enable frontend server and dds to pass through types, method and class information for expression evaluation.
This information is needed for correct expression compilation.
Method name, for example, introduced in 48d8225b17 is needed for the compiler to confirm extension method context.

For flutter https://github.com/flutter/flutter/pull/128084 will be needed.

Fixes https://github.com/dart-lang/sdk/issues/52522

Change-Id: I32e53a953ebfb51afb462f4e2169fac0bcee0a7a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306908
Reviewed-by: Jens Johansen <jensj@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2023-06-02 16:27:28 +00:00
Daco Harkes 3466d6855a [frontend_server] Support @Native assets in compute_kernel
The utils/bazel/kernel_worker.dart entry-point uses
pkg/frontend_server/lib/compute_kernel.dart as entry-point.
So we need to cover this entry point to the frontend_server as well
to pass the native asset yaml file.

Upstreamed from cl/530924434 and cl/512913404.

Change-Id: I86204870807da4ee3e8a7b10c40a17862042da08
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286141
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-05-16 09:50:38 +00:00
Sigmund Cherem 1bd1d9d320 [dart2js] delete bin/dart2js.dart
This entrypoint is practically unused. We normally use
`lib/src/dart2js.dart` instead.

Long ago it wasn't possible to launch a script from the `lib` folder
without running into import canonicalization issues. However, a few
years ago the VM and CFE now normalize the entrypoints. When we
invoke `pkg/compiler/lib/src/dart2js.dart` it gets treataed as
`package:compiler/src/dart2js.dart`. As a result, we really no
longer had a need to have a separate entrypoint in the `bin` folder
that reexports the other entrypoint.

The only remaining references to this file were from documentation
and a couple unit tests. This CL updates those remaining uses.

Change-Id: Ic4c3c5c030db4e497f1d136979f7212db29cc533
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302640
Reviewed-by: Nate Biggs <natebiggs@google.com>
Auto-Submit: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
2023-05-10 17:21:41 +00:00
Johnni Winther 009cbfbeac [kernel] Merge front_end and kernel verifiers
This merges the front_end and kernel verifiers into one Target based
kernel verifier. The RedirectingFactoryBody work-around is moved to
package:kernel to support its verification.

TEST=existing

Change-Id: I0adf4d2c22c4009cf439b3b23fa14192253a2846
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280161
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Reviewed-by: Joshua Litt <joshualitt@google.com>
2023-04-26 14:14:36 +00:00
Lasse R.H. Nielsen 67797ef46f Add more class modifiers to dart:collection.
Pure interfaces made `interface`s
implementation classes which cannot/should not be extended made `final`.

A class like `HasHMap` which provides the `Map` interface,
and no implementation except factory constructors
for internal implementations, is made `final`.

Unified {List,Set,Map}{Base,Mixin} into their `Base` class.
Deprecations are retained in comments for now, to be landed
separately. Search for '// TODO: @Deprecated'.

Tested: No new test, only adding restrictions on use.
CoreLibraryReviewExempt: Everybody's on vacation, everybody everywhere.
Change-Id: Ia83b8a3bb20b5b214546b328d4492de6658253db
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/288240
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Lasse Nielsen <lrn@google.com>
Reviewed-by: Nate Bosch <nbosch@google.com>
2023-04-04 10:39:48 +00:00
Konstantin Shcheglov 895540889e Update SDK constraints for SDK packages.
https://dart-review.googlesource.com/c/sdk/+/287660 implements it.
I want to fix pre-existing violations before enabling.

Bug: https://github.com/dart-lang/sdk/issues/34978
Change-Id: Ie7731162c643018a2312b265f444bc00534c0a51
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/287664
Reviewed-by: Leon Senft <leonsenft@google.com>
Commit-Queue: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2023-03-09 19:52:47 +00:00
Daco Harkes 40787d84fc [frontend_server] Add support for @Native assets
The VM can read native asset mappings from kernel.

Previous CLs already added support to embed native asset mappings for
one-shot compilation.
This CL adds support for adding native assets mappings to kernel files
created by the frontend_server with the incremental compiler.

The frontend_server accepts a `--native-assets=<uri>` at startup and
accepts a `native-assets <uri>` message on stdin as compilation
command.

The frontend_server caches the compiled native assets library.

When a `reset` command is sent to request a full dill from the
incremental compiler, the native assets mapping is taken from the
cache and added to the final dill file.

Split of DartSDK & flutter_tools prototype to land separately.

TEST=pkg/frontend_server/test/native_assets_test.dart

Bug: https://github.com/dart-lang/sdk/issues/49803
Bug: https://github.com/dart-lang/sdk/issues/50565
Change-Id: I6e15f177564b8a962e81261815e951e7c9525513
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-mac-release-try,pkg-win-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/282101
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-02-21 07:29:29 +00:00
Devon Carew 12f0711a5b [pkg/frontend_server] use package:lints/recommended.yaml
Change-Id: I13f6ede17a29972b53283e0bb32960b4ea364aa5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/282388
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2023-02-10 20:29:17 +00:00
Tess Strickland 2c0484c720 [pkg/vm] Initial work on constant operating system fields and getters.
* Add an `targetOS` argument to `pkg/vm`'s `compileToKernel`,
  that contains the target operating system's name.

* Add a new `--target-os` command line argument for all binaries
  that use `compileToKernel` for clients to provide the target
  operating system, if known.

* Add a new`"vm:platform:const"` annotation to certain field and
  getters in the Platform class.

  This annotation is used to annotate static getters and fields with
  initializers where the getter body or field initializer must evaluate
  to a constant value if the target operating system is known. This
  annotation may be used outside the Platform class and in user code.

  For example, this annotation can be used on a static `String` field
  that is initialized with one value if `Platform.isWindows` is true
  and to a different value if `Platform.isWindows` is false.

  Note: If the const functions experimental flag is disabled, then
  any annotated static methods can only contain a single expression
  whose value is returned. If it is enabled, then the static method
  is evaluated as if it is a const function with the special
  handling of annotated static fields and getters above.

* Create a VM constant evaluator that evaluates uses of static getters
  and fields marked with the above annotations when a target operating
  system is provided.

* Use the new VM constant evaluator in the unreachable code elimination
  transformer.

TEST=pkg/vm/test/transformations/platform_use_transformer
     pkg/vm/test/transformations/unreachable_code_elimination

Change-Id: Ie381de70486a767fd7b1d515fd9e6bb58c6bf090
Bug: https://github.com/dart-lang/sdk/issues/31969
Cq-Include-Trybots: luci.dart.try:pkg-linux-release-try
CoreLibraryReviewExempt: Just adding vm-specific annotations.
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/274386
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-02-10 16:29:50 +00:00
Daco Harkes 897acba77f [frontend_server] Cleanup tests
Await unawaited futures and run formatter.

Change-Id: I2808b8cdc44ef9462e0c65d2ece72fdda9e114fe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/281180
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2023-02-07 10:06:50 +00:00
Alexander Markov a9d9b7395b [kernel] Enable sound null safety in TargetFlags by default
TargetFlags.enableNullSafety is set to true by default and
also renamed to TargetFlags.soundNullSafety to better reflect its
meaning.

TEST=ci

Change-Id: I2c2f30c2af6502fd9a96141dc60e4afbf8c524fd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280216
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2023-02-02 14:54:18 +00:00
Josh Soref 77978889eb Spelling
Closes https://github.com/dart-lang/sdk/pull/51143

GitOrigin-RevId: 9e21c99a222d588e4fc95980725a2f8c9784965c
Change-Id: If0870e8936c7649935dce7e23cd783d62aa5610c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/279916
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2023-01-30 18:29:59 +00:00
Josh Soref 01b28894e7 Spelling pkg dev compiler
Closes https://github.com/dart-lang/sdk/pull/50861

GitOrigin-RevId: 71005e6f5bf5a151cb5c1aefb6a2a300fc40f592
Change-Id: Iadfafb5787a62e9a379437f6a3763d31f99ba7c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/277743
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2023-01-26 09:12:41 +00:00
Josh Soref ef42a0b110 Spelling pkg analyzer lib
Closes https://github.com/dart-lang/sdk/pull/50860

GitOrigin-RevId: b27066c37f93c8c6d1123d6ebd6a4c0afcf59844
Change-Id: I15fa4aea1dad45daf168e34d1c4450320ec9b40a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/277742
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2023-01-25 14:08:27 +00:00
Nicholas Shahan c140324e1e [ddc] Rename .dill files
- Makes the names consistent with dart2js and dart2wasm.
- Prepares for the upcoming removal of the weak null safety .dill file.

Change-Id: Idcf694b27cd4731db8f7ed6f0fdab7766ced08a6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/277183
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Mayank Patke <fishythefish@google.com>
2023-01-18 01:34:57 +00:00