This reverts commit 659fdae898.
Reason for revert: Broke the HHH bots.
The fix according to zra@ should be:
"The new dependencies should be added as dependency overrides to flutter_frontend_server's pubspec.yaml: https://github.com/flutter/engine/blob/master/flutter_frontend_server/pubspec.yaml"
Original change's description:
> Move most of the code from u/b/kernel_worker.dart to p/frontend_server
>
> This should allow us to import the code in cases when we want to
> implement the Bazel worker loop ourselves. This opens some possibilities
> of extra optimizations for internal use cases.
>
> I'm leaving some of the code (e.g., `main`) in the old location, so that
> we do not break anything that depends on it.
>
> The only thing that I've done is to pass the input-digest map to
> `computeKernel` instead of using `Input` objects directly, this way
> we can avoid the dependency on `package:bazel_worker` from
> `frontend_server`. So now, if we're using the worker loop, the map will
> be computed by `kernel_worker.dart` (in non-worker mode, we rely on
> the default empty map).
>
> Bug: http://b/187910394
> Change-Id: I408407ebdebc17c7898552c053a8e1c269cf2df5
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/201187
> Commit-Queue: Michal Terepeta <michalt@google.com>
> Reviewed-by: Johnni Winther <johnniwinther@google.com>
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: http://b/187910394
Change-Id: Ieb44d61b668a7d4629a27eef5a6ef61d0db9deaf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/201880
Reviewed-by: Jake Macdonald <jakemac@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Phil Quitslund <pquitslund@google.com>
Auto-Submit: Jake Macdonald <jakemac@google.com>
Commit-Queue: Jake Macdonald <jakemac@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
Commit-Queue: Phil Quitslund <pquitslund@google.com>
This should allow us to import the code in cases when we want to
implement the Bazel worker loop ourselves. This opens some possibilities
of extra optimizations for internal use cases.
I'm leaving some of the code (e.g., `main`) in the old location, so that
we do not break anything that depends on it.
The only thing that I've done is to pass the input-digest map to
`computeKernel` instead of using `Input` objects directly, this way
we can avoid the dependency on `package:bazel_worker` from
`frontend_server`. So now, if we're using the worker loop, the map will
be computed by `kernel_worker.dart` (in non-worker mode, we rely on
the default empty map).
Bug: http://b/187910394
Change-Id: I408407ebdebc17c7898552c053a8e1c269cf2df5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/201187
Commit-Queue: Michal Terepeta <michalt@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Fix `application_snapshot`'s depfile to track the sources of the application instead of the compiler. Split compiling the compiler into a separate GN target with its own depfile.
Bug: https://github.com/flutter/flutter/issues/81074
Change-Id: I0fb23ada40a6241ee3dde7f6cfebdd121b9a4224
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/197020
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This reverts commit a23c31bf8d.
Reason for revert: The bootstrapping compilations are a bit confused about whose deps they're writing
Original change's description:
> [build] Don't list Dart sources up front when creating application snapshots.
>
> GN/Ninja will discover them via the depfile created alongside the snapshot.
>
> Bug: https://github.com/flutter/flutter/issues/81074
> Change-Id: I6e0f07214e8ea29e6d23261c71558da06fd2223a
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196982
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
TBR=bkonyi@google.com,rmacnak@google.com,chinmaygarde@google.com
Change-Id: I267b6bac2676a18f57291c8472fab5c2aaa60284
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/flutter/flutter/issues/81074
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/197000
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Opt an initial batch of files under samples, samples-dev,
utils, and runtime/tools/dartfuzz out of null safety in preparation
for switching the flag on by default.
Change-Id: Icdfd52a5a969e678a7205903332f73fe3841c223
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166960
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Leaf Petersen <leafp@google.com>
doesn't show these formatting changes as diffs.
Change-Id: I69ccbf4adabc66d88371cece785a2c1bce60f133
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138962
Reviewed-by: Ryan Macnak <rmacnak@google.com>
The two fields are already treated the same, so having two is just
confusing.
Change-Id: I649db2a77bced62429dfd94ab300673cf886e367
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138291
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Problem:
We have been having a lot of reports of failures when initializing the incremental compiler
(see https://github.com/dart-lang/sdk/issues/38102).
These appear to be the result of empty kernel files but we have added other logic to check
for that when kernel files are written, which has had zero reports of throwing from users
(https://github.com/dart-lang/build/pull/2387).
We also use the same mechanism for temp directory management as we did with analyzer, but only
see this problem with the switch to kernel, and specifically the incremental compiler.
Solution:
Add retry logic and see if that fixes the problem. Any time there is a retry log a warning
so that we don't just silently do retries all the time.
This is a general defensive mechanism to cover a broad spectrum of potential failures with the
incremental compiler in the wild. I don't intend to remove it as it isn't harmful, and the
warning logs should be enough to encourage issues to be filed if it is happening often.
I have no direct reason to believe this will actually solve the particular linked problem other
than that we only see this issue when the incremental compiler is enabled.
Bug: https://github.com/dart-lang/sdk/issues/39122, https://github.com/dart-lang/sdk/issues/38102
Change-Id: Iaabb4497d6da69684692c1d7b9c030c59ebc6072
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123001
Commit-Queue: Jake Macdonald <jakemac@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Auto-Submit: Jake Macdonald <jakemac@google.com>
Not doing it can lead to the same input producing "different" outputs.
Note that this is a follow-up to cdcec63569
where I forgot about the incremental summary-only case.
Change-Id: Idf76c3839f46c468a62350968b353be7235d91b9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118287
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Not doing it can lead to the same input producing "different" outputs.
Change-Id: I31fa1d728c2d26cbf1ad008472ea8fce4310794f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117725
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
This CL introduces 'tags' as a way to distinguish different setups and
be able to throw away previous state when it cannot be used.
These tags are - for now - basically filled up with the roots used for
the multi root filesystem, the idea being, that if they have changed we
cannot reuse the old state.
Change-Id: I19e069513ce3836f5bc6abf047e4359836fc7e09
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/114945
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Before this CL one could ask to use the incremental compiler, but not
actually get the incremental compiler because one supplied linked inputs
(and not just summary inputs).
This is an artificial limitation, and inside the compiler there are no
real difference between "linked" and "summary" inputs. "Summaries" are
marked as "external" but that's a historical hack to mark it as something
we don't want to serialize --- something that is not used in this package.
Generally, the distinction and hack should go away entirely.
Considering there is no difference in this package, a good start would be
to remove the distinction in this package, but as this package is used
both internally and externally, this CL won't do that, but will just treat
the two 'different' input types the same.
Change-Id: I11a3962d22424387eca83212d08f535da350bd1b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/113022
Reviewed-by: Jake Macdonald <jakemac@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Before this CL, if using the incremental compiler in bazels
kernel_worker, did not do summaries and did ask to "exclude-non-sources"
one would still get the entire thing (including loaded summaries etc)
(i.e. it would behave as if "exclude-non-sources" wasn't set).
This CL fixes it and makes it serialize as in the non-incremental case.
Change-Id: I396b53664a3d1ac32a243e903c8ff5c049fca453
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/113021
Commit-Queue: Jake Macdonald <jakemac@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
This change basically consists of these steps:
* Enable the incremental compiler to trace used inputs.
* Translate used libraries into used dill inputs.
* Output the list of used dills.
Bug: #37788
Change-Id: I08cffe299166cf10e990c9e261f190afd25da8b1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/112384
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
When invoking the kernel worker in worker mode, reusing previous state,
but not using the incremental compiler, we will currently leak,
see dartbug.com/36983.
This was previously fixed in the incremental compiler case, but this CL
moves the fix outside the if to fix it in every configuration.
Change-Id: I993fecbcc053d2bca64af9e65d270f75411f2e68
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103626
Reviewed-by: Kevin Millikin <kmillikin@google.com>
This includes:
* accepting null values like sdk-summary or .packages (which are not used when building kernel for the sdk itself)
* allow enabling language experiments
* only exclude-non-sources on the non-incremental code path if that's requested
Change-Id: I08eeb643676f1f1406f0f3030c341d68179d42a2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103574
Reviewed-by: Nate Bosch <nbosch@google.com>
This CL fixes a number of issues:
* Semi-leaking old components when starting over
(i.e. when loading a new sdk)
* Setting new filesystem not really working because the old one was
cached in the ProcessedOptions
* Leaking old state because of an apparent VM bug (http://dartbug.com/36983)
Change-Id: Ib124257c456b019941e253409c1ef56eedcf66b2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103121
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
This is needed to activate the kernel transformer embedded with the kernel package from the flutter engine.
Change-Id: I3253e01723b662eb48b4b3743ac5bcc1b44c7d46
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/102920
Reviewed-by: Jake Macdonald <jakemac@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Commit-Queue: Jacob Richman <jacobr@google.com>
Had to do a few changes to make this happen:
1. Restored some of the code removed in the transformer, making class
hierarchy optional.
2. Split the "compile" method in bazel_worker.dart into two, since we
need the intermediate Component object in one of the case.
Change-Id: I0ee004467e76dff0025f7979834c62b9db328ecc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100480
Commit-Queue: Jake Macdonald <jakemac@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Reviewed-by: Jacob Richman <jacobr@google.com>
TL;DR: Unbind canonical names doesn't do what you think it does and
probably shouldn't (ever) be used. This CL stops using it in a few places.
Longer version:
When loading a dill file it:
- First loads the table of canonical names. These have no references yet.
- When a canonical name is asked for its reference it creates one if it
doesn't yet have one.
- When loading, for instance, a library, it asks for the reference.
When unbinding a canonical name:
- It removes itself (the canonical name) from the reference
- It removes the reference in itself
- Note: Whatever has a pointer to the reference keeps it, and the
reference points to whatever node it already pointed to.
This also means, that if we have a dill file that's split in two and:
- Load #1
- Load #2
that works fine, but if we
- Load #1
- Unbind canonical names
- Binds canonical names
- Load #2
stuff is not bound correctly (and an error is thrown).
And - the cause of this bug:
- Load #1
- Load #2
everything is fine
- Unbind canonical names
- Binds canonical names
- Load #2'
stuff is not bound correctly --- references points to stuff loaded as #2,
not as #2'. On top of being weird, wrong and confusing it also caused wrong
things to be but into the class hierarchy which ultimatly caused the crash.
This CL fixes it by not calling unbind and force loading of dill files
(at specific call sites) to create new libraries
(and in the process overwriting references ".node").
Revert "[dartdevc] Retry ddc incremental compile on crash"
This reverts commit ecdbdf00b8.
Revert "[kernel_worker] retry on failure"
This reverts commit 43eebea5a3.
Fixes#36644
Bug: #36644
Change-Id: Id8f548179e6a409b01f2ebfa3219f94cb64b1c05
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100380
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Vijay Menon <vsm@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Reviewed-by: Jenny Messerly <jmesserly@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
The value 'devcompiler' is never being passed to this argument so this
will not break any known use cases. The name 'ddc' has better
consistency with how we label SDK platforms in `build_runner`.
Bug: https://github.com/dart-lang/build/issues/2174
Change-Id: I95f6cb75ec86c2f276298781443203d201550815
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99704
Reviewed-by: Jake Macdonald <jakemac@google.com>
Commit-Queue: Nate Bosch <nbosch@google.com>
Without this, if an error occurs during serialization you get this message:
```
Internal problem: Compiler cannot run without a compiler context.
```
Instead of the real error.
Change-Id: I016ec0923a2e01b68f850988724be9d30179f78d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98740
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Auto-Submit: Jake Macdonald <jakemac@google.com>
This allows dartdevc and the bazel worker to user
the incremental compiler with kernel.
Use --reuse-compiler-result --use-incremental-compiler to run in this mode.
Change-Id: I9189ce5f1a51320d9d96e071f8c95c80fe6fca84
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/95386
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Reviewed-by: Jenny Messerly <jmesserly@google.com>
This includes a `--target` flag (that in the future we can use for other targets
too) and a flag to accept the libraries.json file (currently needed to properly
compute constants of the form `bool.fromEnvironment('dart.library.io')`.
Change-Id: I8beea48610f0e1e721b7c76e545c9f662e3255ad
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96849
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
This flag was always true so we can remove it and specialize the
remaining code that used it. The only code that used it was in the
Dart VM's async transformation.
Specializing that code led to some unused fields in the async helper
class which were removed. This triggered me to look at core types
which seems to have a lot of unused stuff.
Change-Id: I7da800ca17ac05478149f4c90337a38d2c2b1459
Reviewed-on: https://dart-review.googlesource.com/c/87960
Reviewed-by: Peter von der Ahé <ahe@google.com>
Commit-Queue: Kevin Millikin <kmillikin@google.com>
- Switch the kernel isolate to run in Dart2 mode
- Refactor build rules so bootstrapping of the front end happens
with a prebuilt dart sdk
- Refactor application snapshot build rules so the kernel application
snapshot rule does not get into a cyclic dependency during the
bootstrap stage
Change-Id: Iccca3233f6cb8d6b307b4cf1ca3262f293f794a6
Reviewed-on: https://dart-review.googlesource.com/70841
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
