In the Flutter engine, we plan to expand and start using Dart actions to
rely more on the GN build system, and less on home-grown (Python/Bash)
tooling. To that end, we'll start generating dart_snapshot and dart_test
rules that are invokved the same way locally (via `et`) and on CI, with
a goal of making all of our tests programatically discoverable.
We need the `metadata` field (a standard GN field) in order to tag our
outputs (i.e. `metadata = { action_type = ['dart_executable'] }`);
see https://gn.googlesource.com/gn/+/main/docs/reference.md#var_metadata.
Bug: https://github.com/flutter/flutter/issues/147013
Change-Id: I0e44209b8aade13cec99b944583a90a7c13d3300
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/386582
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This reverts commit 5cda2a871c.
Reason for revert: broke Flutter build.
Original change's description:
> [tools] Allow precompiling gen_kernel and compile_platform
>
> When iterating on core library changes or changes in the AOT compiler
> many seconds are wasted waiting on gen_kernel/compile_platform to
> parse Dart code. This happens because we are running these tools
> from sources on prebuilt Dart SDK.
>
> This CL allows SDK developer to opt-in into AOT compiling these
> tools by adding `precompile_tools=true` to their DART_GN_ARGS.
>
> AOT compilation is performed using prebuilt SDK - so these
> executables do not need to be recompiled if core libraries or
> VM changes reducing iteration cycles.
>
> pkg/vm/tool/precompiler2 is tweaked to detect when DART_GN_ARGS
> contains `precompile_tools=true` and use precompiled
> gen_kernel.exe instead of running it from source.
>
> Using precompiled compile_platform takes vm_platform_strong.dill
> build from 20 seconds to 3 seconds.
>
> Using precompiled gen_kernel takes small benchmark build from
> ~10 seconds to 2 seconds.
>
> TEST=manually tested
>
> Change-Id: Ieec6ad4e1081023d140eb744f0a3cd0c754414ca
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/367940
> Commit-Queue: Slava Egorov <vegorov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Change-Id: Id3e4eb44d33516f31c165d9a1e55911e8d356e7f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/367960
Commit-Queue: Slava Egorov <vegorov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
When iterating on core library changes or changes in the AOT compiler
many seconds are wasted waiting on gen_kernel/compile_platform to
parse Dart code. This happens because we are running these tools
from sources on prebuilt Dart SDK.
This CL allows SDK developer to opt-in into AOT compiling these
tools by adding `precompile_tools=true` to their DART_GN_ARGS.
AOT compilation is performed using prebuilt SDK - so these
executables do not need to be recompiled if core libraries or
VM changes reducing iteration cycles.
pkg/vm/tool/precompiler2 is tweaked to detect when DART_GN_ARGS
contains `precompile_tools=true` and use precompiled
gen_kernel.exe instead of running it from source.
Using precompiled compile_platform takes vm_platform_strong.dill
build from 20 seconds to 3 seconds.
Using precompiled gen_kernel takes small benchmark build from
~10 seconds to 2 seconds.
TEST=manually tested
Change-Id: Ieec6ad4e1081023d140eb744f0a3cd0c754414ca
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/367940
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This is a reland of commit b3253d9c8b
Fix the gn rules so the RBE offloading only happens in the Dart build
and the conflicting RBE configuration is not included otherwise. This
ensures this change can roll into Flutter while we design a better
solution to compose the projects' RBE usages.
Original change's description:
> [infra] RBE all the things.
>
> This change offloads all Dart actions during the build to RBE using
> the new rewrapper_dart.py script that understands all the different
> command line invocations of Dart programs and translates it into the
> appropriate rewrapper invocation, with a full list of input and output
> files and no absolute paths.
>
> The dart actions are all considered to be expensive RBE compilation
> steps that may be much slower on RBE than locally due to the low
> parallelism of the build during these stages and the comperatively
> slower bots in the RBE pools. The build uses the racing strategy by
> default for these compilations, such that cache hits are still used
> if available, and otherwise a local build is transparently used if it
> happens to be faster. The bots will use the remote strategy by default
> unlike developers, such that the rewrapper_dart.py script does not fail.
>
> rewrapper_dart.py contains a tiny dart import proprocessor written
> in python and a big argument parser that understands every command
> invoked during the build and the semantic meaning of every option.
>
> The absolute paths used during the Dart SDK build is not solved in
> this changelist, which just works around them initially, but these
> will be fixed in follow up changes with the appropriate teams now
> that this change proves they are not needed.
>
> Bug: b/333595242
> Change-Id: I36603ec1bf16f4ac87d56635cc1c98e8686a4028
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335827
> Reviewed-by: William Hesse <whesse@google.com>
> Commit-Queue: Jonas Termansen <sortie@google.com>
Bug: b/333595242
Change-Id: I43bec2e7fcad375e972ca447aa13ae65b0ce09ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362661
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
This reverts commit b3253d9c8b.
Reason for revert: Flutter build is broken, it also has a GN variable rbi_dir, and dart_action.gni now imports two conflicting definitions.
Original change's description:
> [infra] RBE all the things.
>
> This change offloads all Dart actions during the build to RBE using
> the new rewrapper_dart.py script that understands all the different
> command line invocations of Dart programs and translates it into the
> appropriate rewrapper invocation, with a full list of input and output
> files and no absolute paths.
>
> The dart actions are all considered to be expensive RBE compilation
> steps that may be much slower on RBE than locally due to the low
> parallelism of the build during these stages and the comperatively
> slower bots in the RBE pools. The build uses the racing strategy by
> default for these compilations, such that cache hits are still used
> if available, and otherwise a local build is transparently used if it
> happens to be faster. The bots will use the remote strategy by default
> unlike developers, such that the rewrapper_dart.py script does not fail.
>
> rewrapper_dart.py contains a tiny dart import proprocessor written
> in python and a big argument parser that understands every command
> invoked during the build and the semantic meaning of every option.
>
> The absolute paths used during the Dart SDK build is not solved in
> this changelist, which just works around them initially, but these
> will be fixed in follow up changes with the appropriate teams now
> that this change proves they are not needed.
>
> Bug: b/333595242
> Change-Id: I36603ec1bf16f4ac87d56635cc1c98e8686a4028
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335827
> Reviewed-by: William Hesse <whesse@google.com>
> Commit-Queue: Jonas Termansen <sortie@google.com>
Bug: b/333595242
Change-Id: I56c9df11e672e7bd096f2a6e7d0243dccd0396c9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362385
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
This change offloads all Dart actions during the build to RBE using
the new rewrapper_dart.py script that understands all the different
command line invocations of Dart programs and translates it into the
appropriate rewrapper invocation, with a full list of input and output
files and no absolute paths.
The dart actions are all considered to be expensive RBE compilation
steps that may be much slower on RBE than locally due to the low
parallelism of the build during these stages and the comperatively
slower bots in the RBE pools. The build uses the racing strategy by
default for these compilations, such that cache hits are still used
if available, and otherwise a local build is transparently used if it
happens to be faster. The bots will use the remote strategy by default
unlike developers, such that the rewrapper_dart.py script does not fail.
rewrapper_dart.py contains a tiny dart import proprocessor written
in python and a big argument parser that understands every command
invoked during the build and the semantic meaning of every option.
The absolute paths used during the Dart SDK build is not solved in
this changelist, which just works around them initially, but these
will be fixed in follow up changes with the appropriate teams now
that this change proves they are not needed.
Bug: b/333595242
Change-Id: I36603ec1bf16f4ac87d56635cc1c98e8686a4028
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335827
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
The `dartaotruntime` is built in `product` mode even when we do a release build. This CL wires up a product flag so we always build the Dart2Wasm product snapshot when creating the SDK.
Change-Id: Ic8c6a6da180a47a19ba818d7c449f712c9e60123
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/264887
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Joshua Litt <joshualitt@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Tested: Builds on the CQ, built locally.
Change-Id: I25d0286d43199f4012a21e2223b1046a61a86bb9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/251545
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Most of these arguments are flowing through to a `--packages` argument
which allows passing the `package_config.json` file. In the long term we
should remove the `.packages` file entirely.
TEST=None, the change should have no visible impact.
Change-Id: I60a8b175d5e217a85588bbcb91cc5095514066b4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/211503
Auto-Submit: Nate Bosch <nbosch@google.com>
Commit-Queue: Nate Bosch <nbosch@google.com>
Reviewed-by: Kevin Moore <kevmoo@google.com>
Reviewed-by: Lasse R.H. Nielsen <lrn@google.com>
Several people have experienced build issues traced back to
target `//utils/dartanalyzer:generate_summary_strong`.
The issue is triggered by the sdk_hash changing with an old
kernel-service snapshot in the out/ folder.
`generate_summary_strong` uses but does not explicitly depend on
the kernel-service, causing the outdated dependency.
This then causes 'Unexpected Kernel SDK Version' when the new dart
binary tried to load the old kernel-service snapshot.
Tested:
1. Clean build of create_sdk at HEAD.
2. Empty `git commit --amend` to modify commit and thus sdk hash.
3. Rebuild of create_sdk (which failed before).
Cq-Include-Trybots: dart/try:analyzer-analysis-server-linux-try,analyzer-linux-release-try,analyzer-mac-release-try,analyzer-nnbd-linux-release-try,analyzer-win-release-try,benchmark-linux-try,dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,dart-sdk-win-try,flutter-analyze-try,front-end-linux-release-x64-try,pkg-linux-release-try,vm-kernel-linux-release-x64-try
Change-Id: Iba07e9d0c5daa7a93870fb501d9bb57682c88a5d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/156913
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
doesn't show these formatting changes as diffs.
Change-Id: I69ccbf4adabc66d88371cece785a2c1bce60f133
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138962
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Consensus seems to be that there should be different mechanism to support host-targeting-host vs host-targeting-target configuration: comparing toolchains names won't work for that.
Also, dart_host_toolchain was set up to be used by Fuchsia, but it is no longer being used.
Change-Id: Ic2e63d8cef00b18bf6866122199027459eaf32c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118910
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
- Remove all code that generates the corelib and dart:io library sources into C++ arrays and link them into dart_bootstrap
- Remove the executable dart_bootstrap and all uses of it
- Remove bootstrap_nolib.cc and builtin_nocore.cc
- Remove the Dart 1 code in core lib bootstrap path
Change-Id: Ifd33496204285a08b42fe09e39428e7a92b416b6
Reviewed-on: https://dart-review.googlesource.com/c/77241
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
#33659
Change-Id: I94fccf4bbce2f0140bf0d6efa953d44b290df856
Reviewed-on: https://dart-review.googlesource.com/62840
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Updated to pass --dfe to invocations of the
checked-in SDK.
Change-Id: I47e122422926d235f166972669c8c2cb18a89735
Reviewed-on: https://dart-review.googlesource.com/62201
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
This is still using the prebuilts from Dart's bot for building
the Observatory. Instead, we should probably build/take the
Observatory from Fuchsia's bots.
Change-Id: I466061d6f7176e2b76912d303ef0b0c518d19e1e
Reviewed-on: https://dart-review.googlesource.com/55580
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
This avoids conflicts with client repos that have a file
//build/compiled_action.gni with something different in them.
Change-Id: I133f66b0989bcb547cb8eed8710a88e2c1384bd7
Reviewed-on: https://dart-review.googlesource.com/53440
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
