Automatic set coverage when loading snapshot
TEST=Existing tests, manual inspection of reduction in sdk snapshot sizes (see data in comments in https://dart-review.googlesource.com/c/sdk/+/370501)
Change-Id: I044616144c2defeed252a6715eba1abcabffc86d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/371700
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
The agnostic mode was added to allow the platform dill embedded in
the VM to support both weak and strong mode. Since weak mode is no
longer supported in the VM, the agnostic mode can new be deleted.
All uses of the agnostic in Dart and Flutter have been removed prior
to this change.
Change-Id: Iff0f69d9cd64e887e01cd7e7d336a97761bd6d4b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/366801
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
When iterating on core library changes or changes in the AOT compiler
many seconds are wasted waiting on gen_kernel/compile_platform to
parse Dart code. This happens because we are running these tools
from sources on prebuilt Dart SDK.
This CL allows SDK developer to opt-in into AOT compiling these
tools by adding `precompile_tools=true` to their DART_GN_ARGS.
AOT compilation is performed using prebuilt SDK - so these
executables do not need to be recompiled if core libraries or
VM changes reducing iteration cycles.
pkg/vm/tool/precompiler2 is tweaked to detect when DART_GN_ARGS
contains `precompile_tools=true` and use precompiled
gen_kernel.exe instead of running it from source.
Using precompiled compile_platform takes vm_platform_strong.dill
build from 20 seconds to 3 seconds.
Using precompiled gen_kernel takes small benchmark build from
~10 seconds to 2 seconds.
This relands 5cda2a871c with fixes
for Flutter build.
TEST=manually tested
Change-Id: I552861c80c152890655e41baaf6ea3fb3b03a57e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/367961
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
This reverts commit 5cda2a871c.
Reason for revert: broke Flutter build.
Original change's description:
> [tools] Allow precompiling gen_kernel and compile_platform
>
> When iterating on core library changes or changes in the AOT compiler
> many seconds are wasted waiting on gen_kernel/compile_platform to
> parse Dart code. This happens because we are running these tools
> from sources on prebuilt Dart SDK.
>
> This CL allows SDK developer to opt-in into AOT compiling these
> tools by adding `precompile_tools=true` to their DART_GN_ARGS.
>
> AOT compilation is performed using prebuilt SDK - so these
> executables do not need to be recompiled if core libraries or
> VM changes reducing iteration cycles.
>
> pkg/vm/tool/precompiler2 is tweaked to detect when DART_GN_ARGS
> contains `precompile_tools=true` and use precompiled
> gen_kernel.exe instead of running it from source.
>
> Using precompiled compile_platform takes vm_platform_strong.dill
> build from 20 seconds to 3 seconds.
>
> Using precompiled gen_kernel takes small benchmark build from
> ~10 seconds to 2 seconds.
>
> TEST=manually tested
>
> Change-Id: Ieec6ad4e1081023d140eb744f0a3cd0c754414ca
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/367940
> Commit-Queue: Slava Egorov <vegorov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Change-Id: Id3e4eb44d33516f31c165d9a1e55911e8d356e7f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/367960
Commit-Queue: Slava Egorov <vegorov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
When iterating on core library changes or changes in the AOT compiler
many seconds are wasted waiting on gen_kernel/compile_platform to
parse Dart code. This happens because we are running these tools
from sources on prebuilt Dart SDK.
This CL allows SDK developer to opt-in into AOT compiling these
tools by adding `precompile_tools=true` to their DART_GN_ARGS.
AOT compilation is performed using prebuilt SDK - so these
executables do not need to be recompiled if core libraries or
VM changes reducing iteration cycles.
pkg/vm/tool/precompiler2 is tweaked to detect when DART_GN_ARGS
contains `precompile_tools=true` and use precompiled
gen_kernel.exe instead of running it from source.
Using precompiled compile_platform takes vm_platform_strong.dill
build from 20 seconds to 3 seconds.
Using precompiled gen_kernel takes small benchmark build from
~10 seconds to 2 seconds.
TEST=manually tested
Change-Id: Ieec6ad4e1081023d140eb744f0a3cd0c754414ca
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/367940
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This is a reland of commit b3253d9c8b
Fix the gn rules so the RBE offloading only happens in the Dart build
and the conflicting RBE configuration is not included otherwise. This
ensures this change can roll into Flutter while we design a better
solution to compose the projects' RBE usages.
Original change's description:
> [infra] RBE all the things.
>
> This change offloads all Dart actions during the build to RBE using
> the new rewrapper_dart.py script that understands all the different
> command line invocations of Dart programs and translates it into the
> appropriate rewrapper invocation, with a full list of input and output
> files and no absolute paths.
>
> The dart actions are all considered to be expensive RBE compilation
> steps that may be much slower on RBE than locally due to the low
> parallelism of the build during these stages and the comperatively
> slower bots in the RBE pools. The build uses the racing strategy by
> default for these compilations, such that cache hits are still used
> if available, and otherwise a local build is transparently used if it
> happens to be faster. The bots will use the remote strategy by default
> unlike developers, such that the rewrapper_dart.py script does not fail.
>
> rewrapper_dart.py contains a tiny dart import proprocessor written
> in python and a big argument parser that understands every command
> invoked during the build and the semantic meaning of every option.
>
> The absolute paths used during the Dart SDK build is not solved in
> this changelist, which just works around them initially, but these
> will be fixed in follow up changes with the appropriate teams now
> that this change proves they are not needed.
>
> Bug: b/333595242
> Change-Id: I36603ec1bf16f4ac87d56635cc1c98e8686a4028
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335827
> Reviewed-by: William Hesse <whesse@google.com>
> Commit-Queue: Jonas Termansen <sortie@google.com>
Bug: b/333595242
Change-Id: I43bec2e7fcad375e972ca447aa13ae65b0ce09ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362661
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
This reverts commit b3253d9c8b.
Reason for revert: Flutter build is broken, it also has a GN variable rbi_dir, and dart_action.gni now imports two conflicting definitions.
Original change's description:
> [infra] RBE all the things.
>
> This change offloads all Dart actions during the build to RBE using
> the new rewrapper_dart.py script that understands all the different
> command line invocations of Dart programs and translates it into the
> appropriate rewrapper invocation, with a full list of input and output
> files and no absolute paths.
>
> The dart actions are all considered to be expensive RBE compilation
> steps that may be much slower on RBE than locally due to the low
> parallelism of the build during these stages and the comperatively
> slower bots in the RBE pools. The build uses the racing strategy by
> default for these compilations, such that cache hits are still used
> if available, and otherwise a local build is transparently used if it
> happens to be faster. The bots will use the remote strategy by default
> unlike developers, such that the rewrapper_dart.py script does not fail.
>
> rewrapper_dart.py contains a tiny dart import proprocessor written
> in python and a big argument parser that understands every command
> invoked during the build and the semantic meaning of every option.
>
> The absolute paths used during the Dart SDK build is not solved in
> this changelist, which just works around them initially, but these
> will be fixed in follow up changes with the appropriate teams now
> that this change proves they are not needed.
>
> Bug: b/333595242
> Change-Id: I36603ec1bf16f4ac87d56635cc1c98e8686a4028
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335827
> Reviewed-by: William Hesse <whesse@google.com>
> Commit-Queue: Jonas Termansen <sortie@google.com>
Bug: b/333595242
Change-Id: I56c9df11e672e7bd096f2a6e7d0243dccd0396c9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362385
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
This change offloads all Dart actions during the build to RBE using
the new rewrapper_dart.py script that understands all the different
command line invocations of Dart programs and translates it into the
appropriate rewrapper invocation, with a full list of input and output
files and no absolute paths.
The dart actions are all considered to be expensive RBE compilation
steps that may be much slower on RBE than locally due to the low
parallelism of the build during these stages and the comperatively
slower bots in the RBE pools. The build uses the racing strategy by
default for these compilations, such that cache hits are still used
if available, and otherwise a local build is transparently used if it
happens to be faster. The bots will use the remote strategy by default
unlike developers, such that the rewrapper_dart.py script does not fail.
rewrapper_dart.py contains a tiny dart import proprocessor written
in python and a big argument parser that understands every command
invoked during the build and the semantic meaning of every option.
The absolute paths used during the Dart SDK build is not solved in
this changelist, which just works around them initially, but these
will be fixed in follow up changes with the appropriate teams now
that this change proves they are not needed.
Bug: b/333595242
Change-Id: I36603ec1bf16f4ac87d56635cc1c98e8686a4028
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335827
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
Bug: b/296994239
Change-Id: Ic96571d2c16d7c252cb41673e8bdb16cdb4de754
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/360507
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Note that Chromium's copy of zlib now has a dependency on the cpu_features library.
Change-Id: I0f2c306aa7eb0e1deb8fa4c45e4a04f1fee8bb83
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359600
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This change may require a codereview from flutter side before moving
forward. See https://github.com/flutter/engine/pull/51072.
This reverts commit 283051acba.
Reason for revert: Redo the change with the similar one in flutter.
After this change, the next dart -> flutter roll needs extra cares
in https://github.com/flutter/engine/pull/51072. It also contains
more details about the reason why it could not be two-way
compatible.
Original change's description:
> Revert "[Fuchsia] Remove the legacy gn build rules for fuchsia"
>
> This reverts commit 094202bb91.
>
> Reason for revert: Break flutter, this change should happen after
> flutter being migrated to the same gn-sdk.
>
> Original change's description:
> > [Fuchsia] Remove the legacy gn build rules for fuchsia
> >
> > The updated test-scripts contains https://crrev.com/c/5341620 which
> > allows to generate fidl apis with an env var controlled location
> > rather than a hard-coded one. So the existing fuchsia gn build rules
> > in dart/sdk become obsolete and can be fully removed in favor of the
> > one in //third_party/fuchsia/gn-sdk/.
> >
> > Meanwhile the gn-sdk has been updated with
> > https://crrev.com/c/5325282 to use api-level version'ed idk but not
> > the ones in arch/.
> >
> > Cq-Include-Trybots: luci.dart.try:vm-fuchsia-release-arm64-try,vm-fuchsia-release-x64-try
> > Tested: ^^^^^
> > Bug: 40935282
> > Change-Id: I2ce958e6db1ff8221beef7b7ff953c32bb4525ba
> > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/355283
> > Reviewed-by: Ryan Macnak <rmacnak@google.com>
> > Reviewed-by: Alexander Thomas <athom@google.com>
> > Commit-Queue: Zijie He <zijiehe@google.com>
>
> Bug: 40935282
> Change-Id: I7c455d1d362210523671c97d99ef018ede1743f4
> Cq-Include-Trybots: luci.dart.try:vm-fuchsia-release-arm64-try,vm-fuchsia-release-x64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/356307
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Auto-Submit: Zijie He <zijiehe@google.com>
> Reviewed-by: Derek Xu <derekx@google.com>
> Commit-Queue: Zijie He <zijiehe@google.com>
Bug: 40935282
Change-Id: Id5a9a98013350359037b19f8506f24158c9a3120
Cq-Include-Trybots: luci.dart.try:vm-fuchsia-release-arm64-try,vm-fuchsia-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/356924
Commit-Queue: Zijie He <zijiehe@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Part of https://github.com/flutter/flutter/issues/144430
This is a reland of https://dart-review.googlesource.com/c/sdk/+/356481 with the addition of dependencies to prevent races on directory creation.
Previously, this code did backflips to accomplish two goals
1. Collect the paths of all files that would be copied from one place to another after applying regex filters so that they could be listed as the "inputs" to a GN action.
2. Arrange so that the python script doing the above would only be invoked once during GN to reduce the cost of calling out to python.
However, this is exactly the use-case for the "depfile" parameter to a GN action. Instead of running the script during GN to populate the "inputs" list, when we run the copy_tree.py script, we can instead ask it to generate a depfile to collect all the input files that were actually copied after applying the regex filter. Using that, we don't have to run the python script at all during GN.
TEST=it builds
Change-Id: I593e2500544a9fff5dd9852d0d3370f97aafc464
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/356620
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
This reverts commit f60d32167f.
Reason for revert: build failures in post submit
Original change's description:
> [gn] Deprecate copy_trees()
>
> Part of https://github.com/flutter/flutter/issues/144430
>
> Previously, this code did backflips to accomplish two goals
> 1. Collect the paths of all files that would be copied from one place to another after applying regex filters so that they could be listed as the "inputs" to a GN action.
> 2. Arrange so that the python script doing the above would only be invoked once during GN to reduce the cost of calling out to python.
>
> However, this is exactly the use-case for the "depfile" parameter to a GN action. Instead of running the script during GN to populate the "inputs" list, when we run the copy_tree.py script, we can instead ask it to generate a depfile to collect all the input files that were actually copied after applying the regex filter. Using that, we don't have to run the python script at all during GN.
>
> TEST=it builds
>
> Change-Id: I41a251ce4659119cdc3997bb2d6fc7ee0831bb6d
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/356481
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Zach Anderson <zra@google.com>
> Reviewed-by: Alexander Aprelev <aam@google.com>
Change-Id: Ie4409ed93a3c990fbce375dba1f2d9a3c735ea3f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/356308
Auto-Submit: Zach Anderson <zra@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Zach Anderson <zra@google.com>
This reverts commit 094202bb91.
Reason for revert: Break flutter, this change should happen after
flutter being migrated to the same gn-sdk.
Original change's description:
> [Fuchsia] Remove the legacy gn build rules for fuchsia
>
> The updated test-scripts contains https://crrev.com/c/5341620 which
> allows to generate fidl apis with an env var controlled location
> rather than a hard-coded one. So the existing fuchsia gn build rules
> in dart/sdk become obsolete and can be fully removed in favor of the
> one in //third_party/fuchsia/gn-sdk/.
>
> Meanwhile the gn-sdk has been updated with
> https://crrev.com/c/5325282 to use api-level version'ed idk but not
> the ones in arch/.
>
> Cq-Include-Trybots: luci.dart.try:vm-fuchsia-release-arm64-try,vm-fuchsia-release-x64-try
> Tested: ^^^^^
> Bug: 40935282
> Change-Id: I2ce958e6db1ff8221beef7b7ff953c32bb4525ba
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/355283
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Alexander Thomas <athom@google.com>
> Commit-Queue: Zijie He <zijiehe@google.com>
Bug: 40935282
Change-Id: I7c455d1d362210523671c97d99ef018ede1743f4
Cq-Include-Trybots: luci.dart.try:vm-fuchsia-release-arm64-try,vm-fuchsia-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/356307
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Zijie He <zijiehe@google.com>
Reviewed-by: Derek Xu <derekx@google.com>
Commit-Queue: Zijie He <zijiehe@google.com>
Part of https://github.com/flutter/flutter/issues/144430
Previously, this code did backflips to accomplish two goals
1. Collect the paths of all files that would be copied from one place to another after applying regex filters so that they could be listed as the "inputs" to a GN action.
2. Arrange so that the python script doing the above would only be invoked once during GN to reduce the cost of calling out to python.
However, this is exactly the use-case for the "depfile" parameter to a GN action. Instead of running the script during GN to populate the "inputs" list, when we run the copy_tree.py script, we can instead ask it to generate a depfile to collect all the input files that were actually copied after applying the regex filter. Using that, we don't have to run the python script at all during GN.
TEST=it builds
Change-Id: I41a251ce4659119cdc3997bb2d6fc7ee0831bb6d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/356481
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
The updated test-scripts contains https://crrev.com/c/5341620 which
allows to generate fidl apis with an env var controlled location
rather than a hard-coded one. So the existing fuchsia gn build rules
in dart/sdk become obsolete and can be fully removed in favor of the
one in //third_party/fuchsia/gn-sdk/.
Meanwhile the gn-sdk has been updated with
https://crrev.com/c/5325282 to use api-level version'ed idk but not
the ones in arch/.
Cq-Include-Trybots: luci.dart.try:vm-fuchsia-release-arm64-try,vm-fuchsia-release-x64-try
Tested: ^^^^^
Bug: 40935282
Change-Id: I2ce958e6db1ff8221beef7b7ff953c32bb4525ba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/355283
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Zijie He <zijiehe@google.com>
Automatically detect the appropriate configuration file if RBE=1 is
set to request RBE on all projects, or if DART_RBE=1 is set to request
RBE for Dart only, or otherwise respect the explicit configuration file.
Automatically set the server_address location to the build directory
if it has not already been set. This is unfortunately not supported on
Windows due to the environment variable being set during build but it
needed to be set during gn.
Retain the older configuration files during the transitory period.
Bug: b/296994239
Fixes: b/320876546
Change-Id: I62d1fbfed35248477731cceda3f7267c605c4969
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/355400
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Place the windows toolchain in the build root so it can be sent to RBE.
Use relative paths in debug symbols when building with Windows clang,
which is needed for remotely building with RBE.
Set the executable bit when invoking clang-cl remotely as it has not
been set on Windows RBE clients.
Stop including .rst files during build commands as @options-from-file
is not supported by rewrapper at this time.
Forward RBE_cfg into the msvc build environment and rewrite the
INCLUDE, LIB, and LIBPATH environment variables to use relative paths
instead of absolute paths.
Don't download reclient on windows-arm64 where there is no cipd
package and the windows-arm64 python lies and says it's x64 because
they actually packaged a x64 python.
Tune the Windows -j performance to match the historic bot performance
as most casual Windows machines won't handle -j 1000 well.
Bug: b/296994239
Change-Id: I2bd1b4478fca796f0a326c062a5f478f16c2c6ef
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/343440
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
ffx will change the way product lookup prints out the download url,
but the original solution is less ideal and using json is definitely
better.
So using the latest test-scripts would be a must-have before updating
the fuchsia-sdk.
A drive-by change is to return proc.returncode from with_envs.py.
Change-Id: Ic4589597006226aa7f5b0e557fe1f084ad3a0a74
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/345567
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Zijie He <zijiehe@google.com>
Bug: b/296994239
Change-Id: Ie8630bcece5c2f62c5693afe654fb9d6367b33e4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/345286
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
The RBE build happens in a different working directory when run
remotely and the -fdebug-prefix-map option with a local path inhibits
cache sharing between tryjobs and developers. This change fixes the
RBE object files so they don't contain the remote absolute path and
also ensures the build commands can be cache shared.
Bug: b/296994239
Change-Id: I6d2cd42fd775724ccb5c335c85b5f31ad279624f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/342342
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Bug: b/296994239
Change-Id: Ice6ed1bda597e300600d6d6823901ec21f5cfcd2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/342420
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
The sysroot needs to be passed as a relative path inside the exec root
since RBE will fail to compile with an absolute path.
Bug: b/296994239
Change-Id: Idc0e155368daaf904ac4650bd8668b8955d09e28
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/338002
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
This change migrates the Dart linux builders from Goma to RBE via
reclient. Goma is being shut down by the end of the year and we must
migrate to the new replacement, which is much more powerful and can do
things we could never do with Goma. We should already see some mild
build time improvements from simply switching.
This change can be safely reverted if there are any problems:
- If you must revert this change, please reach out with what you
observed, but please first read and try the below instructions.
- Cross-compilation with RBE is not implemented at this time and certain
special builders running on Linux intel may not support RBE. The
existing configuration support to disable Goma remains in place. The
existence of configuration file makes the recipe switch all Linux x64
bots use RBE rather than Goma. Try disabling Goma on any builders that
are failing, whether using `--no-goma` in the test matrix or using
`goma = False` on infra/config. Goma is going away soon and we will
soon bring RBE up to feature parity and Goma isn't really working on
the special cross builds anyway.
- Rare flakes in the remote builds have previously been observed, but I
haven't seen them reappear yet after upgrading reclient. Look for
build log messages mentioning the remote execution failed, or that an
upload blob size digest mysteriously changed. Please alert me if you
see this and I will talk to the Foundry team. If this happens, please
try if you can fix this using `exec_strategy=remote_local_fallback` in
the configuration file. Right now reclient only does remote executions
to catch cases where RBE doesn't work, but if it fails, it can be
configured to silently fall back on a slow local build.
- If RBE is used on a configuration that doesn't support RBE builds, the
default -j 1000 may overload the bot if it falls back on a local
build. If this happens, the bot may explode and fail with BOT_DIED. If
this happens, disable Goma (RBE) on the builder as above and let me
know about the problem.
- We haven't tried sending a full production load to the RBE pool yet.
Flutter is already using the RBE pool partially. It's unclear what
will happen once we turn this on for everyone. Keep an eye out for
anything unusual and let me know.
- If it turns out RBE doesn't cache invalidate properly, it is possible
outdated object files will be linked. If anything really weird
happens, this may the case. As far as I tested, the caching is working
correctly.
RBE is yet to be officially rolled out to our developers. Stay tuned.
Bug: b/296994239
Change-Id: I77e0025197ab98a672efbc0cb04c8ece528c3e29
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335826
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
This is a no-op since RBE is not enabled anywhere yet.
Bug: b/296994239
Change-Id: I2d39419736fca0382a6082f6be64e787348bb825
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/336440
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
build/fuchsia/gn-sdk is a copy of the current GN SDK with paths adjusted.
The rest of build/fuchsia comes from Flutter.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/38752
Change-Id: I192cca955d05991bde3931d053df9835c5a2195d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/334142
Reviewed-by: Zijie He <zijiehe@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
This can be controlled via codesigning_identity GN arg.
For example, setting codesigning_identity="-"
would produce ad-hoc signed binaries.
This CL also includes changes in vm/cc tests which are needed
for tests to be green when running with hardened runtime.
Issue https://github.com/dart-lang/sdk/issues/53928
Tested: enabled ad-hoc signing and tested on bots.
Cq-Include-Trybots: luci.dart.try:vm-mac-debug-arm64-try,vm-mac-release-arm64-try,vm-mac-release-x64-try,vm-mac-debug-x64-try
Change-Id: I3c3a6265c62b2904e43a326b7d8223bcfd393577
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333401
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
- Adds RV64 support
- Toolchain and sysroot will come from the same place
Change-Id: Ic835de7667a334dc450e89b979bf19950f77efc2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331746
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Build one package with three elf_test_runner components: one for each of the binaries that the test harness invokes on the target. (Though currently only tests that use "dart" are setup.)
Create includable CML shards for the capabilites required by the AOT and JIT VMs.
Fold test_runner's fuchsia_cfv2.dart back into fuchsia.dart.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/38752
Change-Id: I7bd8d43d184cbcb11903c7aed77ce31fb30cb894
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332451
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Zijie He <zijiehe@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Bug: b/296994239
Change-Id: I179cc7729846885ee952d26082a1e615a46a30b9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331923
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
This reverts commit 9a11fe517c.
Reason for revert: broke the build for some users.
Original change's description:
> [infra] Begin RBE Linux x64 support.
>
> Bug: b/296994239
> Change-Id: I0ddf9a1beb2996ac67b59779b8b8113432434786
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331282
> Reviewed-by: William Hesse <whesse@google.com>
> Commit-Queue: Jonas Termansen <sortie@google.com>
> Reviewed-by: Alexander Thomas <athom@google.com>
Bug: b/296994239
Change-Id: Id70259b9357790581f11f105516ff0c6c63ab1ac
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331780
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Bug: b/296994239
Change-Id: I0ddf9a1beb2996ac67b59779b8b8113432434786
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331282
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
