Commit graph

154 commits

Author SHA1 Message Date
Jonas Termansen e0b08fadd8 Reland "[infra] RBE all the things."
This is a reland of commit b3253d9c8b

Fix the gn rules so the RBE offloading only happens in the Dart build
and the conflicting RBE configuration is not included otherwise. This
ensures this change can roll into Flutter while we design a better
solution to compose the projects' RBE usages.

Original change's description:
> [infra] RBE all the things.
>
> This change offloads all Dart actions during the build to RBE using
> the new rewrapper_dart.py script that understands all the different
> command line invocations of Dart programs and translates it into the
> appropriate rewrapper invocation, with a full list of input and output
> files and no absolute paths.
>
> The dart actions are all considered to be expensive RBE compilation
> steps that may be much slower on RBE than locally due to the low
> parallelism of the build during these stages and the comperatively
> slower bots in the RBE pools. The build uses the racing strategy by
> default for these compilations, such that cache hits are still used
> if available, and otherwise a local build is transparently used if it
> happens to be faster. The bots will use the remote strategy by default
> unlike developers, such that the rewrapper_dart.py script does not fail.
>
> rewrapper_dart.py contains a tiny dart import proprocessor written
> in python and a big argument parser that understands every command
> invoked during the build and the semantic meaning of every option.
>
> The absolute paths used during the Dart SDK build is not solved in
> this changelist, which just works around them initially, but these
> will be fixed in follow up changes with the appropriate teams now
> that this change proves they are not needed.
>
> Bug: b/333595242
> Change-Id: I36603ec1bf16f4ac87d56635cc1c98e8686a4028
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335827
> Reviewed-by: William Hesse <whesse@google.com>
> Commit-Queue: Jonas Termansen <sortie@google.com>

Bug: b/333595242
Change-Id: I43bec2e7fcad375e972ca447aa13ae65b0ce09ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362661
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
2024-04-12 22:06:32 +00:00
William Hesse 1b249f5496 Revert "[infra] RBE all the things."
This reverts commit b3253d9c8b.

Reason for revert: Flutter build is broken, it also has a GN variable rbi_dir, and dart_action.gni now imports two conflicting definitions.

Original change's description:
> [infra] RBE all the things.
>
> This change offloads all Dart actions during the build to RBE using
> the new rewrapper_dart.py script that understands all the different
> command line invocations of Dart programs and translates it into the
> appropriate rewrapper invocation, with a full list of input and output
> files and no absolute paths.
>
> The dart actions are all considered to be expensive RBE compilation
> steps that may be much slower on RBE than locally due to the low
> parallelism of the build during these stages and the comperatively
> slower bots in the RBE pools. The build uses the racing strategy by
> default for these compilations, such that cache hits are still used
> if available, and otherwise a local build is transparently used if it
> happens to be faster. The bots will use the remote strategy by default
> unlike developers, such that the rewrapper_dart.py script does not fail.
>
> rewrapper_dart.py contains a tiny dart import proprocessor written
> in python and a big argument parser that understands every command
> invoked during the build and the semantic meaning of every option.
>
> The absolute paths used during the Dart SDK build is not solved in
> this changelist, which just works around them initially, but these
> will be fixed in follow up changes with the appropriate teams now
> that this change proves they are not needed.
>
> Bug: b/333595242
> Change-Id: I36603ec1bf16f4ac87d56635cc1c98e8686a4028
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335827
> Reviewed-by: William Hesse <whesse@google.com>
> Commit-Queue: Jonas Termansen <sortie@google.com>

Bug: b/333595242
Change-Id: I56c9df11e672e7bd096f2a6e7d0243dccd0396c9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362385
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2024-04-12 10:08:28 +00:00
Jonas Termansen b3253d9c8b [infra] RBE all the things.
This change offloads all Dart actions during the build to RBE using
the new rewrapper_dart.py script that understands all the different
command line invocations of Dart programs and translates it into the
appropriate rewrapper invocation, with a full list of input and output
files and no absolute paths.

The dart actions are all considered to be expensive RBE compilation
steps that may be much slower on RBE than locally due to the low
parallelism of the build during these stages and the comperatively
slower bots in the RBE pools. The build uses the racing strategy by
default for these compilations, such that cache hits are still used
if available, and otherwise a local build is transparently used if it
happens to be faster. The bots will use the remote strategy by default
unlike developers, such that the rewrapper_dart.py script does not fail.

rewrapper_dart.py contains a tiny dart import proprocessor written
in python and a big argument parser that understands every command
invoked during the build and the semantic meaning of every option.

The absolute paths used during the Dart SDK build is not solved in
this changelist, which just works around them initially, but these
will be fixed in follow up changes with the appropriate teams now
that this change proves they are not needed.

Bug: b/333595242
Change-Id: I36603ec1bf16f4ac87d56635cc1c98e8686a4028
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335827
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
2024-04-11 23:02:16 +00:00
Jonas Termansen 144724e085 [infra] Simplify RBE startup/shutdown logic.
Change-Id: I45d99a1386bfb0d3af94a563cf5626d9bc7b4f68
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/360743
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2024-04-03 09:28:30 +00:00
Jonas Termansen e445389862 [infra] Remove Goma support.
Bug: b/296994239
Change-Id: Ic96571d2c16d7c252cb41673e8bdb16cdb4de754
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/360507
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2024-04-02 22:54:19 +00:00
Jonas Termansen 05936f4d0b [infra] Fail hard if starting RBE fails.
The build isn't going to go through anyway and the slow 3 minute default
timeout is just confusing and annoying when this happens. There isn't
any additional useful context being output like the comment suggested.

Bug: b/331346866
Change-Id: I0475002469c372fefe3a54d104b54692c89ca9f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359800
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2024-03-26 11:10:48 +00:00
Jonas Termansen c0e089af45 [infra] Fix environment during RBE shutdown.
Bug: b/296994239
Change-Id: Id7545ef44a28ebb3c1439c1f980fa95978be1412
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/355780
Reviewed-by: William Hesse <whesse@google.com>
2024-03-05 13:47:17 +00:00
Jonas Termansen a27993d7c9 [infra] Fix not initializing RBE before gn.
Bug: b/296994239
Change-Id: I344f07fa7f389fa5b0f5e65b24168f2d05daacbf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/355683
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
2024-03-05 13:22:39 +00:00
Jonas Termansen 3f1d6a99e7 [infra] Simplify RBE configuration and allow concurrent builds.
Automatically detect the appropriate configuration file if RBE=1 is
set to request RBE on all projects, or if DART_RBE=1 is set to request
RBE for Dart only, or otherwise respect the explicit configuration file.

Automatically set the server_address location to the build directory
if it has not already been set. This is unfortunately not supported on
Windows due to the environment variable being set during build but it
needed to be set during gn.

Retain the older configuration files during the transitory period.

Bug: b/296994239
Fixes: b/320876546
Change-Id: I62d1fbfed35248477731cceda3f7267c605c4969
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/355400
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2024-03-04 14:54:26 +00:00
Jonas Termansen a19d3f3375 [infra] Support testing with RBE on Windows.
Place the windows toolchain in the build root so it can be sent to RBE.

Use relative paths in debug symbols when building with Windows clang,
which is needed for remotely building with RBE.

Set the executable bit when invoking clang-cl remotely as it has not
been set on Windows RBE clients.

Stop including .rst files during build commands as @options-from-file
is not supported by rewrapper at this time.

Forward RBE_cfg into the msvc build environment and rewrite the
INCLUDE, LIB, and LIBPATH environment variables to use relative paths
instead of absolute paths.

Don't download reclient on windows-arm64 where there is no cipd
package and the windows-arm64 python lies and says it's x64 because
they actually packaged a x64 python.

Tune the Windows -j performance to match the historic bot performance
as most casual Windows machines won't handle -j 1000 well.

Bug: b/296994239
Change-Id: I2bd1b4478fca796f0a326c062a5f478f16c2c6ef
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/343440
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
2024-02-29 10:33:09 +00:00
Jonas Termansen f4f557345c [infra] Reland "Begin RBE Linux x64 support."
Bug: b/296994239
Change-Id: I179cc7729846885ee952d26082a1e615a46a30b9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331923
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2023-10-24 14:36:19 +00:00
Alexander Thomas d37620ed03 Revert "[infra] Begin RBE Linux x64 support."
This reverts commit 9a11fe517c.

Reason for revert: broke the build for some users.

Original change's description:
> [infra] Begin RBE Linux x64 support.
>
> Bug: b/296994239
> Change-Id: I0ddf9a1beb2996ac67b59779b8b8113432434786
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331282
> Reviewed-by: William Hesse <whesse@google.com>
> Commit-Queue: Jonas Termansen <sortie@google.com>
> Reviewed-by: Alexander Thomas <athom@google.com>

Bug: b/296994239
Change-Id: Id70259b9357790581f11f105516ff0c6c63ab1ac
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331780
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
2023-10-23 16:45:47 +00:00
Jonas Termansen 9a11fe517c [infra] Begin RBE Linux x64 support.
Bug: b/296994239
Change-Id: I0ddf9a1beb2996ac67b59779b8b8113432434786
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331282
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2023-10-23 14:42:59 +00:00
Vyacheslav Egorov bd2452c7e3 [infra] Workaround Python binary pecularities on Mac OS X
For some reason `python3` binary shipped with Mac OS X
sets LIBRARY_PATH, SDKROOT and CPATH environment variables.

This affects clang if it is spawned in such a modified environment,
because it picks and uses those values.

This CL simply removes these variables from the environment
if they are present.

See also https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4149962

Fixes https://github.com/dart-lang/sdk/issues/52411

R=kustermann@google.com

Change-Id: Id71ece4c3232c3e9ad19dc07d7546d1012d6330f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306540
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2023-05-31 08:51:22 +00:00
Ryan Macnak 60e76a406b [build] Pass targets from build.py to ninja unchanged.
Passing the empty set of targets to build.py now passes the empty set to ninja instead of substituting "all". Because ninja's default target is "default" instead of "all", this changes the meaning of `build.py` to `build.py default` instead of `build.py all`.

Bug: https://github.com/dart-lang/sdk/issues/52442
Change-Id: Ia82b7bdfc54a5dafdfe7a6426e5620ad7532dd87
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/305061
Reviewed-by: Derek Xu <derekx@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-05-23 20:34:00 +00:00
Alexander Thomas e87f2d895c [build] Switch builds to the ninja in DEPS
Note: this "upgrades" ninja to 1.11.1.

Change-Id: Idca0f8a2a67cf5d5dbe75661bb14de174012580f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/261101
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2022-09-27 11:24:03 +00:00
Ryan Macnak 26f50368d2 [build] Check that a second Ninja invocation does nothing.
Not checking Windows, where the set of linker outputs is not constant.

Change-Id: I1241aa4108f7feebc2638ca762743464fcb48a52
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/201165
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-05-26 17:48:30 +00:00
Alexander Thomas b5c63ce757 [infra] Migrate scripts to python3
* Migrate to python3; drop python support.
* Update Windows toolchain support.
* Remove some unused methods.
* Python 2.7 is still needed on Windows.
* Update gsutil to a version that supports python3.

Fixes: https://github.com/dart-lang/sdk/issues/28793

TEST=Manually tested common user journeys.

Change-Id: I663a22b237a548bb82dc2e601e399e3bc3649211
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192182
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-04-15 10:10:20 +00:00
eggfly 318babd5ac [tools] Fix typo in build.py
Closes https://github.com/dart-lang/sdk/pull/44580
https://github.com/dart-lang/sdk/pull/44580

GitOrigin-RevId: 43ebbcec6ee358d04f19bdd2850b2aa8f9718ad5
Change-Id: Ia10816097d43f9ee1062d846326a680767033165
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177520
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2021-01-11 09:58:36 +00:00
William Hesse ef07751c43 [infra] Fix build.py and gn.py to exit on SIGINT
It is very tricky to make scripts using the python multiprocessing
package terminate cleanly on SIGINT (ctrl-c).
Use subprocesses run in parallel instead for the simple uses
in these scripts.

This is necessary when running build.py from within another script,
so that it terminates cleanly when the process group receives
a broadcast SIGINT.

Change-Id: Ifc858225a49f369c4bd9cee62639ac6fc8ff737e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/171120
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2020-11-13 12:35:09 +00:00
Martin Kustermann 89fe12b12a [infra] Always re-run GN in build.py
The workflow of using tools/gn.py followed-by tools/build.py is no
longer working as it used to (the latter will override whatever the
former configured).

Instead one should pick between:

  tools/gn.py & ninja
  tools/build.py

=> Both tools/gn.py and tools/build.py suport the same options.

This CL also makes tools/build.py stop using tools/generate_buildfiles.py (which
would get just overriden by the re-run of GN).

Change-Id: Ie698d7395e8c5862ae04a479c7c820c76ac5565d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154323
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2020-07-15 09:01:28 +00:00
Martin Kustermann 2a11b9b426 [infra] Add using_goma=False (fixes recent tools/build.py change)
Change-Id: I9edf8c4451c3a14597aa0927f5c63e2e62bb5c49
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154326
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2020-07-14 15:28:20 +00:00
Martin Kustermann be9b69553d [infra] Prepare to make tools/build.py always re-runs GN.
Also adds a '-nvh' short-hand for '--no-verify-sdk-hash'

Change-Id: If893953e592128a8fa0b2f2e5720748d4b9b76aa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153770
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2020-07-14 14:10:11 +00:00
Liam Appelbe 764e72800f Reland "Revert "Add an --os=fuchsia option to build.py:""
This reverts commit d39d4d0d55.

Reason for revert: Relanding with fix

Change-Id: I6d6911b6cdc578a3b576b237db94c7f1db9824b9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151600
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
2020-06-18 00:24:31 +00:00
Leaf Petersen d44457f79d [Core Libraries] Eliminate the fork in the core libraries.
Move the nnbd core libraries from sdk_nnbd to sdk, and updates
references in build files and elsewhere accordingly.

Change-Id: I09760fe1e006657aacdfe80f3b22fdf6f7e30a9f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151121
Commit-Queue: Leaf Petersen <leafp@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
2020-06-16 23:37:36 +00:00
George Wright d39d4d0d55 Revert "Add an --os=fuchsia option to build.py:"
This reverts commit 4ce18ab931.

Reason for revert: bad merge, caused build breakages in flutter/engine

Original change's description:
> Add an --os=fuchsia option to build.py:
> 
> tools/build.py --os=fuchsia runtime create_sdk
> 
> This is analogous to --os=android. It cross compiles from Linux x64 to
> Fuchsia.
> 
> A lot of the build rules are just slightly different between the existing
> Fuchsia build rules used by Flutter, and the ones added by GN SDK. For
> example "$fuchsia_sdk_root/pkg:fdio" is now "$fuchsia_sdk_root/pkg/fdio".
> So to support this I had to add a new variable, using_fuchsia_gn_sdk,
> analogous to using_fuchsia_sdk. Flutter will need to set this to false.
> 
> Change-Id: Ief275d65f30a42a801607de93cf2d27a1fe825dd
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150689
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Liam Appelbe <liama@google.com>

TBR=rmacnak@google.com,asiva@google.com,liama@google.com,kaushikiska@google.com,matthewcarroll@google.com

Change-Id: If01ee34eba906c55f2d56ba650748a86e81e701c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151321
Auto-Submit: George Wright <wrightgeorge@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2020-06-15 23:01:53 +00:00
Liam Appelbe 4ce18ab931 Add an --os=fuchsia option to build.py:
tools/build.py --os=fuchsia runtime create_sdk

This is analogous to --os=android. It cross compiles from Linux x64 to
Fuchsia.

A lot of the build rules are just slightly different between the existing
Fuchsia build rules used by Flutter, and the ones added by GN SDK. For
example "$fuchsia_sdk_root/pkg:fdio" is now "$fuchsia_sdk_root/pkg/fdio".
So to support this I had to add a new variable, using_fuchsia_gn_sdk,
analogous to using_fuchsia_sdk. Flutter will need to set this to false.

Change-Id: Ief275d65f30a42a801607de93cf2d27a1fe825dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150689
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
2020-06-15 20:05:16 +00:00
Ryan Macnak cb46dbb237 [infra] Recognize the exit code for UBSAN failures.
This seems to only affect the bots and not local runs of test.py.

Bug: https://github.com/dart-lang/sdk/issues/39427
Change-Id: I3d80fc662a29585416a61f454b21f977853c0e71
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147621
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: William Hesse <whesse@google.com>
2020-05-12 09:03:59 +00:00
asiva 03ce33f80c [BUILD] - Initial CL to unfork the NNBD Dart SDK
- Flips the flag from --nnbd to --no-nnbd so that by default it builds
  the NNBD version
- using the --no-nnbd flag results in the SDK being built in a
  directory which has the 'Legacy' suffix added to it
  (e.g: out/DebugX64Legacy)
- the '--enable-experiment=non-nullable' flag still needs to be passed
  in during execution so that CFE runs in that mode. This is different
  from the 'null_safety' flag

Change-Id: I729630a7bd36ea7ae38134f9285b44e93c283c68
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138902
Reviewed-by: Paul Berry <paulberry@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2020-05-07 21:12:47 +00:00
Siva Annamalai 726d3c7725 Revert "[BUILD] - Initial CL to unfork the NNBD Dart SDK"
This reverts commit cab69e7706.

Reason for revert: It was a temporary submit

Original change's description:
> [BUILD] - Initial CL to unfork the NNBD Dart SDK
> NOTE: This is a trial submit and will be reverted immediately.
> 
> - Flips the flag from --nnbd to --no-nnbd so that by default it builds
>   the NNBD version
> - using the --no-nnbd flag results in the SDK being built in a
>   directory which has the 'Legacy' suffix added to it
>   (e.g: out/DebugX64Legacy)
> - the '--enable-experiment=non-nullable' flag still needs to be passed
>   in during execution so that CFE runs in that mode. This is different
>   from the 'null_safety' flag
> 
> Change-Id: I7d25d9710818af5919c0bb83abe51153172f5886
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144880
> Reviewed-by: Siva Annamalai <asiva@google.com>

TBR=asiva@google.com

Change-Id: Ib0f99fc1a42c9c7a8b21f8542f4a35dba9105d5c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144900
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-04-27 05:00:10 +00:00
asiva cab69e7706 [BUILD] - Initial CL to unfork the NNBD Dart SDK
NOTE: This is a trial submit and will be reverted immediately.

- Flips the flag from --nnbd to --no-nnbd so that by default it builds
  the NNBD version
- using the --no-nnbd flag results in the SDK being built in a
  directory which has the 'Legacy' suffix added to it
  (e.g: out/DebugX64Legacy)
- the '--enable-experiment=non-nullable' flag still needs to be passed
  in during execution so that CFE runs in that mode. This is different
  from the 'null_safety' flag

Change-Id: I7d25d9710818af5919c0bb83abe51153172f5886
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144880
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-04-27 04:59:26 +00:00
Siva Annamalai 63a92a3b00 Revert "This is a trial submit of the unfork CL, it will be reverted immediately."
This reverts commit 393468dc89.

Reason for revert: This was a trial submit and so reverting it.

Original change's description:
> This is a trial submit of the unfork CL, it will be reverted immediately.
> This commit is being done so we can roll it into Flutter and run some device lab testing to see the effect this CL will have on Flutter.
> 
> [BUILD] - Initial CL to unfork the NNBD Dart SDK
> 
> - Flips the flag from --nnbd to --no-nnbd so that by default it builds
>   the NNBD version
> - using the --no-nnbd flag results in the SDK being built in a
>   directory which has the 'Legacy' suffix added to it
>   (e.g: out/DebugX64Legacy)
> - the '--enable-experiment=non-nullable' flag still needs to be passed
>   in during execution so that CFE runs in that mode. This is different
>   from the 'null_safety' flag
> 
> Change-Id: I83c813aee2a5b61ad876d8a7b0988705bb24c839
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144082
> Reviewed-by: Siva Annamalai <asiva@google.com>

TBR=asiva@google.com,liama@google.com

Change-Id: I7b50fae71764eceb17893338d1981e40ea2de520
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144083
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-04-20 03:50:49 +00:00
asiva 393468dc89 This is a trial submit of the unfork CL, it will be reverted immediately.
This commit is being done so we can roll it into Flutter and run some device lab testing to see the effect this CL will have on Flutter.

[BUILD] - Initial CL to unfork the NNBD Dart SDK

- Flips the flag from --nnbd to --no-nnbd so that by default it builds
  the NNBD version
- using the --no-nnbd flag results in the SDK being built in a
  directory which has the 'Legacy' suffix added to it
  (e.g: out/DebugX64Legacy)
- the '--enable-experiment=non-nullable' flag still needs to be passed
  in during execution so that CFE runs in that mode. This is different
  from the 'null_safety' flag

Change-Id: I83c813aee2a5b61ad876d8a7b0988705bb24c839
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144082
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-04-20 03:46:06 +00:00
Martin Kustermann 767943850f [vm] Enable check_initialization_order for ASAN, ensure tools/test.py always passes ASAN options, make Random be initialized during VM startup
Running tools/test.py locally behaves differently than on the bots
because the bots will have special environment variables set.

=> We want also local runs of tools/test.py to report ASAN errors, so
   this CL moves the setting of environment variables to tools/test.py

Furthermore we enable "additional check_initialization_order=true" asan
option to detect ordering issues when initializing global state.

Make the IsolateGroup::isolate_group_random_ heap allocated on VM
startup to avoid initialization ordering issue (turns out our `Random`
can depend on embedder entropy source callback being available)

Also add locking around assigning of isolate group ids -- without it we
could have two threads racing and possibly getting the same id.

See b/149978682

Change-Id: I41b9a34c66934dcc0d3804233f5acf09004ba3bd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136630
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-02-25 10:32:01 +00:00
Clement Skau d765d23746 [Tools] Makes build.py --arch more case-tolerant.
This allows us to do both:
  tools/build.py -a ia32
and
  tools/build.py -a IA32,ARMx64,x64

Also fixes missing arch option in utils.py.

Change-Id: I6f911397dbf52437f5347d41d71cdd3254a29476
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134700
Reviewed-by: Teagan Strickland <sstrickl@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
Auto-Submit: Clement Skau <cskau@google.com>
2020-02-07 08:26:35 +00:00
Martin Kustermann a5bb8b666a [infa] Respect --sanitizer flag in tools/test.py when running GN
Change-Id: I0c9fdc26983381111021bf1265ef58f31deb8ae2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134296
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2020-02-04 18:45:26 +00:00
Ryan Macnak 3678a3bd42 [build] Use separate out directories for each sanitizer.
Bug: https://github.com/dart-lang/sdk/issues/39611
Change-Id: Ie0aeaff758234220c2f0267b462d14f4c076bdf6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/127821
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2019-12-11 18:12:47 +00:00
Samir Jindel c885bdde1d [vm] DBC is obsolete. Remove dead code.
Change-Id: Ica33af158cca53c8e951e4b2582de83660e8a60d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121851
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2019-10-27 18:18:29 +00:00
Martin Kustermann e82fedc345 [vm] Remove deprecated (and untested) ARMv5 support from the VM
Since we have no CQ/CI test coverage of ARMv5 and none of our customers
uses it, a breaking change request was made (see issue further down).

The breaking change has been approved and a corresponding email has been
sent out:

https://groups.google.com/a/dartlang.org/forum/?nomobile=true#!topic/announce/vRQja7di3FQ

This CL removes any armv5 related code from our codebase.

Closes https://github.com/dart-lang/sdk/issues/36666

Change-Id: I9abf32f1980ca19a9cf882b43c9966b2127f89e3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/122540
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2019-10-23 14:44:33 +00:00
Samir Jindel db352304ec [vm] Test AOT on Android with the 64->32 bit cross-compiler.
We introduce a new architecture ARM_X64, which is the same as ARM except that
the host-targeting-host binaries are compiled for X64 instead of X86.

Issue https://github.com/dart-lang/sdk/issues/38709

Change-Id: I00a2b1cc71948adec9f66c7b2ee61e9e5ac53d2a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/120121
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2019-10-07 10:35:45 +00:00
Robert Nystrom e919e89d0b Add an "--nnbd" flag to build.py.
This treats whether or not to use the forked NNBD SDK as a build flag
similar to mode or architecture. When this flag is passed, it appends
"NNBD" to the main output directory, like xcodebuild/ReleaseX64NNBD.

It also defines a "use_nnbd" flag that is available inside BUILD.gn
files to determine whether or not to use the forked SDK. This flag
currently isn't used. Implementation teams will need to edit their
build steps to take that into account.

In order to use this mode on the bots, we'll need to tweak the bot
scripts to figure out whether the mode is enabled or not. There are a
couple of TODOs in the Python scripts for that. I'm not sure how that
should be specified in dart_sdk.py.

This CL does not remove my previous change to allow building the NNBD
fork next to dart-sdk in the same root output directory. I'll revert
those changes in a different CL once this one lands.

Change-Id: I3bd28f273106ee90caf9474b2fadad120b2c2d0b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119602
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2019-10-03 00:11:32 +00:00
Nate Bosch 55f81f2210 Mass format python with yapf
- Add `.style.yapf` with configuration to use Google style.
- Run `yapf` on all `.py` files in this repo.
- Manually fix one trailing space in a doc string.
- Run `git cl format runtime` to satisfy presubmit.

Change-Id: I7e6bd11e91f07926b9188362599af398551eed79
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/111600
Commit-Queue: Nate Bosch <nbosch@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2019-08-05 20:34:31 +00:00
Vyacheslav Egorov 125c5cbf4a [vm/compiler] Instroduce SIMARM_X64 build architecture.
This is variant of SIMARM build (ARM target and non-ARM host) but with a
word size mismatch (32-bit target and 64-bit host). We only expect to
build gen_snapshot binary in this mode.

This is the first step towards enabling AOT compilation targeting ARM
using 64-bit gen_snapshot binary.

This change also introduces --gen-snapshot flag for test.py which allows
to specify which gen_snapshot binary should be used for running tests.

Expected workflow with SIMARM_X64 build:

$ tools/build.py -a simarm_x64 -m release gen_snapshot
$ tools/build.py -a simarm -m release dart_precompiled_runtime
vm_platform
$ tools/test.py -a simarm -m release -c dartkp --gen-snapshot
out/ReleaseSIMARM_X64/gen_snapshot

Note that our ARM simulator can't be built as a 64-binary so we are
going to be using SIMARM runtime to test AOT compiled code produced by
SIMARM_X64 binary.

Issue https://github.com/dart-lang/sdk/issues/36839
Issue https://github.com/flutter/flutter/issues/22598

Change-Id: Id003900e8b46fc9d57975ab82b0c21852a176079
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100968
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2019-05-06 10:48:08 +00:00
Joseph Richey 86fe7ca75c Fix build when python=python3
Right now most of the dart SDK's python is compatible with python2
or python3. This change fixes a few of the build scripts to make
that completely true (at least when building the standard build on
Linux). There are only four types of changes:
  - Bare `print` statements now use the `print ()` function
  - `commands.getoutput` becomes `subprocess.check_output` with `shell=True`
  - `xrange` becomes `range`
  - `print >> sys.stderr` becomes `sys.stderr.write`

Starts work on addressing (but does not completely fix):
https://github.com/dart-lang/sdk/issues/28793

See related issue:
https://fuchsia-review.googlesource.com/c/fuchsia/+/272925

This change applys to both the `dev` and `master` branches.

Change-Id: Ibd3eb9b1f57520d2d745f05c2ac430b1d20943da

Closes #36662
https://github.com/dart-lang/sdk/pull/36662

GitOrigin-RevId: beab165294982a7e369daf6d61aea63efcab1b9b
Change-Id: I6d240749a9ba0889b5a45a08f3c4c2c20291f484
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99707
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2019-04-23 07:48:15 +00:00
Stevie Strickland ba75b4c437 Revert "Change default target to "create_sdk"."
This reverts commit 4afb45f5b8.

Reason for revert: Going to instead file a bug to get the `all' target appropriately fixed, if possible.

Original change's description:
> Change default target to "create_sdk".
> 
> Currently the "all" target can lead to build errors and is unsupported,
> so fall back to a supported target instead.
> 
> Change-Id: I925a785e4b81ab92e99c3842cb81c338c5a1d990
> Reviewed-on: https://dart-review.googlesource.com/c/87403
> Auto-Submit: Stevie Strickland <sstrickl@google.com>
> Reviewed-by: Zach Anderson <zra@google.com>
> Commit-Queue: Stevie Strickland <sstrickl@google.com>

TBR=zra@google.com,sstrickl@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: I27c567ad61e54bd2916027b6e27406bd67eb8330
Reviewed-on: https://dart-review.googlesource.com/c/89140
Reviewed-by: Stevie Strickland <sstrickl@google.com>
Commit-Queue: Stevie Strickland <sstrickl@google.com>
2019-01-11 11:50:13 +00:00
Stevie Strickland fe079b31b0 [tools] Fix list of possible arguments for -a in --help.
Make sure the architectures listed by --help for the -a command
line option reflect all currently available architectures + 'all'.

Change-Id: I2759c7c7e846d38fce5afde1854bfd2db0dd8b70
Reviewed-on: https://dart-review.googlesource.com/c/88821
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Stevie Strickland <sstrickl@google.com>
2019-01-10 10:46:14 +00:00
Stevie Strickland 4afb45f5b8 Change default target to "create_sdk".
Currently the "all" target can lead to build errors and is unsupported,
so fall back to a supported target instead.

Change-Id: I925a785e4b81ab92e99c3842cb81c338c5a1d990
Reviewed-on: https://dart-review.googlesource.com/c/87403
Auto-Submit: Stevie Strickland <sstrickl@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Stevie Strickland <sstrickl@google.com>
2018-12-17 18:09:07 +00:00
Ryan Macnak 49e521c55b [build] Forward load limit from build.py to Ninja.
Change-Id: I8d217147aa8273e8434ee5c4eefbd6266f5a3fec
Reviewed-on: https://dart-review.googlesource.com/77487
Reviewed-by: Zach Anderson <zra@google.com>
2018-10-02 00:40:03 +00:00
Zach Anderson defd68e431 [infra] Don't run generate_buildfiles.py during gclient hooks
There is a race with downloading the prebuilt Dart SDK from CIPD
introduced by:

647e1e79eb

If the prebuilt Dart SDK isn't there, then generate_buildfiles.py will
fail. Instead check whether generate_buildfiles.py should be run
right before we try to build.

Change-Id: I1e25e5280075f9d1c7d96877602617ddfb40291e
Reviewed-on: https://dart-review.googlesource.com/77160
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-09-28 18:11:07 +00:00
Zach Anderson b4004f6476 [vm] Reland: Move the interpreter under a flag.
After this CL, the interpreter is included by default in the
JIT VM under the flag --enable-interpreter.

Reland with fix to NativeArgument setup in simulator_arm.cc

Change-Id: Ib9b4df6eb4d997dfbe361188b8a127828c1d9c6f
Reviewed-on: https://dart-review.googlesource.com/74003
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-09-07 19:25:31 +00:00