Also renames the test configurations for x64 to include the architecture.
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-mac-release-try,pkg-win-release-try
Change-Id: I76e259a059e9fe16f4799d65f2bb5380a9c6fb94
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330028
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Run the seal_native_object.js preamble before any other code when
trying to benchmark dart2js compiled code.
Change-Id: Iec95dd3189869738185bce34f5202be4db0b90f8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330669
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
Many tests under `pkg`, like `pkg/js_ast` and `pkg/analyzer_cli` are
meant to run as unit tests in the Dart VM. These were also being run in
dart2js and causing flakiness because they took a long time to build on
our CQ.
Locally some of these tests compile in under 5s, but we see the
compile-time vary among the bots a lot. I've observed cases close to 30s
and some that reach the 1-minute timeout cutoff. Flakiness data show
that this is happening between 5-7% of the runs, especially on linux
bots.
This CL stops running these tests on the dart2js configuration by
changing which subset of the `pkg` suite is provided to dart2js. It also
adjusts the .status file to skip a few suites for convenience on local
testing only. This allows developers locally to run `test.py -n... pkg`
without worrying about filtering out skipped suites.
Addendum: after discussions on the CL we decided not to cmpletely drop
coverage of js_ast tests on JS to ensure it can be used in browser
environments in the future. To support this we made made the tests
cheaper by removing the dependency on package:test. They will now only
be run in a single configuration, which is all we need for this purpose.
We can reevaluate and remove this if we continue to see timeouts.
Change-Id: Idf0dbdd37e412ef71ba117ec979cb1e52585c431
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330704
Reviewed-by: Stephen Adams <sra@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Notably, this carries us across the gcc->llvm transition.
Change-Id: I46203749c64c7ef0f52de40abcd4a073517cb06f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330949
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
- Remove test directories standalone_2 and dart_2
- Remove all references to standalone_2 and dart_2 in test scripts
TEST=ci
Change-Id: I574868e274558efbbe12fe8b6408bbe77a888583
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330982
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Change-Id: I567732843bc305c4a42f10027cf0e9c705874713
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/328822
Commit-Queue: Ömer Ağacan <omersa@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Delete lots of dead code in bot_utils and utils. The logic has been
migrated into the recipes for a long time already or has otherwise
become obsolete.
This second version restores IncreasedNumberOfFileDescriptors which was
accidentally deleted while still in use.
Bug: b/270022416
Change-Id: Iea3a607663886c24ea7e0d19ea2a8a615f40ec31
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/327867
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
This reverts commit 90300b9ca1.
Reason for revert: A bunch of purple bots mentioning:
`NameError: name 'IncreasedNumberOfFileDescriptors' is not defined`.
Original change's description:
> Forward compatibility for renaming the be channel to main.
>
> Delete lots of dead code in bot_utils and utils. The logic has been
> migrated into the recipes for a long time already or has otherwise
> become obsolete.
>
> Bug: b/270022416
> Change-Id: I7977c7a0d129e6112ad143f10f554aec7d9f653f
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/326780
> Commit-Queue: Jonas Termansen <sortie@google.com>
> Reviewed-by: Alexander Thomas <athom@google.com>
Bug: b/270022416
Change-Id: I3b008eeb2c729fce4e9abc5879c91d2416dfa692
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/327960
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Delete lots of dead code in bot_utils and utils. The logic has been
migrated into the recipes for a long time already or has otherwise
become obsolete.
Bug: b/270022416
Change-Id: I7977c7a0d129e6112ad143f10f554aec7d9f653f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/326780
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Change-Id: I1098392c67da82d97c95b16490eb3beb31601896
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/326760
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
- Update dart2wasm script to accept a `-O` argument. When passed it also
passed `--inlining-limit 10` to dart2wasm (unless it's already passed
by the caller) and runs wasm-opt on the output, with the same
arguments as `compile_benchmark`.
- Add new test configurations to run tests with optimized binaries.
Example configuration: `dart2wasm-linux-optimized-d8`.
Change-Id: Ia7bd8d9c99d3a4baecb72b92b2ba38c0e92807c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/326620
Commit-Queue: Ömer Ağacan <omersa@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
This change created a CFv2 build target and provided two
implementations of the FuchsiaEmulator, so that I can continually work
on both CFv2 build and test script integration independently.
The v2 target can be built with
```
tools/build.py --os fuchsia --mode release fuchsia_ffi_test_package_cfv2
```
And v2 FuchsiaEmulator implementation can be triggered with
FUCHSIA_CFV2 environment variable.
Bug: #38752
Change-Id: I31936a2ca967fbfeb2bc5628e2f005aef6762687
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322583
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Zijie He <zijiehe@google.com>
- Add dart2wasm_options syntax to test_matrix.json.
- Add a new builder for the JavaScript compatbility mode
(`--js-compatbility`).
Change-Id: I32642e932f1d4a8a74b5b75f7783117520d51f02
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322281
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ömer Ağacan <omersa@google.com>
This should signal what is needed to add DDC canary mode
golem configs.
Change-Id: Id1fe8ed016641bda4bfa9e5fabacc78b26a35503
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/315560
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
Provides a signal for what needs to change in the golem configuration.
Change-Id: I1b77ec88ba3076cd4a0296a78950ed209e996661
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316843
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: William Hesse <whesse@google.com>
- Updates test infra to use the assets built by the new "ddc" build
targets.
- The old "dartdevc" targets are still present to avoid breaking
golem when this change lands.
- Old targets are still used when packaging assets for the SDK.
- Golem and the SDK packaging will be updated in following changes.
Change-Id: I1926e0c86833c812f5ed8355d7cd4df8740d79ee
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/315224
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
- Introduces a new output subdirectory `gen/utils/ddc` where the
new build targets output files. This is intended to make the
migration to the new assets easier since landing this change
shouldn't immediately break any dependencies.
- Enables building the canary and stable assets at the same time.
- Changes more names in our workflow from "dartdevc" to "ddc".
- Updates the ddc-canary-linux-chrome and
ddc-canary-linux-chrome-unsound configurations to use the
new assets.
The new outputs appear in directories that are more consistently
named, (no more sound vs kernel confusion). They are named in
preparation for the eventual deletion of the unsound targets
without any lingering cruft to be cleaned up in the sound
directories.
The new structure is:
```
gen/utils/ddc/
|- canary
| |- pkg
| |- sdk
|- canary_unsound
| |- pkg
| |- sdk
|- stable
| |- pkg
| |- sdk
|- stable-unsound
|- pkg
|- sdk
```
The 'pkg' and 'sdk' directories all contain outputs with the same
files names, each compiled in the respective build modes
(sound/unsound and stable/canary).
Change-Id: I66822ebf03bba487b6d359a8e0aa818b9e7b6bef
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/313081
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
- Run the tests on the existing DDC bots.
- In the future we would like to move these to their own bots so
they can be triggered individually.
Change-Id: I2ab022f19323cdfd20f42ef5c41777547ec428f4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/313882
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Anna Gringauze <annagrin@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
This reverts commit f827eb3a78.
Reason for revert: The tests being run, and the results being compared to, are changing dramatically, causing all sorts of test results going from [status] -> Skipped and from New Test -> [status].
Something is really going wrong.
Original change's description:
> Balance tests equally across shards.
>
> The sharded test runner invocations are now passed the previous results
> which contains the test timing, which are used to simulate how long each
> shard would take to run. The shards are now balanced as evenly as
> possible on a test level, taking multiple cores into account.
>
> Sharded tests are now run starting with the slowest test first, such
> that extremely long running tests finish as early as possible. This
> behavior ensures the cores are saturated and can be padded with fast
> tests near the end, rather than waiting for a few slow tests to complete
> while the rest of the system is idle.
>
> The algorithm works very well whenever it's able to accurately predict
> the time to run shards. In a number of cases, the model doesn't quite
> reflect reality and the data, which makes it fairly imperfect but still
> reasonably good. I think a second order feedback loop might kick in once
> it reorders the tests across shards and the test timing data reflects
> the new test timings.
>
> Multitests are no longer always sent to the same shard, since the data
> isn't available at the moment, and the change as-is speeds up the test
> running considerably.
>
> The front end unit test suites currently ignore the feature as there are
> no benefits yet to improving those quick shards.
>
> Upgrade the language version to 3.0.0 so patterns can be used and fix
> a mixin not being a mixin.
>
> Fixes: b/291585137
> Change-Id: I3cc1b1d96038d5b46e836b091e299097717c226c
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/314081
> Reviewed-by: William Hesse <whesse@google.com>
> Commit-Queue: Jonas Termansen <sortie@google.com>
Change-Id: I233e4bfa6d6ecf0cea4f97c1e47f1635f7b9040c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/315060
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: William Hesse <whesse@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
The sharded test runner invocations are now passed the previous results
which contains the test timing, which are used to simulate how long each
shard would take to run. The shards are now balanced as evenly as
possible on a test level, taking multiple cores into account.
Sharded tests are now run starting with the slowest test first, such
that extremely long running tests finish as early as possible. This
behavior ensures the cores are saturated and can be padded with fast
tests near the end, rather than waiting for a few slow tests to complete
while the rest of the system is idle.
The algorithm works very well whenever it's able to accurately predict
the time to run shards. In a number of cases, the model doesn't quite
reflect reality and the data, which makes it fairly imperfect but still
reasonably good. I think a second order feedback loop might kick in once
it reorders the tests across shards and the test timing data reflects
the new test timings.
Multitests are no longer always sent to the same shard, since the data
isn't available at the moment, and the change as-is speeds up the test
running considerably.
The front end unit test suites currently ignore the feature as there are
no benefits yet to improving those quick shards.
Upgrade the language version to 3.0.0 so patterns can be used and fix
a mixin not being a mixin.
Fixes: b/291585137
Change-Id: I3cc1b1d96038d5b46e836b091e299097717c226c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/314081
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
The purpose of the wasm_js_compatibility target is to facilitate experiments with a JS compatibility mode for Dart2Wasm. Initially, we're just going to focus on typed data, but this will give us a place to experiment with moving List and String to JS as well.
In addition, someday down the road we hope to experiment with two additional compatibility changes:
1) Exclusively using double for all Dart numbers
2) Allowing undefined to flow as null.
The two major benefits of this approach are:
1) Much faster JS interop
2) To make it easier to bring up Dart2JS applications on Dart2Wasm
The only downside will be access overhead on the Wasm side, but the JS builtins proposal could potentially bring us close to parity with Wasm builtins someday.
Tested: Wasm specific trivial refactor.
Change-Id: I2c09426b6999507c1de6e584e9bc7072a088bda9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/313240
Commit-Queue: Joshua Litt <joshualitt@google.com>
Reviewed-by: Ömer Ağacan <omersa@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Add test.py --default-suites that includes all the default suites in
addition to the ones explicitly requested, so the test matrix can run
co19 together with the default suites in one sharded test step.
Bug: b/290617138
Change-Id: I5dd5d1aaf3b1ee38adf88c6e9ee6ec13d97fe1ce
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/313567
Reviewed-by: William Hesse <whesse@google.com>
The non-sharded tests takes 10 minutes and each of the sharded tests
takes 14 minutes. It's faster to shard immediately (costing one bot
more) and concurrently run the local tests.
Fix end to end dart2js test that times out when sharded and run outside
a directory called sdk.
Bug: b/290617138
Change-Id: If71f0d301edf565c9f15847098320106ca383635
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/312983
Reviewed-by: William Hesse <whesse@google.com>
The try builder is among the fastest commonly used builders at 21
minutes average, which is less than the current 25 minute best case
goal. Free up 2 of 10 shards to be available for other work during peak
load, which will slow it down by 2 or 3 minutes and finish at the same
time as the other try builders, resulting in an overall faster commit
queue experience.
Bug: b/290617138
Change-Id: Iea2ba71248fa0d3c21f1d80b540b55ce810d96af
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/313565
Reviewed-by: William Hesse <whesse@google.com>
The test steps with the same configurations have been unified and
the shard number balanced so each step finishes as closely to each
other as possible.
Bug: b/290617138
Change-Id: I6616fb78e5cd6db36153db0d9e2b45131eef25fd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/312984
Reviewed-by: William Hesse <whesse@google.com>
Change-Id: I37070079ddbdc86fb0e5144842690877abb1b031
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/313283
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Makes the TSAN skips apply to
`test.py -mrelease --sanitizer=tsan`
and not just
`test.py -n vm-tsan-linux-release-x64`.
Also makes the timeouts agree.
Change-Id: I10315e754a4ebb3020f3c2f6cecfac6b77e77a9b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311828
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This adds basic infrastructure for a stringref implementation in
dart2wasm:
- A `--[no-]stringref` option to the compiler
- An option in the `WasmTarget`, controlling the name of the target
- Separate sets of patch files for the two targets
- Separate platform dill files for the two targets
For now, the patch file contents are the same, and the compiler flag
is not used by the backend (only by the `dart2wasm` script to select
the appropriate platform dill file). Both of these will change as the
implementation progresses.
Tested: ci + manual check that the option selects the correct dill
Change-Id: I2c9bb95ba06fd3de3f7007703ef545e3f0c728ba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/310621
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Ömer Ağacan <omersa@google.com>
This has been replaced by debian-x64 via its own recipe.
Bug: b/283763308
Change-Id: I1ae5950407ba7a4caae7467b1b1e7b6d3d6fb781
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/304882
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: William Hesse <whesse@google.com>
