Commit graph

435 commits

Author SHA1 Message Date
William Hesse 416b43faaa [infra] Fix handling of deflaking a list of tests
An error was introduced by
https://dart-review.googlesource.com/c/sdk/+/313567
when the --default-suites flag is used along with a
list of tests to deflake. The suites in the list of tests
should be the only suites used as selectors, and should
replace any other input selectors.

Bug: b/290617138
Change-Id: Ic43c1409df621bbaf852276fbaa16ce4961a9549
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/314061
Auto-Submit: William Hesse <whesse@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2023-07-17 19:11:00 +00:00
Jonas Termansen f9b8917e30 Balance VM JIT and AOT shards.
Add test.py --default-suites that includes all the default suites in
addition to the ones explicitly requested, so the test matrix can run
co19 together with the default suites in one sharded test step.

Bug: b/290617138
Change-Id: I5dd5d1aaf3b1ee38adf88c6e9ee6ec13d97fe1ce
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/313567
Reviewed-by: William Hesse <whesse@google.com>
2023-07-14 16:01:09 +00:00
Ryan Macnak 3daa47b54e [test_runner] Don't assign all vm/cc tests to the first shard.
Before:
./tools/test.py --shards=10 --shard=1 --list | wc -l  # 3664
./tools/test.py --shards=10 --shard=2 --list | wc -l  # 1047
./tools/test.py --shards=10 --shard=3 --list | wc -l  # 1146

After:
./tools/test.py --shards=10 --shard=1 --list | wc -l  # 1408
./tools/test.py --shards=10 --shard=2 --list | wc -l  # 1306
./tools/test.py --shards=10 --shard=3 --list | wc -l  # 1381

Change-Id: I2107779e79d85976c04db7c01c11581a8d9895b0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/313280
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-07-12 17:56:59 +00:00
Daco Harkes b64ef27dec [pkg/ffi] Roll dart-lang/native
Rolling https://github.com/dart-lang/native/pull/87 manually because
of the c_compiler -> native_toolchain_c rename.

Change-Id: I2592882a7137a40703f96a487a66d31eac7c990d
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-mac-release-try,pkg-win-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/313200
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Devon Carew <devoncarew@google.com>
2023-07-11 19:14:48 +00:00
Ryan Macnak 99ef36064f [infra] Make the sanitizer a first-class status variable.
Makes the TSAN skips apply to
 `test.py -mrelease --sanitizer=tsan`
and not just
 `test.py -n vm-tsan-linux-release-x64`.

Also makes the timeouts agree.

Change-Id: I10315e754a4ebb3020f3c2f6cecfac6b77e77a9b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311828
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-07-10 17:46:31 +00:00
Martin Kustermann daa35fd4fd [gardening] Restore timout multiplier in test runner for linux-arm64
The timeout multipiler for linux-arm64 was originally 4, then lowered to
1 (see [0]), then increased again to 2 (see [1]).

Though it seems that service tests are still flakily timing out, so
let's try restoring the original multilier.

[0] https://dart-review.googlesource.com/c/sdk/+/306662
[1] https://dart-review.googlesource.com/c/sdk/+/307972

Also special case `ia32` in timeout calculations due to not using
an AppJIT trained `kernel-isolate` snapshot and therefore being
very slow, especially in ia32-debug mode.

Issue https://github.com/dart-lang/sdk/issues/52589
TEST=ci

Change-Id: Iab8c768866aec9e77bb83c7a3242cc5de8fb4e2f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/312905
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2023-07-10 12:02:16 +00:00
Daco Harkes bd1bad7afc [deps/ffi] Unbundle package:native_assets_builder
The Dart SDK CL for https://github.com/dart-lang/native/pull/69.

Bug: https://github.com/dart-lang/native/issues/67
Change-Id: I45d7ac691a6aaa41bce5be0e36403021a948bb4f
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-mac-release-try,pkg-win-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311740
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Devon Carew <devoncarew@google.com>
2023-06-28 09:09:09 +00:00
Devon Carew 9216f830c6 [deps] rev package:lints to the latest; address lints
Change-Id: I64032a39c589c291297decade18f6a46c08f9c5e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/310966
Commit-Queue: Devon Carew <devoncarew@google.com>
Reviewed-by: Ömer Ağacan <omersa@google.com>
2023-06-27 17:03:37 +00:00
Vyacheslav Egorov 8f6eaeb883 [infra] Pass TEST_COMPILATION_DIR to Android tests
The whole directory is already copied to the device. This makes
it consistent with tests running on the host and allows us to
access various artefacts (e.g. SO files containing DWARF
information).

Change-Id: I9fe20fdbb48ad5a3742286a84c87f87f580f2c09
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/310600
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2023-06-21 14:17:34 +00:00
William Hesse 98009cf7a7 [test] Report android infra failures correctly when running tests
Any adb failures when test_runner runs tests on android devices,
except an failure of the actual precompiled test when it is run,
should be reported as an infra failure and should shut down the
test runner.

After change https://dart-review.googlesource.com/c/sdk/+/291304
these types of failure were reported as test failures instead.

Fixes: b/281492587
Change-Id: Iea47ccffac81b0b92f3a3fb3780d2181d08fc1d1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/310260
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: William Hesse <whesse@google.com>
2023-06-20 15:49:38 +00:00
Tess Strickland 69172b7eb5 [pkg/test_runner] Add QemuConfig entries for compressed architectures.
Change-Id: Ib40185f3e5dd291c6ba9724089a04e696306a7f8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/307801
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2023-06-12 14:31:04 +00:00
Ryan Macnak 27f5a8c175 [test] Increase timeout for ARM64 Linux (but not Windows).
Several tests seems to be just at the current timeout threshold and flaking between timeout and their normal outcome.

Cf. 87362c6f42

Bug: https://github.com/dart-lang/sdk/issues/52589
Change-Id: I4e89aa71618c51a9ca1d38e8d03fffbcd919a744
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/307972
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-06-08 15:20:11 +00:00
Ryan Macnak 32977d8f43 [infra] Setup Windows ARM64.
Change-Id: I6a9c5beba0ec909b75a463de04586787068fdbcd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/303424
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-06-02 18:18:09 +00:00
Ryan Macnak b43d95a942 [test] Re-establish at least some Fuchsia testing.
Only works with FFI tests that don't use multitest. I.e., tests that only need things inside fuchsia_ffi_test_package.

Change-Id: I17d8cb4eafe02430f64f6ec60601095c49c54810
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306675
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-06-01 22:44:03 +00:00
Ryan Macnak 87362c6f42 [test] Don't apply simulator timeout multipler to ARM hardware.
Change-Id: I686480d405905553fe885c2e1d41946ed9397075
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306662
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2023-06-01 15:16:58 +00:00
Ömer Sinan Ağacan 0f49f23be4 [dart2wasm] Add support for testing in Chrome
d8 doesn't have some of the JS and HTML functions we need for event and
micro-task scheduling. In [1] we add mock functions to allow running in
d8, but that means the mock event loop can run differently than the
actual Chrome event loop, so we need to test in Chrome. This CL adds
support for this.

[1]: https://dart-review.googlesource.com/c/sdk/+/305201

New failing tests with the new configuration:

language/nnbd/static_errors/unchecked_use_of_nullable_test/30 broke (RuntimeError -> Pass, expected RuntimeError)
language/regress/regress24935_test/01 broke (RuntimeError -> Pass, expected RuntimeError)
language/unsorted/flatten_test/01 broke (RuntimeError -> Pass, expected RuntimeError)
language/unsorted/flatten_test/05 broke (RuntimeError -> Pass, expected RuntimeError)
language/unsorted/flatten_test/09 broke (RuntimeError -> Pass, expected RuntimeError)
lib/js/export/static_interop_mock/proto_test is new and failed (CompileTimeError, expected Pass)
lib/js/static_interop_test/constants_test is new and failed (CompileTimeError, expected Pass)
lib/js/static_interop_test/futurevaluetype_test is new and failed (CompileTimeError, expected Pass)
lib/js/static_interop_test/supertype_transform_test is new and failed (CompileTimeError, expected Pass)

New passing tests with the new configuration:

co19/Language/Expressions/Function_Invocation/async_generator_invokation_t10 was fixed (Fail -> Pass)
co19/Language/Libraries_and_Scripts/Scripts/top_level_main_t01 was fixed (RuntimeError -> Pass)
co19/Language/Libraries_and_Scripts/Scripts/top_level_main_t06 was fixed (RuntimeError -> Pass)
co19/Language/Statements/Yield_and_Yield_Each/Yield/execution_async_A01_t07 was fixed (RuntimeError -> Pass)
co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_A03_t03 was fixed (Fail -> Pass)
co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_A03_t07 was fixed (Fail -> Pass)
co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_A03_t08 was fixed (RuntimeError -> Pass)
co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_A03_t09 was fixed (RuntimeError -> Pass)
co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_A03_t11 was fixed (Fail -> Pass)
co19/LibTest/async/Future/Future.delayed_A01_t02 was fixed (RuntimeError -> Pass)
co19/LibTest/async/Stream/Stream.periodic_A01_t01 was fixed (RuntimeError -> Pass)
co19/LibTest/async/Stream/Stream.periodic_all_t01 was fixed (RuntimeError -> Pass)
co19/LibTest/async/Stream/Stream.periodic_all_t02 was fixed (RuntimeError -> Pass)
co19/LibTest/async/Stream/Stream.periodic_all_t03 was fixed (RuntimeError -> Pass)
co19/LibTest/async/StreamController/StreamController.broadcast_Stream_all_A01_t01 was fixed (RuntimeError -> Pass)
co19/LibTest/async/StreamController/StreamController.broadcast_Stream_all_A01_t02 was fixed (RuntimeError -> Pass)
co19/LibTest/async/StreamController/StreamController.broadcast_Stream_all_A01_t03 was fixed (RuntimeError -> Pass)
co19/LibTest/async/StreamController/StreamController.broadcast_Stream_all_A02_t01 was fixed (RuntimeError -> Pass)
co19/LibTest/async/StreamController/StreamController.broadcast_Stream_all_A02_t02 was fixed (RuntimeError -> Pass)
co19/LibTest/async/StreamController/StreamController.broadcast_Stream_all_A02_t03 was fixed (RuntimeError -> Pass)
co19/LibTest/async/StreamController/stream_all_A01_t01 was fixed (RuntimeError -> Pass)
co19/LibTest/async/StreamController/stream_all_A01_t02 was fixed (RuntimeError -> Pass)
co19/LibTest/async/StreamController/stream_all_A02_t01 was fixed (RuntimeError -> Pass)
co19/LibTest/async/StreamController/stream_all_A02_t02 was fixed (RuntimeError -> Pass)
co19/LibTest/async/StreamController/stream_all_A02_t03 was fixed (RuntimeError -> Pass)
co19/LibTest/async/Timer/Timer.periodic_A01_t01 was fixed (RuntimeError -> Pass)
co19/LibTest/async/Timer/Timer_A01_t01 was fixed (RuntimeError -> Pass)
co19/LibTest/core/Stopwatch/elapsedTicks_A01_t01 was fixed (RuntimeError -> Pass)
corelib/throw_half_surrogate_pair_test/01 was fixed (NonUtf8Output -> RuntimeError)
corelib/throw_half_surrogate_pair_test/02 was fixed (NonUtf8Output -> RuntimeError)
language/async/return_throw_test was fixed (RuntimeError -> Pass)
language/async_star/async_star_await_for_test was fixed (RuntimeError -> Pass)
language/async_star/async_star_test was fixed (RuntimeError -> Pass)
language/async_star/yield_test was fixed (Fail -> Pass)
language/async_star/yieldstar_test was fixed (Fail -> Pass)
language/import/conditional_string_test is new and succeeded (Pass)
language/regress/regress21795_test was fixed (RuntimeError -> Pass)
language/regress/regress23244_test was fixed (RuntimeError -> Pass)
lib/async/multiple_timer_test was fixed (RuntimeError -> Pass)
lib/async/periodic_timer2_test was fixed (RuntimeError -> Pass)
lib/async/periodic_timer3_test was fixed (RuntimeError -> Pass)
lib/async/periodic_timer4_test was fixed (RuntimeError -> Pass)
lib/async/schedule_microtask_test was fixed (RuntimeError -> Pass)
lib/async/slow_consumer2_test succeeded again (Pass -> skipped)
lib/async/stream_timeout_test was fixed (RuntimeError -> skipped)
lib/async/timer_isActive_test was fixed (RuntimeError -> Pass)
lib/async/timer_repeat_test was fixed (RuntimeError -> Pass)
lib/async/timer_test was fixed (RuntimeError -> Pass)

Change-Id: I2c2f837b4a6093a1a9aad357df21a04a6907ab6f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/305941
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Jackson Gardner <jacksongardner@google.com>
Commit-Queue: Ömer Ağacan <omersa@google.com>
2023-05-31 18:06:55 +00:00
William Hesse cf07d75b18 [test] Increase internal timeout on test_runner test.
In slow configurations, the test subprocesses could
take more than 10 seconds. Increase their timeout,
and use a smaller timeout for the timing-out test.

Change-Id: I218a3b18b763f4ea9c4868b2841b2519ef3b9746
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306360
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: William Hesse <whesse@google.com>
2023-05-30 12:36:02 +00:00
Ryan Macnak b0170463af [infra] Run JIT FFI tests on QEMU.
Change-Id: I97dd32cff783265de6a431276434e79bb030939b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299780
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-05-26 23:22:32 +00:00
Ryan Macnak 032ace1944 [infra] Change default NNBD to strong.
Change-Id: I2d52bdb1747346d118ef962ab2caf910fe8ebf53
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296703
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-05-24 20:48:36 +00:00
Ömer Sinan Ağacan c74387a3a2 [dart2wasm] New async implementation
This CL re-implements the async function compilation without using JSPI
or any other platform features.

This implementation is faster than the JSPI-based one in all benchmarks,
in some cases up to 200x (benchmark results at the end). So we remove
the JSPI-based implementation as there's no point in maintaining a much
slower implementation and supporting two implementations at the same
time (which is tricky because these implementations need different
libraries, all scripts need to support two modes etc.) that also
requires experimental platform features.

# Main changes

- A new pass `AwaitTransformer` transforms `await` expressions to
  top-level statements in form `var <fresh variable> = await <simple
  expr>`, where `<simple expr>` is an expression without `await`.

  After this pass all `await` expressions have the simple continuation
  of "assign the value of the awaited future to this variable and
  continue with the next statement". This simplifies `await`
  compilation.

- A new code generator `AsyncCodeGenerator` (inherits from
  `CodeGenerator`) compiles `async` functions. The `_YieldFinder` class
  is copied from `sync*` code generator but modified to handle `async`
  expressions.

- Mentions to the V8 flag `--experimental-wasm-stack-switching` is
  removed from all scripts and documents.

# Future work

- Control flow handling in `AsyncCodeGenerator` needs to be implemented
  in a similar way in `SyncStarCodeGenerator`. Doing this without
  duplicating a lot of code will require some refactoring.

# New passing tests

- co19/Language/Statements/Yield_and_Yield_Each/Yield/execution_async_A05_t01
- co19/Language/Statements/For/Asynchronous_For_in/execution_A02_t02
- language/regress/regress23996_test
- language/sync_star/dcall_type_test

# Benchmarks

Current implementation:

```
AsyncLiveVars.LiveObj1(RunTime): 1586000.0 us.
AsyncLiveVars.LiveObj2(RunTime): 2114000.0 us.
AsyncLiveVars.LiveObj4(RunTime): 1972500.0 us.
AsyncLiveVars.LiveObj8(RunTime): 2212000.0 us.
AsyncLiveVars.LiveObj16(RunTime): 2238000.0 us.
AsyncLiveVars.LiveInt1(RunTime): 2362000.0 us.
AsyncLiveVars.LiveInt4(RunTime): 2470000.0 us.
AsyncLiveVars.LiveObj2Int2(RunTime): 2575000.0 us.
AsyncLiveVars.LiveObj4Int4(RunTime): 2820000.0 us.
Calls.AwaitAsyncCall(RunTimeRaw): 35676.15658362989 ns.
Calls.AwaitAsyncCallClosureTargetPolymorphic(RunTimeRaw): 38934.108527131786 ns.
Calls.AwaitAsyncCallInstanceTargetPolymorphic(RunTimeRaw): 42617.02127659575 ns.
Calls.AwaitFutureCall(RunTimeRaw): 2832.058906825262 ns.
Calls.AwaitFutureCallClosureTargetPolymorphic(RunTimeRaw): 3665.8125915080527 ns.
Calls.AwaitFutureCallInstanceTargetPolymorphic(RunTimeRaw): 4420.449537241076 ns.
Calls.AwaitFutureOrCall(RunTimeRaw): 3692.7621861152143 ns.
Calls.AwaitFutureOrCallClosureTargetPolymorphic(RunTimeRaw): 4625.346901017576 ns.
Calls.AwaitFutureOrCallInstanceTargetPolymorphic(RunTimeRaw): 4514.6726862302485 ns.
Calls.AwaitFutureOrCallInstanceTargetPolymorphicManyAwaits(RunTimeRaw): 345172.4137931034 ns.
Calls.AwaitForAsyncStarStreamPolymorphic(RunTimeRaw): 697000.0 ns.
Calls.AwaitForAsyncStarStreamPolymorphicManyYields(RunTimeRaw): 704666.6666666666 ns.
Calls.AwaitForManualStreamPolymorphic(RunTimeRaw): 11010.989010989011 ns.
Calls.SyncCall(RunTimeRaw): 0.40275240996973316 ns.
Calls.SyncCallClosureTarget(RunTimeRaw): 0.3989591156672242 ns.
Calls.SyncCallInstanceTargetPolymorphic(RunTimeRaw): 3.2632549336335526 ns.
Calls.IterableSyncStarIterablePolymorphic(RunTimeRaw): 353.3980582524272 ns.
Calls.IterableManualIterablePolymorphic(RunTimeRaw): 332.1161825726141 ns.
Calls.IterableManualIterablePolymorphicManyYields(RunTimeRaw): 354.28067078552516 ns.
```

New implementation:

```
AsyncLiveVars.LiveObj1(RunTime): 11327.683615819209 us.
AsyncLiveVars.LiveObj2(RunTime): 10923.91304347826 us.
AsyncLiveVars.LiveObj4(RunTime): 10956.284153005465 us.
AsyncLiveVars.LiveObj8(RunTime): 11286.516853932584 us.
AsyncLiveVars.LiveObj16(RunTime): 11445.714285714286 us.
AsyncLiveVars.LiveInt1(RunTime): 11016.483516483517 us.
AsyncLiveVars.LiveInt4(RunTime): 11327.683615819209 us.
AsyncLiveVars.LiveObj2Int2(RunTime): 10918.478260869566 us.
AsyncLiveVars.LiveObj4Int4(RunTime): 10737.967914438503 us.
Calls.AwaitAsyncCall(RunTimeRaw): 1082.2510822510822 ns.
Calls.AwaitAsyncCallClosureTargetPolymorphic(RunTimeRaw): 1056.4124234100993 ns.
Calls.AwaitAsyncCallInstanceTargetPolymorphic(RunTimeRaw): 1134.1726210729273 ns.
Calls.AwaitFutureCall(RunTimeRaw): 865.6509695290858 ns.
Calls.AwaitFutureCallClosureTargetPolymorphic(RunTimeRaw): 841.3967185527977 ns.
Calls.AwaitFutureCallInstanceTargetPolymorphic(RunTimeRaw): 839.066957543212 ns.
Calls.AwaitFutureOrCall(RunTimeRaw): 397.9941096871766 ns.
Calls.AwaitFutureOrCallClosureTargetPolymorphic(RunTimeRaw): 406.17384240454913 ns.
Calls.AwaitFutureOrCallInstanceTargetPolymorphic(RunTimeRaw): 393.7472929873607 ns.
Calls.AwaitFutureOrCallInstanceTargetPolymorphicManyAwaits(RunTimeRaw): 1095.0503723171266 ns.
Calls.AwaitForAsyncStarStreamPolymorphic(RunTimeRaw): 6643.426294820717 ns.
Calls.AwaitForAsyncStarStreamPolymorphicManyYields(RunTimeRaw): 7178.750897343863 ns.
Calls.AwaitForManualStreamPolymorphic(RunTimeRaw): 1456.23998835008 ns.
Calls.SyncCall(RunTimeRaw): 0.3919935321067202 ns.
Calls.SyncCallClosureTarget(RunTimeRaw): 0.3906669661780074 ns.
Calls.SyncCallInstanceTargetPolymorphic(RunTimeRaw): 3.1676143112814583 ns.
Calls.IterableSyncStarIterablePolymorphic(RunTimeRaw): 104.4932079414838 ns.
Calls.IterableManualIterablePolymorphic(RunTimeRaw): 104.57516339869281 ns.
Calls.IterableManualIterablePolymorphicManyYields(RunTimeRaw): 116.92487576731949 ns.
```

TEST=ci
CoreLibraryReviewExempt: Added entry-point pragmas.
Change-Id: I02fbd08141f51c00fb37b6fa0304dc25d6afdb71
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/301020
Commit-Queue: Ömer Ağacan <omersa@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Joshua Litt <joshualitt@google.com>
2023-05-22 08:32:12 +00:00
Vyacheslav Egorov 19e56613cf [infra] Prevent VM AOT tests from poluting SDK source.
Extend test_runner VMOptions support with an ability to specify
paths relative to temporary compilation directory.

    // VMOptions=--foo=$TEST_COMPILATION_DIR/foo.file

The same directory will also be passed as an environment variable
to execution command.

Migrate most of the tests which used to write stuff into the SDK
root to use this feature. I am leaving vm/dart/causal/* tests
unmigrated because migrating requires time consuming manual
update of expectations (which encode raw line numbers). I have
a follow up CL which changes how these tests are written which
will make migration trivial.

Change-Id: Id53008be66de8ff18623efac27ff15750f407749
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/300600
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2023-05-03 13:24:39 +00:00
Daco Harkes 9e0e9a8a9a [pkgs/native] Roll and update env keys
Also picks up keys from `package:native_assets_cli` instead of coding
them which could lead to typos.

Change-Id: I5c72c26c9a45a38ac41f3513c2b5b10e8863ff83
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-mac-release-try,pkg-win-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/300221
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-05-01 13:19:02 +00:00
Daco Harkes 59ff3b75c1 [infra] Fix Windows bots
Change-Id: I90f5f770f73119aa030ff2eb8198c7856d571423
Cq-Include-Trybots: luci.dart.try:vm-win-release-x64-try,pkg-win-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299661
Reviewed-by: Jonas Termansen <sortie@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-04-28 15:25:27 +00:00
Daco Harkes 6c3c7ae331 [pkgs/native] Windows support + CI
We use the MSVC from Depot Tools to build C files.
When using clang from DEPS, we need to pass the right INCLUDE / LIB
environment variables. So we might as well use the MSVC instead.

Moves the `nativeCompilerEnvironmentVariables` to the
`TestConfiguration` so that it's cached.

Bug: https://github.com/dart-lang/sdk/issues/50565
Change-Id: I3576cd0ed07081afb9ae5ba737b8bfe1b21941e5
Cq-Include-Trybots: luci.dart.try:pkg-win-release-try,pkg-mac-release-try,pkg-mac-release-arm64-try,pkg-linux-release-try,pkg-linux-debug-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/298580
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-04-28 13:31:10 +00:00
Robert Nystrom 988642a94e Remove support for "@compile-error" and similar markers in tests.
These were added during the Dart 2 test migration as a way to indicate
why a test was expected to fail. But, like negative tests, they have
very poorly granularity.

Static error tests are strictly superior.

The co19 tests have been migrated off of these markers for a couple of
years, so our own language tests were the only holdouts. I see no uses
of "@syntax-error", "@runtime-error", or "@static-warning". I have now
migrated all of the tests that contained "@compile-error" to be proper
static error tests.

This simplifies the test runner and makes our tests more precise.

Fix #45634.

Change-Id: I0f46d110b6f322d98187e734195ecba7524574af
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296720
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Auto-Submit: Bob Nystrom <rnystrom@google.com>
2023-04-24 18:34:59 +00:00
Robert Nystrom b985fc961a Make the static error updater gracefully skip multitests.
Multitests aren't valid Dart files that can be processed by a Dart
implementation so the updater generally does a poor job if it tries to
update one.

It's probably not worth supporting because, in practice, a test should
either be a multitest or a static error test, but not both.

Change the tool to skip over any multitests it encounters. If this
results in it doing nothing at all, it reports that as an error.
Otherwise, it just lists the multitests it didn't process.

Close #37721.

Change-Id: Icfb1ff9fe63f2c249b3ccfba65166b97654a9918
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296760
Commit-Queue: Leaf Petersen <leafp@google.com>
Reviewed-by: Leaf Petersen <leafp@google.com>
Auto-Submit: Bob Nystrom <rnystrom@google.com>
2023-04-21 04:17:27 +00:00
Daco Harkes 4cb4d5d036 [pkgs/native] MacOS support + CI
Pulls in a new versions of https://github.com/dart-lang/native with
support for MacOS and enables the tests for these on the MacOS pkg
bots.

Fixes the clang paths in pkg/test_runner/lib/src/options.dart for
mac and win.

Since a relatively recent XCode is available on the bots, use XCode
to build rather than the clang from DEPS.

test/native_toolchain/recognizer_test was refactored to only
run tests for tools installed on the host. So it's unskipped.

Bug: https://github.com/dart-lang/sdk/issues/50565
Change-Id: I3804b01da7a4e74d3e49a1b2ba3c8425132417d7
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-mac-release-try,pkg-win-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296180
Reviewed-by: Liam Appelbe <liama@google.com>
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2023-04-20 14:17:39 +00:00
Alexander Markov cf3e602869 [test harness] Handle non-zero exit code of a test as a test failure, not infra failure
Sometimes failing tests crash or terminate with arbitrary exit codes.
For example, MSAN can abort a test with exit code 1.
This should not be treated as an infra failure by the test harness,
and should not prevent running other tests.

Change-Id: I97450d9cfb1e117451c35ab8a8cd01074fbef8db
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/291304
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2023-04-19 17:49:22 +00:00
Daco Harkes f9f1030959 [infra] DEPS and test dart-lang/native packages
This CL pulls in dart-lang/native packages and runs them on the
pkg bots.

Bug: https://github.com/dart-lang/sdk/issues/50565
Change-Id: Ifcd8ba25f8bdfeca8f4c161adfc3c20e0ce500d1
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-mac-release-try,pkg-win-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/295280
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2023-04-18 08:04:49 +00:00
Ryan Macnak 3d8424e9fc [test] Introduce status variable $simulator, and fix some cases missing simarm_x64.
TEST=ci
Change-Id: Ie2a8ac015b2d316527d648956c86d4c7cb319d26
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292962
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-04-04 03:10:26 +00:00
Ryan Macnak fdabbc9d10 [infra] Add configurations to rename AOT NNBD.
Change-Id: I56044e0fcfdb4623595971354a01164d8ae708e6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286360
Reviewed-by: Alexander Thomas <athom@google.com>
2023-04-02 08:34:23 +00:00
Robert Nystrom 0196a5b296 Allow static error tests to detect unreachable case hints from analyzer.
Eventually, these hints should probably be moved over to warnings. But
for now, this makes it possible to write static error language tests
that validate that analyzer produces unreachable case warnings/hints
where expected.

Also updated the patterns/ and switch/ tests now that those errors must
be expected by the test.

Change-Id: If1fb92602c4bde2819b9eec73598033009054947
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/291967
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
2023-03-31 20:50:52 +00:00
Aske Simon Christensen eca8a9a3ee [dart2wasm] Support commandline arguments to Wasm program
Change-Id: Id44fe9b2faf52071c18c81e9be53a4c04eae1735
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/291086
Reviewed-by: Jonas Termansen <sortie@google.com>
Reviewed-by: Joshua Litt <joshualitt@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
2023-03-30 16:20:27 +00:00
Ryan Macnak d01421c837 [test] Recognize exitcode 1 from all sanitizers.
Should avoid test failures triggering spurious infra failures.

TEST=ci
Change-Id: Iece312aa7eba814d7a740b0db16505d480cd3808
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/291582
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-03-29 18:04:20 +00:00
Nicholas Shahan 7c0e4d67c7 [dart2js] Prevent unsound compiles
- Remove unsound platform .dill files from the packaged SDK.
- Throw and error referencing the sound null safety requirements of
  Dart 3 when compiling with `--no-sound-null-safety` and an unsound
  platform .dill file is not found.

Change-Id: I703a446e3e92c76a6b48168577fc0161eea83854
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289280
Commit-Queue: Nate Biggs <natebiggs@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
2023-03-28 21:31:27 +00:00
Devon Carew 85cf783fe8 [pkg/test_runner] analyze with package:lints/recommended.yaml
Change-Id: Iab61ff9f67b40157657fde2d732e970d67c9fcac
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/290607
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2023-03-23 15:23:17 +00:00
Joshua Litt f49db09106 [js] Run web specific tests on Dart2Wasm.
Change-Id: I9facee2ae4357ed0b9f235d6951eafbae84f18dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289700
Reviewed-by: Srujan Gaddam <srujzs@google.com>
Commit-Queue: Joshua Litt <joshualitt@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2023-03-21 15:35:04 +00:00
Devon Carew 6da7792039 [pkg/test_runner] use more lints from package:lints/recommended
Change-Id: Ie617a8bec0ea9432e72fd8b30874298536f57c74
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289021
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2023-03-15 18:33:30 +00:00
Nicholas Shahan de44a23dd1 [test_runner] Cleanup unused compiler names
"dartdevc" and "dartdevk" have been replaced with "ddc".

Change-Id: I823bc029a0bbe1295fd731efcc32961e61c6c175
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/282485
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2023-03-07 02:09:16 +00:00
William Hesse 9aa70fbdf7 Fix test_runner package unit tests
Fix test_runner's handling of tests passed as
relative native file paths to the test runner command.
Fix failing use of a special executable to test a crashing process.

Change-Id: I0837a0c2c4461883a604aa85083c3098dfbaae2e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286821
Reviewed-by: Devon Carew <devoncarew@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2023-03-06 13:57:49 +00:00
Ryan Macnak cbb4881f0d [test_runner] Make AppJIT testing work with QEMU.
Change-Id: I784e24c1fbb04fefc09497cbc6006f5d330afec1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/285344
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2023-03-03 18:59:16 +00:00
Ryan Macnak 92c914d19e [infra] Fix running X64 AOT tests on an ARM64 Mac.
Change-Id: Idd83f28aaab6f0c2cc843618eefa1fd17cb1c05e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/283707
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-02-17 20:19:07 +00:00
Nicholas Shahan 7a6e5e12d0 [test_runner] Equate "legacy" and "weak" flags for DDC
For DDC "legacy" code requires the same flags as compiling
and running modern code with unsound null safety.

During compilation all the nnbd "weak" options are already passed when
the mode is "legacy". Now the same runtime flags are used as well.

Change-Id: I8610a786adabab4d85f47dc3d0989b280376bece
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/283133
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
2023-02-16 00:11:26 +00:00
Nicholas Shahan b9699971dc [test_runner] Unify name for passing DDC options
Use "ddcOptions" in test files to closer resemble the name 'ddc-options'
from the test matrix or command line option.

I don't like that this option is in camelCase when written in a test
file but uses a dash when passed on the command line but that matches 
all the other implementations (dart2jsOptions, sharedOptionn, etc). 
Hopefully it is the most discoverable or least surprising choice.

Change-Id: I8f0d53d50416b952c9ea74406c18bb9c700f98dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/281820
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
2023-02-14 20:13:57 +00:00
Alexander Thomas b9d8103371 [testing] Use migrated suites by default on non-legacy configurations
Make test.py pick migrated suites by default when the `NnbdMode` is not `NnbdMode.legacy`.

* Remove non-existing benchmark_smoke suite.
* Remove broken analyzer_library suite from default suites.
* Deprecated observatory_ui is not added to the default migrated suites.
* Remove remaining references to samples-dev.
* Remove unnecessary suite specifications from the test_matrix.json.
* Remove broken references to observatory_ui from the test_matrix.json.
* Remove defunct observatory_ui hack from test.py.

Fixes: b/268474066
Change-Id: I6c3635bcb396021ca86498d60c9efc77ab39589e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/281701
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2023-02-10 08:43:27 +00:00
Nicholas Shahan 87d1806055 [test_runner] Add new name for ddc compiler
* The new compiler name "ddc" will be used for all configurations
  in an upcoming change.
* Aliases "dartdevc" and "dartdevk" will be removed in the future.

Change-Id: I2fa48e0ec043b1ba7fb70f883b39e40f338b6928
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280862
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2023-02-09 18:32:58 +00:00
pq ad2da86f18 linter 1.33.0 rc
Some notes.

* There are a number of tests that should get decoupled from the linter being pulled into `DEPS`.  Instead of depending on the state of lints there, we should update the tests instead to use a more hermetic environment.  (See for example `options_rule_validator_test.dart` for how that might look.)


Downstream (blocking) fixes:

* https://github.com/flutter/flutter/pull/119736
* https://github.com/flutter/gallery/pull/878


Change-Id: I5671b0abde3eeda75513abaaf9fef3bcd5115f9b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280054
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Phil Quitslund <pquitslund@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
2023-02-07 20:56:35 +00:00
Nicholas Shahan 5717d08a2d [test_runner] Add ddc-options to compiler configuration
Use explicit ddc-options to pass `--canary` flag to the compiler instead
of sniffing for a builder tag.

This just feels more expected and makes more sense if you are trying
to understand how the canary configurations compile tests differently
from the stable configurations.

Change-Id: I59b1e62d3839f6f992eea917ecbae240640fcd76
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280783
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2023-02-06 18:58:34 +00:00
Daco Harkes 2ef0ee4be6 [tools] test.py support Android devices with . in name
Closes: https://github.com/dart-lang/sdk/issues/51243
Change-Id: Ia165c6ecdc4d4ed6ad2799ae75a8ea68b0110757
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280641
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Auto-Submit: Daco Harkes <dacoharkes@google.com>
2023-02-03 11:03:16 +00:00
Nicholas Shahan 55fba34a8a [test_runner] Add detect-host command line flag
The --detect-host flag is provided as a convenience for running tests 
locally with named configurations. It alters the provided configurations
if needed to match the host system and architecture.

It is an error to use the flag without named configurations or as an 
option in the test_matrix.json file.

Add a the `Configuration._cloneHelper()` constructor with all parameters
as required arguments. This provides some compile time feedback when
new fields are added to the class and the clone methods haven't been 
updated.

Change-Id: I58e2ead1c3324c2cc893a4766d7daccd2787f631
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280139
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
2023-02-03 01:54:55 +00:00