This improves usability of the tool when a multiple tests needs updating.
Change-Id: I034e3da7381ae79b2149429e5271904e0dfcae59
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210862
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
macOS arm64 builds only create the x64 gen_snapshot by default because
they are cross-compiled in Rosetta. Adding the gen_snapshot target
ensures that we have a gen_snapshot for arm64 as well.
These bots require clang which currently isn't provisioned on shards, so
they are running on the main bot.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-mac-release-arm64-try
Change-Id: I842f37019a65dfb30e522d2d9bead5e62bb3ff79
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/204505
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
No qemu-using CI bot runs vm/cc tests currently, but this allows
developers to run vm/cc tests locally using qemu. This way, tests
like those from assembler_arm_test.cc can be tested both with and
without our own ARM7/ARM8 simulators.
Change-Id: I7fd97e411936954b59d9f0fc65e16c518e9ee332
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/204781
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Error tests with explicit line/column markers, like
[error line 123, column 456]
currently require both "line" and "column".
All existing occurrences of `[error line 123, column 456]` always
refer to the most recent non-comment line *just like* the `// ^^` lines,
so omitting the line, and having the most recent non-comment line
as the *default* line, seems reasonable.
This CL makes the line optional, the new format without a "line" entry
will be just `[error column 456]`.
This allows files to change the number of lines in the code without
invalidating any later expectations.
(Or, say, remove a comment line at the top of the file!)
Omits the line entry by default when writing/updating expectations.
Change-Id: Ia0c95bf804a1c6b11c0c1621bfd481e43f8fa0c0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/200429
Commit-Queue: Lasse R.H. Nielsen <lrn@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
https://github.com/dart-lang/co19/issues/1090
TEST=Standard tryjobs, all changes are in test code only.
Change-Id: Ie73174447ab165827335652cf035f85b7510ab55
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/201564
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Erik Ernst <eernst@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
There are two ways the test runner might want to parse and show errors
from analyzer:
1. When a static error test's output doesn't match the expected errors.
2. When a normal runtime test produces unexpected errors.
The former were correctly migrated over to parse and read the analyzer's
JSON output. The latter wasn't. It was still trying to parse the old
"|"-separated machine format.
Worse, when migrating to the new JSON output, a try-catch-block was
added to handle any unexpected errors. That had the unintended side
effect of silently catching this parse error and then continuing. The
end result is that if an analyzer test failed on a normal runtime test,
the test runner didn't print any of the analyzer errors. :(
This fixes that. It uses a single code path to parse the JSON output to
AnalyzerError. Then, for static error tests, it converts those to
StaticErrors.
Change-Id: I51c1277f0ed04faead82f579d668f8897f379a49
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/201382
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
There was a race between the subprocess listening on the socket and
the test trying to connect.
Also the Android bits of the test harness needed to be taught about
the abstract_socket_test program, similar to how process_test is
handled.
TEST=Fixes tests
Change-Id: I5e1c5d2e23e4bff2a01cd5c491a45e1c58d2578e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/201861
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
Change-Id: I9fc656bddae68767edd79b768abbf679c86efd6e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/199421
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Karl Klose <karlklose@google.com>
This is a reland of 4e34ea06a9TBR=athom@google.com
Original change's description:
> [pkg/test_runner] Write experiments to results.json
>
> Change-Id: I32423ddf665a24443de1ad60ca8f323f5c6d3aca
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196489
> Reviewed-by: Alexander Thomas <athom@google.com>
> Commit-Queue: Karl Klose <karlklose@google.com>
Change-Id: I894616f77cb47df25c8da46d2bf6ddb87734db41
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/198404
Reviewed-by: Karl Klose <karlklose@google.com>
Commit-Queue: Karl Klose <karlklose@google.com>
This reverts commit 4e34ea06a9.
Reason for revert: The data does not show up in bigquery
Original change's description:
> [pkg/test_runner] Write experiments to results.json
>
> Change-Id: I32423ddf665a24443de1ad60ca8f323f5c6d3aca
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196489
> Reviewed-by: Alexander Thomas <athom@google.com>
> Commit-Queue: Karl Klose <karlklose@google.com>
TBR=karlklose@google.com,athom@google.com
Change-Id: Ifccdf3a6031dd55bebfe5d282bcce21c5ab5632a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/198403
Reviewed-by: Karl Klose <karlklose@google.com>
Commit-Queue: Karl Klose <karlklose@google.com>
Change-Id: I32423ddf665a24443de1ad60ca8f323f5c6d3aca
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196489
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Karl Klose <karlklose@google.com>
A multitest section marked "dynamic type error" or "checked mode
compile time error" didn't actually do anything. It was silently treated
as "ok", which makes for a very confusing looking test.
Fixed the only remaining four tests that used "dynamic type error".
The other outcome was not used by any test.
Change-Id: I9727b3b524cf1effb0dd899bf206aa65dbd60803
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/198180
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Nate Bosch <nbosch@google.com>
Reviewed-by: Nate Bosch <nbosch@google.com>
See: https://dart-review.googlesource.com/c/sdk/+/196026
TEST=Code cleanup exclusively (sorting imports); no new tests.
Change-Id: Ib07a82ff418138c542d6a83cfab9aabbb285f866
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196180
Auto-Submit: Phil Quitslund <pquitslund@google.com>
Reviewed-by: Phil Quitslund <pquitslund@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Phil Quitslund <pquitslund@google.com>
The CFE implementation of "why not promoted" functionality for
non-promotion of `this` doesn't associate the context message with any
location information, because there is no relevant location to cite.
For example, the output can look like this:
tests/language/why_not_promoted/this_error_test.dart:16:10: Error: Property 'isEven' cannot be accessed on 'int?' because it is potentially null.
Try accessing using ?. instead.
this.isEven;
^^^^^^
Context: 'this' can't be promoted.
See http://dart.dev/go/non-promo-this
The test runner assumes that all messages have a location, so it
wasn't picking up on this context message at all. This CL avoids the
problem by having the test runner associate any location-less context
message with the error above it.
(Note that the analyzer doesn't have this problem; all of its context
messages have locations).
Change-Id: Ied52daa8b0090f28617e7d3784233aa44dcc897a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195301
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
* Migrate to python3; drop python support.
* Update Windows toolchain support.
* Remove some unused methods.
* Python 2.7 is still needed on Windows.
* Update gsutil to a version that supports python3.
Fixes: https://github.com/dart-lang/sdk/issues/28793
TEST=Manually tested common user journeys.
Change-Id: I663a22b237a548bb82dc2e601e399e3bc3649211
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192182
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
I'm not sure if the output truncation was ever practically useful. But
it causes problems now that we have a CFE test that wants to validate
that the CFE can produce many errors in one go.
I thought about allowing a test to extend the truncation size, but I
figured the simplest fix was to just not truncate at all and make the
test runner a little simpler. Also did a little clean up on this
ancient code while I was at it.
Fix#44493.
Change-Id: Ie678c80e7114e0fb7f68931335075e3d319fb53a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178940
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
This allows the test runner to support context messages, for example
the context error messages associated with the new "why not promoted"
feature.
Bug: https://github.com/dart-lang/sdk/issues/44905
Change-Id: Ie342bde21c43641eafc5d117f328e9fde23c49bc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193740
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
It can sometimes happen that multiple error messages point to equal
context messages. To prevent the static error test runner from
getting them mixed up, we need to use identity maps to track the
mapping from error message to context number.
Change-Id: I6d03484a37519d5db1f92750273daf898cb4381a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193527
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
I apparently broke this four years ago (in
https://codereview.chromium.org/2919573003) and never realized. Caught
it today because of a lint warning.
Change-Id: I00d71ccde581e685620800b603b0ffb9e8e3a1b7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192951
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Devon Carew <devoncarew@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Currently only CFE ("Fasta") tests have their context message output
parsed. It should be easy to extend that to dart2js and DDC if that's
useful. Analyzer might be more work.
This also adds support to the test updater for inserting context
messages when updating tests. By default, that flag is off, so the
existing behavior is preserved where context messages are ignnored. If
you want them, pass "-c" when updating a test.
When validating test output, if the test file contains context messages,
then they are validated. Otherwise, any context messages in the CFE
output are ignored. This way existing tests still pass.
Change StaticError to represent a single error for a single front end.
Before, the data model collapsed errors for different front-ends at the
same location into a single StaticError object which tracked different
messages for each front end. The idea was to move towards a world where
they really are the "same" error with eventually the same message.
But this adds a lot of complexity with things like merging errors and
doesn't reflect the reality that each error from each front end is
basically its own thing. Also, critically, it makes it much harder to
attach context messages to a specific front end's error object.
This changes it so that an instance of StaticError represents a single
error for a single front end. The test file syntax is unchanged and the
updated tool behaves the same. In a static error test, multiple
expectations can still share the same "// ^^^" marker line. They are
just expanded to multiple StaticError objects at parse time.
This eliminates all of the complexity around merging and simplifying
errors.
Change-Id: Ida1736bfcde436fc2d1ce2963d91fa9cb154afa8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193281
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
For some reason, the regexp to strip off multitest comments was very
slow. On a couple of co_19 tests with pathologically long lines, it
would hang practically forever. Even on shorter lines, it was noticeably
slow. This fixes that.
Change-Id: I04f2894f474dcc593e982dd691945421396274a6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193222
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
Before, the data model collapsed errors for different front-ends at the
same location into a single StaticError object which tracked different
messages for each front end. The idea was to move towards a world where
they really are the "same" error with eventually the same message.
But this adds a lot of complexity with things like merging errors and
doesn't reflect the reality that each error from each front end is
basically its own thing. Also, critically, it makes it much harder to
attach context messages to a specific front end's error object.
This changes it so that an instance of StaticError represents a single
error for a single front end. The test file syntax is unchanged and the
updated tool behaves the same. In a static error test, multiple
expectations can still share the same "// ^^^" marker line. They are
just expanded to multiple StaticError objects at parse time.
This eliminates all of the complexity around merging and simplifying
errors.
Change-Id: I1d55a6e885e12cc9c438f928297fc0db7dd5ce85
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193280
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
Follow up to 3b64509 to apply post-submit review comments.
Change-Id: I4a573312777fea4701abc3b3c9efbb2f8b7971ee
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192929
Auto-Submit: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: William Hesse <whesse@google.com>
The buildbucket v1 API is deprecated and may stop working at any time.
This also fixes the buildset tag to use the latest format.
Change-Id: I7fe889c1293b002d65a201a8ea4c9d75b9489074
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192927
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Allows removing a branch when decompressing pointers and removing fixup sign-extension after Smi ops that use 32-bit ops to detect overflow. Increases the code size of indexed and stores for some widths due to addressing mode limitations.
TEST=ci
Change-Id: Ia48353cccbbc586dd0d2e055a843c65e37c63a30
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185660
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
This makes the TestRunner's ADB class able to recognise TCP/IP ADB
devices, e.g. "localhost:1234 device".
TEST=Tested 'tools/test.py -n dartk-android-release-arm ..' with
remote device.
Change-Id: I1ee8d27736e30c61cf057e0bc34a75667a0f9bb0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184465
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
Change-Id: Ibfcad1997c79b1839b1842a1b7f7b14b328c3599
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183720
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
- Added support for --define to the VM and dart2js
- Added support for -D and --define for `dart run` and `dart compile js`
Remaining improvements:
- Add support for providing multiple comma separated values for `dart
run`, `dart`, and `dart2js`
Related issue: https://github.com/dart-lang/sdk/issues/44562
TEST=Updated CLI tests and added new dart2js tests.
This reverts commit e49937769f.
Change-Id: I5f9275b829665eb5e8695403d67f230e752ab0e6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183180
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
The directories where generated tests and compilation output are stored
by the test runner were named based on all the relevant configuration
options. Change this so that the directory path is based on the
name of the tested configuration instead. This allows simultaneous
test runs of configurations that don't differ on relevant options.
Change-Id: Iaad8d0212cb900bc91d7472d99e266183d29a0b2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178986
Reviewed-by: Karl Klose <karlklose@google.com>
Commit-Queue: William Hesse <whesse@google.com>
This change adds support for the 'enable-experiments' option to control
the experiments used in the analyzer and CFE.
It also adds support for multiple input files.
Change-Id: I15b9082e96f276249165efc66734ac9506b93ca1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178560
Commit-Queue: Karl Klose <karlklose@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
