Commit graph

66 commits

Author SHA1 Message Date
Robert Nystrom 99e8a9fba5 Move the test runner (i.e. "test.dart"/"test.py") to pkg/.
This makes it an actual Pub package like most other code inside the SDK
repo. The main goal is to make it easier to write tests for the test
runner itself.

This change:

- Moves all of the code from tools/testing/dart/ over to
  pkg/test_runner. Most of it ends up under test_runner/lib/src.

- Move tools/testing/dart/main.dart to
  pkg/test_runner/bin/test_runner.dart.

- Move standalone_2/io/test_runner_test.dart to
  pkg/test_runner/test/test_runner_test.dart. I don't think it currently
  works, but it wasn't being run in its old location either.

- Add test_runner to the analysis-server bot. This ensures the
  test_runner package is static error clean.

- Remove standalone_2/io/test_runner_analyze_test.dart which used to
  attempt to do the above and is no longer needed.

- Update test.py to look for the test runner at its new location.

- Add test_runner to the repo .packages file and remove the weird
  test_dart pseudo-package. (I think this fixes #35279.)

- Remove status file entries for the removed standalone_2 tests.

There are no code changes to the test runner itself aside from fixing
up import paths.

Change-Id: I3d05d50d222b291848fa5a30de2846e803bc81e6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105821
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2019-06-14 23:35:10 +00:00
Peter von der Ahé f3cee0147c Fast versions of legacy and strong tests.
These versions don't run the VM on the generated .dill files.

These tests are still "Slow" in status-file lingo, as they run for
about 30 seconds on a fast machine.

Change-Id: I7bd326a14c8f1f1905d4c6b4d5f850a95b135325
Reviewed-on: https://dart-review.googlesource.com/c/88967
Reviewed-by: Aske Simon Christensen <askesc@google.com>
2019-01-17 14:23:07 +00:00
Peter von der Ahé 09d2740247 Honor verbose mode when running test directly
Change-Id: Ice090b54789c011e627715c5e6ecb58901b304e3
Reviewed-on: https://dart-review.googlesource.com/c/82995
Commit-Queue: Peter von der Ahé <ahe@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Auto-Submit: Peter von der Ahé <ahe@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2018-11-06 13:39:20 +00:00
Daniel Hillerström 71eaf3209b Fix broken 'fasta testing sdk'.
This CL fixes some bugs and regressions in the 'fasta testing sdk' tool:

  * Plugs the excessive memory leak in suite.dart (thanks to jensj
    for helping tracking it down)
  * Fixes the signature of MultitestTransformer.bind such that it no
    longer causes a type error to happen at runtime.
  * Removes the code that disallows test name "none".
  * Updates the exclude list in testing.json

Change-Id: Ie850139c2228550c9847883d4fdaed839a052200
Reviewed-on: https://dart-review.googlesource.com/76840
Reviewed-by: Peter von der Ahé <ahe@google.com>
Commit-Queue: Daniel Hillerström <hillerstrom@google.com>
2018-09-27 13:08:47 +00:00
Peter von der Ahé a3bb529eec Update package:testing wrt. void
Change-Id: I7bb99b192410c0fc966cff06bea3a938b1d0351b
Reviewed-on: https://dart-review.googlesource.com/65504
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
2018-07-18 10:18:52 +00:00
Alexander Thomas 66c9109caf [infra] Download checked in SDKs from CIPD
#33659

Change-Id: I94fccf4bbce2f0140bf0d6efa953d44b290df856
Reviewed-on: https://dart-review.googlesource.com/62840
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2018-06-28 16:08:14 +00:00
Kevin Millikin 2d46ebd6a5 Fix Dart 2 runtime errors in the front end
Fix all the Dart 2 runtime errors revealed by compiling dart2js, the
front end itself, the front end tests.

Change-Id: Ic6e6dd9f85db845b6a351ebbcfea9a6045843fc2
Reviewed-on: https://dart-review.googlesource.com/56322
Commit-Queue: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
2018-05-24 10:14:20 +00:00
Zach Anderson afb490adbc [dart:io] Provide modern Dart-styled constants
This CL updates dart:io but not dart:_http. It updates the sdk sources,
the patch files, and tests.

Change-Id: I64c3da407f09fa2bc6eec582049c4ae3a8afbe6d
Reviewed-on: https://dart-review.googlesource.com/52990
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Lasse R.H. Nielsen <lrn@google.com>
2018-05-03 17:00:14 +00:00
Peter von der Ahé b2d7397a0c Add testing chain to validate messages.yaml
Change-Id: I34bf0a4bbbd9dab1133a69c844033035b7edb8db
Reviewed-on: https://dart-review.googlesource.com/43969
Commit-Queue: Peter von der Ahé <ahe@google.com>
Reviewed-by: Dan Rubel <danrubel@google.com>
2018-02-28 11:02:23 +00:00
Peter von der Ahé fc5e9938e6 Tweak output from package:testing
* Ensure that chain paths have a trailing slash.
* Don't use ANSI escapes when terminal doesn't support them.

Both changes combined should change the logged output on build bots from running pkg/dev_compiler/test/sourcemap/sourcemaps_ddk_suite.dart from:

<ESC>[2K[ 00:00 |   0.0% | +    0 | -    0 ]: sourcemaps_ddk//breakpoint_on_class_all_in_one_line.unformatte setup<ESC>[1A

To:

[ 00:00 |   0.0% | +    0 | -    0 ]: sourcemaps_ddk/breakpoint_on_class_all_in_one_line.unformatte setup

Change-Id: I0f79e7cefb4e5bcba3002aa49f624cc0a2f7032c
Reviewed-on: https://dart-review.googlesource.com/43402
Commit-Queue: Peter von der Ahé <ahe@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2018-02-23 12:50:04 +00:00
Leaf Petersen 0b2a0ed73c Update package testing for Dart 2.0 core lib changes (wave 2.2)
Change-Id: Ie64d3bcc0609b9d6c36a84815694c4aa71d20851
Reviewed-on: https://dart-review.googlesource.com/37864
Reviewed-by: Emily Fortuna <efortuna@google.com>
Commit-Queue: Leaf Petersen <leafp@google.com>
2018-01-31 21:12:28 +00:00
Johnni Winther 513ac0ffd7 Implement runDiff for Windows
Change-Id: Ic9191e3844e5bcd47c97867f4a7cec52c161fd7a
Reviewed-on: https://dart-review.googlesource.com/35280
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
2018-01-18 12:57:45 +00:00
Peter von der Ahé 1bca76d012 Convert Platform.resolvedExecutable to URI correctly
Change-Id: I583b98277b1c62328885c775fcb712823c9ff405
Reviewed-on: https://dart-review.googlesource.com/34860
Reviewed-by: Alexander Thomas <athom@google.com>
2018-01-16 14:24:52 +00:00
Jens Johansen d8d9bb5613 [pkg testing] Fix off-by-one in cmdline parsing
Running a test via pkg/testing where the command line starts with "--"
would throw a RangeError (it's asking for a range from 0 to -1).
Having the "--" somewhere in the middle of the arguments list would not
throw, but instead "eat" an argument.

Bug:
Change-Id: Ie13052a378746c17b343dff7704002a6929088bf
Reviewed-on: https://dart-review.googlesource.com/21920
Reviewed-by: Peter von der Ahé <ahe@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2017-11-20 11:35:57 +00:00
Jens Johansen 0633d0bfc0 Add cleanup method to pkg/testing
This CL adds a cleanup method to pkg/testing.
This logically allows one to have a "finally" step in a chain to perform
cleanup even upon step failure.

Bug:
Change-Id: I934e97354d733df32f186ebfe370d0307515bebb
Reviewed-on: https://dart-review.googlesource.com/21980
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
2017-11-20 11:10:07 +00:00
Lasse R.H. Nielsen 0b58c4bd10 Change some constant declarations to lowerCase.
Retain the old values.

Reapply of https://dart-review.googlesource.com/c/sdk/+/20680 with fixes
for VM method fingerprints.

Change-Id: Ie14e7ccc3194d5561983348e6b6752728913ff4d
Reviewed-on: https://dart-review.googlesource.com/20664
Reviewed-by: Erik Ernst <eernst@google.com>
Commit-Queue: Lasse R.H. Nielsen <lrn@google.com>
2017-11-14 12:59:14 +00:00
Lasse R.H. Nielsen e61fc07b58 Revert "Change some constant declarations to lowerCase."
This reverts commit c048cfb178.

Reason for revert: VM code fingerprints needs update.

Original change's description:
> Change some constant declarations to lowerCase.
> 
> Retain the old values.
> 
> Change-Id: I03bd326f379fe5f96d8f6081a7133de956d745c0
> Reviewed-on: https://dart-review.googlesource.com/11520
> Commit-Queue: Lasse R.H. Nielsen <lrn@google.com>
> Reviewed-by: Florian Loitsch <floitsch@google.com>

TBR=lrn@google.com,floitsch@google.com

Change-Id: I095e8198304ca4e59b39d30b0fdf0af4945e5231
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/20680
Reviewed-by: Lasse R.H. Nielsen <lrn@google.com>
Commit-Queue: Lasse R.H. Nielsen <lrn@google.com>
2017-11-14 11:23:08 +00:00
Lasse R.H. Nielsen c048cfb178 Change some constant declarations to lowerCase.
Retain the old values.

Change-Id: I03bd326f379fe5f96d8f6081a7133de956d745c0
Reviewed-on: https://dart-review.googlesource.com/11520
Commit-Queue: Lasse R.H. Nielsen <lrn@google.com>
Reviewed-by: Florian Loitsch <floitsch@google.com>
2017-11-14 11:13:04 +00:00
Aske Simon Christensen 306668bf0a Support syntax error outcome in sdk test suite.
Also updated test expectations.

Change-Id: I25fe9f47904beb89b9e18cfafd2e889c6a8106a2
Reviewed-on: https://dart-review.googlesource.com/18800
Reviewed-by: Erik Ernst <eernst@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
2017-11-06 12:22:51 +00:00
Sigmund Cherem 5462885dd8 Process arguments in runMe to extract the environment and selectors
Bug: https://github.com/dart-lang/sdk/issues/31177
Change-Id: I7c3190c95f81fb580bfa6c142d6ea9726548e343
Reviewed-on: https://dart-review.googlesource.com/15765
Reviewed-by: Peter von der Ahé <ahe@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
2017-10-23 21:27:48 +00:00
Aske Simon Christensen 60be83aed4 Reland "Suite for Fasta crashes and verification errors in language tests"
Remove obsolete "language" suite.

Original review: https://dart-review.googlesource.com/13741

Change-Id: I713fa2684fe478f6e9123d33ee7b7d2a961b1c00
Reviewed-on: https://dart-review.googlesource.com/14460
Reviewed-by: Peter von der Ahé <ahe@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
2017-10-17 13:54:40 +00:00
Martin Kustermann 25dc42f5d8 Revert "Error on setter having not exactly one parameter"
This reverts commit 4bb83e9311
This reverts commit f1b07b2789.

Reason for revert on analyzer bots:

    Analyzing pkg/analyzer...
      error • Undefined name 'source' at pkg/analyzer/test/generated/compile_time_error_code_kernel_test.dart:625:9 • undefined_identifier
      error • Undefined name 'CompileTimeErrorCode' at pkg/analyzer/test/generated/compile_time_error_code_kernel_test.dart:625:18 • undefined_identifier
      error • Undefined name 'source' at pkg/analyzer/test/generated/compile_time_error_code_kernel_test.dart:626:13 • undefined_identifier
      error • Undefined name 'source' at pkg/analyzer/test/generated/compile_time_error_code_kernel_test.dart:635:9 • undefined_identifier
      error • Undefined name 'CompileTimeErrorCode' at pkg/analyzer/test/generated/compile_time_error_code_kernel_test.dart:635:18 • undefined_identifier
      error • Undefined name 'source' at pkg/analyzer/test/generated/compile_time_error_code_kernel_test.dart:636:13 • undefined_identifier
      error • Undefined name 'source' at pkg/analyzer/test/generated/compile_time_error_code_kernel_test.dart:645:9 • undefined_identifier
      error • Undefined name 'CompileTimeErrorCode' at pkg/analyzer/test/generated/compile_time_error_code_kernel_test.dart:645:18 • undefined_identifier
      error • Undefined name 'source' at pkg/analyzer/test/generated/compile_time_error_code_kernel_test.dart:646:13 • undefined_identifier
      error • Undefined name 'source' at pkg/analyzer/test/generated/compile_time_error_code_kernel_test.dart:655:9 • undefined_identifier
      error • Undefined name 'CompileTimeErrorCode' at pkg/analyzer/test/generated/compile_time_error_code_kernel_test.dart:655:18 • undefined_identifier
      error • Undefined name 'source' at pkg/analyzer/test/generated/compile_time_error_code_kernel_test.dart:656:13 • undefined_identifier
    12 errors found.

Change-Id: Ie2035bf40e50c48f85b514c7a3786141cd0f2453
Reviewed-on: https://dart-review.googlesource.com/14061
Reviewed-by: Martin Kustermann <kustermann@google.com>
2017-10-16 16:32:13 +00:00
Aske Simon Christensen 4bb83e9311 Suite for Fasta crashes and verification errors in language tests
Remove obsolete "language" suite.

Change-Id: I8daad8b7fd2d6e19124850626a5815aa9dcca347
Reviewed-on: https://dart-review.googlesource.com/13741
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
2017-10-16 16:01:55 +00:00
Leaf Petersen 3124001cc7 Eliminate uses of fuzzy arrows in pkg/testing (fix bots).
The analyzer now issues hints on uses of the deprecated behavior in
which dynamic is treated as bottom in certain positions.  These
changes eliminate cases from pkg/testing that were relying on the
deprecated behavior.

TBR=ahe@google.com

BUG=

Review-Url: https://codereview.chromium.org/3010803002 .
2017-08-30 16:24:56 -07:00
Erik Corry 029b1cb948 Spelling fixes e to i.
R=kevmoo@google.com
BUG=

Review-Url: https://codereview.chromium.org/2957593002 .
2017-06-24 13:41:39 +02:00
Peter von der Ahé 69d6e8e55a All strong mode cleaning of dart2js.
Fixes https://github.com/dart-lang/sdk/issues/29886

Supersedes CLs 2939193003, 2950433002, and 2947583002.

R=johnniwinther@google.com

Review-Url: https://codereview.chromium.org/2944843002 .
2017-06-19 14:06:38 +02:00
Florian Loitsch 71ad537fbf Fix "?." usage in analyze.dart.
Review-Url: https://codereview.chromium.org/2941123005 .
2017-06-16 13:29:15 +02:00
Peter von der Ahé e24970ef40 dart2js isn't strong mode clean yet.
R=johnniwinther@google.com

Review-Url: https://codereview.chromium.org/2939313003 .
2017-06-16 12:21:54 +02:00
Peter von der Ahé dcf133589a Trim the String, not the Future.
R=johnniwinther@google.com

Review-Url: https://codereview.chromium.org/2940653002 .
2017-06-13 16:16:23 +02:00
Peter von der Ahé 964384424a Analyze package:testing.
R=johnniwinther@google.com

Review-Url: https://codereview.chromium.org/2941493002 .
2017-06-13 15:46:04 +02:00
Peter von der Ahé 74b037fbf9 Use Fasta scanner when running analyzer.
R=danrubel@google.com

Review-Url: https://codereview.chromium.org/2925823002 .
2017-06-06 20:06:41 +02:00
Konstantin Shcheglov 433b7904c2 Revert "Fix another warning in pkg/testing."
This reverts commit d935a7fd00.

R=aam@google.com, ahe@google.com
BUG=

Review-Url: https://codereview.chromium.org/2922303002 .
2017-06-05 18:44:34 -07:00
Konstantin Shcheglov d935a7fd00 Fix another warning in pkg/testing.
R=ahe@google.com, paulberry@google.com
BUG=

Review-Url: https://codereview.chromium.org/2925583002 .
2017-06-05 10:32:29 -07:00
Konstantin Shcheglov 64ed9c942c Fix type argument for Result.
R=ahe@google.com, paulberry@google.com
BUG=

Review-Url: https://codereview.chromium.org/2920283002 .
2017-06-05 09:43:07 -07:00
Peter von der Ahé c4481e2384 Add RuntimeError to default expectations and handle expectation groups correctly.
R=johnniwinther@google.com

Review-Url: https://codereview.chromium.org/2913153002 .
2017-05-31 15:02:51 +02:00
Peter von der Ahé 8f1a69826a Fix problems with closures_test.
R=dmitryas@google.com

Review-Url: https://codereview.chromium.org/2898173005 .
2017-05-30 12:10:01 +02:00
Peter von der Ahé ddf5fc23f1 Improve analysis coverage.
R=johnniwinther@google.com

Review-Url: https://codereview.chromium.org/2904483002 .
2017-05-24 15:52:42 +02:00
Peter von der Ahé 547b2cdcce Implement generalized function types.
Fixes https://github.com/dart-lang/sdk/issues/6972

Fixes https://github.com/dart-lang/sdk/issues/26581

R=johnniwinther@google.com

Review-Url: https://codereview.chromium.org/2876813002 .
2017-05-19 11:03:24 +02:00
Paul Berry 8ea1906e54 Handle diagnostic messages from analyzer that lack a URI.
R=sigmund@google.com

Review-Url: https://codereview.chromium.org/2886193003 .
2017-05-17 14:17:49 -07:00
Erik Corry aa6353b6da Dart SDK Spelling b, c, and d.
R=kmillikin@google.com
BUG=

Review-Url: https://codereview.chromium.org/2850783002 .
2017-05-01 08:28:10 +02:00
Erik Corry 2fb5d08102 Spelling a
R=ahe@google.com
BUG=

Review-Url: https://codereview.chromium.org/2841543002 .
2017-04-24 14:53:51 +02:00
Erik Corry 6617737ff5 Replace 'the the' with 'the'
R=kmillikin@google.com
BUG=

Review-Url: https://codereview.chromium.org/2830353002 .
2017-04-24 08:50:37 +02:00
Erik Corry 4b9bba55b4 Stoppp using trippple consonants
R=ahe@google.com
BUG=

Review-Url: https://codereview.chromium.org/2833073002 .
2017-04-21 17:50:13 +02:00
Peter von der Ahé f1a6b12341 Parse escapes in analyzer machine output and handle Windows file names.
R=johnniwinther@google.com

Review-Url: https://codereview.chromium.org/2823293003 .
2017-04-19 14:36:09 +02:00
Paul Berry 84ef5ed2da Add the ability to select a subset of tests using "...".
For example, the following command:

    pkg/front_end/tool/fasta testing compile/type...

Now runs all tests in the "compile" suite whose name begins with
"type".

R=ahe@google.com

Review-Url: https://codereview.chromium.org/2824373002 .
2017-04-19 01:45:34 -07:00
Peter von der Ahé 5a81cba61a Specify location of SDK when running analyzer.
R=whesse@google.com

Review-Url: https://codereview.chromium.org/2794003002 .
2017-04-03 17:47:21 +02:00
Peter von der Ahé 0e82fb44c2 Run analyzer from sources instead of built SDK.
R=johnniwinther@google.com

Review-Url: https://codereview.chromium.org/2790143002 .
2017-04-03 16:13:51 +02:00
Peter von der Ahé 5bdc1199ca Fix dart2js warnings and add test to ensure it stays clean.
Filed https://github.com/dart-lang/sdk/issues/29224 to track all the problems that require ignore.

R=johnniwinther@google.com

Review-Url: https://codereview.chromium.org/2791993002 .
2017-04-03 15:08:40 +02:00
Jacob Richman de54e18369 Run dartfmt on remaining unformated pkg packages
BUG=
R=efortuna@google.com

Review-Url: https://codereview.chromium.org/2743423009 .
2017-03-21 18:21:32 -07:00
Jacob Richman 88eb557b65 Fix warnings_checker.dart handling of multitests
Update all tests

Support //# multitests for better dartfmt compatibility and fewer multitest false positives

All files under tests were manually updated with

find . -iregex '.*\.dart$' -print0 | xargs -0 perl -pi -e 's/(\S\s+)\/\/\/ /$1\/\/# /'

For now both old and new styles are allowed to accommodate CO19 tests.

R=efortuna@google.com
BUG=

Review-Url: https://codereview.chromium.org/2765693002 .
Review-Url: https://codereview.chromium.org/2765893003 .
2017-03-21 17:29:56 -07:00