mirror of
https://github.com/dart-lang/sdk
synced 2024-09-04 16:03:44 +00:00
Move package:testing to SDK.
R=karlklose@google.com Review-Url: https://codereview.chromium.org/2624373003 .
This commit is contained in:
Peter von der Ahé
parent
1be77216f1
commit
f2faa44730
5
DEPS
5
DEPS
|
|
@ -27,8 +27,6 @@ vars = {
|
|||
# Only use this temporarily while waiting for a mirror for a new package.
|
||||
"github_dartlang": "https://github.com/dart-lang/%s.git",
|
||||
|
||||
"github_testing": "https://github.com/peter-ahe-google/testing.git",
|
||||
|
||||
"gyp_rev": "@6ee91ad8659871916f9aa840d42e1513befdf638",
|
||||
"co19_rev": "@cf831f58ac65f68f14824c0b1515f6b7814d94b8",
|
||||
|
||||
|
|
@ -111,7 +109,6 @@ vars = {
|
|||
"stream_channel_tag": "@1.5.0",
|
||||
"string_scanner_tag": "@1.0.0",
|
||||
"sunflower_rev": "@879b704933413414679396b129f5dfa96f7a0b1e",
|
||||
"testing_rev": "@2e196d51c147411a93a949109656be93626bda49",
|
||||
"test_reflective_loader_tag": "@0.1.0",
|
||||
"test_tag": "@0.12.15+6",
|
||||
"typed_data_tag": "@1.1.3",
|
||||
|
|
@ -318,8 +315,6 @@ deps = {
|
|||
Var("sunflower_rev"),
|
||||
Var("dart_root") + "/third_party/pkg/test":
|
||||
(Var("github_mirror") % "test") + Var("test_tag"),
|
||||
Var("dart_root") + "/third_party/testing":
|
||||
Var("github_testing") + Var("testing_rev"),
|
||||
Var("dart_root") + "/third_party/pkg/test_reflective_loader":
|
||||
(Var("github_mirror") % "test_reflective_loader") +
|
||||
Var("test_reflective_loader_tag"),
|
||||
|
|
|
|||
265
pkg/testing/README.md
Normal file
265
pkg/testing/README.md
Normal file
|
|
@ -0,0 +1,265 @@
|
|||
<!--
|
||||
Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
for details. All rights reserved. Use of this source code is governed by a
|
||||
BSD-style license that can be found in the LICENSE file.
|
||||
-->
|
||||
# Test Infrastructure without Batteries
|
||||
|
||||
This package:
|
||||
|
||||
* Provides a way to test a compiler in multiple steps.
|
||||
|
||||
* Provides a way to run standalone tests. A standalone test is a test that has a `main` method, and can be run as a standalone program.
|
||||
|
||||
* Ensures all tests and implementations are free of warnings (using dartanalyzer).
|
||||
|
||||
This package is not:
|
||||
|
||||
* A replacement for `package:test`. This package can be used to run `package:test` tests, and `package:test` can be viewed as the batteries that aren't included in this package.
|
||||
|
||||
## Motivation
|
||||
|
||||
We want to test tool chains, for example, a Dart compiler. Depending on the tool chain, it may comprise several individual steps. For example, to test dart2js, you have these steps:
|
||||
|
||||
1. Run dart2js on a Dart source file to produce a Javascript output file.
|
||||
|
||||
2. Run the Javascript file from step 1 on a Javascript interpreter and report if the program threw an exception.
|
||||
|
||||
On the other hand, to test a Dart VM, there's only one step:
|
||||
|
||||
1. Run the Dart source file in the Dart VM and report if the program threw an exception.
|
||||
|
||||
Similarly, to test dartanalyzer, there's also a single step:
|
||||
|
||||
1. Analyze the Dart source file and report if there were any problems.
|
||||
|
||||
In general, a tool chain can have more steps, for example, a pub transformer.
|
||||
|
||||
Furthermore, multiple tool chains may share the input sources and should agree on the behavior. For example, you should be able to compile `hello-world.dart` with dart2js and run it on d8 and it shouldn't throw an exception, running `hello-world.dart` on the Dart VM shouldn't throw an exception, and analysing it with dartanalyzer should report nothing.
|
||||
|
||||
In addition, parts of the tool chain may have been implemented in Dart and have unit tests written in Dart, for example, using [package:test](https://github.com/dart-lang/test). We want to run these unit tests, and have noticed that compiler unit tests in general run faster when run from the same Dart VM process (due to dynamic optimizations kicking in). For this reason, it's convenient to have a single Dart program that runs all tests. On the other hand, when developing, it's often convenient to run just a single test.
|
||||
|
||||
For this reason, we want to support running unit tests individually, or combined in one program. And we also want the Dart-based implementation to be free of problems with respect to dartanalyzer.
|
||||
|
||||
## Test Suites
|
||||
|
||||
A test suite is a collection of tests. Based on the above motivation, we have two kinds of suites:
|
||||
|
||||
1. [Chain](#Chain), a test suite for tool chains.
|
||||
|
||||
2. [Dart](#Dart), a test suite for Dart-based unit tests.
|
||||
|
||||
## Getting Started
|
||||
|
||||
1. Create a [configuration file](#Configuration) named `testing.json`.
|
||||
|
||||
2. Run `bin/run_tests.dart`.
|
||||
|
||||
## Configuration
|
||||
|
||||
The test runner is configured using a JSON file. A minimal configuration file is:
|
||||
|
||||
```json
|
||||
{
|
||||
}
|
||||
```
|
||||
|
||||
### Chain
|
||||
|
||||
A `Chain` suite is a suite that's designed to test a tool chain and can be used to test anything that can be divided into one or more steps.
|
||||
|
||||
Here a complete example of a `Chain` suite:
|
||||
|
||||
```json
|
||||
{
|
||||
"suites": [
|
||||
{
|
||||
"name": "golden",
|
||||
"kind": "Chain",
|
||||
"source": "test/golden_suite.dart",
|
||||
"path": "test/golden/",
|
||||
"status": "test/golden.status",
|
||||
"pattern": [
|
||||
"\\.dart$"
|
||||
],
|
||||
"exclude": [
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
The properties of a `Chain` suite are:
|
||||
|
||||
*name*: a name for the suite. For simple packages, `test` or the package name are good candidates. In the Dart SDK, for example, it would be things like `language`, `corelib`, etc.
|
||||
|
||||
*kind*: always `Chain` for this kind of suite.
|
||||
|
||||
*source*: a relative URI to a Dart program that implements the steps in the suite. See [below](#Implementing-a-Chain-Suite).
|
||||
|
||||
*path*: a URI relative to the configuration file which is the root directory of all files in this suite. For now, only file URIs are supported. Each file is passed to the first step in the suite.
|
||||
|
||||
*status*: a URI relative to the configuration file which lists the status of tests.
|
||||
|
||||
*pattern*: a list of regular expressions that match file names that are tests.
|
||||
|
||||
*exclude*: a list of regular expressions that exclude files from being included in this suite.
|
||||
|
||||
#### Implementing a Chain Suite
|
||||
|
||||
The `source` property of a `Chain` suite is a Dart program that must provide a top-level method with this name and signature:
|
||||
|
||||
```dart
|
||||
Future<ChainContext> createContext(Chain suite) async { ... }
|
||||
```
|
||||
|
||||
A suite is expected to implement a subclass of `ChainContext` which defines the steps that make up the chain and return it from `createContext`.
|
||||
|
||||
A step is a subclass of `Step`. The input to the first step is a `TestDescription`. The input to step n+1 is the output of step n.
|
||||
|
||||
Here is an example of a suite that runs tests on the Dart VM:
|
||||
|
||||
```dart
|
||||
import 'testing.dart';
|
||||
|
||||
Future<ChainContext> createContext(
|
||||
Chain suite, Map<String, String> enviroment) async {
|
||||
return new VmContext();
|
||||
}
|
||||
|
||||
class VmContext extends ChainContext {
|
||||
final List<Step> steps = const <Step>[const DartVmStep()];
|
||||
}
|
||||
|
||||
class DartVmStep extends Step<TestDescription, int, VmContext> {
|
||||
const DartVmStep();
|
||||
|
||||
String get name => "Dart VM";
|
||||
|
||||
Future<Result<int>> run(TestDescription input, VmContext context) async {
|
||||
StdioProcess process = await StdioProcess.run("dart", [input.file.path]);
|
||||
return process.toResult();
|
||||
}
|
||||
}
|
||||
|
||||
main(List<String> arguments) => runMe(arguments, createContext);
|
||||
```
|
||||
|
||||
An example with multiple steps in the chain can be found in the Kernel package's [suite](https://github.com/dart-lang/kernel/blob/closure_conversion/test/closures/suite.dart). Notice how this suite stores an `AnalysisContext` in its `TestContext` and is this way able to reuse the same `AnalysisContext` in all tests.
|
||||
|
||||
### Dart
|
||||
|
||||
The `Dart` suite is for running unit tests written in Dart. Each test is a Dart program with a main method that can be run directly from the command line.
|
||||
|
||||
The suite generates a new Dart program which combines all the tests included in the suite, so they can all be run (in sequence) in the same process. Such tests must be co-operative and must clean up after themselves.
|
||||
|
||||
You can use any test-framework, for example, `package:test` in these individual programs, as long as the frameworks are well-behaved with respect to global state, see [below](#Well-Behaved-Tests).
|
||||
|
||||
Here is a complete example of a `Dart` suite:
|
||||
|
||||
```json
|
||||
{
|
||||
"suites": [
|
||||
{
|
||||
"name": "my-package",
|
||||
"path": "test/",
|
||||
"kind": "Dart",
|
||||
"pattern": [
|
||||
"_test\\.dart$"
|
||||
],
|
||||
"exclude": [
|
||||
"/test/golden/"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
The properties of a `Dart` suite are:
|
||||
|
||||
*name*: a name for the suite. For simple packages, `test` or the package name are good candidates. In the Dart SDK, for example, the names could be the name of the component that's tested by this suite's unit tests, for example, `dart2js`.
|
||||
|
||||
*path*: a URI relative to the configuration file which is the root directory of all files in this suite. For now, only file URIs are supported.
|
||||
|
||||
*kind*: always `Dart` for this kind of suite.
|
||||
|
||||
*pattern*: a list of regular expressions that match file names that are tests.
|
||||
|
||||
*exclude*: a list of regular expressions that exclude files from being included in this suite.
|
||||
|
||||
#### Well-Behaved Tests
|
||||
|
||||
The `Dart` suite makes certain assumptions about the tests it runs.
|
||||
|
||||
* All tests use the same packages configuration file.
|
||||
|
||||
* An asynchronous test returns a `Future` from its `main`.
|
||||
|
||||
* Tests manages global state.
|
||||
|
||||
All tests are imported into the same program as individual libraries, which is why they all must use the same `.packages` file. The tests aren't concatenated, so they have the lexical scope you'd normally expect from a Dart library.
|
||||
|
||||
Tests are run in order. In particular, the test framework will not start the next test until any future returned from the current test's `main` method complete. In addition, asynchronous tests are expected to have finished all asynchronous operations when the future returned from their `main` method completes (with or without an error).
|
||||
|
||||
Tests are expected to manage global state (aka static state). Simply put: clean up after yourself. But if it's simpler to ensure global state is reset before running a test and not clean up afterwards, that's also fine as long as all tests agree on how to manage their shared global state.
|
||||
|
||||
### Configuring Analyzed Programs
|
||||
|
||||
By default, all tests in `Dart` suites are analyzed by the `dartanalyzer`. It is possible to exclude tests from analysis, and it's possible to add additional files to be analyzed. Here is a complete example of a `Dart` suite and analyzer configuration:
|
||||
|
||||
```json
|
||||
{
|
||||
"suites": [
|
||||
{
|
||||
"name": "my-package",
|
||||
"path": "test/",
|
||||
"kind": "Dart",
|
||||
"pattern": [
|
||||
"_test\\.dart$"
|
||||
],
|
||||
"exclude": [
|
||||
"/test/golden/"
|
||||
]
|
||||
}
|
||||
],
|
||||
"analyze": {
|
||||
"uris": [
|
||||
"lib/",
|
||||
],
|
||||
"exclude": [
|
||||
"/third_party/"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The properties of the `analyze` section are:
|
||||
|
||||
*uris*: a list of URIs relative to the configuration file that should also be analyzed. For now, only file URIs are supported.
|
||||
|
||||
*exclude*: a list of regular expression that matches file names that should be excluded from analysis. For now, the files are still analyzed but diagnostics are suppressed and ignored.
|
||||
|
||||
## Integrating with Other Test Runners
|
||||
|
||||
### `test.dart`
|
||||
|
||||
To run the suite `my_suite` from `test.dart`, create a file named `mysuite_test.dart` with this content:
|
||||
|
||||
import 'package:async_helper/async_helper.dart' show asyncTest;
|
||||
|
||||
import 'package:testing/testing.dart' show run;
|
||||
|
||||
main(List<String> arguments) => asyncTest(run(arguments, ["my_suite"]));
|
||||
|
||||
### `package:test`
|
||||
|
||||
To run the suite `my_suite` from `package:test`, create a file named `mysuite_test.dart` with this content:
|
||||
|
||||
import 'package:test/test.dart' show Timeout, test;
|
||||
|
||||
import 'package:testing/testing.dart' show run;
|
||||
|
||||
main() {
|
||||
test("my_suite", () => run([], ["my_suite"]),
|
||||
timeout: new Timeout(new Duration(minutes: 5)));
|
||||
}
|
||||
8
pkg/testing/bin/run_tests.dart
Executable file
8
pkg/testing/bin/run_tests.dart
Executable file
|
|
@ -0,0 +1,8 @@
|
|||
#!/usr/bin/env dart -c
|
||||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
import "package:testing/src/run_tests.dart" as run_tests;
|
||||
|
||||
main(List<String> arguments) => run_tests.main(arguments);
|
||||
7
pkg/testing/bin/testing.dart
Normal file
7
pkg/testing/bin/testing.dart
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
import "package:testing/src/run_tests.dart" as run_tests;
|
||||
|
||||
main(List<String> arguments) => run_tests.main(arguments);
|
||||
29
pkg/testing/lib/dart_vm_suite.dart
Normal file
29
pkg/testing/lib/dart_vm_suite.dart
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
library testing.dart_vm_suite;
|
||||
|
||||
import 'testing.dart';
|
||||
|
||||
Future<ChainContext> createContext(
|
||||
Chain suite, Map<String, String> enviroment) async {
|
||||
return new VmContext();
|
||||
}
|
||||
|
||||
class VmContext extends ChainContext {
|
||||
final List<Step> steps = const <Step>[const DartVmStep()];
|
||||
}
|
||||
|
||||
class DartVmStep extends Step<TestDescription, int, VmContext> {
|
||||
const DartVmStep();
|
||||
|
||||
String get name => "Dart VM";
|
||||
|
||||
Future<Result<int>> run(TestDescription input, VmContext context) async {
|
||||
StdioProcess process = await StdioProcess.run("dart", [input.file.path]);
|
||||
return process.toResult();
|
||||
}
|
||||
}
|
||||
|
||||
main(List<String> arguments) => runMe(arguments, createContext);
|
||||
150
pkg/testing/lib/src/analyze.dart
Normal file
150
pkg/testing/lib/src/analyze.dart
Normal file
|
|
@ -0,0 +1,150 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
library testing.analyze;
|
||||
|
||||
import 'dart:async' show
|
||||
Stream,
|
||||
Future;
|
||||
|
||||
import 'dart:convert' show
|
||||
LineSplitter,
|
||||
UTF8;
|
||||
|
||||
import 'dart:io' show
|
||||
File,
|
||||
Process;
|
||||
|
||||
import '../testing.dart' show
|
||||
dartSdk;
|
||||
|
||||
import 'log.dart' show
|
||||
isVerbose;
|
||||
|
||||
import 'suite.dart' show
|
||||
Suite;
|
||||
|
||||
class Analyze extends Suite {
|
||||
final List<Uri> uris;
|
||||
|
||||
final List<RegExp> exclude;
|
||||
|
||||
Analyze(this.uris, this.exclude)
|
||||
: super("analyze", "analyze", null);
|
||||
|
||||
Future<Null> run(Uri packages, List<Uri> extraUris) {
|
||||
List<Uri> allUris = new List<Uri>.from(uris);
|
||||
if (extraUris != null) {
|
||||
allUris.addAll(extraUris);
|
||||
}
|
||||
return analyzeUris(packages, allUris, exclude);
|
||||
}
|
||||
|
||||
static Future<Analyze> fromJsonMap(
|
||||
Uri base, Map json, List<Suite> suites) async {
|
||||
List<Uri> uris = new List<Uri>.from(
|
||||
json["uris"].map((String relative) => base.resolve(relative)));
|
||||
|
||||
List<RegExp> exclude =
|
||||
new List<RegExp>.from(json["exclude"].map((String p) => new RegExp(p)));
|
||||
|
||||
return new Analyze(uris, exclude);
|
||||
}
|
||||
|
||||
String toString() => "Analyze($uris, $exclude)";
|
||||
}
|
||||
|
||||
class AnalyzerDiagnostic {
|
||||
final String kind;
|
||||
|
||||
final String detailedKind;
|
||||
|
||||
final String code;
|
||||
|
||||
final Uri uri;
|
||||
|
||||
final int line;
|
||||
|
||||
final int startColumn;
|
||||
|
||||
final int endColumn;
|
||||
|
||||
final String message;
|
||||
|
||||
AnalyzerDiagnostic(this.kind, this.detailedKind, this.code, this.uri,
|
||||
this.line, this.startColumn, this.endColumn, this.message);
|
||||
|
||||
factory AnalyzerDiagnostic.fromLine(String line) {
|
||||
List<String> parts = line.split("|");
|
||||
if (parts.length != 8) {
|
||||
throw "Malformed output: $line";
|
||||
}
|
||||
return new AnalyzerDiagnostic(parts[0], parts[1], parts[2],
|
||||
Uri.base.resolve(parts[3]),
|
||||
int.parse(parts[4]), int.parse(parts[5]), int.parse(parts[6]),
|
||||
parts[7]);
|
||||
}
|
||||
|
||||
String toString() {
|
||||
return "$uri:$line:$startColumn: "
|
||||
"${kind == 'INFO' ? 'warning: hint' : kind.toLowerCase()}:\n$message";
|
||||
}
|
||||
}
|
||||
|
||||
Stream<AnalyzerDiagnostic> parseAnalyzerOutput(
|
||||
Stream<List<int>> stream) async* {
|
||||
Stream<String> lines =
|
||||
stream.transform(UTF8.decoder).transform(new LineSplitter());
|
||||
await for (String line in lines) {
|
||||
yield new AnalyzerDiagnostic.fromLine(line);
|
||||
}
|
||||
}
|
||||
|
||||
/// Run dartanalyzer on all tests in [uris].
|
||||
Future<Null> analyzeUris(
|
||||
Uri packages, List<Uri> uris, List<RegExp> exclude) async {
|
||||
if (uris.isEmpty) return;
|
||||
const String analyzerPath = "bin/dartanalyzer";
|
||||
Uri analyzer = dartSdk.resolve(analyzerPath);
|
||||
if (!await new File.fromUri(analyzer).exists()) {
|
||||
throw "Couldn't find '$analyzerPath' in '${dartSdk.toFilePath()}'";
|
||||
}
|
||||
List<String> arguments = <String>[
|
||||
"--packages=${packages.toFilePath()}",
|
||||
"--package-warnings",
|
||||
"--format=machine",
|
||||
];
|
||||
arguments.addAll(uris.map((Uri uri) => uri.toFilePath()));
|
||||
if (isVerbose) {
|
||||
print("Running:\n ${analyzer.toFilePath()} ${arguments.join(' ')}");
|
||||
} else {
|
||||
print("Running dartanalyzer.");
|
||||
}
|
||||
Stopwatch sw = new Stopwatch()..start();
|
||||
Process process = await Process.start(analyzer.toFilePath(), arguments);
|
||||
process.stdin.close();
|
||||
Future stdoutFuture = parseAnalyzerOutput(process.stdout).toList();
|
||||
Future stderrFuture = parseAnalyzerOutput(process.stderr).toList();
|
||||
await process.exitCode;
|
||||
List<AnalyzerDiagnostic> diagnostics = <AnalyzerDiagnostic>[];
|
||||
diagnostics.addAll(await stdoutFuture);
|
||||
diagnostics.addAll(await stderrFuture);
|
||||
bool hasOutput = false;
|
||||
Set<String> seen = new Set<String>();
|
||||
for (AnalyzerDiagnostic diagnostic in diagnostics) {
|
||||
String path = diagnostic.uri.path;
|
||||
if (exclude.any((RegExp r) => path.contains(r))) continue;
|
||||
String message = "$diagnostic";
|
||||
if (seen.add(message)) {
|
||||
hasOutput = true;
|
||||
print(message);
|
||||
}
|
||||
}
|
||||
if (hasOutput) {
|
||||
throw "Non-empty output from analyzer.";
|
||||
}
|
||||
sw.stop();
|
||||
print("Running analyzer took: ${sw.elapsed}.");
|
||||
}
|
||||
|
||||
330
pkg/testing/lib/src/chain.dart
Normal file
330
pkg/testing/lib/src/chain.dart
Normal file
|
|
@ -0,0 +1,330 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
library testing.chain;
|
||||
|
||||
import 'dart:async' show
|
||||
Future,
|
||||
Stream;
|
||||
|
||||
import 'dart:convert' show
|
||||
JSON,
|
||||
JsonEncoder;
|
||||
|
||||
import 'dart:io' show
|
||||
Directory,
|
||||
File,
|
||||
FileSystemEntity,
|
||||
exitCode;
|
||||
|
||||
import 'suite.dart' show
|
||||
Suite;
|
||||
|
||||
import '../testing.dart' show
|
||||
TestDescription;
|
||||
|
||||
import 'test_dart/status_file_parser.dart' show
|
||||
Expectation,
|
||||
ReadTestExpectations,
|
||||
TestExpectations;
|
||||
|
||||
import 'zone_helper.dart' show
|
||||
runGuarded;
|
||||
|
||||
import 'error_handling.dart' show
|
||||
withErrorHandling;
|
||||
|
||||
import 'log.dart' show
|
||||
logMessage,
|
||||
logStepComplete,
|
||||
logStepStart,
|
||||
logSuiteComplete,
|
||||
logTestComplete,
|
||||
logUnexpectedResult,
|
||||
splitLines;
|
||||
|
||||
import 'multitest.dart' show
|
||||
MultitestTransformer;
|
||||
|
||||
typedef Future<ChainContext> CreateContext(
|
||||
Chain suite, Map<String, String> environment);
|
||||
|
||||
/// A test suite for tool chains, for example, a compiler.
|
||||
class Chain extends Suite {
|
||||
final Uri source;
|
||||
|
||||
final Uri uri;
|
||||
|
||||
final List<RegExp> pattern;
|
||||
|
||||
final List<RegExp> exclude;
|
||||
|
||||
final bool processMultitests;
|
||||
|
||||
Chain(String name, String kind, this.source, this.uri, Uri statusFile,
|
||||
this.pattern, this.exclude, this.processMultitests)
|
||||
: super(name, kind, statusFile);
|
||||
|
||||
factory Chain.fromJsonMap(
|
||||
Uri base, Map json, String name, String kind) {
|
||||
Uri source = base.resolve(json["source"]);
|
||||
Uri uri = base.resolve(json["path"]);
|
||||
Uri statusFile = base.resolve(json["status"]);
|
||||
List<RegExp> pattern = new List<RegExp>.from(
|
||||
json["pattern"].map((String p) => new RegExp(p)));
|
||||
List<RegExp> exclude = new List<RegExp>.from(
|
||||
json["exclude"].map((String p) => new RegExp(p)));
|
||||
bool processMultitests = json["process-multitests"] ?? false;
|
||||
return new Chain(
|
||||
name, kind, source, uri, statusFile, pattern, exclude, processMultitests);
|
||||
}
|
||||
|
||||
void writeImportOn(StringSink sink) {
|
||||
sink.write("import '");
|
||||
sink.write(source);
|
||||
sink.write("' as ");
|
||||
sink.write(name);
|
||||
sink.writeln(";");
|
||||
}
|
||||
|
||||
void writeClosureOn(StringSink sink) {
|
||||
sink.write("await runChain(");
|
||||
sink.write(name);
|
||||
sink.writeln(".createContext, environment, selectors, r'''");
|
||||
const String jsonExtraIndent = " ";
|
||||
sink.write(jsonExtraIndent);
|
||||
sink.writeAll(splitLines(new JsonEncoder.withIndent(" ").convert(this)),
|
||||
jsonExtraIndent);
|
||||
sink.writeln("''');");
|
||||
}
|
||||
|
||||
Map toJson() {
|
||||
return {
|
||||
"name": name,
|
||||
"kind": kind,
|
||||
"source": "$source",
|
||||
"path": "$uri",
|
||||
"status": "$statusFile",
|
||||
"process-multitests": processMultitests,
|
||||
"pattern": []..addAll(pattern.map((RegExp r) => r.pattern)),
|
||||
"exclude": []..addAll(exclude.map((RegExp r) => r.pattern)),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
abstract class ChainContext {
|
||||
const ChainContext();
|
||||
|
||||
List<Step> get steps;
|
||||
|
||||
Future<Null> run(Chain suite, Set<String> selectors) async {
|
||||
TestExpectations expectations = await ReadTestExpectations(
|
||||
<String>[suite.statusFile.toFilePath()], {});
|
||||
Stream<TestDescription> stream = list(suite);
|
||||
if (suite.processMultitests) {
|
||||
stream = stream.transform(new MultitestTransformer());
|
||||
}
|
||||
List<TestDescription> descriptions = await stream.toList();
|
||||
descriptions.sort();
|
||||
Map<TestDescription, Result> unexpectedResults =
|
||||
<TestDescription, Result>{};
|
||||
Map<TestDescription, Set<Expectation>> unexpectedOutcomes =
|
||||
<TestDescription, Set<Expectation>>{};
|
||||
int completed = 0;
|
||||
List<Future> futures = <Future>[];
|
||||
for (TestDescription description in descriptions) {
|
||||
String selector = "${suite.name}/${description.shortName}";
|
||||
if (selectors.isNotEmpty &&
|
||||
!selectors.contains(selector) &&
|
||||
!selectors.contains(suite.name)) {
|
||||
continue;
|
||||
}
|
||||
Set<Expectation> expectedOutcomes =
|
||||
expectations.expectations(description.shortName);
|
||||
Result result;
|
||||
StringBuffer sb = new StringBuffer();
|
||||
// Records the outcome of the last step that was run.
|
||||
Step lastStep = null;
|
||||
Iterator<Step> iterator = steps.iterator;
|
||||
|
||||
/// Performs one step of [iterator].
|
||||
///
|
||||
/// If `step.isAsync` is true, the corresponding step is said to be
|
||||
/// asynchronous.
|
||||
///
|
||||
/// If a step is asynchrouns the future returned from this function will
|
||||
/// complete after the the first asynchronous step is scheduled. This
|
||||
/// allows us to start processing the next test while an external process
|
||||
/// completes as steps can be interleaved. To ensure all steps are
|
||||
/// completed, wait for [futures].
|
||||
///
|
||||
/// Otherwise, the future returned will complete when all steps are
|
||||
/// completed. This ensures that tests are run in sequence without
|
||||
/// interleaving steps.
|
||||
Future doStep(dynamic input) async {
|
||||
Future future;
|
||||
bool isAsync = false;
|
||||
if (iterator.moveNext()) {
|
||||
Step step = iterator.current;
|
||||
lastStep = step;
|
||||
isAsync = step.isAsync;
|
||||
logStepStart(completed, unexpectedResults.length, descriptions.length,
|
||||
suite, description, step);
|
||||
future = runGuarded(() async {
|
||||
try {
|
||||
return await step.run(input, this);
|
||||
} catch (error, trace) {
|
||||
return step.unhandledError(error, trace);
|
||||
}
|
||||
}, printLineOnStdout: sb.writeln);
|
||||
} else {
|
||||
future = new Future.value(null);
|
||||
}
|
||||
future = future.then((Result currentResult) {
|
||||
if (currentResult != null) {
|
||||
logStepComplete(completed, unexpectedResults.length,
|
||||
descriptions.length, suite, description, lastStep);
|
||||
result = currentResult;
|
||||
if (currentResult.outcome == Expectation.PASS) {
|
||||
// The input to the next step is the output of this step.
|
||||
return doStep(result.output);
|
||||
}
|
||||
}
|
||||
if (steps.isNotEmpty && steps.last == lastStep &&
|
||||
description.shortName.endsWith("negative_test")) {
|
||||
if (result.outcome == Expectation.PASS) {
|
||||
result.addLog("Negative test didn't report an error.\n");
|
||||
} else if (result.outcome == Expectation.FAIL) {
|
||||
result.addLog("Negative test reported an error as expeceted.\n");
|
||||
}
|
||||
result = result.toNegativeTestResult();
|
||||
}
|
||||
if (!expectedOutcomes.contains(result.outcome)) {
|
||||
result.addLog("$sb");
|
||||
unexpectedResults[description] = result;
|
||||
unexpectedOutcomes[description] = expectedOutcomes;
|
||||
logUnexpectedResult(suite, description, result, expectedOutcomes);
|
||||
} else {
|
||||
logMessage(sb);
|
||||
}
|
||||
logTestComplete(++completed, unexpectedResults.length,
|
||||
descriptions.length, suite, description);
|
||||
});
|
||||
if (isAsync) {
|
||||
futures.add(future);
|
||||
return null;
|
||||
} else {
|
||||
return future;
|
||||
}
|
||||
}
|
||||
// The input of the first step is [description].
|
||||
await doStep(description);
|
||||
}
|
||||
await Future.wait(futures);
|
||||
logSuiteComplete();
|
||||
if (unexpectedResults.isNotEmpty) {
|
||||
unexpectedResults.forEach((TestDescription description, Result result) {
|
||||
exitCode = 1;
|
||||
logUnexpectedResult(suite, description, result,
|
||||
unexpectedOutcomes[description]);
|
||||
});
|
||||
print("${unexpectedResults.length} failed:");
|
||||
unexpectedResults.forEach((TestDescription description, Result result) {
|
||||
print("${suite.name}/${description.shortName}: ${result.outcome}");
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
Stream<TestDescription> list(Chain suite) async* {
|
||||
Directory testRoot = new Directory.fromUri(suite.uri);
|
||||
if (await testRoot.exists()) {
|
||||
Stream<FileSystemEntity> files =
|
||||
testRoot.list(recursive: true, followLinks: false);
|
||||
await for (FileSystemEntity entity in files) {
|
||||
if (entity is! File) continue;
|
||||
String path = entity.uri.path;
|
||||
if (suite.exclude.any((RegExp r) => path.contains(r))) continue;
|
||||
if (suite.pattern.any((RegExp r) => path.contains(r))) {
|
||||
yield new TestDescription(suite.uri, entity);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
throw "${suite.uri} isn't a directory";
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
abstract class Step<I, O, C extends ChainContext> {
|
||||
const Step();
|
||||
|
||||
String get name;
|
||||
|
||||
bool get isAsync => false;
|
||||
|
||||
Future<Result<O>> run(I input, C context);
|
||||
|
||||
Result<O> unhandledError(error, StackTrace trace) {
|
||||
return new Result<O>.crash(error, trace);
|
||||
}
|
||||
|
||||
Result<O> pass(O output) => new Result<O>.pass(output);
|
||||
|
||||
Result<O> crash(error, StackTrace trace) => new Result<O>.crash(error, trace);
|
||||
|
||||
Result<O> fail(O output, [error, StackTrace trace]) {
|
||||
return new Result<O>.fail(output, error, trace);
|
||||
}
|
||||
}
|
||||
|
||||
class Result<O> {
|
||||
final O output;
|
||||
|
||||
final Expectation outcome;
|
||||
|
||||
final error;
|
||||
|
||||
final StackTrace trace;
|
||||
|
||||
final List<String> logs = <String>[];
|
||||
|
||||
Result(this.output, this.outcome, this.error, this.trace);
|
||||
|
||||
Result.pass(O output)
|
||||
: this(output, Expectation.PASS, null, null);
|
||||
|
||||
Result.crash(error, StackTrace trace)
|
||||
: this(null, Expectation.CRASH, error, trace);
|
||||
|
||||
Result.fail(O output, [error, StackTrace trace])
|
||||
: this(output, Expectation.FAIL, error, trace);
|
||||
|
||||
String get log => logs.join();
|
||||
|
||||
void addLog(String log) {
|
||||
logs.add(log);
|
||||
}
|
||||
|
||||
Result<O> toNegativeTestResult() {
|
||||
Expectation outcome = this.outcome;
|
||||
if (outcome == Expectation.PASS) {
|
||||
outcome = Expectation.FAIL;
|
||||
} else if (outcome == Expectation.FAIL) {
|
||||
outcome = Expectation.PASS;
|
||||
}
|
||||
return new Result<O>(output, outcome, error, trace)
|
||||
..logs.addAll(logs);
|
||||
}
|
||||
}
|
||||
|
||||
/// This is called from generated code.
|
||||
Future<Null> runChain(
|
||||
CreateContext f, Map<String, String> environment, Set<String> selectors,
|
||||
String json) {
|
||||
return withErrorHandling(() async {
|
||||
Chain suite = new Suite.fromJsonMap(Uri.base, JSON.decode(json));
|
||||
print("Running ${suite.name}");
|
||||
ChainContext context = await f(suite, environment);
|
||||
return context.run(suite, selectors);
|
||||
});
|
||||
}
|
||||
95
pkg/testing/lib/src/discover.dart
Normal file
95
pkg/testing/lib/src/discover.dart
Normal file
|
|
@ -0,0 +1,95 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
library testing.discover;
|
||||
|
||||
import 'dart:io' show
|
||||
Directory,
|
||||
FileSystemEntity,
|
||||
Platform,
|
||||
Process;
|
||||
|
||||
import 'dart:async' show
|
||||
Future,
|
||||
Stream,
|
||||
StreamController,
|
||||
StreamSubscription;
|
||||
|
||||
import '../testing.dart' show
|
||||
TestDescription;
|
||||
|
||||
final Uri packageConfig = computePackageConfig();
|
||||
|
||||
final Uri dartSdk = computeDartSdk();
|
||||
|
||||
/// Common arguments when running a dart program. Returns a copy that can
|
||||
/// safely be modified by caller.
|
||||
List<String> get dartArguments => <String>["-c", "--packages=$packageConfig"];
|
||||
|
||||
Stream<TestDescription> listTests(List<Uri> testRoots, {Pattern pattern}) {
|
||||
StreamController<TestDescription> controller =
|
||||
new StreamController<TestDescription>();
|
||||
Map<Uri, StreamSubscription> subscriptions = <Uri, StreamSubscription>{};
|
||||
for (Uri testRootUri in testRoots) {
|
||||
subscriptions[testRootUri] = null;
|
||||
Directory testRoot = new Directory.fromUri(testRootUri);
|
||||
testRoot.exists().then((bool exists) {
|
||||
if (exists) {
|
||||
Stream<FileSystemEntity> stream =
|
||||
testRoot.list(recursive: true, followLinks: false);
|
||||
var subscription = stream.listen((FileSystemEntity entity) {
|
||||
TestDescription description =
|
||||
TestDescription.from(testRootUri, entity, pattern: pattern);
|
||||
if (description != null) {
|
||||
controller.add(description);
|
||||
}
|
||||
}, onError: (error, StackTrace trace) {
|
||||
controller.addError(error, trace);
|
||||
}, onDone: () {
|
||||
subscriptions.remove(testRootUri);
|
||||
if (subscriptions.isEmpty) {
|
||||
controller.close(); // TODO(ahe): catchError???
|
||||
}
|
||||
});
|
||||
subscriptions[testRootUri] = subscription;
|
||||
} else {
|
||||
controller.addError("$testRootUri isn't a directory");
|
||||
subscriptions.remove(testRootUri);
|
||||
}
|
||||
if (subscriptions.isEmpty) {
|
||||
controller.close(); // TODO(ahe): catchError???
|
||||
}
|
||||
});
|
||||
}
|
||||
return controller.stream;
|
||||
}
|
||||
|
||||
Uri computePackageConfig() {
|
||||
String path = Platform.packageConfig;
|
||||
if (path != null) return Uri.base.resolve(path);
|
||||
return Uri.base.resolve(".packages");
|
||||
}
|
||||
|
||||
Uri computeDartSdk() {
|
||||
String dartSdkPath = Platform.environment["DART_SDK"]
|
||||
?? const String.fromEnvironment("DART_SDK");
|
||||
if (dartSdkPath != null) {
|
||||
return Uri.base.resolveUri(new Uri.file(dartSdkPath));
|
||||
} else {
|
||||
return Uri.base.resolve(Platform.resolvedExecutable).resolve("../");
|
||||
}
|
||||
}
|
||||
|
||||
Future<Process> startDart(
|
||||
Uri program,
|
||||
[List<String> arguments,
|
||||
List<String> vmArguments]) {
|
||||
List<String> allArguments = <String>[];
|
||||
allArguments.addAll(vmArguments ?? dartArguments);
|
||||
allArguments.add(program.toFilePath());
|
||||
if (arguments != null) {
|
||||
allArguments.addAll(arguments);
|
||||
}
|
||||
return Process.start(Platform.resolvedExecutable, allArguments);
|
||||
}
|
||||
30
pkg/testing/lib/src/error_handling.dart
Normal file
30
pkg/testing/lib/src/error_handling.dart
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
library testing.error_handling;
|
||||
|
||||
import 'dart:async' show
|
||||
Future;
|
||||
|
||||
import 'dart:io' show
|
||||
exitCode,
|
||||
stderr;
|
||||
|
||||
import 'dart:isolate' show
|
||||
ReceivePort;
|
||||
|
||||
Future withErrorHandling(Future f()) async {
|
||||
final ReceivePort port = new ReceivePort();
|
||||
try {
|
||||
return await f();
|
||||
} catch (e, trace) {
|
||||
exitCode = 1;
|
||||
stderr.writeln(e);
|
||||
if (trace != null) {
|
||||
stderr.writeln(trace);
|
||||
}
|
||||
} finally {
|
||||
port.close();
|
||||
}
|
||||
}
|
||||
164
pkg/testing/lib/src/log.dart
Normal file
164
pkg/testing/lib/src/log.dart
Normal file
|
|
@ -0,0 +1,164 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
library testing.log;
|
||||
|
||||
import 'chain.dart' show
|
||||
Result,
|
||||
Step;
|
||||
|
||||
import 'suite.dart' show
|
||||
Suite;
|
||||
|
||||
import 'test_description.dart' show
|
||||
TestDescription;
|
||||
|
||||
import 'test_dart/status_file_parser.dart' show
|
||||
Expectation;
|
||||
|
||||
/// ANSI escape code for moving cursor one line up.
|
||||
/// See [CSI codes](https://en.wikipedia.org/wiki/ANSI_escape_code#CSI_codes).
|
||||
const String cursorUp = "\u001b[1A";
|
||||
|
||||
/// ANSI escape code for erasing the entire line.
|
||||
/// See [CSI codes](https://en.wikipedia.org/wiki/ANSI_escape_code#CSI_codes).
|
||||
const String eraseLine = "\u001b[2K";
|
||||
|
||||
final Stopwatch wallclock = new Stopwatch()..start();
|
||||
|
||||
bool _isVerbose = const bool.fromEnvironment("verbose");
|
||||
|
||||
bool get isVerbose => _isVerbose;
|
||||
|
||||
void enableVerboseOutput() {
|
||||
_isVerbose = true;
|
||||
}
|
||||
|
||||
void logTestComplete(int completed, int failed, int total,
|
||||
Suite suite, TestDescription description) {
|
||||
String message = formatProgress(completed, failed, total);
|
||||
if (suite != null) {
|
||||
message += ": ${formatTestDescription(suite, description)}";
|
||||
}
|
||||
logProgress(message);
|
||||
}
|
||||
|
||||
void logStepStart(int completed, int failed, int total,
|
||||
Suite suite, TestDescription description, Step step) {
|
||||
String message = formatProgress(completed, failed, total);
|
||||
if (suite != null) {
|
||||
message += ": ${formatTestDescription(suite, description)} ${step.name}";
|
||||
if (step.isAsync) {
|
||||
message += "...";
|
||||
}
|
||||
}
|
||||
logProgress(message);
|
||||
}
|
||||
|
||||
void logStepComplete(int completed, int failed, int total,
|
||||
Suite suite, TestDescription description, Step step) {
|
||||
if (!step.isAsync) return;
|
||||
String message = formatProgress(completed, failed, total);
|
||||
if (suite != null) {
|
||||
message += ": ${formatTestDescription(suite, description)} ${step.name}!";
|
||||
}
|
||||
logProgress(message);
|
||||
}
|
||||
|
||||
void logProgress(String message) {
|
||||
if (isVerbose) {
|
||||
print(message);
|
||||
} else {
|
||||
print("$eraseLine$message$cursorUp");
|
||||
}
|
||||
}
|
||||
|
||||
String formatProgress(int completed, int failed, int total) {
|
||||
Duration elapsed = wallclock.elapsed;
|
||||
String percent = pad((completed / total * 100.0).toStringAsFixed(1), 5);
|
||||
String good = pad(completed, 5);
|
||||
String bad = pad(failed, 5);
|
||||
String minutes = pad(elapsed.inMinutes, 2, filler: "0");
|
||||
String seconds = pad(elapsed.inSeconds % 60, 2, filler: "0");
|
||||
return "[ $minutes:$seconds | $percent% | +$good | -$bad ]";
|
||||
}
|
||||
|
||||
String formatTestDescription(Suite suite, TestDescription description) {
|
||||
return "${suite.name}/${description.shortName}";
|
||||
}
|
||||
|
||||
void logMessage(Object message) {
|
||||
if (isVerbose) {
|
||||
print("$message");
|
||||
}
|
||||
}
|
||||
|
||||
void logNumberedLines(String text) {
|
||||
if (isVerbose) {
|
||||
print(numberedLines(text));
|
||||
}
|
||||
}
|
||||
|
||||
void logUnexpectedResult(Suite suite, TestDescription description,
|
||||
Result result, Set<Expectation> expectedOutcomes) {
|
||||
print("${eraseLine}UNEXPECTED: ${suite.name}/${description.shortName}");
|
||||
Uri statusFile = suite.statusFile;
|
||||
if (statusFile != null) {
|
||||
String path = statusFile.toFilePath();
|
||||
if (result.outcome == Expectation.PASS) {
|
||||
print("The test unexpectedly passed, please update $path.");
|
||||
} else {
|
||||
print("The test had the outcome ${result.outcome}, but the status file "
|
||||
"($path) allows these outcomes: ${expectedOutcomes.join(' ')}");
|
||||
}
|
||||
}
|
||||
String log = result.log;
|
||||
if (log.isNotEmpty) {
|
||||
print(log);
|
||||
}
|
||||
if (result.error != null) {
|
||||
print(result.error);
|
||||
if (result.trace != null) {
|
||||
print(result.trace);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void logSuiteComplete() {
|
||||
if (!isVerbose) {
|
||||
print("");
|
||||
}
|
||||
}
|
||||
|
||||
void logUncaughtError(error, StackTrace stackTrace) {
|
||||
logMessage(error);
|
||||
if (stackTrace != null) {
|
||||
logMessage(stackTrace);
|
||||
}
|
||||
}
|
||||
|
||||
String pad(Object o, int pad, {String filler: " "}) {
|
||||
String result = (filler * pad) + "$o";
|
||||
return result.substring(result.length - pad);
|
||||
}
|
||||
|
||||
String numberedLines(String text) {
|
||||
StringBuffer result = new StringBuffer();
|
||||
int lineNumber = 1;
|
||||
List<String> lines = splitLines(text);
|
||||
int pad = "${lines.length}".length;
|
||||
String fill = " " * pad;
|
||||
for (String line in lines) {
|
||||
String paddedLineNumber = "$fill$lineNumber";
|
||||
paddedLineNumber =
|
||||
paddedLineNumber.substring(paddedLineNumber.length - pad);
|
||||
result.write("$paddedLineNumber: $line");
|
||||
lineNumber++;
|
||||
}
|
||||
return '$result';
|
||||
}
|
||||
|
||||
List<String> splitLines(String text) {
|
||||
return text.split(new RegExp('^', multiLine: true));
|
||||
}
|
||||
124
pkg/testing/lib/src/multitest.dart
Normal file
124
pkg/testing/lib/src/multitest.dart
Normal file
|
|
@ -0,0 +1,124 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
library testing.multitest;
|
||||
|
||||
import 'dart:async' show
|
||||
Stream,
|
||||
StreamTransformer;
|
||||
|
||||
import 'dart:io' show
|
||||
Directory,
|
||||
File;
|
||||
|
||||
import 'log.dart' show
|
||||
splitLines;
|
||||
|
||||
import 'test_description.dart' show
|
||||
TestDescription;
|
||||
|
||||
class MultitestTransformer
|
||||
implements StreamTransformer<TestDescription, TestDescription> {
|
||||
static const String multitestMarker = "///";
|
||||
|
||||
static const List<String> validOutcomesList = const <String>[
|
||||
'ok',
|
||||
'compile-time error',
|
||||
'runtime error',
|
||||
'static type warning',
|
||||
'dynamic type error',
|
||||
'checked mode compile-time error',
|
||||
];
|
||||
|
||||
static final Set<String> validOutcomes =
|
||||
new Set<String>.from(validOutcomesList);
|
||||
|
||||
Stream<TestDescription> bind(Stream<TestDescription> stream) async* {
|
||||
List<String> errors = <String>[];
|
||||
reportError(String error) {
|
||||
errors.add(error);
|
||||
print(error);
|
||||
}
|
||||
nextTest: await for (TestDescription test in stream) {
|
||||
String contents = await test.file.readAsString();
|
||||
if (!contents.contains(multitestMarker)) {
|
||||
yield test;
|
||||
continue nextTest;
|
||||
}
|
||||
// Note: this is modified in the loop below.
|
||||
List<String> linesWithoutAnnotations = <String>[];
|
||||
Map<String, List<String>> testsAsLines = <String, List<String>>{
|
||||
"none": linesWithoutAnnotations,
|
||||
};
|
||||
Map<String, Set<String>> outcomes = <String, Set<String>>{
|
||||
"none": new Set<String>(),
|
||||
};
|
||||
int lineNumber = 0;
|
||||
for (String line in splitLines(contents)) {
|
||||
lineNumber++;
|
||||
int index = line.indexOf(multitestMarker);
|
||||
String subtestName;
|
||||
List<String> subtestOutcomesList;
|
||||
if (index != -1) {
|
||||
String annotationText =
|
||||
line.substring(index + multitestMarker.length).trim();
|
||||
index = annotationText.indexOf(":");
|
||||
if (index != -1) {
|
||||
subtestName = annotationText.substring(0, index).trim();
|
||||
subtestOutcomesList = annotationText.substring(index + 1).split(",")
|
||||
.map((s) => s.trim()).toList();
|
||||
if (subtestName == "none") {
|
||||
reportError(test.formatError(
|
||||
"$lineNumber: $subtestName can't be used as test name."));
|
||||
continue nextTest;
|
||||
}
|
||||
if (subtestOutcomesList.isEmpty) {
|
||||
reportError(test.formatError(
|
||||
"$lineNumber: Expected <testname>:<outcomes>"));
|
||||
continue nextTest;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (subtestName != null) {
|
||||
List<String> lines = testsAsLines.putIfAbsent(subtestName,
|
||||
() => new List<String>.from(linesWithoutAnnotations));
|
||||
lines.add(line);
|
||||
Set<String> subtestOutcomes = outcomes.putIfAbsent(subtestName,
|
||||
() => new Set<String>());
|
||||
if (subtestOutcomesList.length != 1 ||
|
||||
subtestOutcomesList.single != "continued") {
|
||||
for (String outcome in subtestOutcomesList) {
|
||||
if (validOutcomes.contains(outcome)) {
|
||||
subtestOutcomes.add(outcome);
|
||||
} else {
|
||||
reportError(test.formatError(
|
||||
"$lineNumber: '$outcome' isn't a recognized outcome."));
|
||||
continue nextTest;
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
for (List<String> lines in testsAsLines.values) {
|
||||
// This will also modify [linesWithoutAnnotations].
|
||||
lines.add(line);
|
||||
}
|
||||
}
|
||||
}
|
||||
Uri root = Uri.base.resolve("generated/");
|
||||
Directory generated = new Directory.fromUri(root.resolve(test.shortName));
|
||||
generated = await generated.create(recursive: true);
|
||||
for (String name in testsAsLines.keys) {
|
||||
List<String> lines = testsAsLines[name];
|
||||
Uri uri = generated.uri.resolve("${name}_generated.dart");
|
||||
TestDescription subtest =
|
||||
new TestDescription(root, new File.fromUri(uri));
|
||||
await subtest.file.writeAsString(lines.join(""));
|
||||
yield subtest;
|
||||
}
|
||||
}
|
||||
if (errors.isNotEmpty) {
|
||||
throw "Error: ${errors.join("\n")}";
|
||||
}
|
||||
}
|
||||
}
|
||||
229
pkg/testing/lib/src/run.dart
Normal file
229
pkg/testing/lib/src/run.dart
Normal file
|
|
@ -0,0 +1,229 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
library testing.run;
|
||||
|
||||
import 'dart:async' show
|
||||
Future,
|
||||
Stream;
|
||||
|
||||
import 'dart:convert' show
|
||||
JSON;
|
||||
|
||||
import 'dart:io' show
|
||||
Platform;
|
||||
|
||||
import 'dart:isolate' show
|
||||
Isolate,
|
||||
ReceivePort;
|
||||
|
||||
import 'test_root.dart' show
|
||||
TestRoot;
|
||||
|
||||
import 'test_description.dart' show
|
||||
TestDescription;
|
||||
|
||||
import 'error_handling.dart' show
|
||||
withErrorHandling;
|
||||
|
||||
import 'chain.dart' show
|
||||
CreateContext;
|
||||
|
||||
import '../testing.dart' show
|
||||
Chain,
|
||||
ChainContext,
|
||||
TestDescription,
|
||||
listTests;
|
||||
|
||||
import 'analyze.dart' show
|
||||
Analyze;
|
||||
|
||||
import 'log.dart' show
|
||||
isVerbose,
|
||||
logMessage,
|
||||
logNumberedLines,
|
||||
splitLines;
|
||||
|
||||
import 'suite.dart' show
|
||||
Dart,
|
||||
Suite;
|
||||
|
||||
import 'zone_helper.dart' show
|
||||
acknowledgeControlMessages;
|
||||
|
||||
Future<TestRoot> computeTestRoot(String configurationPath, Uri base) {
|
||||
Uri configuration = configurationPath == null
|
||||
? Uri.base.resolve("testing.json")
|
||||
: base.resolve(configurationPath);
|
||||
return TestRoot.fromUri(configuration);
|
||||
}
|
||||
|
||||
/// This is called from a Chain suite, and helps implement main. In most cases,
|
||||
/// main will look like this:
|
||||
///
|
||||
/// main(List<String> arguments) => runMe(arguments, createContext);
|
||||
///
|
||||
/// The optional argument [configurationPath] should be used when
|
||||
/// `testing.json` isn't located in the current working directory and is a path
|
||||
/// relative to `Platform.script`.
|
||||
Future<Null> runMe(
|
||||
List<String> arguments, CreateContext f, [String configurationPath]) {
|
||||
return withErrorHandling(() async {
|
||||
TestRoot testRoot =
|
||||
await computeTestRoot(configurationPath, Platform.script);
|
||||
for (Chain suite in testRoot.toolChains) {
|
||||
if (Platform.script == suite.source) {
|
||||
print("Running suite ${suite.name}...");
|
||||
ChainContext context = await f(suite, <String, String>{});
|
||||
await context.run(suite, new Set<String>());
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/// This is called from a `_test.dart` file, and helps integration in other
|
||||
/// test runner frameworks.
|
||||
///
|
||||
/// For example, to run the suite `my_suite` from `test.dart`, create a file
|
||||
/// with this content:
|
||||
///
|
||||
/// import 'package:async_helper/async_helper.dart' show asyncTest;
|
||||
///
|
||||
/// import 'package:testing/testing.dart' show run;
|
||||
///
|
||||
/// main(List<String> arguments) => asyncTest(run(arguments, ["my_suite"]));
|
||||
///
|
||||
/// To run run the same suite from `package:test`, create a file with this
|
||||
/// content:
|
||||
///
|
||||
/// import 'package:test/test.dart' show Timeout, test;
|
||||
///
|
||||
/// import 'package:testing/testing.dart' show run;
|
||||
///
|
||||
/// main() {
|
||||
/// test("my_suite", () => run([], ["my_suite"]),
|
||||
/// timeout: new Timeout(new Duration(minutes: 5)));
|
||||
/// }
|
||||
///
|
||||
/// The optional argument [configurationPath] should be used when
|
||||
/// `testing.json` isn't located in the current working directory and is a path
|
||||
/// relative to `Uri.base`.
|
||||
Future<Null> run(
|
||||
List<String> arguments, List<String> suiteNames,
|
||||
[String configurationPath]) {
|
||||
return withErrorHandling(() async {
|
||||
TestRoot root = await computeTestRoot(configurationPath, Uri.base);
|
||||
List<Suite> suites = root.suites.where(
|
||||
(Suite suite) => suiteNames.contains(suite.name)).toList();
|
||||
SuiteRunner runner = new SuiteRunner(suites, <String, String>{}, null);
|
||||
String program = await runner.generateDartProgram();
|
||||
await runner.analyze(root.packages);
|
||||
if (program != null) {
|
||||
await runProgram(program, root.packages);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
Future<Null> runProgram(String program, Uri packages) async {
|
||||
logMessage("Running:");
|
||||
logNumberedLines(program);
|
||||
Uri dataUri = new Uri.dataFromString(program);
|
||||
ReceivePort exitPort = new ReceivePort();
|
||||
Isolate isolate = await Isolate.spawnUri(dataUri, <String>[], null,
|
||||
paused: true, onExit: exitPort.sendPort, errorsAreFatal: false,
|
||||
checked: true, packageConfig: packages);
|
||||
List error;
|
||||
var subscription = isolate.errors.listen((data) {
|
||||
error = data;
|
||||
exitPort.close();
|
||||
});
|
||||
await acknowledgeControlMessages(isolate, resume: isolate.pauseCapability);
|
||||
await for (var _ in exitPort) {
|
||||
exitPort.close();
|
||||
}
|
||||
subscription.cancel();
|
||||
return error == null
|
||||
? null
|
||||
: new Future<Null>.error(error[0], new StackTrace.fromString(error[1]));
|
||||
}
|
||||
|
||||
class SuiteRunner {
|
||||
final List<Suite> suites;
|
||||
|
||||
final Map<String, String> environment;
|
||||
|
||||
final List<String> selectors;
|
||||
|
||||
List<Uri> testUris;
|
||||
|
||||
SuiteRunner(this.suites, this.environment, Iterable<String> selectors)
|
||||
: selectors = selectors.toList(growable: false);
|
||||
|
||||
Future<String> generateDartProgram() async {
|
||||
List<TestDescription> descriptions = await list().toList();
|
||||
testUris = <Uri>[];
|
||||
StringBuffer imports = new StringBuffer();
|
||||
StringBuffer dart = new StringBuffer();
|
||||
StringBuffer chain = new StringBuffer();
|
||||
|
||||
for (TestDescription description in descriptions) {
|
||||
testUris.add(await Isolate.resolvePackageUri(description.uri));
|
||||
description.writeImportOn(imports);
|
||||
description.writeClosureOn(dart);
|
||||
}
|
||||
|
||||
for (Chain suite in suites.where((Suite suite) => suite is Chain)) {
|
||||
testUris.add(await Isolate.resolvePackageUri(suite.source));
|
||||
suite.writeImportOn(imports);
|
||||
suite.writeClosureOn(chain);
|
||||
}
|
||||
|
||||
if (testUris.isEmpty) return null;
|
||||
|
||||
return """
|
||||
library testing.generated;
|
||||
|
||||
import 'dart:async' show Future;
|
||||
|
||||
import 'dart:convert' show JSON;
|
||||
|
||||
import 'package:testing/src/run_tests.dart' show runTests;
|
||||
|
||||
import 'package:testing/src/chain.dart' show runChain;
|
||||
|
||||
import 'package:testing/src/log.dart' show enableVerboseOutput, isVerbose;
|
||||
|
||||
${imports.toString().trim()}
|
||||
|
||||
Future<Null> main() async {
|
||||
if ($isVerbose) enableVerboseOutput();
|
||||
Map<String, String> environment = JSON.decode('${JSON.encode(environment)}');
|
||||
Set<String> selectors = JSON.decode('${JSON.encode(selectors)}').toSet();
|
||||
await runTests(<String, Function> {
|
||||
${splitLines(dart.toString().trim()).join(' ')}
|
||||
});
|
||||
${splitLines(chain.toString().trim()).join(' ')}
|
||||
}
|
||||
""";
|
||||
}
|
||||
|
||||
Future<Null> analyze(Uri packages) async {
|
||||
for (Analyze suite in suites.where((Suite suite) => suite is Analyze)) {
|
||||
await suite.run(packages, testUris);
|
||||
}
|
||||
}
|
||||
|
||||
Stream<TestDescription> list() async* {
|
||||
for (Dart suite in suites.where((Suite suite) => suite is Dart)) {
|
||||
await for (TestDescription description in
|
||||
listTests(<Uri>[suite.uri], pattern: "")) {
|
||||
String path = description.file.uri.path;
|
||||
if (suite.exclude.any((RegExp r) => path.contains(r))) continue;
|
||||
if (suite.pattern.any((RegExp r) => path.contains(r))) {
|
||||
yield description;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
205
pkg/testing/lib/src/run_tests.dart
Normal file
205
pkg/testing/lib/src/run_tests.dart
Normal file
|
|
@ -0,0 +1,205 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
library testing.run_tests;
|
||||
|
||||
import 'dart:async' show
|
||||
Future;
|
||||
|
||||
import 'dart:io' show
|
||||
Directory,
|
||||
File,
|
||||
FileSystemEntity;
|
||||
|
||||
import 'dart:io' as io show
|
||||
exitCode;
|
||||
|
||||
import 'dart:isolate' show
|
||||
Isolate;
|
||||
|
||||
import 'error_handling.dart' show
|
||||
withErrorHandling;
|
||||
|
||||
import 'test_root.dart' show
|
||||
TestRoot;
|
||||
|
||||
import 'zone_helper.dart' show
|
||||
runGuarded;
|
||||
|
||||
import 'log.dart' show
|
||||
enableVerboseOutput,
|
||||
isVerbose,
|
||||
logMessage,
|
||||
logSuiteComplete,
|
||||
logTestComplete;
|
||||
|
||||
import 'run.dart' show
|
||||
SuiteRunner,
|
||||
runProgram;
|
||||
|
||||
import 'suite.dart' show
|
||||
Suite;
|
||||
|
||||
class CommandLine {
|
||||
final Set<String> options;
|
||||
final List<String> arguments;
|
||||
|
||||
CommandLine(this.options, this.arguments);
|
||||
|
||||
bool get verbose => options.contains("--verbose") || options.contains("-v");
|
||||
|
||||
Set<String> get skip {
|
||||
return options.expand((String s) {
|
||||
const String prefix = "--skip=";
|
||||
if (!s.startsWith(prefix)) return const [];
|
||||
s = s.substring(prefix.length);
|
||||
return s.split(",");
|
||||
}).toSet();
|
||||
}
|
||||
|
||||
Map<String, String> get environment {
|
||||
Map<String, String> result = <String, String>{};
|
||||
for (String option in options) {
|
||||
if (option.startsWith("-D")) {
|
||||
int equalIndex = option.indexOf("=");
|
||||
if (equalIndex != -1) {
|
||||
String key = option.substring(2, equalIndex);
|
||||
String value = option.substring(equalIndex + 1);
|
||||
result[key] = value;
|
||||
}
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
Set<String> get selectedSuites {
|
||||
return selectors.map((String selector) {
|
||||
int index = selector.indexOf("/");
|
||||
return index == -1 ? selector : selector.substring(0, index);
|
||||
}).toSet();
|
||||
}
|
||||
|
||||
Iterable<String> get selectors => arguments;
|
||||
|
||||
Future<Uri> get configuration async {
|
||||
const String configPrefix = "--config=";
|
||||
List<String> configurationPaths = options
|
||||
.where((String option) => option.startsWith(configPrefix))
|
||||
.map((String option) => option.substring(configPrefix.length))
|
||||
.toList();
|
||||
if (configurationPaths.length > 1) {
|
||||
return fail("Only one --config option is supported");
|
||||
}
|
||||
String configurationPath;
|
||||
if (configurationPaths.length == 1) {
|
||||
configurationPath = configurationPaths.single;
|
||||
} else {
|
||||
configurationPath = "testing.json";
|
||||
if (!await new File(configurationPath).exists()) {
|
||||
Directory test = new Directory("test");
|
||||
if (await test.exists()) {
|
||||
List<FileSystemEntity> candiates =
|
||||
await test.list(recursive: true, followLinks: false)
|
||||
.where((FileSystemEntity entity) {
|
||||
return entity is File &&
|
||||
entity.uri.path.endsWith("/testing.json");
|
||||
}).toList();
|
||||
switch (candiates.length) {
|
||||
case 0:
|
||||
return fail("Couldn't locate: '$configurationPath'.");
|
||||
|
||||
case 1:
|
||||
configurationPath = candiates.single.path;
|
||||
break;
|
||||
|
||||
default:
|
||||
return fail(
|
||||
"Usage: run_tests.dart [$configPrefix=configuration_file]\n"
|
||||
"Where configuration_file is one of:\n "
|
||||
"${candiates.map((File file) => file.path).join('\n ')}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
logMessage("Reading configuration file '$configurationPath'.");
|
||||
Uri configuration =
|
||||
await Isolate.resolvePackageUri(Uri.base.resolve(configurationPath));
|
||||
if (configuration == null ||
|
||||
!await new File.fromUri(configuration).exists()) {
|
||||
return fail("Couldn't locate: '$configurationPath'.");
|
||||
}
|
||||
return configuration;
|
||||
}
|
||||
|
||||
static CommandLine parse(List<String> arguments) {
|
||||
int index = arguments.indexOf("--");
|
||||
Set<String> options;
|
||||
if (index != -1) {
|
||||
options = new Set<String>.from(arguments.getRange(0, index - 1));
|
||||
arguments = arguments.sublist(index + 1);
|
||||
} else {
|
||||
options =
|
||||
arguments.where((argument) => argument.startsWith("-")).toSet();
|
||||
arguments =
|
||||
arguments.where((argument) => !argument.startsWith("-")).toList();
|
||||
}
|
||||
return new CommandLine(options, arguments);
|
||||
}
|
||||
}
|
||||
|
||||
fail(String message) {
|
||||
print(message);
|
||||
io.exitCode = 1;
|
||||
return null;
|
||||
}
|
||||
|
||||
main(List<String> arguments) => withErrorHandling(() async {
|
||||
CommandLine cl = CommandLine.parse(arguments);
|
||||
if (cl.verbose) {
|
||||
enableVerboseOutput();
|
||||
}
|
||||
Map<String, String> environment = cl.environment;
|
||||
Uri configuration = await cl.configuration;
|
||||
if (configuration == null) return;
|
||||
if (!isVerbose) {
|
||||
print("Use --verbose to display more details.");
|
||||
}
|
||||
TestRoot root = await TestRoot.fromUri(configuration);
|
||||
Set<String> skip = cl.skip;
|
||||
Set<String> selectedSuites = cl.selectedSuites;
|
||||
List<Suite> suites = root.suites.where((s) {
|
||||
return !skip.contains(s.name) &&
|
||||
(selectedSuites.isEmpty || selectedSuites.contains(s.name));
|
||||
}).toList();
|
||||
SuiteRunner runner = new SuiteRunner(suites, environment, cl.selectors);
|
||||
String program = await runner.generateDartProgram();
|
||||
await runner.analyze(root.packages);
|
||||
Stopwatch sw = new Stopwatch()..start();
|
||||
if (program == null) {
|
||||
fail("No tests configured.");
|
||||
} else {
|
||||
await runProgram(program, root.packages);
|
||||
}
|
||||
print("Running tests took: ${sw.elapsed}.");
|
||||
});
|
||||
|
||||
Future<Null> runTests(Map<String, Function> tests) =>
|
||||
withErrorHandling(() async {
|
||||
int completed = 0;
|
||||
for (String name in tests.keys) {
|
||||
StringBuffer sb = new StringBuffer();
|
||||
try {
|
||||
await runGuarded(() {
|
||||
print("Running test $name");
|
||||
return tests[name]();
|
||||
}, printLineOnStdout: sb.writeln);
|
||||
logMessage(sb);
|
||||
} catch (e) {
|
||||
print(sb);
|
||||
rethrow;
|
||||
}
|
||||
logTestComplete(++completed, 0, tests.length, null, null);
|
||||
}
|
||||
logSuiteComplete();
|
||||
});
|
||||
70
pkg/testing/lib/src/stdio_process.dart
Normal file
70
pkg/testing/lib/src/stdio_process.dart
Normal file
|
|
@ -0,0 +1,70 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
library testing.stdio_process;
|
||||
|
||||
import 'dart:async' show
|
||||
Future,
|
||||
Timer;
|
||||
|
||||
import 'dart:convert' show
|
||||
UTF8;
|
||||
|
||||
import 'dart:io' show
|
||||
Process,
|
||||
ProcessSignal;
|
||||
|
||||
import 'chain.dart' show
|
||||
Result;
|
||||
|
||||
class StdioProcess {
|
||||
final int exitCode;
|
||||
|
||||
final String output;
|
||||
|
||||
StdioProcess(this.exitCode, this.output);
|
||||
|
||||
Result<int> toResult({int expected: 0}) {
|
||||
if (exitCode == expected) {
|
||||
return new Result<int>.pass(exitCode);
|
||||
} else {
|
||||
return new Result<int>.fail(exitCode, output);
|
||||
}
|
||||
}
|
||||
|
||||
static Future<StdioProcess> run(
|
||||
String executable, List<String> arguments,
|
||||
{String input, Duration timeout: const Duration(seconds: 60)}) async {
|
||||
Process process = await Process.start(executable, arguments);
|
||||
Timer timer;
|
||||
StringBuffer sb = new StringBuffer();
|
||||
timer = new Timer(timeout, () {
|
||||
sb.write("Process timed out: ");
|
||||
sb.write(executable);
|
||||
sb.write(" ");
|
||||
sb.writeAll(arguments, " ");
|
||||
sb.writeln();
|
||||
sb.writeln("Sending SIGTERM to process");
|
||||
process.kill();
|
||||
timer = new Timer(const Duration(seconds: 10), () {
|
||||
sb.writeln("Sending SIGKILL to process");
|
||||
process.kill(ProcessSignal.SIGKILL);
|
||||
});
|
||||
});
|
||||
if (input != null) {
|
||||
process.stdin.write(input);
|
||||
}
|
||||
Future closeFuture = process.stdin.close();
|
||||
Future<List<String>> stdoutFuture =
|
||||
process.stdout.transform(UTF8.decoder).toList();
|
||||
Future<List<String>> stderrFuture =
|
||||
process.stderr.transform(UTF8.decoder).toList();
|
||||
int exitCode = await process.exitCode;
|
||||
timer.cancel();
|
||||
sb.writeAll(await stdoutFuture);
|
||||
sb.writeAll(await stderrFuture);
|
||||
await closeFuture;
|
||||
return new StdioProcess(exitCode, "$sb");
|
||||
}
|
||||
}
|
||||
84
pkg/testing/lib/src/suite.dart
Normal file
84
pkg/testing/lib/src/suite.dart
Normal file
|
|
@ -0,0 +1,84 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
library testing.suite;
|
||||
|
||||
import 'chain.dart' show
|
||||
Chain;
|
||||
|
||||
/// Records the properties of a test suite.
|
||||
abstract class Suite {
|
||||
final String name;
|
||||
|
||||
final String kind;
|
||||
|
||||
final Uri statusFile;
|
||||
|
||||
Suite(this.name, this.kind, this.statusFile);
|
||||
|
||||
factory Suite.fromJsonMap(Uri base, Map json) {
|
||||
String kind = json["kind"].toLowerCase();
|
||||
String name = json["name"];
|
||||
switch (kind) {
|
||||
case "dart":
|
||||
return new Dart.fromJsonMap(base, json, name);
|
||||
|
||||
case "chain":
|
||||
return new Chain.fromJsonMap(base, json, name, kind);
|
||||
|
||||
default:
|
||||
throw "Suite '$name' has unknown kind '$kind'.";
|
||||
}
|
||||
}
|
||||
|
||||
String toString() => "Suite($name, $kind)";
|
||||
}
|
||||
|
||||
/// A suite of standalone tests. The tests are combined and run as one program.
|
||||
///
|
||||
/// A standalone test is a test with a `main` method. The test is considered
|
||||
/// successful if main doesn't throw an error (or if `main` returns a future,
|
||||
/// that future completes without errors).
|
||||
///
|
||||
/// The tests are combined by generating a Dart file which imports all the main
|
||||
/// methods and calls them sequentially.
|
||||
///
|
||||
/// Example JSON configuration:
|
||||
///
|
||||
/// {
|
||||
/// "name": "test",
|
||||
/// "kind": "Dart",
|
||||
/// # Root directory of tests in this suite.
|
||||
/// "path": "test/",
|
||||
/// # Files in `path` that match any of the following regular expressions
|
||||
/// # are considered to be part of this suite.
|
||||
/// "pattern": [
|
||||
/// "_test.dart$"
|
||||
/// ],
|
||||
/// # Except if they match any of the following regular expressions.
|
||||
/// "exclude": [
|
||||
/// "/golden/"
|
||||
/// ]
|
||||
/// }
|
||||
class Dart extends Suite {
|
||||
final Uri uri;
|
||||
|
||||
final List<RegExp> pattern;
|
||||
|
||||
final List<RegExp> exclude;
|
||||
|
||||
Dart(String name, this.uri, this.pattern, this.exclude)
|
||||
: super(name, "dart", null);
|
||||
|
||||
factory Dart.fromJsonMap(Uri base, Map json, String name) {
|
||||
Uri uri = base.resolve(json["path"]);
|
||||
List<RegExp> pattern = new List<RegExp>.from(
|
||||
json["pattern"].map((String p) => new RegExp(p)));
|
||||
List<RegExp> exclude = new List<RegExp>.from(
|
||||
json["exclude"].map((String p) => new RegExp(p)));
|
||||
return new Dart(name, uri, pattern, exclude);
|
||||
}
|
||||
|
||||
String toString() => "Dart($name, $uri, $pattern, $exclude)";
|
||||
}
|
||||
7
pkg/testing/lib/src/test_dart/README.md
Normal file
7
pkg/testing/lib/src/test_dart/README.md
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
<!--
|
||||
Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
for details. All rights reserved. Use of this source code is governed by a
|
||||
BSD-style license that can be found in the LICENSE file.
|
||||
-->
|
||||
Files in this directory are copied from
|
||||
[test.dart](https://github.com/dart-lang/sdk/tree/master/tools/testing/dart).
|
||||
311
pkg/testing/lib/src/test_dart/path.dart
Normal file
311
pkg/testing/lib/src/test_dart/path.dart
Normal file
|
|
@ -0,0 +1,311 @@
|
|||
// Copyright (c) 2012, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE file.
|
||||
|
||||
library legacy_path;
|
||||
|
||||
import 'dart:io';
|
||||
import 'dart:math';
|
||||
|
||||
// TODO: Remove this class, and use the URI class for all path manipulation.
|
||||
class Path {
|
||||
final String _path;
|
||||
final bool isWindowsShare;
|
||||
|
||||
Path(String source)
|
||||
: _path = _clean(source),
|
||||
isWindowsShare = _isWindowsShare(source);
|
||||
|
||||
Path.raw(String source)
|
||||
: _path = source,
|
||||
isWindowsShare = false;
|
||||
|
||||
Path._internal(String this._path, bool this.isWindowsShare);
|
||||
|
||||
static String _clean(String source) {
|
||||
if (Platform.operatingSystem == 'windows') return _cleanWindows(source);
|
||||
// Remove trailing slash from directories:
|
||||
if (source.length > 1 && source.endsWith('/')) {
|
||||
return source.substring(0, source.length - 1);
|
||||
}
|
||||
return source;
|
||||
}
|
||||
|
||||
static String _cleanWindows(String source) {
|
||||
// Change \ to /.
|
||||
var clean = source.replaceAll('\\', '/');
|
||||
// Add / before intial [Drive letter]:
|
||||
if (clean.length >= 2 && clean[1] == ':') {
|
||||
clean = '/$clean';
|
||||
}
|
||||
if (_isWindowsShare(source)) {
|
||||
return clean.substring(1, clean.length);
|
||||
}
|
||||
return clean;
|
||||
}
|
||||
|
||||
static bool _isWindowsShare(String source) {
|
||||
return Platform.operatingSystem == 'windows' && source.startsWith('\\\\');
|
||||
}
|
||||
|
||||
int get hashCode => _path.hashCode;
|
||||
bool get isEmpty => _path.isEmpty;
|
||||
bool get isAbsolute => _path.startsWith('/');
|
||||
bool get hasTrailingSeparator => _path.endsWith('/');
|
||||
|
||||
String toString() => _path;
|
||||
|
||||
Path relativeTo(Path base) {
|
||||
// Returns a path "relative" such that
|
||||
// base.join(relative) == this.canonicalize.
|
||||
// Throws exception if an impossible case is reached.
|
||||
if (base.isAbsolute != isAbsolute ||
|
||||
base.isWindowsShare != isWindowsShare) {
|
||||
throw new ArgumentError("Invalid case of Path.relativeTo(base):\n"
|
||||
" Path and base must both be relative, or both absolute.\n"
|
||||
" Arguments: $_path.relativeTo($base)");
|
||||
}
|
||||
|
||||
var basePath = base.toString();
|
||||
// Handle drive letters specially on Windows.
|
||||
if (base.isAbsolute && Platform.operatingSystem == 'windows') {
|
||||
bool baseHasDrive =
|
||||
basePath.length >= 4 && basePath[2] == ':' && basePath[3] == '/';
|
||||
bool pathHasDrive =
|
||||
_path.length >= 4 && _path[2] == ':' && _path[3] == '/';
|
||||
if (baseHasDrive && pathHasDrive) {
|
||||
int baseDrive = basePath.codeUnitAt(1) | 32; // Convert to uppercase.
|
||||
if (baseDrive >= 'a'.codeUnitAt(0) &&
|
||||
baseDrive <= 'z'.codeUnitAt(0) &&
|
||||
baseDrive == (_path.codeUnitAt(1) | 32)) {
|
||||
if (basePath[1] != _path[1]) {
|
||||
// Replace the drive letter in basePath with that from _path.
|
||||
basePath = '/${_path[1]}:/${basePath.substring(4)}';
|
||||
base = new Path(basePath);
|
||||
}
|
||||
} else {
|
||||
throw new ArgumentError("Invalid case of Path.relativeTo(base):\n"
|
||||
" Base path and target path are on different Windows drives.\n"
|
||||
" Arguments: $_path.relativeTo($base)");
|
||||
}
|
||||
} else if (baseHasDrive != pathHasDrive) {
|
||||
throw new ArgumentError("Invalid case of Path.relativeTo(base):\n"
|
||||
" Base path must start with a drive letter if and "
|
||||
"only if target path does.\n"
|
||||
" Arguments: $_path.relativeTo($base)");
|
||||
}
|
||||
}
|
||||
if (_path.startsWith(basePath)) {
|
||||
if (_path == basePath) return new Path('.');
|
||||
// There must be a '/' at the end of the match, or immediately after.
|
||||
int matchEnd = basePath.length;
|
||||
if (_path[matchEnd - 1] == '/' || _path[matchEnd] == '/') {
|
||||
// Drop any extra '/' characters at matchEnd
|
||||
while (matchEnd < _path.length && _path[matchEnd] == '/') {
|
||||
matchEnd++;
|
||||
}
|
||||
return new Path(_path.substring(matchEnd)).canonicalize();
|
||||
}
|
||||
}
|
||||
|
||||
List<String> baseSegments = base.canonicalize().segments();
|
||||
List<String> pathSegments = canonicalize().segments();
|
||||
if (baseSegments.length == 1 && baseSegments[0] == '.') {
|
||||
baseSegments = [];
|
||||
}
|
||||
if (pathSegments.length == 1 && pathSegments[0] == '.') {
|
||||
pathSegments = [];
|
||||
}
|
||||
int common = 0;
|
||||
int length = min(pathSegments.length, baseSegments.length);
|
||||
while (common < length && pathSegments[common] == baseSegments[common]) {
|
||||
common++;
|
||||
}
|
||||
final segments = new List<String>();
|
||||
|
||||
if (common < baseSegments.length && baseSegments[common] == '..') {
|
||||
throw new ArgumentError("Invalid case of Path.relativeTo(base):\n"
|
||||
" Base path has more '..'s than path does.\n"
|
||||
" Arguments: $_path.relativeTo($base)");
|
||||
}
|
||||
for (int i = common; i < baseSegments.length; i++) {
|
||||
segments.add('..');
|
||||
}
|
||||
for (int i = common; i < pathSegments.length; i++) {
|
||||
segments.add('${pathSegments[i]}');
|
||||
}
|
||||
if (segments.isEmpty) {
|
||||
segments.add('.');
|
||||
}
|
||||
if (hasTrailingSeparator) {
|
||||
segments.add('');
|
||||
}
|
||||
return new Path(segments.join('/'));
|
||||
}
|
||||
|
||||
Path join(Path further) {
|
||||
if (further.isAbsolute) {
|
||||
throw new ArgumentError(
|
||||
"Path.join called with absolute Path as argument.");
|
||||
}
|
||||
if (isEmpty) {
|
||||
return further.canonicalize();
|
||||
}
|
||||
if (hasTrailingSeparator) {
|
||||
var joined = new Path._internal('$_path${further}', isWindowsShare);
|
||||
return joined.canonicalize();
|
||||
}
|
||||
var joined = new Path._internal('$_path/${further}', isWindowsShare);
|
||||
return joined.canonicalize();
|
||||
}
|
||||
|
||||
// Note: The URI RFC names for canonicalize, join, and relativeTo
|
||||
// are normalize, resolve, and relativize. But resolve and relativize
|
||||
// drop the last segment of the base path (the filename), on URIs.
|
||||
Path canonicalize() {
|
||||
if (isCanonical) return this;
|
||||
return makeCanonical();
|
||||
}
|
||||
|
||||
bool get isCanonical {
|
||||
// Contains no consecutive path separators.
|
||||
// Contains no segments that are '.'.
|
||||
// Absolute paths have no segments that are '..'.
|
||||
// All '..' segments of a relative path are at the beginning.
|
||||
if (isEmpty) return false; // The canonical form of '' is '.'.
|
||||
if (_path == '.') return true;
|
||||
List segs = _path.split('/'); // Don't mask the getter 'segments'.
|
||||
if (segs[0] == '') {
|
||||
// Absolute path
|
||||
segs[0] = null; // Faster than removeRange().
|
||||
} else {
|
||||
// A canonical relative path may start with .. segments.
|
||||
for (int pos = 0; pos < segs.length && segs[pos] == '..'; ++pos) {
|
||||
segs[pos] = null;
|
||||
}
|
||||
}
|
||||
if (segs.last == '') segs.removeLast(); // Path ends with /.
|
||||
// No remaining segments can be ., .., or empty.
|
||||
return !segs.any((s) => s == '' || s == '.' || s == '..');
|
||||
}
|
||||
|
||||
Path makeCanonical() {
|
||||
bool isAbs = isAbsolute;
|
||||
List segs = segments();
|
||||
String drive;
|
||||
if (isAbs && !segs.isEmpty && segs[0].length == 2 && segs[0][1] == ':') {
|
||||
drive = segs[0];
|
||||
segs.removeRange(0, 1);
|
||||
}
|
||||
List newSegs = [];
|
||||
for (String segment in segs) {
|
||||
switch (segment) {
|
||||
case '..':
|
||||
// Absolute paths drop leading .. markers, including after a drive.
|
||||
if (newSegs.isEmpty) {
|
||||
if (isAbs) {
|
||||
// Do nothing: drop the segment.
|
||||
} else {
|
||||
newSegs.add('..');
|
||||
}
|
||||
} else if (newSegs.last == '..') {
|
||||
newSegs.add('..');
|
||||
} else {
|
||||
newSegs.removeLast();
|
||||
}
|
||||
break;
|
||||
case '.':
|
||||
case '':
|
||||
// Do nothing - drop the segment.
|
||||
break;
|
||||
default:
|
||||
newSegs.add(segment);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
List segmentsToJoin = [];
|
||||
if (isAbs) {
|
||||
segmentsToJoin.add('');
|
||||
if (drive != null) {
|
||||
segmentsToJoin.add(drive);
|
||||
}
|
||||
}
|
||||
|
||||
if (newSegs.isEmpty) {
|
||||
if (isAbs) {
|
||||
segmentsToJoin.add('');
|
||||
} else {
|
||||
segmentsToJoin.add('.');
|
||||
}
|
||||
} else {
|
||||
segmentsToJoin.addAll(newSegs);
|
||||
if (hasTrailingSeparator) {
|
||||
segmentsToJoin.add('');
|
||||
}
|
||||
}
|
||||
return new Path._internal(segmentsToJoin.join('/'), isWindowsShare);
|
||||
}
|
||||
|
||||
String toNativePath() {
|
||||
if (isEmpty) return '.';
|
||||
if (Platform.operatingSystem == 'windows') {
|
||||
String nativePath = _path;
|
||||
// Drop '/' before a drive letter.
|
||||
if (nativePath.length >= 3 &&
|
||||
nativePath.startsWith('/') &&
|
||||
nativePath[2] == ':') {
|
||||
nativePath = nativePath.substring(1);
|
||||
}
|
||||
nativePath = nativePath.replaceAll('/', '\\');
|
||||
if (isWindowsShare) {
|
||||
return '\\$nativePath';
|
||||
}
|
||||
return nativePath;
|
||||
}
|
||||
return _path;
|
||||
}
|
||||
|
||||
List<String> segments() {
|
||||
List result = _path.split('/');
|
||||
if (isAbsolute) result.removeRange(0, 1);
|
||||
if (hasTrailingSeparator) result.removeLast();
|
||||
return result;
|
||||
}
|
||||
|
||||
Path append(String finalSegment) {
|
||||
if (isEmpty) {
|
||||
return new Path._internal(finalSegment, isWindowsShare);
|
||||
} else if (hasTrailingSeparator) {
|
||||
return new Path._internal('$_path$finalSegment', isWindowsShare);
|
||||
} else {
|
||||
return new Path._internal('$_path/$finalSegment', isWindowsShare);
|
||||
}
|
||||
}
|
||||
|
||||
String get filenameWithoutExtension {
|
||||
var name = filename;
|
||||
if (name == '.' || name == '..') return name;
|
||||
int pos = name.lastIndexOf('.');
|
||||
return (pos < 0) ? name : name.substring(0, pos);
|
||||
}
|
||||
|
||||
String get extension {
|
||||
var name = filename;
|
||||
int pos = name.lastIndexOf('.');
|
||||
return (pos < 0) ? '' : name.substring(pos + 1);
|
||||
}
|
||||
|
||||
Path get directoryPath {
|
||||
int pos = _path.lastIndexOf('/');
|
||||
if (pos < 0) return new Path('');
|
||||
while (pos > 0 && _path[pos - 1] == '/') --pos;
|
||||
var dirPath = (pos > 0) ? _path.substring(0, pos) : '/';
|
||||
return new Path._internal(dirPath, isWindowsShare);
|
||||
}
|
||||
|
||||
String get filename {
|
||||
int pos = _path.lastIndexOf('/');
|
||||
return _path.substring(pos + 1);
|
||||
}
|
||||
}
|
||||
314
pkg/testing/lib/src/test_dart/status_expression.dart
Normal file
314
pkg/testing/lib/src/test_dart/status_expression.dart
Normal file
|
|
@ -0,0 +1,314 @@
|
|||
// Copyright (c) 2011, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE file.
|
||||
|
||||
library status_expression;
|
||||
|
||||
/**
|
||||
* Parse and evaluate expressions in a .status file for Dart and V8.
|
||||
* There are set expressions and Boolean expressions in a .status file.
|
||||
* The grammar is:
|
||||
* BooleanExpression := $variableName == value | $variableName != value |
|
||||
* $variableName | (BooleanExpression) |
|
||||
* BooleanExpression && BooleanExpression |
|
||||
* BooleanExpression || BooleanExpression
|
||||
*
|
||||
* SetExpression := value | (SetExpression) |
|
||||
* SetExpression || SetExpression |
|
||||
* SetExpression if BooleanExpression |
|
||||
* SetExpression , SetExpression
|
||||
*
|
||||
* Productions are listed in order of precedence, and the || and , operators
|
||||
* both evaluate to set union, but with different precedence.
|
||||
*
|
||||
* Values and variableNames are non-empty strings of word characters, matching
|
||||
* the RegExp \w+.
|
||||
*
|
||||
* Expressions evaluate as expected, with values of variables found in
|
||||
* an environment passed to the evaluator. The SetExpression "value"
|
||||
* evaluates to a singleton set containing that value. "A if B" evaluates
|
||||
* to A if B is true, and to the empty set if B is false.
|
||||
*/
|
||||
|
||||
class ExprEvaluationException {
|
||||
String error;
|
||||
|
||||
ExprEvaluationException(this.error);
|
||||
|
||||
toString() => error;
|
||||
}
|
||||
|
||||
class Token {
|
||||
static const String LEFT_PAREN = "(";
|
||||
static const String RIGHT_PAREN = ")";
|
||||
static const String DOLLAR_SYMBOL = r"$";
|
||||
static const String UNION = ",";
|
||||
static const String EQUALS = "==";
|
||||
static const String NOT_EQUALS = "!=";
|
||||
static const String AND = "&&";
|
||||
static const String OR = "||";
|
||||
}
|
||||
|
||||
class Tokenizer {
|
||||
String expression;
|
||||
List<String> tokens;
|
||||
|
||||
Tokenizer(String this.expression) : tokens = new List<String>();
|
||||
|
||||
// Tokens are : "(", ")", "$", ",", "&&", "||", "==", "!=", and (maximal) \w+.
|
||||
static final testRegexp =
|
||||
new RegExp(r"^([()$\w\s,]|(\&\&)|(\|\|)|(\=\=)|(\!\=))+$");
|
||||
static final regexp = new RegExp(r"[()$,]|(\&\&)|(\|\|)|(\=\=)|(\!\=)|\w+");
|
||||
|
||||
List<String> tokenize() {
|
||||
if (!testRegexp.hasMatch(expression)) {
|
||||
throw new FormatException("Syntax error in '$expression'");
|
||||
}
|
||||
for (Match match in regexp.allMatches(expression)) tokens.add(match[0]);
|
||||
return tokens;
|
||||
}
|
||||
}
|
||||
|
||||
abstract class BooleanExpression {
|
||||
bool evaluate(Map<String, String> environment);
|
||||
}
|
||||
|
||||
abstract class SetExpression {
|
||||
Set<String> evaluate(Map<String, String> environment);
|
||||
}
|
||||
|
||||
class Comparison implements BooleanExpression {
|
||||
TermVariable left;
|
||||
TermConstant right;
|
||||
bool negate;
|
||||
|
||||
Comparison(this.left, this.right, this.negate);
|
||||
|
||||
bool evaluate(environment) {
|
||||
return negate !=
|
||||
(left.termValue(environment) == right.termValue(environment));
|
||||
}
|
||||
|
||||
String toString() =>
|
||||
"(\$${left.name} ${negate ? '!=' : '=='} ${right.value})";
|
||||
}
|
||||
|
||||
class TermVariable {
|
||||
String name;
|
||||
|
||||
TermVariable(this.name);
|
||||
|
||||
String termValue(environment) {
|
||||
var value = environment[name];
|
||||
if (value == null) {
|
||||
throw new ExprEvaluationException("Could not find '$name' in environment "
|
||||
"while evaluating status file expression.");
|
||||
}
|
||||
return value.toString();
|
||||
}
|
||||
}
|
||||
|
||||
class TermConstant {
|
||||
String value;
|
||||
|
||||
TermConstant(String this.value);
|
||||
|
||||
String termValue(environment) => value;
|
||||
}
|
||||
|
||||
class BooleanVariable implements BooleanExpression {
|
||||
TermVariable variable;
|
||||
|
||||
BooleanVariable(this.variable);
|
||||
|
||||
bool evaluate(environment) => variable.termValue(environment) == 'true';
|
||||
String toString() => "(bool \$${variable.name})";
|
||||
}
|
||||
|
||||
class BooleanOperation implements BooleanExpression {
|
||||
String op;
|
||||
BooleanExpression left;
|
||||
BooleanExpression right;
|
||||
|
||||
BooleanOperation(this.op, this.left, this.right);
|
||||
|
||||
bool evaluate(environment) => (op == Token.AND)
|
||||
? left.evaluate(environment) && right.evaluate(environment)
|
||||
: left.evaluate(environment) || right.evaluate(environment);
|
||||
String toString() => "($left $op $right)";
|
||||
}
|
||||
|
||||
class SetUnion implements SetExpression {
|
||||
SetExpression left;
|
||||
SetExpression right;
|
||||
|
||||
SetUnion(this.left, this.right);
|
||||
|
||||
// Overwrites left.evaluate(env).
|
||||
// Set.addAll does not return this.
|
||||
Set<String> evaluate(environment) {
|
||||
Set<String> result = left.evaluate(environment);
|
||||
result.addAll(right.evaluate(environment));
|
||||
return result;
|
||||
}
|
||||
|
||||
String toString() => "($left || $right)";
|
||||
}
|
||||
|
||||
class SetIf implements SetExpression {
|
||||
SetExpression left;
|
||||
BooleanExpression right;
|
||||
|
||||
SetIf(this.left, this.right);
|
||||
|
||||
Set<String> evaluate(environment) => right.evaluate(environment)
|
||||
? left.evaluate(environment)
|
||||
: new Set<String>();
|
||||
String toString() => "($left if $right)";
|
||||
}
|
||||
|
||||
class SetConstant implements SetExpression {
|
||||
String value;
|
||||
|
||||
SetConstant(String v) : value = v.toLowerCase();
|
||||
|
||||
Set<String> evaluate(environment) => new Set<String>.from([value]);
|
||||
String toString() => value;
|
||||
}
|
||||
|
||||
// An iterator that allows peeking at the current token.
|
||||
class Scanner {
|
||||
List<String> tokens;
|
||||
Iterator tokenIterator;
|
||||
String current;
|
||||
|
||||
Scanner(this.tokens) {
|
||||
tokenIterator = tokens.iterator;
|
||||
advance();
|
||||
}
|
||||
|
||||
bool hasMore() => current != null;
|
||||
|
||||
void advance() {
|
||||
current = tokenIterator.moveNext() ? tokenIterator.current : null;
|
||||
}
|
||||
}
|
||||
|
||||
class ExpressionParser {
|
||||
Scanner scanner;
|
||||
|
||||
ExpressionParser(this.scanner);
|
||||
|
||||
SetExpression parseSetExpression() => parseSetUnion();
|
||||
|
||||
SetExpression parseSetUnion() {
|
||||
SetExpression left = parseSetIf();
|
||||
while (scanner.hasMore() && scanner.current == Token.UNION) {
|
||||
scanner.advance();
|
||||
SetExpression right = parseSetIf();
|
||||
left = new SetUnion(left, right);
|
||||
}
|
||||
return left;
|
||||
}
|
||||
|
||||
SetExpression parseSetIf() {
|
||||
SetExpression left = parseSetOr();
|
||||
while (scanner.hasMore() && scanner.current == "if") {
|
||||
scanner.advance();
|
||||
BooleanExpression right = parseBooleanExpression();
|
||||
left = new SetIf(left, right);
|
||||
}
|
||||
return left;
|
||||
}
|
||||
|
||||
SetExpression parseSetOr() {
|
||||
SetExpression left = parseSetAtomic();
|
||||
while (scanner.hasMore() && scanner.current == Token.OR) {
|
||||
scanner.advance();
|
||||
SetExpression right = parseSetAtomic();
|
||||
left = new SetUnion(left, right);
|
||||
}
|
||||
return left;
|
||||
}
|
||||
|
||||
SetExpression parseSetAtomic() {
|
||||
if (scanner.current == Token.LEFT_PAREN) {
|
||||
scanner.advance();
|
||||
SetExpression value = parseSetExpression();
|
||||
if (scanner.current != Token.RIGHT_PAREN) {
|
||||
throw new FormatException("Missing right parenthesis in expression");
|
||||
}
|
||||
scanner.advance();
|
||||
return value;
|
||||
}
|
||||
if (!new RegExp(r"^\w+$").hasMatch(scanner.current)) {
|
||||
throw new FormatException(
|
||||
"Expected identifier in expression, got ${scanner.current}");
|
||||
}
|
||||
SetExpression value = new SetConstant(scanner.current);
|
||||
scanner.advance();
|
||||
return value;
|
||||
}
|
||||
|
||||
BooleanExpression parseBooleanExpression() => parseBooleanOr();
|
||||
|
||||
BooleanExpression parseBooleanOr() {
|
||||
BooleanExpression left = parseBooleanAnd();
|
||||
while (scanner.hasMore() && scanner.current == Token.OR) {
|
||||
scanner.advance();
|
||||
BooleanExpression right = parseBooleanAnd();
|
||||
left = new BooleanOperation(Token.OR, left, right);
|
||||
}
|
||||
return left;
|
||||
}
|
||||
|
||||
BooleanExpression parseBooleanAnd() {
|
||||
BooleanExpression left = parseBooleanAtomic();
|
||||
while (scanner.hasMore() && scanner.current == Token.AND) {
|
||||
scanner.advance();
|
||||
BooleanExpression right = parseBooleanAtomic();
|
||||
left = new BooleanOperation(Token.AND, left, right);
|
||||
}
|
||||
return left;
|
||||
}
|
||||
|
||||
BooleanExpression parseBooleanAtomic() {
|
||||
if (scanner.current == Token.LEFT_PAREN) {
|
||||
scanner.advance();
|
||||
BooleanExpression value = parseBooleanExpression();
|
||||
if (scanner.current != Token.RIGHT_PAREN) {
|
||||
throw new FormatException("Missing right parenthesis in expression");
|
||||
}
|
||||
scanner.advance();
|
||||
return value;
|
||||
}
|
||||
|
||||
// The only atomic booleans are of the form $variable == value or
|
||||
// of the form $variable.
|
||||
if (scanner.current != Token.DOLLAR_SYMBOL) {
|
||||
throw new FormatException(
|
||||
"Expected \$ in expression, got ${scanner.current}");
|
||||
}
|
||||
scanner.advance();
|
||||
if (!new RegExp(r"^\w+$").hasMatch(scanner.current)) {
|
||||
throw new FormatException(
|
||||
"Expected identifier in expression, got ${scanner.current}");
|
||||
}
|
||||
TermVariable left = new TermVariable(scanner.current);
|
||||
scanner.advance();
|
||||
if (scanner.current == Token.EQUALS ||
|
||||
scanner.current == Token.NOT_EQUALS) {
|
||||
bool negate = scanner.current == Token.NOT_EQUALS;
|
||||
scanner.advance();
|
||||
if (!new RegExp(r"^\w+$").hasMatch(scanner.current)) {
|
||||
throw new FormatException(
|
||||
"Expected value in expression, got ${scanner.current}");
|
||||
}
|
||||
TermConstant right = new TermConstant(scanner.current);
|
||||
scanner.advance();
|
||||
return new Comparison(left, right, negate);
|
||||
} else {
|
||||
return new BooleanVariable(left);
|
||||
}
|
||||
}
|
||||
}
|
||||
341
pkg/testing/lib/src/test_dart/status_file_parser.dart
Normal file
341
pkg/testing/lib/src/test_dart/status_file_parser.dart
Normal file
|
|
@ -0,0 +1,341 @@
|
|||
// Copyright (c) 2012, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE file.
|
||||
|
||||
library status_file_parser;
|
||||
|
||||
import "dart:async";
|
||||
import "dart:convert" show LineSplitter, UTF8;
|
||||
import "dart:io";
|
||||
|
||||
import "path.dart";
|
||||
import "status_expression.dart";
|
||||
|
||||
class Expectation {
|
||||
// Possible outcomes of running a test.
|
||||
static Expectation PASS = byName('Pass');
|
||||
static Expectation CRASH = byName('Crash');
|
||||
static Expectation TIMEOUT = byName('Timeout');
|
||||
static Expectation FAIL = byName('Fail');
|
||||
|
||||
// Special 'FAIL' cases
|
||||
static Expectation RUNTIME_ERROR = byName('RuntimeError');
|
||||
static Expectation COMPILETIME_ERROR = byName('CompileTimeError');
|
||||
static Expectation MISSING_RUNTIME_ERROR = byName('MissingRuntimeError');
|
||||
static Expectation MISSING_COMPILETIME_ERROR =
|
||||
byName('MissingCompileTimeError');
|
||||
static Expectation STATIC_WARNING = byName('StaticWarning');
|
||||
static Expectation MISSING_STATIC_WARNING = byName('MissingStaticWarning');
|
||||
static Expectation PUB_GET_ERROR = byName('PubGetError');
|
||||
|
||||
// "meta expectations"
|
||||
static Expectation OK = byName('Ok');
|
||||
static Expectation SLOW = byName('Slow');
|
||||
static Expectation SKIP = byName('Skip');
|
||||
static Expectation SKIP_SLOW = byName('SkipSlow');
|
||||
static Expectation SKIP_BY_DESIGN = byName('SkipByDesign');
|
||||
|
||||
static Expectation byName(String name) {
|
||||
_initialize();
|
||||
name = name.toLowerCase();
|
||||
if (!_AllExpectations.containsKey(name)) {
|
||||
throw new Exception("Expectation.byName(name='$name'): Invalid name.");
|
||||
}
|
||||
return _AllExpectations[name];
|
||||
}
|
||||
|
||||
// Keep a map of all possible Expectation objects, initialized lazily.
|
||||
static Map<String, Expectation> _AllExpectations;
|
||||
static void _initialize() {
|
||||
if (_AllExpectations == null) {
|
||||
_AllExpectations = new Map<String, Expectation>();
|
||||
|
||||
Expectation build(prettyName, {group: null, isMetaExpectation: false}) {
|
||||
var expectation = new Expectation._(prettyName,
|
||||
group: group, isMetaExpectation: isMetaExpectation);
|
||||
assert(!_AllExpectations.containsKey(expectation.name));
|
||||
return _AllExpectations[expectation.name] = expectation;
|
||||
}
|
||||
|
||||
var fail = build("Fail");
|
||||
build("Pass");
|
||||
build("Crash");
|
||||
build("Timeout");
|
||||
|
||||
build("MissingCompileTimeError", group: fail);
|
||||
build("MissingRuntimeError", group: fail);
|
||||
build("CompileTimeError", group: fail);
|
||||
build("RuntimeError", group: fail);
|
||||
|
||||
build("MissingStaticWarning", group: fail);
|
||||
build("StaticWarning", group: fail);
|
||||
|
||||
build("PubGetError", group: fail);
|
||||
|
||||
var skip = build("Skip", isMetaExpectation: true);
|
||||
build("SkipByDesign", isMetaExpectation: true);
|
||||
build("SkipSlow", group: skip, isMetaExpectation: true);
|
||||
build("Ok", isMetaExpectation: true);
|
||||
build("Slow", isMetaExpectation: true);
|
||||
}
|
||||
}
|
||||
|
||||
final String prettyName;
|
||||
final String name;
|
||||
final Expectation group;
|
||||
// Indicates whether this expectation cannot be a test outcome (i.e. it is a
|
||||
// "meta marker").
|
||||
final bool isMetaExpectation;
|
||||
|
||||
Expectation._(prettyName,
|
||||
{Expectation this.group: null, bool this.isMetaExpectation: false})
|
||||
: prettyName = prettyName,
|
||||
name = prettyName.toLowerCase();
|
||||
|
||||
bool canBeOutcomeOf(Expectation expectation) {
|
||||
Expectation outcome = this;
|
||||
while (outcome != null) {
|
||||
if (outcome == expectation) {
|
||||
return true;
|
||||
}
|
||||
outcome = outcome.group;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
String toString() => prettyName;
|
||||
}
|
||||
|
||||
final RegExp SplitComment = new RegExp("^([^#]*)(#.*)?\$");
|
||||
final RegExp HeaderPattern = new RegExp(r"^\[([^\]]+)\]");
|
||||
final RegExp RulePattern = new RegExp(r"\s*([^: ]*)\s*:(.*)");
|
||||
final RegExp IssueNumberPattern = new RegExp("[Ii]ssue ([0-9]+)");
|
||||
|
||||
class StatusFile {
|
||||
final Path location;
|
||||
|
||||
StatusFile(this.location);
|
||||
}
|
||||
|
||||
// TODO(whesse): Implement configuration_info library that contains data
|
||||
// structures for test configuration, including Section.
|
||||
class Section {
|
||||
final StatusFile statusFile;
|
||||
|
||||
final BooleanExpression condition;
|
||||
final List<TestRule> testRules;
|
||||
final int lineNumber;
|
||||
|
||||
Section.always(this.statusFile, this.lineNumber)
|
||||
: condition = null,
|
||||
testRules = new List<TestRule>();
|
||||
Section(this.statusFile, this.condition, this.lineNumber)
|
||||
: testRules = new List<TestRule>();
|
||||
|
||||
bool isEnabled(environment) =>
|
||||
condition == null || condition.evaluate(environment);
|
||||
|
||||
String toString() {
|
||||
return "Section: $condition";
|
||||
}
|
||||
}
|
||||
|
||||
Future<TestExpectations> ReadTestExpectations(
|
||||
List<String> statusFilePaths, Map environment) {
|
||||
var testExpectations = new TestExpectations();
|
||||
return Future.wait(statusFilePaths.map((String statusFile) {
|
||||
return ReadTestExpectationsInto(testExpectations, statusFile, environment);
|
||||
})).then((_) => testExpectations);
|
||||
}
|
||||
|
||||
Future ReadTestExpectationsInto(
|
||||
TestExpectations expectations, String statusFilePath, environment) {
|
||||
var completer = new Completer();
|
||||
List<Section> sections = new List<Section>();
|
||||
|
||||
void sectionsRead() {
|
||||
for (Section section in sections) {
|
||||
if (section.isEnabled(environment)) {
|
||||
for (var rule in section.testRules) {
|
||||
expectations.addRule(rule, environment);
|
||||
}
|
||||
}
|
||||
}
|
||||
completer.complete();
|
||||
}
|
||||
|
||||
ReadConfigurationInto(new Path(statusFilePath), sections, sectionsRead);
|
||||
return completer.future;
|
||||
}
|
||||
|
||||
void ReadConfigurationInto(Path path, sections, onDone) {
|
||||
StatusFile statusFile = new StatusFile(path);
|
||||
File file = new File(path.toNativePath());
|
||||
if (!file.existsSync()) {
|
||||
throw new Exception('Cannot find test status file $path');
|
||||
}
|
||||
int lineNumber = 0;
|
||||
Stream<String> lines =
|
||||
file.openRead().transform(UTF8.decoder).transform(new LineSplitter());
|
||||
|
||||
Section currentSection = new Section.always(statusFile, -1);
|
||||
sections.add(currentSection);
|
||||
|
||||
lines.listen((String line) {
|
||||
lineNumber++;
|
||||
Match match = SplitComment.firstMatch(line);
|
||||
line = (match == null) ? "" : match[1];
|
||||
line = line.trim();
|
||||
if (line.isEmpty) return;
|
||||
|
||||
// Extract the comment to get the issue number if needed.
|
||||
String comment = (match == null || match[2] == null) ? "" : match[2];
|
||||
|
||||
match = HeaderPattern.firstMatch(line);
|
||||
if (match != null) {
|
||||
String condition_string = match[1].trim();
|
||||
List<String> tokens = new Tokenizer(condition_string).tokenize();
|
||||
ExpressionParser parser = new ExpressionParser(new Scanner(tokens));
|
||||
currentSection =
|
||||
new Section(statusFile, parser.parseBooleanExpression(), lineNumber);
|
||||
sections.add(currentSection);
|
||||
return;
|
||||
}
|
||||
|
||||
match = RulePattern.firstMatch(line);
|
||||
if (match != null) {
|
||||
String name = match[1].trim();
|
||||
// TODO(whesse): Handle test names ending in a wildcard (*).
|
||||
String expression_string = match[2].trim();
|
||||
List<String> tokens = new Tokenizer(expression_string).tokenize();
|
||||
SetExpression expression =
|
||||
new ExpressionParser(new Scanner(tokens)).parseSetExpression();
|
||||
|
||||
// Look for issue number in comment.
|
||||
String issueString = null;
|
||||
match = IssueNumberPattern.firstMatch(comment);
|
||||
if (match != null) {
|
||||
issueString = match[1];
|
||||
if (issueString == null) issueString = match[2];
|
||||
}
|
||||
int issue = issueString != null ? int.parse(issueString) : null;
|
||||
currentSection.testRules
|
||||
.add(new TestRule(name, expression, issue, lineNumber));
|
||||
return;
|
||||
}
|
||||
|
||||
print("unmatched line: $line");
|
||||
}, onDone: onDone);
|
||||
}
|
||||
|
||||
class TestRule {
|
||||
String name;
|
||||
SetExpression expression;
|
||||
int issue;
|
||||
int lineNumber;
|
||||
|
||||
TestRule(this.name, this.expression, this.issue, this.lineNumber);
|
||||
|
||||
bool get hasIssue => issue != null;
|
||||
|
||||
String toString() => 'TestRule($name, $expression, $issue)';
|
||||
}
|
||||
|
||||
class TestExpectations {
|
||||
// Only create one copy of each Set<Expectation>.
|
||||
// We just use .toString as a key, so we may make a few
|
||||
// sets that only differ in their toString element order.
|
||||
static Map _cachedSets = new Map();
|
||||
|
||||
Map _map;
|
||||
bool _preprocessed = false;
|
||||
Map _regExpCache;
|
||||
Map _keyToRegExps;
|
||||
|
||||
/**
|
||||
* Create a TestExpectations object. See the [expectations] method
|
||||
* for an explanation of matching.
|
||||
*/
|
||||
TestExpectations() : _map = new Map();
|
||||
|
||||
/**
|
||||
* Add a rule to the expectations.
|
||||
*/
|
||||
void addRule(testRule, environment) {
|
||||
// Once we have started using the expectations we cannot add more
|
||||
// rules.
|
||||
if (_preprocessed) {
|
||||
throw "TestExpectations.addRule: cannot add more rules";
|
||||
}
|
||||
var names = testRule.expression.evaluate(environment);
|
||||
var expectations = names.map((name) => Expectation.byName(name));
|
||||
_map.putIfAbsent(testRule.name, () => new Set()).addAll(expectations);
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute the expectations for a test based on the filename.
|
||||
*
|
||||
* For every (key, expectation) pair. Match the key with the file
|
||||
* name. Return the union of the expectations for all the keys
|
||||
* that match.
|
||||
*
|
||||
* Normal matching splits the key and the filename into path
|
||||
* components and checks that the anchored regular expression
|
||||
* "^$keyComponent\$" matches the corresponding filename component.
|
||||
*/
|
||||
Set<Expectation> expectations(String filename) {
|
||||
var result = new Set();
|
||||
var splitFilename = filename.split('/');
|
||||
|
||||
// Create mapping from keys to list of RegExps once and for all.
|
||||
_preprocessForMatching();
|
||||
|
||||
_map.forEach((key, expectation) {
|
||||
List regExps = _keyToRegExps[key];
|
||||
if (regExps.length > splitFilename.length) return;
|
||||
for (var i = 0; i < regExps.length; i++) {
|
||||
if (!regExps[i].hasMatch(splitFilename[i])) return;
|
||||
}
|
||||
// If all components of the status file key matches the filename
|
||||
// add the expectations to the result.
|
||||
result.addAll(expectation);
|
||||
});
|
||||
|
||||
// If no expectations were found the expectation is that the test
|
||||
// passes.
|
||||
if (result.isEmpty) {
|
||||
result.add(Expectation.PASS);
|
||||
}
|
||||
return _cachedSets.putIfAbsent(result.toString(), () => result);
|
||||
}
|
||||
|
||||
// Preprocess the expectations for matching against
|
||||
// filenames. Generate lists of regular expressions once and for all
|
||||
// for each key.
|
||||
void _preprocessForMatching() {
|
||||
if (_preprocessed) return;
|
||||
|
||||
_keyToRegExps = new Map();
|
||||
_regExpCache = new Map();
|
||||
|
||||
_map.forEach((key, expectations) {
|
||||
if (_keyToRegExps[key] != null) return;
|
||||
var splitKey = key.split('/');
|
||||
var regExps = new List(splitKey.length);
|
||||
for (var i = 0; i < splitKey.length; i++) {
|
||||
var component = splitKey[i];
|
||||
var regExp = _regExpCache[component];
|
||||
if (regExp == null) {
|
||||
var pattern = "^${splitKey[i]}\$".replaceAll('*', '.*');
|
||||
regExp = new RegExp(pattern);
|
||||
_regExpCache[component] = regExp;
|
||||
}
|
||||
regExps[i] = regExp;
|
||||
}
|
||||
_keyToRegExps[key] = regExps;
|
||||
});
|
||||
|
||||
_regExpCache = null;
|
||||
_preprocessed = true;
|
||||
}
|
||||
}
|
||||
67
pkg/testing/lib/src/test_description.dart
Normal file
67
pkg/testing/lib/src/test_description.dart
Normal file
|
|
@ -0,0 +1,67 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
library testing.test_description;
|
||||
|
||||
import 'dart:io' show
|
||||
File,
|
||||
FileSystemEntity;
|
||||
|
||||
class TestDescription implements Comparable<TestDescription> {
|
||||
final Uri root;
|
||||
final File file;
|
||||
final Uri output;
|
||||
|
||||
TestDescription(this.root, this.file, {this.output});
|
||||
|
||||
Uri get uri => file.uri;
|
||||
|
||||
String get shortName {
|
||||
String baseName = "$uri".substring("$root".length);
|
||||
return baseName.substring(0, baseName.length - ".dart".length);
|
||||
}
|
||||
|
||||
String get escapedName => shortName.replaceAll("/", "__");
|
||||
|
||||
void writeImportOn(StringSink sink) {
|
||||
sink.write("import '");
|
||||
sink.write(uri);
|
||||
sink.write("' as ");
|
||||
sink.write(escapedName);
|
||||
sink.writeln(" show main;");
|
||||
}
|
||||
|
||||
void writeClosureOn(StringSink sink) {
|
||||
sink.write(' "');
|
||||
sink.write(shortName);
|
||||
sink.write('": ');
|
||||
sink.write(escapedName);
|
||||
sink.writeln('.main,');
|
||||
}
|
||||
|
||||
static TestDescription from(
|
||||
Uri root, FileSystemEntity entity, {Pattern pattern}) {
|
||||
if (entity is! File) return null;
|
||||
pattern ??= "_test.dart";
|
||||
String path = entity.uri.path;
|
||||
bool hasMatch = false;
|
||||
if (pattern is String) {
|
||||
if (path.endsWith(pattern)) hasMatch = true;
|
||||
} else if (path.contains(pattern)) {
|
||||
hasMatch = true;
|
||||
}
|
||||
return hasMatch ? new TestDescription(root, entity) : null;
|
||||
}
|
||||
|
||||
int compareTo(TestDescription other) => "$uri".compareTo("${other.uri}");
|
||||
|
||||
String formatError(String message) {
|
||||
String base = Uri.base.toFilePath();
|
||||
String path = uri.toFilePath();
|
||||
if (path.startsWith(base)) {
|
||||
path = path.substring(base.length);
|
||||
}
|
||||
return "$path:$message";
|
||||
}
|
||||
}
|
||||
104
pkg/testing/lib/src/test_root.dart
Normal file
104
pkg/testing/lib/src/test_root.dart
Normal file
|
|
@ -0,0 +1,104 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
library testing.test_root;
|
||||
|
||||
import 'dart:async' show
|
||||
Future;
|
||||
|
||||
import 'dart:convert' show
|
||||
JSON;
|
||||
|
||||
import 'dart:io' show
|
||||
File;
|
||||
|
||||
import '../testing.dart' show
|
||||
Chain;
|
||||
|
||||
import 'analyze.dart' show
|
||||
Analyze;
|
||||
|
||||
import 'suite.dart' show
|
||||
Dart,
|
||||
Suite;
|
||||
|
||||
/// Records properties of a test root. The information is read from a JSON file.
|
||||
///
|
||||
/// Example with comments:
|
||||
/// {
|
||||
/// # Path to the `.packages` file used.
|
||||
/// "packages": "test/.packages",
|
||||
/// # A list of test suites (collection of tests).
|
||||
/// "suites": [
|
||||
/// # A list of suite objects. See the subclasses of [Suite] below.
|
||||
/// ],
|
||||
/// "analyze": {
|
||||
/// # Uris to analyze.
|
||||
/// "uris": [
|
||||
/// "lib/",
|
||||
/// "bin/dartk.dart",
|
||||
/// "bin/repl.dart",
|
||||
/// "test/log_analyzer.dart",
|
||||
/// "third_party/testing/lib/"
|
||||
/// ],
|
||||
/// # Regular expressions of file names to ignore when analyzing.
|
||||
/// "exclude": [
|
||||
/// "/third_party/dart-sdk/pkg/compiler/",
|
||||
/// "/third_party/kernel/"
|
||||
/// ]
|
||||
/// }
|
||||
/// }
|
||||
class TestRoot {
|
||||
final Uri packages;
|
||||
|
||||
final List<Suite> suites;
|
||||
|
||||
TestRoot(this.packages, this.suites);
|
||||
|
||||
Analyze get analyze => suites.last;
|
||||
|
||||
List<Uri> get urisToAnalyze => analyze.uris;
|
||||
|
||||
List<RegExp> get excludedFromAnalysis => analyze.exclude;
|
||||
|
||||
Iterable<Dart> get dartSuites {
|
||||
return new List<Dart>.from(
|
||||
suites.where((Suite suite) => suite is Dart));
|
||||
}
|
||||
|
||||
Iterable<Chain> get toolChains {
|
||||
return new List<Chain>.from(
|
||||
suites.where((Suite suite) => suite is Chain));
|
||||
}
|
||||
|
||||
String toString() {
|
||||
return "TestRoot($suites, $urisToAnalyze)";
|
||||
}
|
||||
|
||||
static Future<TestRoot> fromUri(Uri uri) async {
|
||||
String json = await new File.fromUri(uri).readAsString();
|
||||
Map data = JSON.decode(json);
|
||||
|
||||
addDefaults(data);
|
||||
|
||||
Uri packages = uri.resolve(data["packages"]);
|
||||
|
||||
List<Suite> suites = new List<Suite>.from(
|
||||
data["suites"].map((Map json) => new Suite.fromJsonMap(uri, json)));
|
||||
|
||||
Analyze analyze = await Analyze.fromJsonMap(uri, data["analyze"], suites);
|
||||
|
||||
suites.add(analyze);
|
||||
|
||||
return new TestRoot(packages, suites);
|
||||
}
|
||||
|
||||
static void addDefaults(Map data) {
|
||||
data.putIfAbsent("packages", () => ".packages");
|
||||
data.putIfAbsent("suites", () => []);
|
||||
Map analyze = data.putIfAbsent("analyze", () => {});
|
||||
analyze.putIfAbsent("uris", () => []);
|
||||
analyze.putIfAbsent("exclude", () => []);
|
||||
}
|
||||
}
|
||||
89
pkg/testing/lib/src/zone_helper.dart
Normal file
89
pkg/testing/lib/src/zone_helper.dart
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
/// Helper functions for running code in a Zone.
|
||||
library testing.zone_helper;
|
||||
|
||||
import 'dart:async' show
|
||||
Completer,
|
||||
Future,
|
||||
ZoneSpecification,
|
||||
runZoned;
|
||||
|
||||
import 'dart:isolate' show
|
||||
Capability,
|
||||
Isolate,
|
||||
ReceivePort;
|
||||
|
||||
import 'log.dart' show
|
||||
logUncaughtError;
|
||||
|
||||
Future runGuarded(
|
||||
Future f(),
|
||||
{void printLineOnStdout(line),
|
||||
void handleLateError(error, StackTrace stackTrace)}) {
|
||||
|
||||
var printWrapper;
|
||||
if (printLineOnStdout != null) {
|
||||
printWrapper = (_1, _2, _3, String line) {
|
||||
printLineOnStdout(line);
|
||||
};
|
||||
}
|
||||
|
||||
Completer completer = new Completer();
|
||||
|
||||
handleUncaughtError(error, StackTrace stackTrace) {
|
||||
logUncaughtError(error, stackTrace);
|
||||
if (!completer.isCompleted) {
|
||||
completer.completeError(error, stackTrace);
|
||||
} else if (handleLateError != null) {
|
||||
handleLateError(error, stackTrace);
|
||||
} else {
|
||||
// Delegate to parent.
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
ZoneSpecification specification = new ZoneSpecification(print: printWrapper);
|
||||
|
||||
ReceivePort errorPort = new ReceivePort();
|
||||
Future errorFuture = errorPort.listen((List errors) {
|
||||
Isolate.current.removeErrorListener(errorPort.sendPort);
|
||||
errorPort.close();
|
||||
var error = errors[0];
|
||||
var stackTrace = errors[1];
|
||||
if (stackTrace != null) {
|
||||
stackTrace = new StackTrace.fromString(stackTrace);
|
||||
}
|
||||
handleUncaughtError(error, stackTrace);
|
||||
}).asFuture();
|
||||
|
||||
Isolate.current.addErrorListener(errorPort.sendPort);
|
||||
return acknowledgeControlMessages(Isolate.current).then((_) {
|
||||
runZoned(
|
||||
() => new Future(f).then(completer.complete),
|
||||
zoneSpecification: specification,
|
||||
onError: handleUncaughtError);
|
||||
|
||||
return completer.future.whenComplete(() {
|
||||
errorPort.close();
|
||||
Isolate.current.removeErrorListener(errorPort.sendPort);
|
||||
return acknowledgeControlMessages(Isolate.current)
|
||||
.then((_) => errorFuture);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/// Ping [isolate] to ensure control messages have been delivered. Control
|
||||
/// messages are things like [Isolate.addErrorListener] and
|
||||
/// [Isolate.addOnExitListener].
|
||||
Future acknowledgeControlMessages(Isolate isolate, {Capability resume}) {
|
||||
ReceivePort ping = new ReceivePort();
|
||||
Isolate.current.ping(ping.sendPort);
|
||||
if (resume == null) {
|
||||
return ping.first;
|
||||
} else {
|
||||
return ping.first.then((_) => isolate.resume(resume));
|
||||
}
|
||||
}
|
||||
25
pkg/testing/lib/testing.dart
Normal file
25
pkg/testing/lib/testing.dart
Normal file
|
|
@ -0,0 +1,25 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
library testing;
|
||||
|
||||
export 'dart:async' show
|
||||
Future;
|
||||
|
||||
export 'src/discover.dart';
|
||||
|
||||
export 'src/test_description.dart';
|
||||
|
||||
export 'src/chain.dart' show
|
||||
Chain,
|
||||
ChainContext,
|
||||
Result,
|
||||
Step;
|
||||
|
||||
export 'src/stdio_process.dart' show
|
||||
StdioProcess;
|
||||
|
||||
export 'src/run.dart' show
|
||||
run,
|
||||
runMe;
|
||||
7
pkg/testing/test/dart_sdk_negative_test.dart
Normal file
7
pkg/testing/test/dart_sdk_negative_test.dart
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
main() {
|
||||
throw "This is a negative test.";
|
||||
}
|
||||
3
pkg/testing/test/dart_vm_suite.status
Normal file
3
pkg/testing/test/dart_vm_suite.status
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
# Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
# for details. All rights reserved. Use of this source code is governed by a
|
||||
# BSD-style license that can be found in the LICENSE.md file.
|
||||
7
pkg/testing/test/hello_test.dart
Normal file
7
pkg/testing/test/hello_test.dart
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
// Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file
|
||||
// for details. All rights reserved. Use of this source code is governed by a
|
||||
// BSD-style license that can be found in the LICENSE.md file.
|
||||
|
||||
main() {
|
||||
print("Hello, World!");
|
||||
}
|
||||
24
pkg/testing/testing.json
Normal file
24
pkg/testing/testing.json
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
{ "Copyright (c) 2016, the Dart project authors. Please see the AUTHORS file":0,
|
||||
"for details. All rights reserved. Use of this source code is governed by a":0,
|
||||
"BSD-style license that can be found in the LICENSE.md file.":0,
|
||||
"suites": [
|
||||
{
|
||||
"name": "dart_vm",
|
||||
"kind": "Chain",
|
||||
"source": "package:testing/dart_vm_suite.dart",
|
||||
"path": "test/",
|
||||
"status": "test/dart_vm_suite.status",
|
||||
"pattern": [
|
||||
"\\.dart$"
|
||||
],
|
||||
"exclude": [
|
||||
]
|
||||
}
|
||||
],
|
||||
"analyze": {
|
||||
"uris": [
|
||||
"lib/",
|
||||
"bin/run_tests.dart"
|
||||
]
|
||||
}
|
||||
}
|
||||
Loading…
Reference in a new issue