The old code to report the difference between a tests's expected and
actual errors was easily confused by missing or unexpected errors.
Those would misalign the lists of errors causing all subsequent errors
to be reported as wrong. This fixes that. Before, this test containing
a single unexpected error:
main() {
int a = "s";
// ^^^
// [analyzer] STATIC_TYPE_WARNING.INVALID_ASSIGNMENT
unknownFunction();
int b = "s";
// ^^^
// [analyzer] STATIC_TYPE_WARNING.INVALID_ASSIGNMENT
int c = "s";
// ^^^
// [analyzer] STATIC_TYPE_WARNING.INVALID_ASSIGNMENT
}
Would cause the test runner to output:
incorrect static errors:
Error at line 6, column 3, length 15
- Expected on line 8 but was on 6.
- Expected on column 11 but was on 3.
- Expected length 3 but was 15.
- Expected error code STATIC_TYPE_WARNING.INVALID_ASSIGNMENT but was STATIC_TYPE_WARNING.UNDEFINED_FUNCTION.
Error at line 8, column 11, length 3
- Expected on line 12 but was on 8.
reported unexpected static errors:
Error at line 12, column 11, length 3
STATIC_TYPE_WARNING.INVALID_ASSIGNMENT
Oof. Now it prints:
static error failures:
Unexpected static error at line 6, column 3, length 15:
- Had error code STATIC_TYPE_WARNING.UNDEFINED_FUNCTION.
Change-Id: I1b22b2a542c6f8049f92fda3627ae69144bfd52a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/114240
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Leaf Petersen <leafp@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Move existing test status from the dart2js status file.
Change-Id: I8c9f296119ae23da40896581c1f1a1a9cba11494
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/112646
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Previously we would not have any VM logs because they are were sent to the Android
system logs, which are not captured by the bots.
Also added a flag --android-log-to-stderr, which is useful for local testing
and debugging.
Change-Id: I1968b4c230d70b5546d2dd48e40fe3cfe5196def
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/112250
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Note: Changes in this CL will require some google3 changes (mostly in code related to test entrypoints).
We encountered an issue whereby cross-module programs importing files
with the same name (but separate identities/locations) were being
overridden throughout all of DDK's module builders. For example, a
module importing "x.dart" and "y/x.dart" would export {x: x, x: x$} -
the latter renamed module shadowing the first. DDC handles this by
prepending a formatted module-root-relative identifier string, so
"y/x.dart" becomes something like y__x, and the export becomes {x: x,
y__x: y__x}. However, this results in some code bloat, especially as
paths become deeply nested. More importantly, the $-renaming scheme is
not consistent across modules. Were we to export x$ instead of x, how
would the importing library know whether x or x$ referred to "y/x.dart"?
To have the best of both worlds, we're introducing the concept of a
module alias that exists at the boundary of module imports and exports.
We use this module-root-relative alias to differentiate modules with the
same name but use the shorter $-renaming scheme throughout the generated
code. Relevant points:
1) jsLibraryAlias has been introduced alongside jsLibraryName.
jsLibraryName no longer must be unique - that property having been
transferred to the alias. DDK no longer just outputs pathSegments.last,
instead adopting DDC's scheme of "creating" a special alias from the
module root. jsLibraryName's temporary IDs are renamed using the
existing DDK scheme.
2) Libraries are now associated with NameSpecifiers. The name and asName
fields represent jsName and jsAlias, respectively, for exports
(vice-versa for imports).
3) The test frameworks have been taking advantage of DDK's previous
naming scheme (throwing way the module root, keeping just the name). A
common pattern is to expect DDK's output symbol to be "someexport", then
forcing DDC's output (normally "some__path__someexport") to resemble
that with the deprecated "library-root" flag set to the path of the
output JS file. We resolve this by updating the scaffolding code to look
for the module-root-relative (long) name, which requires a bunch of
path-to-identifier subsitution logic. The "absoluteRoot" bool in the
sourcemap test runner is a way to reenable this DDC-specific deprecated
behavior. The test runner now requires this alias in the html generator.
Fixes https://github.com/dart-lang/sdk/issues/37473
Change-Id: Iaaa82f3350d424af195967b55c87cb32b30a3d85
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108942
Reviewed-by: Nate Bosch <nbosch@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
The previous system let you use this to define an "unspecified error":
int i = "bad";
// [unspecified error]
The marker comment meant that *both* front ends had to report some error
on the previous line, but any error at any position was considered
acceptable.
Unfortunately, this doesn't let us incrementally specify an error for
one front end while leaving it unspecified for the other. This means
that when, say, the analyzer implementation lands first, we can't pin
down its errors while still leaving the CFE errors unspecified until
its ready.
This addresses that by making "unspecified" a property of each front
end, not the error itself. The new syntax is:
int i = "bad";
// ^^^
// [analyzer] unspecified
// [cfe] unspecified
This means the same thing as the previous example: both front ends must
report some error on the previous line. The column information is
authored as a best effort, but ignored.
Now, let's say the CFE implementation lands. The above can be changed
to:
int i = "bad";
// ^^^
// [analyzer] unspecified
// [cfe] Real error message.
At this point, the CFE test must report that exact error at that exact
position and message. But the analyzer is still free to report any
error code anywhere on that line.
This syntax is a little more verbose, but it's simpler and more
flexible.
Change-Id: I37f937d245fd8ec8054acb6128256f9fff6241e1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/109728
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
I also removed a field that did nothing useful, fixed a couple of types,
and removed some unnecessary getters.
Change-Id: I95af91850f5971a22046db4a7d746bc0a43a643c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108700
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
I pulled this list from the build package and turned on the ones that
only had a handful of violations. Left the others that I eventually want
to enable commented out.
Change-Id: I09ec037982e29d3682bc0eaa6af28adfad91b941
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108563
Reviewed-by: Nate Bosch <nbosch@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
If a test has the static error markers and is run on analyzer, the
test runner verifies that the reported errors exactly match the expected
ones. A test with error markers is skipped on all other configurations.
Change-Id: Ib921acde517688f5b47937a5b100c0507e0e9138
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107823
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Move generation of Function objects for native trampolines to the Precompiler, so they can be generated during AOT and tree-shaken if possible.
Issue dartbug.com/35765
Change-Id: I0e69b7e0b22db73e3a40f2fe445660e57ddb6fa9
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-bare-linux-release-simarm64-try, vm-kernel-precomp-bare-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-dartkb-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107407
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
These were probably added in Dart 1 because --checked carried a much
higher performance penalty. Today, most tests finish within 1 minute. If
certain configs need longer timeouts these should be specified in the
named configuration in test_matrix.json.
Change-Id: I91c456475971059297e16b9902dc16c2b23f9c9e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107880
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
They don't do anything yet, but the syntax is supported and tested.
Change-Id: I67e3c72babd8e272b28b434dfb0cf6028f472676
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107568
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Nate Bosch <nbosch@google.com>
Also a few minor tweaks to TestFile to make it more testable and pin
down some edge case behavior.
Change-Id: Icf56be74b68b1e9073b892bc441f04a433e437a8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107366
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Nate Bosch <nbosch@google.com>
- Make tests using eval fail instead of timing out.
- Collect allocation stats in the interpreter.
- And various EnsureDeclarationLoaded.
Change-Id: I17377216cd6dde2615b205f1b28071e12e50e252
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107442
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
There's a bunch of information that gets pulled from the contents of a
test file itself: whether it's a multitest, VMOptions, etc. Previously,
that lived in some combination of TestInformation objects and a big
stringly-typed "optionsFromFile" map that got passed everywhere. Also,
the map got mutated in a couple of choice places.
This replaces all of that with a single TestFile class that represents
all of the metadata gleaned from a given test file. This cleans up the
code base and also should pave the way for supporting static error
tests using that same class.
There should be no behavioral changes in this patch.
I didn't remove the packageRoot stuff in this change since there's
another change in flight for that. I'll merge those two together once
one of them lands.
Change-Id: Ia6d4c926afb342b71cb0041db7219586a792ac80
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/106581
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Towards #37318
- appJit has been replaced by appJitk and is no longer.
- precompiled has been replaced by dartkp and is no longer used.
- For the precompiled runtime, default to dartkp.
- Remove status file references to these compiler options and normalize
status files.
Change-Id: I48728db13dc84737092a92314e2f474a9309f4d9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/106942
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
As number of bytecode generation options grows, it becomes cumbersome
to add them and propagate from the place where they are parsed to
the place where they are used. In order to make it easier to
change and add new bytecode generation options, this CL introduces
BytecodeOptions class which consolidates all options for bytecode
generation. Also, command line options --emit-bytecode-*** are gathered
into a single multi-option --bytecode-options=opt1,opt2,...
Also, unused --use-future-bytecode-format option is cleaned up. If needed,
it could be easily re-introduced in the new BytecodeOptions.
Change-Id: I637bf28ceb4233ead2562afe7ad51c69a99f2d60
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/106965
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
I didn't realize they were being gitignored across the entire SDK repo.
I'm not sure why they are ignored at all, but based on the linked bug,
I think it's only a top-level analysis_options.yaml file that would
cause problems?
This change to the .gitignore file prohibits that case while still
allowing analysis_options.yaml files in subdirectories.
I very strongly want to have an options file for test_runner to ensure
that everyone on the team keeps it "no-implicit-cast" clean.
Change-Id: I9ceac507b08a063e71e2fe39501161723651e6b8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/106840
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
- Run dartfmt --fix. This converts JavaDoc comments to "///", removes
"new" and extraneous "const", and a couple of other things.
- Fix SCREAMING_CAPS constants to lowerCamelCase.
- Use collection literals where possible.
- Use UI-as-code in a couple of places where it seemed obvious.
- Use "var" for more local variables.
- Use "const" instead of "final" when possible.
- Make members private when possible. Deleted a few that then became
obviously unused.
- ".length > 0" -> ".isNotEmpty".
There are no meaningful changes.
Change-Id: Ic6c5a74b2af9b3ebcbe881dbed69f65488bdef09
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105880
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
This makes it an actual Pub package like most other code inside the SDK
repo. The main goal is to make it easier to write tests for the test
runner itself.
This change:
- Moves all of the code from tools/testing/dart/ over to
pkg/test_runner. Most of it ends up under test_runner/lib/src.
- Move tools/testing/dart/main.dart to
pkg/test_runner/bin/test_runner.dart.
- Move standalone_2/io/test_runner_test.dart to
pkg/test_runner/test/test_runner_test.dart. I don't think it currently
works, but it wasn't being run in its old location either.
- Add test_runner to the analysis-server bot. This ensures the
test_runner package is static error clean.
- Remove standalone_2/io/test_runner_analyze_test.dart which used to
attempt to do the above and is no longer needed.
- Update test.py to look for the test runner at its new location.
- Add test_runner to the repo .packages file and remove the weird
test_dart pseudo-package. (I think this fixes #35279.)
- Remove status file entries for the removed standalone_2 tests.
There are no code changes to the test runner itself aside from fixing
up import paths.
Change-Id: I3d05d50d222b291848fa5a30de2846e803bc81e6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105821
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
