dart-sdk/tests/language
Sigmund Cherem 050afd650f [expect] introduce Expect.throwsWhen and Expect.throwsTypeErrorWhen
Today, most tests that touch on a behavior variation end up
skipping expectations or the entirety of a test for some
testing configurations.  Moving forward, we'd like skip less
and try to account for the behavior variations if that's
reasonable.

This CL shows an approach to improve our test coverage for
behavior variations. We introduce two new methods to
[Expect] that allow us to conditionally check that a
function throws, depending on variation predicates.

The CL changes expectations for errors that don't occur
when dart2js omits parameter type checks or implicit
downcasts.

Note: originally I had the intention to introduce a name
parameter to `Expect.throws` and `Expect.throwsTypeError` to
avoid introducing a new API. However, because these APIs are
used for testing core language features, such as function
parameters themselves, we decided to keep the use of
features in these APIs as simple as it can be.

CoreLibraryReviewExempt: no public library semantic change - only improving test coverage under variations
Change-Id: I531657622655778491eaca8b37ba69ffaab559fc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/351340
Reviewed-by: Lasse Nielsen <lrn@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
2024-02-14 20:12:05 +00:00
..
abstract Migrate "a" directory language tests off @compile-error. 2023-04-20 20:50:39 +00:00
accessor_conflict
argument [analyzer] NOT_ENOUGH_POSITIONAL_ARGUMENTS 2022-10-11 17:56:11 +00:00
assert [dart2wasm] Emit code for assert initializers. 2023-01-05 00:23:40 +00:00
assign [analyzer] Issue 37238: Multiple errors for type mismatch assignments. 2023-05-19 16:27:58 +00:00
async [vm] Adjust return value of async/async*/sync* after the language spec change 2023-12-19 16:14:39 +00:00
async_nested
async_star [tests] Remove obsolete Dart 3.0 experiments from language tests 2023-06-15 08:26:27 +00:00
augmentation_libraries fix typo 2024-02-12 23:02:51 +00:00
await [tests] Avoid small --optimization-counter-threshold in tests 2022-10-10 21:04:18 +00:00
bool [tests] Fix language/bool/has_environment_not_new 2023-11-28 12:50:05 +00:00
call [tests] Cleanup remnants of multi-test migration 2024-01-10 18:35:50 +00:00
canonicalize Spelling tests language 2022-12-19 16:30:06 +00:00
cascade Remove nbsp 2022-10-17 08:18:49 +00:00
class Remove left-over patch declarations for List constructor. 2023-04-22 00:38:28 +00:00
class_modifiers Breaking changes for analyzer version 6.0.0 2023-06-16 16:31:27 +00:00
closure [dart2js] Generate parameter stubs during codegen. 2023-12-29 07:13:09 +00:00
compile_time_constant [analyzer] Separate exception and evaluation errors in the const evaluator. 2023-08-22 18:21:10 +00:00
const [analyzer] Report cycles in constant constructors. 2023-08-29 19:02:38 +00:00
const_functions [cfe] Error on variable use before declaration, not on declaration 2023-10-03 10:51:15 +00:00
constants_2018 [tests] Remove obsolete Dart 3.0 experiments from language tests 2023-06-15 08:26:27 +00:00
constructor Reland "[dart2wasm] Replace struct.new_default with struct.new for object allocation." 2023-09-26 10:07:42 +00:00
control_flow_collections [tests] update tests to use variations 2024-02-08 17:00:59 +00:00
covariant [expect] introduce Expect.throwsWhen and Expect.throwsTypeErrorWhen 2024-02-14 20:12:05 +00:00
covariant_override Clean up a number of 2.19 opt outs in language tests. 2023-03-17 16:17:50 +00:00
deferred [tests] Delete inert tests from multi-test migration 2023-12-28 17:16:09 +00:00
double [tests] update to use variations.dart#jsNumbers 2024-02-08 17:00:59 +00:00
double_literals
dynamic [dart2wasm] Fix invocation forwarder named argument adjustment 2023-01-10 13:37:33 +00:00
enum [flip-modifiers] opt out from class modifiers some failing front end language tests 2023-03-02 20:40:20 +00:00
exception Migrate "e" and "f" directory language tests off @compile-error. 2023-04-20 20:49:17 +00:00
export Migrate "e" and "f" directory language tests off @compile-error. 2023-04-20 20:49:17 +00:00
extension_methods Fix typos 2022-06-10 15:48:54 +00:00
extension_type Flow analysis: Use extension type erasure for implicit is reachability. 2024-01-24 15:23:27 +00:00
external_abstract_fields [cfe] Implemenet abstract fields errors 2020-07-20 12:05:54 +00:00
factory Fix incorrect file reference in license headers 2023-09-18 06:44:08 +00:00
field Migrate "e" and "f" directory language tests off @compile-error. 2023-04-20 20:49:17 +00:00
final [analyzer] Issue 53927: Disallow final fields to be used in a const context. 2023-11-02 20:46:20 +00:00
function [expect] introduce Expect.throwsWhen and Expect.throwsTypeErrorWhen 2024-02-14 20:12:05 +00:00
function_subtype [expect] introduce Expect.throwsWhen and Expect.throwsTypeErrorWhen 2024-02-14 20:12:05 +00:00
function_type [tests] update tests to use variations 2024-02-08 17:00:59 +00:00
generic [web-fixit] Make generic/mixin language tests minification-agnostic 2023-11-29 21:40:21 +00:00
generic_methods Fix tearoff of methods from promotable variables. 2023-04-03 19:39:59 +00:00
getter Migrate "g" - "i" directory language tests off @compile-error. 2023-04-19 22:09:18 +00:00
identifier [cfe] Remove extra conditions on await-for in non-async body check 2023-05-09 06:56:44 +00:00
identity [tests] update to use variations.dart#jsNumbers 2024-02-08 17:00:59 +00:00
if
if_null Issue 48004. Report when null-aware operator is used on type. 2022-05-29 19:03:15 +00:00
implicit_creation [tests] Remove obsolete Dart 3.0 experiments from language tests 2023-06-15 08:26:27 +00:00
implicit_downcast_during Reland "Flow analysis: promote to non-nullable on initialization" 2020-10-07 13:14:55 +00:00
import [tests] Update two conditional import tests 2023-11-28 14:27:39 +00:00
inference Migrate "g" - "i" directory language tests off @compile-error. 2023-04-19 22:09:18 +00:00
inference_update_1 Retire experiment flags introduced in 2.18. 2022-12-15 13:18:50 +00:00
inference_update_2 Fix assertion failure computing "why not promoted" for a public getter. 2024-01-31 15:40:38 +00:00
initializing_formal Migrate "g" - "i" directory language tests off @compile-error. 2023-04-19 22:09:18 +00:00
instance [cfe] Add UnresolvedKind for fine grained unresolved reporting 2021-08-25 09:51:54 +00:00
instantiate_to_bound master branch to main 2021-09-15 06:22:23 +00:00
interceptor
interface Remove left-over patch declarations for List constructor. 2023-04-22 00:38:28 +00:00
invalid_returns Remove language tests that opt out of null safety 2024-01-23 18:23:32 +00:00
is [tests] Delete inert tests from multi-test migration 2023-12-28 17:16:09 +00:00
label [parser] Add beforeSynthetic to SyntheticStringToken 2021-08-30 11:56:30 +00:00
lazy Spelling tests 2023-01-19 16:24:29 +00:00
least_upper_bound Correct 4 static type expectations in test 2023-11-27 10:38:31 +00:00
library Spelling tests language 2022-12-19 16:30:06 +00:00
list [parser] More specific error messages when recovering new/const/Map/Set/List with said literals 2021-03-25 13:27:03 +00:00
loop [cfe] Error on variable use before declaration, not on declaration 2023-10-03 10:51:15 +00:00
macros handle deserializing metadata diagnostic targets 2024-02-14 00:03:35 +00:00
main [cfe] Support injected members in export scope 2023-05-24 09:21:20 +00:00
malbounded Migrate "l" and "m" directory language tests off @compile-error. 2023-04-20 17:51:42 +00:00
malformed Migrate "l" and "m" directory language tests off @compile-error. 2023-04-20 17:51:42 +00:00
map [dart2js] Better const Maps and Sets 2023-06-20 23:44:08 +00:00
metadata [analyzer] Report more specific diagnostic codes instead of INVALID_CONSTANT. 2023-08-29 22:36:02 +00:00
method [analyzer] Change the const evaluation result of variables to be Constant. 2023-09-22 16:13:10 +00:00
mixin [web-fixit] Make generic/mixin language tests minification-agnostic 2023-11-29 21:40:21 +00:00
mixin_constructor_forwarding Set tests that have mixin errors as 2.19. 2023-03-01 15:03:39 +00:00
mixin_declaration Migrate "l" and "m" directory language tests off @compile-error. 2023-04-20 17:51:42 +00:00
mixin_legacy [web-fixit] Make generic/mixin language tests minification-agnostic 2023-11-29 21:40:21 +00:00
named_arguments_anywhere [analyzer] Initial refactor of _InstanceCreationEvaluator to return Constant value. 2023-07-25 19:23:02 +00:00
new [cfe] Add UnresolvedKind for fine grained unresolved reporting 2021-08-25 09:51:54 +00:00
nnbd Remove language tests that opt out of null safety 2024-01-23 18:23:32 +00:00
no_such_method Migrate "n" through "p" directory language tests off @compile-error. 2023-04-20 17:56:18 +00:00
nonfunction_type_aliases Remove language tests that opt out of null safety 2024-01-23 18:23:32 +00:00
nosuchmethod_forwarding Remove uses of : as default value separator in some tests/ directories. 2022-09-07 14:49:17 +00:00
null [flip-modifiers] opt out from class modifiers some failing front end language tests 2023-03-02 20:40:20 +00:00
null_aware Flag additional code as unreachable due to types Null and Never. 2022-08-22 16:50:19 +00:00
number [tests] update to use variations.dart#jsNumbers 2024-02-08 17:00:59 +00:00
operator [tests] update to use variations.dart#jsNumbers 2024-02-08 17:00:59 +00:00
optimize Migrate "n" through "p" directory language tests off @compile-error. 2023-04-20 17:56:18 +00:00
override Move codes from StaticTypeWarningCode into CompileTimeErrorCode. 2020-07-27 16:44:15 +00:00
parameter [cfe] Correct PrivateNamedParameter problem message 2023-12-01 17:32:48 +00:00
part [cfe] Include import paths in error for unavailable dart: libraries 2022-11-08 14:54:17 +00:00
patterns Fix pattern context type schema for cast patterns. 2024-01-31 17:23:20 +00:00
prefix [cfe] Add UnresolvedKind for fine grained unresolved reporting 2021-08-25 09:51:54 +00:00
private pre-emptively fix some tests for class modifiers flag flip 2023-03-01 20:11:25 +00:00
propagate
records Language test of record demotion behavior. 2024-02-13 23:28:19 +00:00
redirecting Remove uses of : as default value separator in some tests/ directories. 2022-09-07 14:49:17 +00:00
reg_exp
regress [cfe] Infer mixins by directly solving for equality 2024-01-08 11:42:39 +00:00
resolution
return Migrate "r" directory language tests off @compile-error. 2023-04-19 23:35:16 +00:00
rewrite Move codes from StaticTypeWarningCode into CompileTimeErrorCode. 2020-07-27 16:44:15 +00:00
scaling [vm] Increase supported range of TypeParameters to 16 bits. 2022-12-14 23:34:18 +00:00
script Migrate "s" directory language tests off @compile-error. 2023-04-20 17:59:44 +00:00
sealed_class Breaking changes for analyzer version 6.0.0 2023-06-16 16:31:27 +00:00
set_literals [tests] Add const-big-set-literal test 2020-10-20 11:07:23 +00:00
setter Migrate "s" directory language tests off @compile-error. 2023-04-20 17:59:44 +00:00
spread_collections Make the type schema for null-aware spread operations consistent. 2024-02-13 22:25:41 +00:00
stack_trace [gardening] Fix recent change to language/stack_trace/* tests in CSP mode 2024-01-31 14:38:43 +00:00
static Migrate "s" directory language tests off @compile-error. 2023-04-20 17:59:44 +00:00
string Remove language tests that opt out of null safety 2024-01-23 18:23:32 +00:00
subtyping_static [dart2wasm] Fix isSubtype case of type parameter vs FutureOr 2023-08-11 09:08:32 +00:00
super Migrate "s" directory language tests off @compile-error. 2023-04-20 17:59:44 +00:00
superinterface_variance Set tests that have mixin errors as 2.19. 2023-03-01 15:03:39 +00:00
switch [tests] Remove obsolete Dart 3.0 experiments from language tests 2023-06-15 08:26:27 +00:00
symbol Move codes from StaticTypeWarningCode into CompileTimeErrorCode. 2020-07-27 16:44:15 +00:00
sync_star Test for yield* of Iterable with throwing get:iterator 2023-04-14 22:04:28 +00:00
syntax [tests] Remove obsolete Dart 3.0 experiments from language tests 2023-06-15 08:26:27 +00:00
this [expect] introduce Expect.throwsWhen and Expect.throwsTypeErrorWhen 2024-02-14 20:12:05 +00:00
top_level Migrate "t" directory language tests off @compile-error. 2023-04-20 19:55:31 +00:00
type [analyzer] Separate exception and evaluation errors in the const evaluator. 2023-08-22 18:21:10 +00:00
type_object Remove language tests that opt out of null safety 2024-01-23 18:23:32 +00:00
type_promotion Move codes from StaticTypeWarningCode into CompileTimeErrorCode. 2020-07-27 16:44:15 +00:00
type_variable Migrate "t" directory language tests off @compile-error. 2023-04-20 19:55:31 +00:00
typedef Migrate "t" directory language tests off @compile-error. 2023-04-20 19:55:31 +00:00
unsorted Issue 54485. Report error when external constructor has a field formal parameter. 2024-01-03 19:59:53 +00:00
value_class [cfe] Add Jenkins SMI hashcode combiner 2020-09-07 12:36:13 +00:00
variable [cfe] Error on variable use before declaration, not on declaration 2023-10-03 10:51:15 +00:00
variance [cfe,analyzer] Share experimental flags for the parser 2022-11-17 11:32:01 +00:00
vm [vm/compiler] Catch non-Smi mask in 32-bit AOT ~/ call replacement. 2023-10-19 12:30:06 +00:00
void [dart2wasm] Fix compiler bug regarding void values that can be observed 2024-02-07 08:05:19 +00:00
why_not_promoted Allow "field promotion" to apply to abstract getters. 2023-12-07 23:40:27 +00:00
await_type_error_test.dart
await_type_test.dart
dynamic_type_helper.dart
explicit_type_instantiation_parsing_test.dart Remove references to constructor-tearoff experiment. 2021-11-11 14:04:43 +00:00
language.status Skip test that hangs on reload bots. 2024-01-31 12:08:34 +00:00
language_analyzer.status [tests] Cleanup test with absent deferred library 2022-11-03 17:06:42 +00:00
language_dart2js.status [tests] increase timeout for long dart2js test. 2023-11-02 17:22:39 +00:00
language_dart2wasm.status Add "why not promoted" context messages when field promotion fails. 2023-09-27 21:31:49 +00:00
language_dartdevc.status [tests] Mark closure/cycles_test as slow on chrome, skip on firefox. 2023-11-08 18:43:51 +00:00
language_kernel.status [test] Don't expect disassemble_test to crash. 2024-02-07 21:30:10 +00:00
language_precompiled.status [test] Remove ARM64-specific status entries. 2023-10-09 19:30:29 +00:00
language_spec_parser.status Spelling tests language 2022-12-19 16:30:06 +00:00
language_vm.status [test] Remove ARM64-specific status entries. 2023-10-09 19:30:29 +00:00
library1.dart
library2.dart
library10.dart
library11.dart
library12.dart
OWNERS Use OWNERS files to restrict editing of tests/language to language team. 2023-11-16 18:18:00 +00:00
record_literal_problems_test.dart [tests] Remove obsolete Dart 3.0 experiments from language tests 2023-06-15 08:26:27 +00:00
record_literal_test.dart [tests] Remove obsolete Dart 3.0 experiments from language tests 2023-06-15 08:26:27 +00:00
record_type_empty_problems_test.dart [tests] Remove obsolete Dart 3.0 experiments from language tests 2023-06-15 08:26:27 +00:00
record_type_problems_test.dart [tests] Remove obsolete Dart 3.0 experiments from language tests 2023-06-15 08:26:27 +00:00
record_type_test.dart [tests] Remove obsolete Dart 3.0 experiments from language tests 2023-06-15 08:26:27 +00:00
static_type_helper.dart inference-update-3: use unpromoted type as context for local variable assignments. 2023-12-07 23:30:06 +00:00
static_weak_reference_error_test.dart [dart2js] Static weak references to method tearoffs 2023-03-02 20:04:19 +00:00
static_weak_reference_function_apply_test.dart [dart2js] Static weak references to method tearoffs 2023-03-02 20:04:19 +00:00
static_weak_reference_test.dart [dart2js] Static weak references to method tearoffs 2023-03-02 20:04:19 +00:00
syntax_helper.dart Additional language tests for parsing behaviors around type arguments. 2021-05-18 14:06:13 +00:00