All tests skipped here are flaking or timing out. Most of them fail for
a common reason tracked in
https://github.com/dart-lang/sdk/issues/53985: Chrome inactive tabs do
not run behave the same as active tabs. This means that interactions like
css transitions, requestAnimationFrame, and video play don't work as
expected.
This CL skips the tests only on Chrome, but continues to run them in
other browsers where the expectation is met. If we can in the future
ensure tests are run on an active tab, we can consider reneabling these
tests. That said, the value of this tests was higher when we mantained
Dartium, but these days we may consider deleting them instead.
Change-Id: I9c0ea230fecca16fa008b64c2cf316ccdd0f53e4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/335030
Reviewed-by: Srujan Gaddam <srujzs@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
This test appears to flake 88% of the time in FF. One
potential reason could be that FF requires user interaction
(similar to the notification_permissions test), however
I don't have an explanation why it doesn't consistently time-out.
One potential theory is whether this behavior is different depending on
which FF browser version is used, but I don't have the data at the
moment to confirm.
Change-Id: Ib5b2759e0d289e44ff06ab11345c597af6cab0f2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332626
Reviewed-by: Srujan Gaddam <srujzs@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
These tests had several issues. The fixes are aimed to reduce flakiness
and remove invalid configurations where tests are not providing any
value. Some of the issues include:
* some tests interferered with each other causing a 18% flakiness rate
in chrome: they used a common file system that ended up being shared
in a single browser instance. This is why some tests failed when they
find a file exists before it was created [1].
* some tests were not properly using async_helper/async_minitest, so
they may complete before they finish executing
* DDC no longer supports these APIs due to
https://github.com/dart-lang/sdk/issues/53864. We don't intend to fix
those issues, so there is no purpose in running these tests in DDC at this
time.
[1]: https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket/8766313714239949697/+/u/test_results/ignored_flaky_test_failure_logs
Change-Id: I1dbb28eba2d962000a4d425563dbc1eb6b55dbed
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332266
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Srujan Gaddam <srujzs@google.com>
This test provides very little value. It requests the notification
permissions and waits for an answer. What's happening is that the test
launches a prompt and then waiting for a user to click on the browser
dialog to accept or reject the request.
On the bots, this is consistently timing out in chrome and safari.
Turns out this issue was [identified][1] 5 years ago and the test was marked
as Timeout in the status files for that reason (dart2js) or skipped
(DDC). Back then there were discussions
about whether we could make chrome automatically accept or reject the
request, but there was no traction on that.
If we thought this test was providing value, we could explore whether
there is a way to make bots automatically accept/reject the prompt by
saving some preferences on the test machines. That said, I'm honestly
not seeing much value here. For that reason, I'm inclined to delete the
test instead.
Other reasons to delete mentioned in the review:
* this API may require a secure context, but our test runner doesn't default to use https yet
* this API requires a [user gesture in FF][2]
[1]: https://github.com/dart-lang/sdk/issues/32002
[2]: https://developer.mozilla.org/en-US/docs/Web/API/Notification/requestPermission_static
Change-Id: Ib8d372b2ebfb5859a0a9bc82d50a14ad8badeab1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332320
Reviewed-by: Srujan Gaddam <srujzs@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Now that the canary configurations are more reliable we can start
running these tests again. They were originally skipped when the
async implementation in the new type system was not working correctly.
The `web/async_stacktrace_test` has actually been timing out in the
stable configs as well. The timeout is consistent and reproducible
locally so I'm filing a new issue and skipping it on all DDC configs
for now.
Fixes: https://github.com/dart-lang/sdk/issues/50666
Issue: https://github.com/dart-lang/sdk/issues/53555
Change-Id: I7306ba66afe595fb213f06c0ad884d376240addc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/326721
Reviewed-by: Mark Zhou <markzipan@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
These tests are flaky or timing out on the stable DDC configurations.
This change adds skips for the canary configurations to avoid crashing
the infra when too many tests that timeout are attempted to be
deflaked.
When the canary mode stabilizes we should manually mark all these
tests as flaky if they are still not passing on the stable DDC
configurations.
Issue: https://github.com/dart-lang/sdk/issues/50666
Change-Id: I9b1cb8fe466624480767fea05610331bf0f4ed87
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302843
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
"dartdevc" and "dartdevk" have been replaced with "ddc".
Change-Id: I823bc029a0bbe1295fd731efcc32961e61c6c175
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/282485
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
The test either times out or fails on all operating systems.
Change-Id: Ibef092bf808f6562a2eb571cd41cc555fd889dbe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/283863
Reviewed-by: Nicholas Shahan <nshahan@google.com>
* The new compiler name "ddc" will be used for all configurations
in an upcoming change.
* Aliases "dartdevc" and "dartdevk" will be removed in the future.
Change-Id: I2fa48e0ec043b1ba7fb70f883b39e40f338b6928
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280862
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
The timeouts are crashing the test infra when trying to deflake
the failures and preventing more results from getting uploaded.
These tests are timing out right now on the canary bot because
support for the new runtime types hasn't been fully landed. They
should be enabled again after all the support has landed and
should have results at parity with the non-canary configurations.
Change-Id: Ia8814c02445acccb906c4b376c82da88480155b9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/274483
Reviewed-by: Mark Zhou <markzipan@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Compilers named "dartdevc" and "dartdevk" should be listed together
at all locations. These are now aliases for the same compiler
across different configurations.
In the future these names should converge on "dartdevc".
Some skipped tests for "dartdevc" no longer need to be skipped but
currently there are no configurations are running those tests with
that compiler name so removing the skips shows no change in test
results.
Change-Id: Ieca27fbe497c4768293de125b14c694666dc8a8b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/253821
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Split out async, jsify, and html tests. Also split JS$ prefix test and
skipped by design for ddc which doesn't support the prefix.
Change-Id: I85c673d791731bbae3ff4015a7e134d9180170de
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185281
Commit-Queue: Riley Porter <rileyporter@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Srujan Gaddam <srujzs@google.com>
Copies over dart2js and ddc status files to lib.
Change-Id: I946a7fbb7a8aadc0912b709560c33af85e701045
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/140182
Commit-Queue: Srujan Gaddam <srujzs@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Change-Id: I9a51ae6a1f8f4d018da289b6f29ab18053304c19
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/132341
Commit-Queue: Erik Ernst <eernst@google.com>
Reviewed-by: Lasse R.H. Nielsen <lrn@google.com>
- Copies corelib_2/a* -> corelib/
- Copies language_2/ab* -> language/
- Copies lib_2/math/ -> lib/math/
- Copies standalone_2/a* -> standalone/
And also copies over and renames all of the status files in those
directories.
Then it migrates those tests to be static error free in NNBD.
Finally, adds support to the test_runner for the new suites.
Note that this review is split into multiple patchsets. The first
patchset is a straight copy of the existing files. Then the later
patchsets have the interesting changes.
Change-Id: Icec2ff850a3aee30b653066ac184495d1e3814d0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125467
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Leaf Petersen <leafp@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
