Commit graph

123 commits

Author SHA1 Message Date
Mikkel Nygaard Ravn ca0e144f9f
Mark non-flaky tests as such (#18927) 2018-06-29 10:55:31 +02:00
Danny Tuppeny e87b83bfd8 Mark flutter_gallery_instrumentation_test as flaky (#18887)
See #18879.
2018-06-27 15:48:22 -07:00
Ian Hickson 3ecd4301bf
Test performance of "flutter test" (#18810) 2018-06-26 17:34:25 -07:00
Ian Hickson 73498a54bc
Make flaky test as such. (#18815) 2018-06-25 19:25:01 -07:00
Mikkel Nygaard Ravn d4db748047
Add2app devicelab test (#18795) 2018-06-25 22:37:47 +02:00
Chris Bracken 7213a7805a
Mark flutter_attach_test flaky (#18801)
Introduced in d248725e15, but has been red
since landing. Marking flaky rather than reverting since it's the only
failure.
2018-06-25 11:04:51 -07:00
Florian Loitsch d248725e15
Flutter attach (#18677)
Add `flutter attach` command.

This command allows Flutter to connect to applications that haven't been launched using `flutter run`.
2018-06-25 15:33:42 +02:00
Michael Goderbauer ebb174b0bf
Add size benchmark for hello_world (#18522) 2018-06-16 00:08:33 -07:00
Ian Hickson d1cc8b6de8
Clean up output of "flutter run --release" (#18380)
(second attempt)
2018-06-14 12:30:20 -07:00
Chris Bracken 2ae48845a8
Revert elimination of Dart 1 (#18460)
fuchsia_tester.dart still assumes Dart 1. Previously, it ran tests directly
from source, flutter_platform.dart automatically runs a kernel compile when
operating in Dart 2 mode, but this assumes a functional Dart SDK is available
in the artifacts directly, and fuchsia_tester.dart mocks out the artifacts
directory with an empty temp dir.

Remaining work is:
1. Get the frontend server building as a dependency on Fuchsia.
2. Patch fuchsia_tester.dart to use a valid Dart SDK and frontend server.

This also reverts migration to Dart 2 typedef syntax.

This reverts commit 6c56bb2. (#18362)
This reverts commit 3daebd0. (#18316)
2018-06-13 12:46:39 -07:00
Chris Bracken 3daebd059c
It's time to #deleteDart1 (again) (#18316)
* It's time to #deleteDart1 (#18293)

Eliminates support for Dart 1 in flutter_tools, and drops our Dart 1
benchmarks. All commands now run in Dart 1 mode only.

Eliminates --preview-dart-2 / --no-preview-dart-2 support.

* Fix indentation, remove no longer necessary .toList()

* Only push udpated kernel if >0 invalidated srcs
2018-06-08 15:49:24 -07:00
Chris Bracken 9495df4bc3
Revert "It's time to #deleteDart1 (#18293)" (#18313)
Several benchmarks started failing during kernel compile:
1. Timeouts:
   * commands_test
   * hot_mode_dev_cycle__benchmark
   * run_machine_concurrent_hot_reload
   * service_extensions_test

2. Kernel compiler crash:
   * hot_mode_dev_cycle_linux__benchmark
   * routing_test

3. Unexpected termination:
   * hot_mode_dev_cycle_win__benchmark

This reverts commit ed63e7082a.
2018-06-08 12:43:11 -07:00
Chris Bracken ed63e7082a
It's time to #deleteDart1 (#18293)
Eliminates support for Dart 1 in flutter_tools, and drops our Dart 1
benchmarks. All commands now run in Dart 1 mode only.

Eliminates --preview-dart-2 / --no-preview-dart-2 support.
2018-06-08 11:04:57 -07:00
Ian Hickson 944f0a4cc1
Revert "Clean up output of "flutter run --release" (#18049)" (#18184)
This reverts commit 4f1b66044c.
2018-06-04 16:17:08 -07:00
Ian Hickson 4f1b66044c
Clean up output of "flutter run --release" (#18049) 2018-06-04 15:22:51 -07:00
Mikkel Nygaard Ravn f986048094
Mark flaky test as such (#18019) 2018-05-30 12:54:46 +02:00
Todd Volkert 70772ea5a0
Remove flaky marker on some tests. (#17882)
They've been mostly green since https://github.com/flutter/flutter/pull/17812
2018-05-24 13:57:52 -07:00
Chris Bracken efc657aabf
Mark 32-bit iOS gallery transitions test unflaky (#17662)
Remaining issue was fixed in 580c844c2f.
2018-05-16 11:57:06 -07:00
Chris Bracken 0cbda7d68d
Mark 32-bit iOS gallery startup test unflaky (#17660) 2018-05-16 11:13:44 -07:00
Chris Bracken 20512d458e
Add 32-bit iOS gallery startup test (#17653) 2018-05-16 09:29:10 -07:00
Sarah Zakarias a3e71bf11f
Add startup test for Platform View example (#17645) 2018-05-16 13:24:47 +02:00
Chris Bracken 52246c048d
Add 32-bit iOS transition perf integration test (#17582) 2018-05-14 15:05:38 -07:00
Todd Volkert 33dd44bc98
Mark test as flaky (#17486) 2018-05-10 14:35:53 -07:00
Chris Bracken 4509b15d2d
Mark integration_ui flaky (#17319)
This test fails consistently on mac2 and mac3 with the attached Moto G4
devices but passes consistently on other machines.

Adding a delay of 1s right after driver.connect() in setUpAll() causes
it to pass on the machines in question, which suggests a race condition.
Specifically it looks like connect returns the moment Flutter Driver
identifies that the isolate is up and running, but empirically it looks
like we start running the first test before the UI is actually up. This
triggers a failure wherein we start looking for elements before they're
onstage.

Link to viewport.dart:213 at HEAD:
b2b4665926/packages/flutter/lib/src/widgets/viewport.dart (L213)

Stack trace:
FlutterDriver waitFor should find text "present"

```
  DriverError: Error in Flutter application: Uncaught extension error while executing waitFor: NoSuchMethodError: The getter 'visible' was called on null.
  Receiver: null
  Tried calling: visible
  #0      Object.noSuchMethod (dart:core/runtime/libobject_patch.dart:46:5)
  #1      _ViewportElement.debugVisitOnstageChildren. (package:flutter/src/widgets/viewport.dart:213:36)
  #2      WhereIterator.moveNext (dart:_internal/iterable.dart:439:11)
  #3      Iterable.forEach (dart:core/iterable.dart)
  #4      _ViewportElement.debugVisitOnstageChildren (package:flutter/src/widgets/viewport.dart:214:8)
  #5      _DepthFirstChildIterator._reverseChildrenOf (package:flutter_test/src/all_elements.dart:54:15)
  #6      _DepthFirstChildIterator.moveNext (package:flutter_test/src/all_elements.dart:45:19)
  #7      CachingIterable._fillNext (package:flutter/src/foundation/basic_types.dart:252:27)
  #8      _LazyListIterator.moveNext (package:flutter/src/foundation/basic_types.dart:279:21)
  #9      WhereIterator.moveNext (dart:_internal/iterable.dart:438:22)
  #10     CachingIterable._fillNext (package:flutter/src/foundation/basic_types.dart:252:27)
  #11     _LazyListIterator.moveNext (package:flutter/src/foundation/basic_types.dart:279:21)
  #12     Iterable.isEmpty (dart:core/iterable.dart:449:33)
  #13     Iterable.isNotEmpty (dart:core/iterable.dart:456:27)
  #14     FlutterDriverExtension._waitForElement. (package:flutter_driver/src/extension/extension.dart:215:51)
  #15     FlutterDriverExtension._waitUntilFrame (package:flutter_driver/src/extension/extension.dart:197:19)
  #16     FlutterDriverExtension._waitForElement (package:flutter_driver/src/extension/extension.dart:215:11)

  #17     FlutterDriverExtension._waitFor (package:flutter_driver/src/extension/extension.dart:286:11)

  #18     FlutterDriverExtension.call (package:flutter_driver/src/extension/extension.dart:168:51)

  #19     BindingBase.registerServiceExtension. (package:flutter/src/foundation/binding.dart:370:32)
```

Removes a previous hack that no longer appears to help (adding a 1
second delay in setUpAll() does seem to work around this issue though).
2018-05-06 18:00:35 -07:00
Todd Volkert ae62a85699
Mark flaky tests as flaky (#17004) 2018-04-26 07:55:57 -07:00
Yegor a90a850462
bump benchmark timeouts (#16817) 2018-04-20 13:57:06 -07:00
Mikkel Nygaard Ravn 29eebcbe88
Remove flaky flag from non-flaky test (#16075) 2018-03-29 22:02:29 +02:00
Todd Volkert 31049a5a73
Mark test as non-flaky now that it's been fixed (#16033) 2018-03-28 10:58:48 -07:00
Hans Muller 3fd0423a11
Marked flutter_gallery_instrumentation_test flaky (#15891) 2018-03-23 18:16:44 -07:00
Alexander Aprelev 21ad09fe57
With dart2 being default, move preview_dart_2 tests to dart1. (#15667)
* With dart being default, move preview_dart_2 tests to dart1.

* Swap out defaults for preview-dart-2 option in tests. Make dart1 explicit

* Move suffix around

* Rename files
2018-03-19 12:11:18 -07:00
Mikkel Nygaard Ravn 04de8c0a43
Adjust flaky flags to reality (#15608)
Get to green build.
2018-03-16 11:17:23 +00:00
Mikkel Nygaard Ravn a61592793c Silence broken test (#15446)
* Silence broken test

* Mark test as flaky instead
2018-03-12 12:02:58 -07:00
Danny Tuppeny 465929f71e Remove flaky flag from all tests that are green for the whole dashboard
Plus run_machine_concurrent_hot_reload which isn't all green, but is
green since the fix was pushed.
2018-03-12 17:03:21 +00:00
Todd Volkert 5dca13dea5
Remove flaky marker from android_sample_catalog_generator (#14923) 2018-02-27 10:30:45 -08:00
Todd Volkert 50b7fad6a3
Mark android_sample_catalog_generator as flaky (#14799)
https://github.com/flutter/flutter/issues/14798
2018-02-20 22:14:33 -08:00
Mikkel Nygaard Ravn 1e242ebc7c
Mark plugin tests as stable (#14760) 2018-02-17 22:46:29 +01:00
Mikkel Nygaard Ravn 2000435728
Move plugin injection to just after pub get (#14743) 2018-02-16 10:17:28 +01:00
Mikkel Nygaard Ravn c2e1bc0322
Revert "Move plugin injection to just after pub get (#14560)" (#14734)
This reverts commit b3e4976ca2.
2018-02-15 22:55:36 +01:00
Mikkel Nygaard Ravn b3e4976ca2
Move plugin injection to just after pub get (#14560) 2018-02-15 22:17:12 +01:00
Danny Tuppeny 0885866265
Mark run_machine_concurrent_hot_reload as flaky (#14563)
Sometimes fails writing to stdout as flutter terminates. Marking as
flaky until resolved.
2018-02-08 19:03:13 +00:00
Danny Tuppeny efb88a0380 Reject requests for hot reload if a hot reload is already in progress. (#14494)
* Reject requests for hot reload if a hot reload is already in progress.

Fixes #14184

* Implement TODO, verifying further hot reloads complete sucessfully.

* Fix year on new file.

* Add missing type annotations to fix lints

* Add run_machine_concurrent_hot_reload to manifest for CI

* Reformat document

... but undo things that cause lints (like single-line ifs)

* Extract std stream transformations

* Make inProgressHotReload private

* Disallow all types of reload while hot reload in progress

* Simplify code handling in-progress hot reloads
2018-02-07 08:58:23 -08:00
Yegor 6028fe85bb
mark complex_layout_scroll_perf_ios__timeline_summary as flaky\ (#14080) 2018-01-12 13:33:54 -08:00
Alexander Aprelev f1e5551c45
Fix stage for hot_mode ios test. (#14034) 2018-01-10 17:05:33 -08:00
Alexander Aprelev 0c5dd8d31e
Add hot_mode preview_dart_2 tests for ios. (#14022)
* Add hot_mode preview_dart_2 tests for ios.

* Explicitly request ios os in the test.

* 2018
2018-01-10 13:06:53 -08:00
Yegor d589df4697
modernize capability names in devicelab task manifest (#13858)
* modernize capability names in devicelab task manifest

* replace windows too
2018-01-02 19:53:11 -08:00
Yegor ee65db11d4
Revert engine (#13547)
* Revert "Include a directory with Flutter assets (#12944)"

This reverts commit 3af6b9cbf5.

* Revert "Upgrade project.pbxproj to include flutter_assets (#13011)"

This reverts commit 08128cb29b.

* Revert "Upgrade complex_layout project.pbxproj to include flutter_assets (#13544)"

This reverts commit 35f1a04195.

* mark complex_layout_ios__start_up as flaky
2017-12-13 16:25:25 -08:00
Yegor 0df37f2f97
mark flutter_gallery_ios__start_up as flaky (#13542) 2017-12-13 15:45:40 -08:00
Ian Hickson 920a726003
Mark good devicelab tests as non-flaky (#13271) 2017-11-30 14:32:49 -08:00
Ian Hickson 216ff81cd2
Fix devicelab (#13239) 2017-11-28 17:41:40 -08:00
Ian Hickson d104106e78
Refactor compilation devicelab tests (#13218)
This fills out the following matrix:

App: `flutter create` default app, Complex Layout, and Gallery.
Platforms: Linux/Android, Mac/iOS, Windows/Android
Build modes: full release, aot blobs, debug
Dart: normal, 2.0 preview

It also renames all the tests that used "build" to mean compilation
build to use "compile" instead, so that it's much clearer on the
dashboard.

Also, mark some flaky tests that are no longer flaky as not flaky.
Also, remove uses of call() since that's probably going to go away at
some point.
2017-11-28 17:21:01 -08:00