Commit graph

164 commits

Author SHA1 Message Date
Chris Bracken 2ae48845a8
Revert elimination of Dart 1 (#18460)
fuchsia_tester.dart still assumes Dart 1. Previously, it ran tests directly
from source, flutter_platform.dart automatically runs a kernel compile when
operating in Dart 2 mode, but this assumes a functional Dart SDK is available
in the artifacts directly, and fuchsia_tester.dart mocks out the artifacts
directory with an empty temp dir.

Remaining work is:
1. Get the frontend server building as a dependency on Fuchsia.
2. Patch fuchsia_tester.dart to use a valid Dart SDK and frontend server.

This also reverts migration to Dart 2 typedef syntax.

This reverts commit 6c56bb2. (#18362)
This reverts commit 3daebd0. (#18316)
2018-06-13 12:46:39 -07:00
Chris Bracken 3daebd059c
It's time to #deleteDart1 (again) (#18316)
* It's time to #deleteDart1 (#18293)

Eliminates support for Dart 1 in flutter_tools, and drops our Dart 1
benchmarks. All commands now run in Dart 1 mode only.

Eliminates --preview-dart-2 / --no-preview-dart-2 support.

* Fix indentation, remove no longer necessary .toList()

* Only push udpated kernel if >0 invalidated srcs
2018-06-08 15:49:24 -07:00
Chris Bracken 9495df4bc3
Revert "It's time to #deleteDart1 (#18293)" (#18313)
Several benchmarks started failing during kernel compile:
1. Timeouts:
   * commands_test
   * hot_mode_dev_cycle__benchmark
   * run_machine_concurrent_hot_reload
   * service_extensions_test

2. Kernel compiler crash:
   * hot_mode_dev_cycle_linux__benchmark
   * routing_test

3. Unexpected termination:
   * hot_mode_dev_cycle_win__benchmark

This reverts commit ed63e7082a.
2018-06-08 12:43:11 -07:00
Chris Bracken ed63e7082a
It's time to #deleteDart1 (#18293)
Eliminates support for Dart 1 in flutter_tools, and drops our Dart 1
benchmarks. All commands now run in Dart 1 mode only.

Eliminates --preview-dart-2 / --no-preview-dart-2 support.
2018-06-08 11:04:57 -07:00
Ian Hickson 944f0a4cc1
Revert "Clean up output of "flutter run --release" (#18049)" (#18184)
This reverts commit 4f1b66044c.
2018-06-04 16:17:08 -07:00
Ian Hickson 4f1b66044c
Clean up output of "flutter run --release" (#18049) 2018-06-04 15:22:51 -07:00
Mikkel Nygaard Ravn f986048094
Mark flaky test as such (#18019) 2018-05-30 12:54:46 +02:00
Todd Volkert 70772ea5a0
Remove flaky marker on some tests. (#17882)
They've been mostly green since https://github.com/flutter/flutter/pull/17812
2018-05-24 13:57:52 -07:00
Chris Bracken efc657aabf
Mark 32-bit iOS gallery transitions test unflaky (#17662)
Remaining issue was fixed in 580c844c2f.
2018-05-16 11:57:06 -07:00
Chris Bracken 0cbda7d68d
Mark 32-bit iOS gallery startup test unflaky (#17660) 2018-05-16 11:13:44 -07:00
Chris Bracken 20512d458e
Add 32-bit iOS gallery startup test (#17653) 2018-05-16 09:29:10 -07:00
Sarah Zakarias a3e71bf11f
Add startup test for Platform View example (#17645) 2018-05-16 13:24:47 +02:00
Chris Bracken 52246c048d
Add 32-bit iOS transition perf integration test (#17582) 2018-05-14 15:05:38 -07:00
Todd Volkert 33dd44bc98
Mark test as flaky (#17486) 2018-05-10 14:35:53 -07:00
Chris Bracken 4509b15d2d
Mark integration_ui flaky (#17319)
This test fails consistently on mac2 and mac3 with the attached Moto G4
devices but passes consistently on other machines.

Adding a delay of 1s right after driver.connect() in setUpAll() causes
it to pass on the machines in question, which suggests a race condition.
Specifically it looks like connect returns the moment Flutter Driver
identifies that the isolate is up and running, but empirically it looks
like we start running the first test before the UI is actually up. This
triggers a failure wherein we start looking for elements before they're
onstage.

Link to viewport.dart:213 at HEAD:
b2b4665926/packages/flutter/lib/src/widgets/viewport.dart (L213)

Stack trace:
FlutterDriver waitFor should find text "present"

```
  DriverError: Error in Flutter application: Uncaught extension error while executing waitFor: NoSuchMethodError: The getter 'visible' was called on null.
  Receiver: null
  Tried calling: visible
  #0      Object.noSuchMethod (dart:core/runtime/libobject_patch.dart:46:5)
  #1      _ViewportElement.debugVisitOnstageChildren. (package:flutter/src/widgets/viewport.dart:213:36)
  #2      WhereIterator.moveNext (dart:_internal/iterable.dart:439:11)
  #3      Iterable.forEach (dart:core/iterable.dart)
  #4      _ViewportElement.debugVisitOnstageChildren (package:flutter/src/widgets/viewport.dart:214:8)
  #5      _DepthFirstChildIterator._reverseChildrenOf (package:flutter_test/src/all_elements.dart:54:15)
  #6      _DepthFirstChildIterator.moveNext (package:flutter_test/src/all_elements.dart:45:19)
  #7      CachingIterable._fillNext (package:flutter/src/foundation/basic_types.dart:252:27)
  #8      _LazyListIterator.moveNext (package:flutter/src/foundation/basic_types.dart:279:21)
  #9      WhereIterator.moveNext (dart:_internal/iterable.dart:438:22)
  #10     CachingIterable._fillNext (package:flutter/src/foundation/basic_types.dart:252:27)
  #11     _LazyListIterator.moveNext (package:flutter/src/foundation/basic_types.dart:279:21)
  #12     Iterable.isEmpty (dart:core/iterable.dart:449:33)
  #13     Iterable.isNotEmpty (dart:core/iterable.dart:456:27)
  #14     FlutterDriverExtension._waitForElement. (package:flutter_driver/src/extension/extension.dart:215:51)
  #15     FlutterDriverExtension._waitUntilFrame (package:flutter_driver/src/extension/extension.dart:197:19)
  #16     FlutterDriverExtension._waitForElement (package:flutter_driver/src/extension/extension.dart:215:11)

  #17     FlutterDriverExtension._waitFor (package:flutter_driver/src/extension/extension.dart:286:11)

  #18     FlutterDriverExtension.call (package:flutter_driver/src/extension/extension.dart:168:51)

  #19     BindingBase.registerServiceExtension. (package:flutter/src/foundation/binding.dart:370:32)
```

Removes a previous hack that no longer appears to help (adding a 1
second delay in setUpAll() does seem to work around this issue though).
2018-05-06 18:00:35 -07:00
Todd Volkert ae62a85699
Mark flaky tests as flaky (#17004) 2018-04-26 07:55:57 -07:00
Yegor a90a850462
bump benchmark timeouts (#16817) 2018-04-20 13:57:06 -07:00
Mikkel Nygaard Ravn 29eebcbe88
Remove flaky flag from non-flaky test (#16075) 2018-03-29 22:02:29 +02:00
Todd Volkert 31049a5a73
Mark test as non-flaky now that it's been fixed (#16033) 2018-03-28 10:58:48 -07:00
Hans Muller 3fd0423a11
Marked flutter_gallery_instrumentation_test flaky (#15891) 2018-03-23 18:16:44 -07:00
Alexander Aprelev 21ad09fe57
With dart2 being default, move preview_dart_2 tests to dart1. (#15667)
* With dart being default, move preview_dart_2 tests to dart1.

* Swap out defaults for preview-dart-2 option in tests. Make dart1 explicit

* Move suffix around

* Rename files
2018-03-19 12:11:18 -07:00
Mikkel Nygaard Ravn 04de8c0a43
Adjust flaky flags to reality (#15608)
Get to green build.
2018-03-16 11:17:23 +00:00
Mikkel Nygaard Ravn a61592793c Silence broken test (#15446)
* Silence broken test

* Mark test as flaky instead
2018-03-12 12:02:58 -07:00
Danny Tuppeny 465929f71e Remove flaky flag from all tests that are green for the whole dashboard
Plus run_machine_concurrent_hot_reload which isn't all green, but is
green since the fix was pushed.
2018-03-12 17:03:21 +00:00
Todd Volkert 5dca13dea5
Remove flaky marker from android_sample_catalog_generator (#14923) 2018-02-27 10:30:45 -08:00
Todd Volkert 50b7fad6a3
Mark android_sample_catalog_generator as flaky (#14799)
https://github.com/flutter/flutter/issues/14798
2018-02-20 22:14:33 -08:00
Mikkel Nygaard Ravn 1e242ebc7c
Mark plugin tests as stable (#14760) 2018-02-17 22:46:29 +01:00
Mikkel Nygaard Ravn 2000435728
Move plugin injection to just after pub get (#14743) 2018-02-16 10:17:28 +01:00
Mikkel Nygaard Ravn c2e1bc0322
Revert "Move plugin injection to just after pub get (#14560)" (#14734)
This reverts commit b3e4976ca2.
2018-02-15 22:55:36 +01:00
Mikkel Nygaard Ravn b3e4976ca2
Move plugin injection to just after pub get (#14560) 2018-02-15 22:17:12 +01:00
Danny Tuppeny 0885866265
Mark run_machine_concurrent_hot_reload as flaky (#14563)
Sometimes fails writing to stdout as flutter terminates. Marking as
flaky until resolved.
2018-02-08 19:03:13 +00:00
Danny Tuppeny efb88a0380 Reject requests for hot reload if a hot reload is already in progress. (#14494)
* Reject requests for hot reload if a hot reload is already in progress.

Fixes #14184

* Implement TODO, verifying further hot reloads complete sucessfully.

* Fix year on new file.

* Add missing type annotations to fix lints

* Add run_machine_concurrent_hot_reload to manifest for CI

* Reformat document

... but undo things that cause lints (like single-line ifs)

* Extract std stream transformations

* Make inProgressHotReload private

* Disallow all types of reload while hot reload in progress

* Simplify code handling in-progress hot reloads
2018-02-07 08:58:23 -08:00
Yegor 6028fe85bb
mark complex_layout_scroll_perf_ios__timeline_summary as flaky\ (#14080) 2018-01-12 13:33:54 -08:00
Alexander Aprelev f1e5551c45
Fix stage for hot_mode ios test. (#14034) 2018-01-10 17:05:33 -08:00
Alexander Aprelev 0c5dd8d31e
Add hot_mode preview_dart_2 tests for ios. (#14022)
* Add hot_mode preview_dart_2 tests for ios.

* Explicitly request ios os in the test.

* 2018
2018-01-10 13:06:53 -08:00
Yegor d589df4697
modernize capability names in devicelab task manifest (#13858)
* modernize capability names in devicelab task manifest

* replace windows too
2018-01-02 19:53:11 -08:00
Yegor ee65db11d4
Revert engine (#13547)
* Revert "Include a directory with Flutter assets (#12944)"

This reverts commit 3af6b9cbf5.

* Revert "Upgrade project.pbxproj to include flutter_assets (#13011)"

This reverts commit 08128cb29b.

* Revert "Upgrade complex_layout project.pbxproj to include flutter_assets (#13544)"

This reverts commit 35f1a04195.

* mark complex_layout_ios__start_up as flaky
2017-12-13 16:25:25 -08:00
Yegor 0df37f2f97
mark flutter_gallery_ios__start_up as flaky (#13542) 2017-12-13 15:45:40 -08:00
Ian Hickson 920a726003
Mark good devicelab tests as non-flaky (#13271) 2017-11-30 14:32:49 -08:00
Ian Hickson 216ff81cd2
Fix devicelab (#13239) 2017-11-28 17:41:40 -08:00
Ian Hickson d104106e78
Refactor compilation devicelab tests (#13218)
This fills out the following matrix:

App: `flutter create` default app, Complex Layout, and Gallery.
Platforms: Linux/Android, Mac/iOS, Windows/Android
Build modes: full release, aot blobs, debug
Dart: normal, 2.0 preview

It also renames all the tests that used "build" to mean compilation
build to use "compile" instead, so that it's much clearer on the
dashboard.

Also, mark some flaky tests that are no longer flaky as not flaky.
Also, remove uses of call() since that's probably going to go away at
some point.
2017-11-28 17:21:01 -08:00
Chris Bracken 112df6efb0
Mark flutter_gallery__preview_dart_2_bulid unflaky (#13170)
This was fixed in #13150.

That commit rolled the engine to:
  flutter/engine@93b2179597,

which rolled Dart to:
  dart-lang/sdk@70e5deacb5

which includes:
  dart-lang/sdk@d38c08d973

Which fixes the breakage introduced in #13134.

This reverts commit f5d9c7775d.
2017-11-22 18:17:43 -08:00
Mikkel Nygaard Ravn dbfd0bef9e
Move iOS test to devicelab_ios (#13148) 2017-11-22 15:14:14 +01:00
Mikkel Nygaard Ravn d06482cc79
Add integration test of textures (#13122) 2017-11-22 14:16:25 +01:00
Alexander Aprelev f5d9c7775d
Mark flutter_gallery__preview_dart_2_bulid flaky. It was broken in last Dart engine roll (#13142) 2017-11-21 18:45:44 -08:00
Yegor c027aa159b
remove flaky flag from stable benchmarks; reintroduce iOS benchmarks (#12851) 2017-11-02 14:20:14 -07:00
Alexander Aprelev 1a3097025d Clone hot reload benchmark for --preview-dart-2 option. (#12251)
* Clone hot reload benchmark for --preview-dart-2 option.

* Get rid of linux and win preview_dart_2 (only android would be sufficient for now). Refactor code into lib/tasks

* Revert 2016 to 2017

* Mark new test as flaky
2017-09-28 12:45:54 -07:00
Alexander Aprelev 42bd30dd3f Add preview_dart_2 builds to the dashboard (#11889) 2017-08-31 18:06:17 -07:00
Carlo Bernaschina 1328fd043a Mark commands_test as not flaky (#11880) 2017-08-31 18:04:49 -07:00
Devon Carew 8c36ccf534 More frame events (#11747)
* Revert "Revert "fire service protocol events for frames (#11565)" (#11727)"

This reverts commit f25e2f5213.

* move the postEvent() call into a separate method
2017-08-31 11:38:21 -07:00
Hans Muller b682b78105 Mark android_sample_catalog_generator flaky (#11867) 2017-08-31 08:41:02 -07:00
Mikkel Nygaard Ravn 9496e6dfa7 Support for app flavors in flutter tooling, #11676 retake (#11734) 2017-08-23 10:55:35 +02:00
Mikkel Nygaard Ravn 7c3a45dc6e Revert "Support for app flavors in flutter tooling (#11676)" (#11729)
This reverts commit 8d07d3f652.
2017-08-22 12:24:29 +02:00
Mikkel Nygaard Ravn 8d07d3f652 Support for app flavors in flutter tooling (#11676) 2017-08-22 10:38:23 +02:00
Devon Carew f25e2f5213 Revert "fire service protocol events for frames (#11565)" (#11727)
This reverts commit 5f9acc4125.
2017-08-22 01:06:48 -07:00
Devon Carew 5f9acc4125 fire service protocol events for frames (#11565) 2017-08-21 22:12:19 -07:00
Carlo Bernaschina a24c90d789 Set commands_test flaky (#11710) 2017-08-21 15:31:32 -07:00
Carlo Bernaschina 937b98e1b9 Fix flakiness of commands_test (#11705)
- Wait for full Hot Reload
- Wait for full Restart
- Fallback if checkHealth throws METHOD_NOT_FOUND.
  We try to wait for the service extensions to be registered and retry.
2017-08-21 13:21:00 -07:00
Sarah Zakarias f1f5d4f3a8 Add integration test for platform interaction (#11580) 2017-08-11 10:38:12 +02:00
Mikkel Nygaard Ravn 87eec719e2 Support for custom build types on Android (#11354) 2017-07-31 13:57:24 +02:00
Devon Carew ba5bb57a52 Revert "fire service protocol extension events for frames (#10966)" (#11430)
This reverts commit 4b4cabb761.
2017-07-27 16:14:30 -07:00
Devon Carew 4b4cabb761 fire service protocol extension events for frames (#10966)
* fire service protocol extension events for frames

* start time in micros

* introduce a profile() function; only send frame events when in profile (or debug) modes

* moved the profile() function to foundation/profile.dart

* refactor to make the change more testable; test the change

* fire service protocol events by listening to onFrameInfo

* remove the frame event stream; add a devicelab test

* remove a todo

* final
2017-07-27 15:34:53 -07:00
Ian Hickson e1adc525d8 Option to enable the performance overlay from 'flutter run'. (#11288) 2017-07-19 12:57:22 -07:00
Todd Volkert c464dd5d2d Mark flutter_gallery__transition_perf_with_semantics as flaky (#11281) 2017-07-18 14:57:01 -07:00
Yegor 18d9b20ffb add Android instrumentation test (#11063)
* add Android instrumentation test

* add devicelab test

* add to manifest.yaml

* rename _smoke_test.dart to _smoketest.dart to prevent flutter test from picking it up

* volatile fields; style fixes

* use ConditionVariable; fix sh script
2017-07-10 17:20:49 -07:00
Ian Hickson bbea0c3d1a enable routing_test in devicelab (#10958) 2017-06-23 18:13:40 -07:00
Ian Hickson 9adb4a78a6 Deep linking: automatically push the route hiearchy on load. (#10894)
The main purpose of this PR is to make it so that when you set the
initial route and it's a hierarchical route (e.g. `/a/b/c`), it
implies multiple pushes, one for each step of the route (so in that
case, `/`, `/a`, `/a/b`, and `/a/b/c`, in that order). If any of those
routes don't exist, it falls back to '/'.

As part of doing that, I:

 * Changed the default for MaterialApp.initialRoute to honor the
   actual initial route.

 * Added a MaterialApp.onUnknownRoute for handling bad routes.

 * Added a feature to flutter_driver that allows the host test script
   and the device test app to communicate.

 * Added a test to make sure `flutter drive --route` works.
   (Hopefully that will also prove `flutter run --route` works, though
   this isn't testing the `flutter` tool's side of that. My main
   concern is over whether the engine side works.)

 * Fixed `flutter drive` to output the right target file name.

 * Changed how the stocks app represents its data, so that we can
   show a page for a stock before we know if it exists.

 * Made it possible to show a stock page that doesn't exist. It shows
   a progress indicator if we're loading the data, or else shows a
   message saying it doesn't exist.

 * Changed the pathing structure of routes in stocks to work more
   sanely.

 * Made search in the stocks app actually work (before it only worked
   if we happened to accidentally trigger a rebuild). Added a test.

 * Replaced some custom code in the stocks app with a BackButton.

 * Added a "color" feature to BackButton to support the stocks use case.

 * Spaced out the ErrorWidget text a bit more.

 * Added `RouteSettings.copyWith`, which I ended up not using.

 * Improved the error messages around routing.

While I was in some files I made a few formatting fixes, fixed some
code health issues, and also removed `flaky: true` from some devicelab
tests that have been stable for a while. Also added some documentation
here and there.
2017-06-23 14:58:29 -07:00
Yegor 480d628635 mark Linux tasks as stable (#10410) 2017-06-14 16:26:52 -07:00
Hans Muller ede575a92e Refactor sample catalog screenshot production (#10676) 2017-06-14 09:05:53 -07:00
Michael Goderbauer e2d4f9242e Benchmark for semantic overhead during transitions (#10678)
* Benchmark for semantic overhead during transitions

* review comments
2017-06-14 08:09:05 -07:00
Yegor fde985b37d resurrect analyzer benchmarks (#10668)
* resurrect analyzer benchmarks

* move analyzer_benchmark to the more stable linux/android

* report average rather than best result
2017-06-13 13:14:16 -07:00
Michael Goderbauer 8bf17192f6 Adding first semantics perf test (#10649)
* Adding first semantics perf test

* review commnts and analyzer fixes

* fix analyzer warning
2017-06-13 12:49:07 -07:00
Chris Bracken 81eb140411 Mark all iOS devicelab tests flaky (#10609)
Host test runner is flaky. These should be re-enabled once the host
machine has been deflaked.
2017-06-09 17:30:04 -07:00
Hans Muller dfc7788c8e update_devicelab_manifest (#10583) 2017-06-08 11:14:02 -07:00
Yegor 1b5b929c73 mark flutter_gallery_ios__transition_perf as flaky (#10420) 2017-05-31 15:40:14 -07:00
Yegor 06deba25ee move more tests to Linux hosts (#10413)
* move more tests to Linux hosts

* fix test
2017-05-31 15:13:08 -07:00
Yegor 7397bb9071 add tests to linux hosts (#10409) 2017-05-31 12:43:18 -07:00
Hans Muller b4e41ff7d3 Add a devicelab task that generates sample app screenshots, etc (#10303) 2017-05-25 11:47:36 -07:00
Ian Hickson a2235d910c Switch off my new test since it broke the build (#10231) 2017-05-20 11:23:19 -07:00
Ian Hickson ab8202dc5a Track missing dartdocs (#10213)
This will help catch when we accidentally break --dartdocs.
2017-05-19 21:48:51 -07:00
Chris Bracken 7160ecfb07 Mark integration_ui, integration_ui_ios tests non-flaky (#10013) 2017-05-11 17:20:46 -07:00
Chris Bracken 2d437f51c3 Add end-to-end test to verify resize on keyboard dismissal (#10005)
Adds an initial integration test that:
1. captures laid-out widget size
2. focuses a text field to show the keyboard and resize the view
3. unfocuses the text field to dismiss the keyboard and resize the view
4. verify that the final widget size matches the initial size
2017-05-11 14:29:20 -07:00
Yegor 5ad06a502b re-enable devicelab tests that are no longer flaky (#9996) 2017-05-11 09:59:50 -07:00
Ian Hickson bc32847fd4 Remove the flaky tests. (#9988)
They are causing nothing but trouble. We should fix these before turning them back on.

See https://github.com/flutter/flutter/issues/9986 https://github.com/flutter/flutter/issues/9987
2017-05-11 08:59:10 -07:00
Ian Hickson f2e89681d9 Mark microbenchmarks_ios as flaky (#9656)
...because it clearly is.
2017-04-27 19:55:54 -07:00
Mikkel Nygaard Ravn 60bc1cfde7 Remove personal dev team from ios code sigining (#9631)
* Mark channels_integration_test_ios flaky

* Removed personal dev team from ios code siging

* Removed provisioning profile diffs against other samples
2017-04-27 11:07:52 +02:00
Mikkel Nygaard Ravn 70ff50f929 Integration test for channel communication (#9621) 2017-04-27 08:44:28 +02:00
Ian Hickson 96d8a2a3ad Mark some tests as flaky (#9489)
These tests have recently been turning the build red incorrectly. Until they are reliable, let's mark them flaky.
2017-04-20 00:11:31 -07:00
Yegor 7e3bb94d5f Mark complex_layout_scroll_perf__timeline_summary as flaky (#9405) 2017-04-14 17:15:34 -07:00
Yegor 9dfcc6ec79 Revert "temporarily mark iOS tests as flaky (#9314)" (#9333)
This reverts commit ff23a1ebfe.

Our iOS lab hardware is back to normal.
2017-04-11 12:26:15 -07:00
Yegor ff23a1ebfe temporarily mark iOS tests as flaky (#9314) 2017-04-10 10:51:31 -07:00
Yegor d538c11486 update devicelab docs (#9274) 2017-04-06 16:27:29 -07:00
Yegor 57ca2b5ce8 obey custom timeout supplied by cocoon (#9260) 2017-04-06 10:48:45 -07:00
Chris Bracken abfee824cd Re-enable iOS transitions test in devicelab (#9097)
This reverts commit 326355f0f9.
2017-03-29 19:19:57 -07:00
Ian Hickson 326355f0f9 Turn off this flaky test for now. (#9065)
cc @cbracken
2017-03-28 16:40:13 -07:00
Michael Goderbauer 506aea0316 enable windows tests with workaround (#9058) 2017-03-28 14:22:03 -07:00
Michael Goderbauer cea7c66dfd Disable flaky WIndows test :( (#9029) 2017-03-27 14:04:50 -07:00
Sarah Zakarias d274888be6 Rename platform_services to platform_channel (#9018)
* Rename platform_services to platform_channel

* rename in README

* renamed tests

* update modules.xml
2017-03-27 11:17:31 +02:00
Sarah Zakarias 47fc81f14d Revert "Remove flutter_view_ios__start_up.dart (#8978)" (#8996)
This reverts commit 38e4e48d06.
2017-03-24 11:45:57 +01:00
Sarah Zakarias 38e4e48d06 Remove flutter_view_ios__start_up.dart (#8978) 2017-03-23 16:06:09 +01:00