Commit graph

899 commits

Author SHA1 Message Date
Jonah Williams a106b81cc5
[framework] restore old zoom page transition for benchmarking. (#133346)
In the past I switched the implementation of the zoom page transition because the performance of the old transition was terrible, but I'm hopeful that with Impeller we'll be able to identify and fix the issues that made it so slow. In order to evaluate this though, we need to be able to opt into the old transition for benchmarks on CI.

https://github.com/flutter/flutter/issues/129742
https://github.com/flutter/flutter/issues/121325
2023-10-10 22:27:07 +00:00
Loïc Sharma 39b9d8fd6d
[Windows] Add first Arm64 plugin tests (#135512)
Part of: https://github.com/flutter/flutter/issues/129806

Test runs kicked off using `led`:
* ✅ [plugin_test_windows](277ffb2c7b/+/build.proto?server=chromium-swarm.appspot.com)
* ⚠️ [plugin_test](c9daccd5b3/+/build.proto?server=chromium-swarm.appspot.com) - This test fails as no existing Windows Arm64 builder install the `android_sdk` and `open_jdk` dependencies

I will add `plugin_dependencies_test` and `gradle_plugin_bundle_test` in a follow-up pull request. These tests also depend on `android_sdk` and `open_jdk`, I'll be able to test them manually using the new `plugin_test` builder added by this pull request. 

/cc @pbo-linaro
2023-10-06 22:28:08 +00:00
Zachary Anderson 68d47e55d4
Revert "Marks Linux_samsung_a02 new_gallery__transition_perf to be unflaky" (#136097)
Reverts flutter/flutter#135566
2023-10-06 15:10:49 -07:00
Flutter GitHub Bot 420ec92e72
Marks Linux_samsung_a02 new_gallery__transition_perf to be unflaky (#135566)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_samsung_a02 new_gallery__transition_perf"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_samsung_a02%20new_gallery__transition_perf%22).
This test can be marked as unflaky.
2023-10-06 20:27:15 +00:00
Flutter GitHub Bot 8796955e98
Marks Windows module_custom_host_app_name_test to be unflaky (#135961)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows module_custom_host_app_name_test"
}
-->
The issue https://github.com/flutter/flutter/issues/134644 has been
closed, and the test has been passing for [50 consecutive
runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows%20module_custom_host_app_name_test%22).
This test can be marked as unflaky.
2023-10-06 11:19:14 -07:00
Flutter GitHub Bot c6d884ea7e
Marks Linux_android opacity_peephole_grid_of_alpha_savelayers_perf__e2e_summary to be unflaky (#135960)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_android opacity_peephole_grid_of_alpha_savelayers_perf__e2e_summary"
}
-->
The issue https://github.com/flutter/flutter/issues/135118 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_android%20opacity_peephole_grid_of_alpha_savelayers_perf__e2e_summary%22).
This test can be marked as unflaky.
2023-10-04 22:46:40 +00:00
Ricardo Amador d7739dff68
Temporarily disable NexusLowRes, version=29 (#135785)
There are too many flakes to leave this version enabled. Temporarily disabled and following up with FTL.

*List which issues are fixed by this PR. You must list at least one issue.*
Related to https://github.com/flutter/flutter/issues/135784

*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
2023-09-30 01:57:05 +00:00
keyonghan 951653a1c0
Add arch property for windows_arm64 platform (#135725)
Window arm64 bots are using x64 built python, which gave incorrect arch info. This PR adds the arch info explicitly so that recipes can get this value and populate to benchmark tags.

Part of https://github.com/flutter/flutter/issues/135722
2023-09-29 00:57:06 +00:00
Flutter GitHub Bot 55c7a2efc4
Marks Linux_android flutter_gallery__start_up_delayed to be unflaky (#135565)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_android flutter_gallery__start_up_delayed"
}
-->
The issue https://github.com/flutter/flutter/issues/134631 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_android%20flutter_gallery__start_up_delayed%22).
This test can be marked as unflaky.
2023-09-28 22:44:52 +00:00
auto-submit[bot] f76c150cc8
Reverts "Marks Windows module_custom_host_app_name_test to be unflaky" (#135692)
Reverts flutter/flutter#135567
Initiated by: ricardoamador
This change reverts the following previous change:
Original Description:
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows module_custom_host_app_name_test"
}
-->
The issue https://github.com/flutter/flutter/issues/134644 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows%20module_custom_host_app_name_test%22).
This test can be marked as unflaky.
2023-09-28 18:36:26 +00:00
Flutter GitHub Bot c35515a079
Marks Windows module_custom_host_app_name_test to be unflaky (#135567)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows module_custom_host_app_name_test"
}
-->
The issue https://github.com/flutter/flutter/issues/134644 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows%20module_custom_host_app_name_test%22).
This test can be marked as unflaky.
2023-09-28 17:38:51 +00:00
godofredoc d3c45f1499
Config changes for linux coverage. (#135604)
This build runs for over and hour and it does not need to run on presubmit or release candidate branches.
2023-09-27 16:42:54 +00:00
Flutter GitHub Bot 9fedb1a6ce
Marks Linux_pixel_7pro hello_world_impeller to be unflaky (#135564)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_pixel_7pro hello_world_impeller"
}
-->
The test has been passing for [50 consecutive
runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_pixel_7pro%20hello_world_impeller%22).
This test can be marked as unflaky.
2023-09-27 08:35:32 -07:00
Loïc Sharma 7143284102
[Windows] Add more Arm64 compile and run tests (#135475)
Part of: https://github.com/flutter/flutter/issues/129806

Test runs kicked off using `led`:
* Compile tests
    * ✅ [flutter_gallery_win_desktop__compile](a23b73ca25/+/build.proto?server=chromium-swarm.appspot.com&fbclid=IwAR2Yv8iS9YbTxMZKS2i5zmAz9E8FCNjW6waLq4e8HwtaFtMlapl66UHf73w)
* Start up tests
    * ✅ [windows_startup_test](e615b28993/+/build.proto?server=chromium-swarm.appspot.com&fbclid=IwAR1YtPTpAr6FRq71fy392orp6PO8t9jN8n-mpWvlDF0xRnnOsq3zsAPabWg)
    * ✅ [flutter_gallery_win_desktop__start_up](a0148122f8/+/build.proto?server=chromium-swarm.appspot.com&fbclid=IwAR29O3jW-LyaVAwD54jUmnhfZ2mswqtaZascRoGvteT5gaHihxKI4IastgM)
    * ✅ [flutter_view_win_desktop__start_up](6dac552365/+/build.proto?server=chromium-swarm.appspot.com&fbclid=IwAR2_irHDuoZ0V-PSjYDW082yx8ZeDVhOIk0yjQBAZNVa5iKVIGAWZs8v4lo)
    * ✅ [platform_view_win_desktop__start_up](e6522c77bc/+/build.proto?server=chromium-swarm.appspot.com&fbclid=IwAR2X7KJDGjQUWJtpq7KY-nUFK-LA6LvmrAinzj0QuF8pEbpwfsAz6Kv3OkM)
    * ✅ [complex_layout_win_desktop__start_up](a811334185/+/build.proto?server=chromium-swarm.appspot.com&fbclid=IwAR0gIiXFMnB_MSaqUxVyTpgpJqc37xSKqOzEsf2oD_oU-AhKPqrGQ-ZD7IY)
* `flutter run` console output tests
    * ✅ [run_debug_test_windows](1b275a8b16/+/build.proto?server=chromium-swarm.appspot.com&fbclid=IwAR1IlBr0824cq9UKi46j_11eto6HTEQ9KVXF5Is1ZxcSIO7G6IlN-slsyI8)
    * ✅ [run_release_test_windows](5b259bb5e2/+/build.proto?server=chromium-swarm.appspot.com&fbclid=IwAR3DSHzCGL-KN26E-7hQL58qxV2p0Dtb9hE4AZ0HvEY4hp8mCMzX9jznsFQ)

/cc @pbo-linaro
2023-09-26 19:27:00 +00:00
Zachary Anderson 16ae8c0109
Revert "Marks Linux_pixel_7pro hello_world_impeller to be unflaky" (#135388)
Reverts flutter/flutter#135372
2023-09-25 14:13:30 +00:00
godofredoc 07ce6a76a0
Revert "Marks Linux_samsung_a02 new_gallery__transition_perf to be unflaky" (#135399)
Reverts flutter/flutter#135373
2023-09-25 04:29:39 +00:00
Flutter GitHub Bot 2f56288a4e
Marks Linux_samsung_a02 new_gallery__transition_perf to be unflaky (#135373)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_samsung_a02 new_gallery__transition_perf"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_samsung_a02%20new_gallery__transition_perf%22).
This test can be marked as unflaky.
2023-09-25 01:49:07 +00:00
Flutter GitHub Bot 797d34e5af
Marks Linux_pixel_7pro hello_world_impeller to be unflaky (#135372)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_pixel_7pro hello_world_impeller"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_pixel_7pro%20hello_world_impeller%22).
This test can be marked as unflaky.
2023-09-24 16:40:10 +00:00
Flutter GitHub Bot a6e98d3006
Marks Windows_android channels_integration_test_win to be unflaky (#135374)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_android channels_integration_test_win"
}
-->
The issue https://github.com/flutter/flutter/issues/134636 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_android%20channels_integration_test_win%22).
This test can be marked as unflaky.
2023-09-24 16:40:07 +00:00
Flutter GitHub Bot 8bc223e3e5
Marks Mac_ios draw_atlas_perf_ios__timeline_summary to be unflaky (#134626)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_ios draw_atlas_perf_ios__timeline_summary"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_ios%20draw_atlas_perf_ios__timeline_summary%22).
This test can be marked as unflaky.
2023-09-24 05:14:20 +00:00
Flutter GitHub Bot 2c1f517d10
Marks Mac_ios draw_vertices_perf_ios__timeline_summary to be unflaky (#134625)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_ios draw_vertices_perf_ios__timeline_summary"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_ios%20draw_vertices_perf_ios__timeline_summary%22).
This test can be marked as unflaky.
2023-09-24 05:06:26 +00:00
Flutter GitHub Bot 417aaf5b2b
Marks Windows module_test to be unflaky (#134121)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows module_test"
}
-->
The issue https://github.com/flutter/flutter/issues/133639 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows%20module_test%22).
This test can be marked as unflaky.
2023-09-24 04:12:53 +00:00
Flutter GitHub Bot c0bde99e0f
Marks Mac_ios dynamic_path_tessellation_perf_ios__timeline_summary to be unflaky (#133128)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_ios dynamic_path_tessellation_perf_ios__timeline_summary"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_ios%20dynamic_path_tessellation_perf_ios__timeline_summary%22).
This test can be marked as unflaky.
2023-09-24 04:11:19 +00:00
Flutter GitHub Bot 6240c899b9
Marks Mac_ios draw_points_perf_ios__timeline_summary to be unflaky (#127933)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_ios draw_points_perf_ios__timeline_summary"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_ios%20draw_points_perf_ios__timeline_summary%22).
This test can be marked as unflaky.
2023-09-24 04:07:20 +00:00
Flutter GitHub Bot 260c16cfb8
Marks Linux_pixel_7pro picture_cache_perf__timeline_summary to be unflaky (#134118)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_pixel_7pro picture_cache_perf__timeline_summary"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_pixel_7pro%20picture_cache_perf__timeline_summary%22).
This test can be marked as unflaky.
2023-09-24 04:04:54 +00:00
Flutter GitHub Bot 82a972bf1a
Marks Mac_ios very_long_picture_scrolling_perf_ios__e2e_summary to be unflaky (#134629)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_ios very_long_picture_scrolling_perf_ios__e2e_summary"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_ios%20very_long_picture_scrolling_perf_ios__e2e_summary%22).
This test can be marked as unflaky.
2023-09-24 03:52:33 +00:00
Flutter GitHub Bot 389d482aac
Marks Mac_ios animated_advanced_blend_perf_ios__timeline_summary to be unflaky (#132630)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_ios animated_advanced_blend_perf_ios__timeline_summary"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_ios%20animated_advanced_blend_perf_ios__timeline_summary%22).
This test can be marked as unflaky.
2023-09-24 02:49:37 +00:00
Flutter GitHub Bot 3ba8472a15
Marks Mac_ios static_path_tessellation_perf_ios__timeline_summary to be unflaky (#133127)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_ios static_path_tessellation_perf_ios__timeline_summary"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_ios%20static_path_tessellation_perf_ios__timeline_summary%22).
This test can be marked as unflaky.
2023-09-24 02:49:35 +00:00
Flutter GitHub Bot 4f5c815559
Marks Linux_android platform_channels_benchmarks to be flaky (#135106)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_android platform_channels_benchmarks"
}
-->
Issue link: https://github.com/flutter/flutter/issues/135105
2023-09-23 23:07:08 +00:00
Flutter GitHub Bot 6d4a5e3d31
Marks Linux_android flutter_gallery__start_up_delayed to be flaky (#134632)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_android flutter_gallery__start_up_delayed"
}
-->
Issue link: https://github.com/flutter/flutter/issues/134631
2023-09-23 23:07:06 +00:00
Flutter GitHub Bot 6d88ade70a
Marks Linux_android opacity_peephole_grid_of_alpha_savelayers_perf__e2e_summary to be flaky (#135119)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_android opacity_peephole_grid_of_alpha_savelayers_perf__e2e_summary"
}
-->
Issue link: https://github.com/flutter/flutter/issues/135118
2023-09-23 23:03:13 +00:00
Zachary Anderson 70fd335041
Move some tests from staging to prod (#135333)
This PR moves tests running on Pixel 7 Pro that use Skia from staging to prod.
2023-09-23 00:32:16 +00:00
Ricardo Amador 76a938bfa6
Reenable 'Linux packages_autoroller' in prod with kms key update (#135316)
The Github token was recreated and uploaded to GCS. I have verified a passing run with the new key here: 
bc5d1e6662/+/build.proto

And the bot is now generating pull requests: https://github.com/flutter/flutter/pull/135315

*List which issues are fixed by this PR. You must list at least one issue.*
Fixes https://github.com/flutter/flutter/issues/131968

*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
2023-09-22 18:23:14 +00:00
Daco Harkes 2def951924
Reland "Native assets support for Linux" (#135097)
Reland of #134031. (Reverted in #135069.) Contains the fix for b/301051367 together with cl/567233346.

Support for FFI calls with `@Native external` functions through Native assets on Linux. This enables bundling native code without any build-system boilerplate code.

For more info see:

* https://github.com/flutter/flutter/issues/129757

### Implementation details for Linux.

Mainly follows the design of https://github.com/flutter/flutter/pull/130494.

Some differences are:

* Linux does not support cross compiling or compiling for multiple architectures, so this has not been implemented.
* Linux has no add2app.

The assets copying is done in the install-phase of the CMake build of a flutter app.
CMake requires the native assets folder to exist, so we create it also when the feature is disabled or there are no assets.

### Tests

This PR adds new tests to cover the various use cases.

* packages/flutter_tools/test/general.shard/linux/native_assets_test.dart
  * Unit tests the Linux-specific part of building native assets.

It also extends various existing tests:

* packages/flutter_tools/test/integration.shard/native_assets_test.dart
  * Runs (incl hot reload/hot restart), builds, builds frameworks for Linux and flutter-tester.
2023-09-22 06:07:06 +00:00
Gray Mackall b433b17539
Upgrade AGP version in tracing_tests (#134671)
Related to https://github.com/flutter/flutter/issues/134419
2023-09-20 20:56:25 +00:00
Xilai Zhang 52ef9d8827
[flutter roll] Revert "Native assets support for Linux" (#135069)
Reverts flutter/flutter#134031

context: b/301051367
Looked at the error message from the broken TAP target, but seems like the failure might be non trivial to resolve. Would it be okay if we revert this for now while it is being triaged?
2023-09-20 00:06:45 +00:00
Zachary Anderson 5c6b031c10
Retry Linux web tests 1 time on roll presubmit (#135073)
Reland of https://github.com/flutter/flutter/pull/134552
2023-09-19 23:17:25 +00:00
Jackson Gardner d3c60569d1
Update CI to Chrome 117. (#134761) 2023-09-19 16:26:16 +00:00
Daco Harkes f246dc7ae0
Bring native_assets ios tests out of staging (#135015)
Move test added in https://github.com/flutter/flutter/pull/130494 out of staging.

* https://ci.chromium.org/ui/p/flutter/builders/luci.flutter.staging/Mac_ios%20native_assets_ios
* https://ci.chromium.org/ui/p/flutter/builders/luci.flutter.staging/Mac_ios%20native_assets_ios_simulator

> Monitor the CI results of the new shard on the [Flutter build dashboard](https://flutter-dashboard.appspot.com/#/build). After 50 consecutive passing builds without any flakes, the flake bot will create a PR to remove the bringup: true parameter from .ci.yaml in the Framework tree. This will allow the test to block the tree, preventing breakages. With this change, the new shard will start running in presubmit automatically, unless specify presubmit: false.

https://github.com/flutter/flutter/wiki/Adding-a-new-Test-Shard#steps-to-add-a-new-framework-test-shard

Umbrella issue:

* https://github.com/flutter/flutter/issues/129757
2023-09-19 15:48:26 +00:00
Daco Harkes 2337c64d0c
Native assets support for Linux (#134031)
Support for FFI calls with `@Native external` functions through Native assets on Linux. This enables bundling native code without any build-system boilerplate code.

For more info see:

* https://github.com/flutter/flutter/issues/129757

### Implementation details for Linux.

Mainly follows the design of https://github.com/flutter/flutter/pull/130494.

Some differences are:

* Linux does not support cross compiling or compiling for multiple architectures, so this has not been implemented.
* Linux has no add2app.

The assets copying is done in the install-phase of the CMake build of a flutter app.
CMake requires the native assets folder to exist, so we create it also when the feature is disabled or there are no assets.

### Tests

This PR adds new tests to cover the various use cases.

* packages/flutter_tools/test/general.shard/linux/native_assets_test.dart
  * Unit tests the Linux-specific part of building native assets.

It also extends various existing tests:

* packages/flutter_tools/test/integration.shard/native_assets_test.dart
  * Runs (incl hot reload/hot restart), builds, builds frameworks for Linux and flutter-tester.
2023-09-18 11:13:37 +00:00
gaaclarke 43260cced9
moved hello_world_impeller to a 7pro (#134830)
arm64 is necessary for this test

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2023-09-15 12:28:16 -07:00
Zachary Anderson 1e4a1be681
Move two tests on Pixel 7 from staging to prod (#134784)
Related https://github.com/flutter/flutter/issues/134698
2023-09-14 19:10:37 -07:00
Victoria Ashworth eb4b97d63c
Set xcode version to older compatible version for microbenchmarks_ios_xcode_debug test (#134693)
Test added in https://github.com/flutter/flutter/pull/134493 requires an
older version of Xcode because default version of Xcode is not
compatible with macOS version in CI.

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2023-09-14 13:56:00 -05:00
gaaclarke 57ad4ff018
Added a devicelab test for vulkan validation layers (#134685)
This makes sure validation layers are being used and that there are no
validation errors

fixes https://github.com/flutter/flutter/issues/134175

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2023-09-14 09:13:35 -07:00
Victoria Ashworth af5ac930d8
Set the CONFIGURATION_BUILD_DIR in generated xcconfig when debugging core device (#134493)
Xcode uses the CONFIGURATION_BUILD_DIR build setting to determine the location of the bundle to build and install. When launching an app via Xcode with the Xcode debug workflow (for iOS 17 physical devices), temporarily set the CONFIGURATION_BUILD_DIR to the location of the bundle so Xcode can find it.

Also, added a Xcode Debug version of the `microbenchmarks_ios` integration test since it uses `flutter run --profile` without using `--use-application-binary`.

Fixes https://github.com/flutter/flutter/issues/134186.
2023-09-13 18:08:35 +00:00
Flutter GitHub Bot f38dcb1229
Marks Windows module_custom_host_app_name_test to be flaky (#134645)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows module_custom_host_app_name_test"
}
-->
Issue link: https://github.com/flutter/flutter/issues/134644
2023-09-13 09:36:18 -07:00
Flutter GitHub Bot 639c092b5d
Marks Windows_android channels_integration_test_win to be flaky (#134637)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_android channels_integration_test_win"
}
-->
Issue link: https://github.com/flutter/flutter/issues/134636
2023-09-13 09:35:22 -07:00
keyonghan 3e1e869a54
Revert "Retry Linux web tests 1 time on roll presubmit" (#134586)
Reverts flutter/flutter#134552
2023-09-12 22:20:39 +00:00
Zachary Anderson 64c1223b9c
Retry Linux web tests 1 time on roll presubmit (#134552)
A flake in this shard caused an engine roll to fail here https://github.com/flutter/flutter/pull/134526
2023-09-12 18:29:02 +00:00
Flutter GitHub Bot bdd96032a0
Marks Windows_android channels_integration_test_win to be unflaky (#133129)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_android channels_integration_test_win"
}
-->
The issue https://github.com/flutter/flutter/issues/130732 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_android%20channels_integration_test_win%22).
This test can be marked as unflaky.
2023-09-12 15:37:30 +00:00
Daco Harkes aa36db1d29
Native assets support for MacOS and iOS (#130494)
Support for FFI calls with `@Native external` functions through Native assets on MacOS and iOS. This enables bundling native code without any build-system boilerplate code.

For more info see:

* https://github.com/flutter/flutter/issues/129757

### Implementation details for MacOS and iOS.

Dylibs are bundled by (1) making them fat binaries if multiple architectures are targeted, (2) code signing these, and (3) copying them to the frameworks folder. These steps are done manual rather than via CocoaPods. CocoaPods would have done the same steps, but (a) needs the dylibs to be there before the `xcodebuild` invocation (we could trick it, by having a minimal dylib in the place and replace it during the build process, that works), and (b) can't deal with having no dylibs to be bundled (we'd have to bundle a dummy dylib or include some dummy C code in the build file).

The dylibs are build as a new target inside flutter assemble, as that is the moment we know what build-mode and architecture to target.

The mapping from asset id to dylib-path is passed in to every kernel compilation path. The interesting case is hot-restart where the initial kernel file is compiled by the "inner" flutter assemble, while after hot restart the "outer" flutter run compiled kernel file is pushed to the device. Both kernel files need to contain the mapping. The "inner" flutter assemble gets its mapping from the NativeAssets target which builds the native assets. The "outer" flutter run get its mapping from a dry-run invocation. Since this hot restart can be used for multiple target devices (`flutter run -d all`) it contains the mapping for all known targets.

### Example vs template

The PR includes a new template that uses the new native assets in a package and has an app importing that. Separate discussion in: https://github.com/flutter/flutter/issues/131209.

### Tests

This PR adds new tests to cover the various use cases.

* dev/devicelab/bin/tasks/native_assets_ios.dart
  * Runs an example app with native assets in all build modes, doing hot reload and hot restart in debug mode.
* dev/devicelab/bin/tasks/native_assets_ios_simulator.dart
  * Runs an example app with native assets, doing hot reload and hot restart.
* packages/flutter_tools/test/integration.shard/native_assets_test.dart
  * Runs (incl hot reload/hot restart), builds, builds frameworks for iOS, MacOS and flutter-tester.
* packages/flutter_tools/test/general.shard/build_system/targets/native_assets_test.dart
  * Unit tests the new Target in the backend.
* packages/flutter_tools/test/general.shard/ios/native_assets_test.dart
* packages/flutter_tools/test/general.shard/macos/native_assets_test.dart
  * Unit tests the native assets being packaged on a iOS/MacOS build.

It also extends various existing tests:

* dev/devicelab/bin/tasks/module_test_ios.dart
   * Exercises the add2app scenario.
* packages/flutter_tools/test/general.shard/features_test.dart
   * Unit test the new feature flag.
2023-09-10 08:07:13 +00:00
Flutter GitHub Bot da676f79aa
Marks Linux_android very_long_picture_scrolling_perf__e2e_summary to be unflaky (#134116)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_android very_long_picture_scrolling_perf__e2e_summary"
}
-->
The test has been passing for [50 consecutive
runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_android%20very_long_picture_scrolling_perf__e2e_summary%22).
This test can be marked as unflaky.
2023-09-08 10:08:21 -07:00
Gray Mackall 827b5dff02
[integration_test] Allow capture of screenshots for FlutterFragmentActivitys (#132406)
Fixes https://github.com/flutter/flutter/issues/89683.

The changes to `getFlutterView` in `FlutterDeviceScreenshot` are the fix that was required, everything else was done to get tests running (such as re-generating some lockfiles and modifying the android manifest).

The code was all currently not unit tested, and there were no other easy examples to base these java unit tests off in flutter/flutter, so let me know if this approach to testing is wrong.
2023-09-07 22:32:48 +00:00
Loïc Sharma 2032fcc571
[Windows Arm64] Remove device_type property (#134181)
The Windows Arm64 tests are failing to start as no machines match
desired properties. This removes the `device_type: none` property as no
Windows Arm64 machines match this ([with
`device_type`](https://chromium-swarm.appspot.com/botlist?c=id&c=task&c=os&c=status&d=asc&f=cpu%3Aarm64&f=os%3AWindows&f=pool%3Aluci.flutter.staging&f=device_type%3Anone&s=id),
[without
`device_type`](https://chromium-swarm.appspot.com/botlist?c=id&c=task&c=os&c=status&d=asc&f=cpu%3Aarm64&f=os%3AWindows&f=pool%3Aluci.flutter.staging&s=id)).

This allows the tests to run on the Windows Arm64 machine pool. [I
tested this manually using
`led`](d96efacb0c/+/build.proto?server=chromium-swarm.appspot.com).

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [ ] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2023-09-07 13:25:36 -07:00
Loïc Sharma d792c664a2
[Windows Arm64] Also use Windows 11 for Devicelab tests (#134082)
Currently, the Windows Arm64 tests try to run on Windows 10 devices.
However, [all Windows Arm64 devices use Windows
11](https://chromium-swarm.appspot.com/botlist?c=id&c=task&c=os&c=status&d=asc&f=cpu%3Aarm64&f=os%3AWindows-11&f=pool%3Aluci.flutter.staging&s=id)
which result in the Windows Arm64 tests not running. This updates the
configuration to accept both Windows 10 and 11 (ideally the tests would
run on both OSes).

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2023-09-06 11:59:20 -07:00
Loïc Sharma df8ee8d63a
[Windows Arm64] Add first device lab test (#133895)
Adds the first Windows Arm64 devicelab test. The builder for this test won't be created until after this lands on master, so we can't actually test this until this is merged.
2023-09-01 23:55:36 +00:00
Gray Mackall 033274c66a
Change android_virtual_device to version 34 (#133809)
Change from AVD 33 to [AVD 34](https://chrome-infra-packages.appspot.com/p/chromium/tools/android/avd/linux-amd64) in CI. Only affects two tests.

We should also update the firebase virtual and physical devices when 34 is available there (for example [here](e98d3929ca/.ci.yaml (L441))), as that will allow us to update a lot more tests.
2023-09-01 16:48:20 +00:00
Victoria Ashworth 37c55bab2c
Run Mac_arm64_ios run_debug_test_macos in presubmit (#133869)
Reland https://github.com/flutter/flutter/pull/133788.

Fixes https://github.com/flutter/flutter/issues/118830.
Fixes https://github.com/flutter/flutter/issues/125342.
2023-09-01 16:20:46 +00:00
Victoria Ashworth e98d3929ca
Revert "Run Mac_arm64_ios run_debug_test_macos in presubmit" (#133808)
Reverts flutter/flutter#133788

Something is wrong with bot https://chromium-swarm.appspot.com/bot?id=flutter-devicelab-mac-38, it's unable to open the project in Xcode. Failing consistently. 

https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8771213845302282993/+/u/ios_debug_symbol_doctor/recover_with_120_second_timeout/stdout

So reverting until we can get the bot fixed.
2023-08-31 21:34:48 +00:00
Victoria Ashworth 8128b2e2f6
Run Mac_arm64_ios run_debug_test_macos in presubmit (#133788)
Fixes https://github.com/flutter/flutter/issues/118830.
2023-08-31 19:41:07 +00:00
Jim Graham b51859ec33
Add benchmark for scrolling very long pictures (#133434)
This benchmark will track the performance of the RTree implementation to cull very large pictures down to just the portion visible on the screen.
2023-08-31 19:03:14 +00:00
Flutter GitHub Bot 7368da037a
Marks Windows module_test to be flaky (#133640)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows module_test"
}
-->
Issue link: https://github.com/flutter/flutter/issues/133639
2023-08-30 08:00:14 -07:00
Zachary Anderson d8470f713c
Don't run on Samsung S10, shift to Pixel 7 Pro (#133085) 2023-08-22 19:18:00 -07:00
Victoria Ashworth b3005a318b
Re-enable Xcode Debug tests (#132523)
Re-enables Xcode Debug tests (still in bringup). It also has them use an older version of Xcode that is compatible with the host OS.

Fixes https://github.com/flutter/flutter/issues/132309.
2023-08-22 17:22:06 +00:00
Zachary Anderson 6a02483044
Run new_gallery__transition_perf on Pixel 7 and Samsung A02 (#132772) 2023-08-17 11:42:57 -07:00
keyonghan 33b66f6436
Skip backfiller for docs_publish (#132510)
This is to avoid old docs published by backfilling.
2023-08-15 16:53:17 +00:00
Jonah Williams f7bd03202f
[devicelab] boot up benchmarks. (#132148)
Enable Impeller benchmarks for drawAtlas/drawVertices on iOS/Metal, Android/GLES, and Android/Vulkan.

Enable impeller tessellation benchmarks on iOS/Metal and Android/Vulkan - not GLES as this is measuring backend agnostic performance.
2023-08-14 18:47:08 +00:00
Victoria Ashworth fff08248b3
Disable Xcode Debug tests (#132398)
Temporarily removes Xcode debug tests. 

Xcode tests are currently failing in CI for some infra-related issues. I have a fix for it (https://github.com/flutter/flutter/pull/132318) that I confirmed worked in presubmit, but I think the logic is better suited to live in recipes. However, it's going to be a bit of a learning curve to update recipes and might take me a couple days so disabling them in the meantime.

Tracking issue to readd when ready: https://github.com/flutter/flutter/issues/132309
2023-08-11 18:24:23 +00:00
Zachary Anderson bd1bb675b3
Remove bringup from Windows build_tests shards (#132306)
Context:
aff8ef13d4
2023-08-10 09:25:02 -07:00
Victoria Ashworth d631b26285
New tooling for iOS 17 physical devices (#131865)
This PR includes the following changes. These changes only apply to iOS 17 physical devices.

| Command | Change Description  | Changes to User Experience |
| ------------- | ------------- | ------------- |
| `flutter run --release` | Uses `devicectl` to install and launch application in release mode.  | No change.  |
| `flutter run`  | Uses Xcode via automation scripting to run application in debug and profile mode. | Xcode will be opened in the background. Errors/crashes may be caught in Xcode and therefore may not show in terminal. |
| `flutter run --use-application-binary=xxxx` | Creates temporary empty Xcode project and use Xcode to run via automation scripting in debug and profile. | Xcode will be opened in the background. Errors/crashes may be caught in Xcode and therefore may not show in terminal.  |
| `flutter install` | Uses `devicectl` to check installed apps, install app, uninstall app.  | No change.  |
| `flutter screenshot` | Will return error.  | Will return error.  |

Other changes include:
* Using `devicectl` to get information about the device
* Using `idevicesyslog` and Dart VM logging for device logs

Note:
Xcode automation scripting (used in `flutter run` for debug and profile) does not work in a headless (without a UI) interface. No known workaround.

Fixes https://github.com/flutter/flutter/issues/128827, https://github.com/flutter/flutter/issues/128531.
2023-08-09 19:25:12 +00:00
gaaclarke 1ff82af540
[Impeller] Adds advanced blend benchmark (#131893)
issue: https://github.com/flutter/flutter/issues/131784

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2023-08-09 10:57:35 -07:00
Ricardo Amador 45383deffa
Mark the ci_yaml roller task so that it is not backfilled. (#131966)
In order to avoid new tasks from not being scheduled we are not going to backfill the autoroller because it can sometimes play transactions out of order causing issues with tasks that are scheduled but not known.

*List which issues are fixed by this PR. You must list at least one issue.*
Part of https://github.com/flutter/flutter/issues/127063

*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
2023-08-05 01:18:10 +00:00
Christopher Fujino a2fa60cc4d
mark linux packages_autoroller flaky because of token issue (#131970)
unblock tree from https://github.com/flutter/flutter/issues/131968
2023-08-04 18:00:14 -07:00
gaaclarke f0e39349c9
[Impeller] Add animated_blur_backdrop_filter_perf execution on samsung vulkan and opengles (#131864)
issue: https://github.com/flutter/flutter/issues/131784
2023-08-03 22:31:06 +00:00
gaaclarke 4247e1eff1
Added new gallery benchmark test that forces opengles. (#131796)
fixes https://github.com/flutter/flutter/issues/131782

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat

---------

Co-authored-by: Xilai Zhang <xilaizhang@google.com>
2023-08-03 12:25:29 -07:00
Chris Yang 13ff16cba9
remove Mac_ios ios_app_with_extensions_test (#131806)
Removing the extension test that unnecessarily running with devices.

The new hostonly tests were added in https://github.com/flutter/flutter/pull/131441
2023-08-03 16:28:58 +00:00
Mahmoud Zayad 22f029e582
Remove target Staging_build_linux framework_tests_misc (#126959)
I deleted the target mentioned in the issue listed below. This is my first time contributing to open source please explain if I did anything wrong in the process or wrong with the changes. 

Best

*List which issues are fixed by this PR. You must list at least one issue.*
Resolves [#126278](https://github.com/flutter/flutter/issues/126278)
2023-08-03 15:50:59 +00:00
Chris Yang 12b8e0cb0e
add gem dependency to the ios_app_with_extensions_test (#131713)
Adds gem dependency to the tests 

Fixes https://github.com/flutter/flutter/issues/131693
2023-08-02 18:37:06 +00:00
Elias Yishak aff8ef13d4
Update .ci.yaml to add new shard to prevent timeouts (#131712)
Fixes:
- https://github.com/flutter/flutter/issues/131327

It was observed that the tests would hit their 30 minute timeout so this PR adds another shard to reduce how many tests are run per shard to reduce test run time
2023-08-02 17:26:44 +00:00
Chris Yang 3c8c3004f3
Move ios_app_with_extensions_test to host only (#131441)
`ios_app_with_extensions_test` only builds the artifact, it does not need a device to run.
This PR adds the copies of these tests to run hostonly. When the new tests pass in post submit, we should remove the old ones.
2023-07-28 15:42:35 +00:00
Jackson Gardner e81907e07a
Update Chrome to 115, and remove bringup: true from skwasm benchmarks. (#131430)
Recent changes to binaryen cause us to need to update to Chrome 115 for our wasm benchmarks.
2023-07-28 15:04:19 +00:00
Loïc Sharma 81b0e9f111
Revert "Run benchmarks with --omit-type-checks" (#131353)
Reverts flutter/flutter#131102 as it closed the tree due to `Linux web_benchmarks_skwasm` failures. The test seems to be timing out without completing.

Example failure: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_benchmarks_skwasm/1/overview
2023-07-26 19:23:37 +00:00
Jackson Gardner 8046e13373
Run benchmarks with --omit-type-checks (#131102)
We've decided to use the `--omit-type-checks` flag for our dart2wasm benchmarks. Right now, many of the benchmark results are dominated by type checks and most of what we are actually trying to measure get drowned out in the noise.
2023-07-26 17:10:07 +00:00
Dan Field 764095ce1a
[CI/FTL] Oriole to Panther, presubmit false (#130912)
Alternative to https://github.com/flutter/flutter/pull/130868

Oriole is the Pixel 6, Panther is the Pixel 7. Panther is more available in FTL at this point.

There's less value in running this on presubmit, since those can spawn many jobs multiple times over as people push new commits to branches. Let's just run it post submit to avoid overloading the capacity of FTL.
2023-07-21 18:21:22 +00:00
Zachary Anderson 3b0b670d91
Mark Windows_android channels_integration_test_win flaky (#130893)
For https://github.com/flutter/flutter/issues/130732
2023-07-19 08:39:43 -07:00
Reid Baker 7a42ed7ef6
Update to valid build tools variant and update lockfiles (#125825)
https://github.com/flutter/flutter/issues/125331
2023-07-17 15:57:08 +00:00
Ricardo Amador 4d7bffe7f5
Remove use_emulator flag (#130634)
MInor cleanup to remove the use_emulator flag as adding the virtual device is enough to signal use of an emulator.

Land after https://flutter-review.googlesource.com/c/recipes/+/47341

*List which issues are fixed by this PR. You must list at least one issue.*
Fixes https://github.com/flutter/flutter/issues/130522

*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
2023-07-14 23:52:49 +00:00
Zachary Anderson e1d025ae54
Move straggling Android Impeller test to staging. (#130529) 2023-07-13 15:31:02 -07:00
Jonah Williams 0e543889ec
mark android impeller tests as bringup (#130525)
These should not close the tree as this backend is a WIP.
2023-07-13 14:35:30 -07:00
Dan Field 13283f86cb
Revert "Marks Linux firebase_oriol33_abstract_method_smoke_test to be unflaky" (#130497)
Reverts flutter/flutter#128398

See e.g.
https://ci.chromium.org/ui/p/flutter/builders/try/Linux%20firebase_oriol33_abstract_method_smoke_test/350/overview
2023-07-13 08:55:36 -07:00
Christopher Fujino b89bfd8e37
re-enable "Linux packages_autoroller" (#130088)
Fixes https://github.com/flutter/flutter/issues/129744

This change:

1. re-enables the Linux packages_autoroller
2. ensures we redact the token from appearing in any logs (in local testing I realized some failure logs might still expose the token)

What actually fixed authentication however was creating and uploading a new GitHub personal access token, not this change.

It's currently failing post-submit because being marked `bringup` it is running in the try pool, which does not have permissions to access the cloud KMS. However, I ran a LED build in the prod pool that succeeded:

3a8f128c35/+/build.proto?server=chromium-swarm.appspot.com
2023-07-10 19:32:56 +00:00
Flutter GitHub Bot c8265f81cf
Marks Linux firebase_oriol33_abstract_method_smoke_test to be unflaky (#128398)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux firebase_oriol33_abstract_method_smoke_test"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux%20firebase_oriol33_abstract_method_smoke_test%22).
This test can be marked as unflaky.
2023-07-06 19:42:24 +00:00
Christopher Fujino 97fce9aa19
mark packages-autoroller bringup again (#129859)
Even after updating the account (and the cloudKMS GitHub token), when
the bot tries to push to
https://github.com/flutter-pub-roller-bot/flutter, it gets a 403, which
probably means authentication failed. I will have to investigate next
week.

```
Exception: GitException: Exception on command "push https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git packages-autoroller-branch-1:packages-autoroller-branch-1": Command "git push https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git packages-autoroller-branch-1:packages-autoroller-branch-1" failed in directory "/b/s/w/ir/x/w/flutter_conductor_checkouts/framework" to update the release branch with the commit. Git exited with error code 128.
stderr from git:
16:33:21.199510 exec-cmd.c:90           trace: resolved executable path from procfs: /b/s/w/ir/cipd_bin_packages/bin/git
16:33:21.199587 exec-cmd.c:237          trace: resolved executable dir: /b/s/w/ir/cipd_bin_packages/bin
16:33:21.200155 git.c:460               trace: built-in: git push https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git packages-autoroller-branch-1:packages-autoroller-branch-1
16:33:21.200542 run-command.c:655       trace: run_command: GIT_DIR=.git git remote-https https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git
16:33:21.201787 exec-cmd.c:90           trace: resolved executable path from procfs: /b/s/w/ir/cipd_bin_packages/libexec/git-core/git
16:33:21.201845 exec-cmd.c:237          trace: resolved executable dir: /b/s/w/ir/cipd_bin_packages/libexec/git-core
16:33:21.202175 git.c:750               trace: exec: git-remote-https https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git
16:33:21.202195 run-command.c:655       trace: run_command: git-remote-https https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git
16:33:21.203510 exec-cmd.c:90           trace: resolved executable path from procfs: /b/s/w/ir/cipd_bin_packages/libexec/git-core/git-remote-http
16:33:21.203567 exec-cmd.c:237          trace: resolved executable dir: /b/s/w/ir/cipd_bin_packages/libexec/git-core
remote: Permission to flutter-pub-roller-bot/flutter.git denied to flutter-pub-roller-bot.
fatal: unable to access 'https://github.com/flutter-pub-roller-bot/flutter.git/': The requested URL returned error: 403
```


https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20packages_autoroller/6031/overview

This is still being tracked in
https://github.com/flutter/flutter/issues/129744
2023-06-30 16:54:55 -07:00
Christopher Fujino 67b7aa22c2
Change pub roller bot to push to flutter-pub-roller-bot/flutter.git (#129844)
Fixes https://github.com/flutter/flutter/issues/129744
Part of https://github.com/flutter/flutter/issues/129847

Migrate the packages_autoroller script to push to
flutter-pub-roller-bot/flutter.git, rather than
fluttergithubbot/flutter.git.
2023-06-30 16:28:00 -07:00
gmackall d1eb1f0a57
Upgrade integration tests to use AGP 7.3/Gradle 7.4 (#129642)
Upgrades tests under `dev/integration_tests` to use the latest values [in the templates](944d6c8fef/dev/tools/bin/generate_gradle_lockfiles.dart (L135)). Most of this PR was auto-generated, by running `find dev/integration_tests/ -type d -name 'android' | dart dev/tools/bin/generate_gradle_lockfiles.dart` from the root of the flutter directory.

The pieces that were not are:
1. Upgrading the Gradle versions used in integration tests to be >=7.4, in places where it was currently lower.
2. Upgrading the mac, windows, and linux build_tests .ci.yaml configuration to use jdk 17 on all shards. It currently was using a mix of 17 and 11. This isn't desirable, because some of the tests require 17, and the distribution is [random across shards](cef9a8c54b/dev/bots/test.dart (L553)) (so they were only passing because they were getting randomly placed on shards using jdk 17).
3. Adding a dependency on jdk 11 for the tests based on the `firebaselab/firebaselab` recipe. Previously they had no java dependency, and were therefore defaulting to java 1.8. The newer AGP versions necessitated an upgrade to 11 to run.

Note that it also ended up downgrading the AGP version in two places (in the [hybrid_android_views](https://github.com/flutter/flutter/pull/129642/files#diff-d7cfaa1c96d5c2383e7fc253ac8c9df062b544c24f7d80aa8a02290c01d17205L17) and [platform_interaction](https://github.com/flutter/flutter/pull/129642/files#diff-b2fba811efadd1cfb097bebd93be99fe4b6d6f17aefa863267290fc704290766L17) tests), because those had been manually edited to a newer version than the template.

Related to: https://github.com/flutter/flutter/issues/123636, https://github.com/flutter/flutter/pull/123910
2023-06-30 19:34:20 +00:00
Jackson Gardner 1b887c72b5
Skwasm benchmarks. (#129681)
This enables benchmarks for the Skwasm renderer, compiled with
dart2wasm.

Platform views aren't supported in Skwasm yet, so we are skipping those
benchmarks for now.
2023-06-28 17:16:19 -07:00
Christopher Fujino c1b764a80f
mark autoroller flaky (#129745)
Unblock tree while I investigate: https://github.com/flutter/flutter/issues/129744
2023-06-28 23:26:56 +00:00
Victoria Ashworth 9531cd4000
Update Xcode to 14.3.1 (#129024)
Framework part of https://github.com/flutter/flutter/issues/129019
2023-06-27 18:21:33 +00:00
Brandon DeRosier b898d1d270
[devicelab] Add platform view scroll benchmarks for Impeller on Android (#129455)
For tracking perf improvements with changes like
https://github.com/flutter/engine/pull/43152.
2023-06-23 18:40:17 -07:00
godofredoc 929468e171
Add a devicelab test to recipes cq. (#129411)
This is to ensure the devicelab_drone recipe is tested on presubmit.

Bug: https://github.com/flutter/flutter/issues/126111
2023-06-23 17:38:38 +00:00
Kate Lovett 309e50173e
Run misc test shard when examples/api/** change (#129433)
Fixes https://github.com/flutter/flutter/issues/129390

Found in https://github.com/flutter/flutter/pull/129381, changing sample code in `examples/api` did not trigger the misc test shard that executes the tests for `examples/api`.
2023-06-23 16:46:04 +00:00
godofredoc f582c0c573
Add comments to .ci.yaml that were removed from recipes. (#129321)
Firebaselab recipe was cleaned up and comments removed. This is adding the still valid comments to the .ci.yaml file.
2023-06-23 05:10:54 +00:00
gmackall 51f659825e
Unpin path_provider_android (#129205)
Unpins path_provider_android where it is pinned. Follows the same steps as https://github.com/flutter/flutter/pull/128898.

Fixes https://github.com/flutter/flutter/issues/116376
2023-06-22 21:22:49 +00:00
godofredoc dffef2715e
Move all the firebase lab device configs to .ci.yaml. (#129219)
Bug: https://github.com/flutter/flutter/issues/129211
2023-06-21 01:10:07 +00:00
Jackson Gardner 54e27e3ed5
Update Chrome version for testing (#128447)
Most of the tests are using Chrome 96.2, which is quite old. Update to Chrome 114.
2023-06-08 15:37:06 +00:00
Reid Baker b4796915b3
i127682 android api 33 test support (#127689)
- try to set emulator to version 33 to see what happens

#127682
2023-05-31 16:38:16 +00:00
godofredoc 7d310e3b1b
Add oriol_33 firebaselab configuration. (#127894)
This allows to run the api 33 as a separate build.

Bug: https://github.com/flutter/flutter/issues/127884
Bug: https://github.com/flutter/flutter/issues/118736
2023-05-31 00:19:58 +00:00
keyonghan 2b4e303b5a
Remove legacy xcode properties/dependencies (#127724)
Now only $flutter/osx_sdk property is being used, and it's safe to remove deprecated entries from ci.yaml.

Part of https://github.com/flutter/flutter/issues/127534
2023-05-30 23:14:02 +00:00
keyonghan 3eecc217cf
Append $flutter/osx sdk property to existing platforms/targets relying on xcode (#127537)
Part of https://github.com/flutter/flutter/issues/127534
2023-05-25 22:47:31 +00:00
keyonghan 784ebdf2a3
Run Mac intel only targets on both intel and arm (#127230)
The slowness issue was resolved with latest xcode: https://github.com/flutter/flutter/issues/119750, and all these targets should resume on both platforms as before. This will help mitigate high queue time on intel bots.

This PR updates these targets:

- Mac plugin_test_ios: https://github.com/flutter/flutter/issues/119764
- Mac build_tests: https://github.com/flutter/flutter/pull/120620
- Mac plugin_test: https://github.com/flutter/flutter/pull/120714
- Mac plugin_test_macos: https://github.com/flutter/flutter/pull/122212
- Mac framework_tests_misc: https://github.com/flutter/flutter/pull/122618
2023-05-19 23:22:25 +00:00
Zachary Anderson 31a16f342e
Roll goldctl to f808dcff91b221ae313e540c09d79696cd08b8de (#127218)
goldctl hasn't been updated in about a year, and an update may be needed to reland https://github.com/flutter/flutter/pull/127110.
2023-05-19 18:52:50 +00:00
Christopher Fujino 344c9c6af2
mark windows build tests non-bringup (#127059)
This is a follow-up to aa0a1a7ae1. I was forced to mark these targets bringup in that PR because the targets were technically new (since each name changed 3 -> 4), so this is marking them non-bringup.
2023-05-18 17:08:21 +00:00
Christopher Fujino aa0a1a7ae1
shard windows build tests 3 -> 4 (#127057)
Time outs are currently making the tree red.

Fixes https://github.com/flutter/flutter/issues/127004
2023-05-17 15:49:35 -07:00
godofredoc 25a9efe994
Fix drone_dimensions. (#126953)
Use drone_dimensions instead of dimensions for builds running on dart-internal. This is required for those builders to select the correct drone type.
2023-05-16 22:11:21 +00:00
keyonghan 3f01c7e019
Move Mac_build_test flutter_gallery__transition_perf_e2e_ios to prod (#126941)
Target `Mac_build_test flutter_gallery__transition_perf_e2e_ios` was enabled in staging: https://github.com/flutter/flutter/pull/111164, and it has passed more than 50 runs: https://ci.chromium.org/p/flutter/builders/staging/Mac_build_test%20flutter_gallery__transition_perf_e2e_ios?limit=50. 

Manually enabling it in prod and removing the old `Mac_ios flutter_gallery__transition_perf_e2e_ios`.

The `Mac_build_test` one does the same thing as `Mac_ios` one, but separating build and test steps in separate targets.

Context: https://github.com/flutter/flutter/issues/103542
2023-05-16 21:46:06 +00:00
Jonah Williams 41abe998ee
[devicelab] add drawPoints benchmark (#126728)
Add a benchmark that measures the improvements from https://github.com/flutter/engine/pull/41803
2023-05-15 19:14:58 +00:00
Zachary Anderson 674b1ad0ae
Remove obsolete iOS benchmark (#126862)
Fixes https://github.com/flutter/flutter/issues/126861
2023-05-15 19:12:56 +00:00
Reid Baker 159ebf736a
Mark no longer flakey tests as bringup:false (#126281)
This is a result of auditing the ci.yaml tests marked as bringup true and determining which ones were still flakey. 
Each commit markes one tests as non flakey along with a link of proof of the lack of flakes. 

- Mark new_gallery_impeller__transition_perf as not flakey, #124699 closed, not recent flakes
- No failures in 500 runs for Staging_build_linux analyze https://ci.chromium.org/p/flutter/builders/staging/Staging_build_linux%20analyze\?cursor\=id%3E8782771663962116337\&limit\=200
- No flakes in 600 builds https://ci.chromium.org/p/flutter/builders/staging/Linux%20flutter_packaging_test\?cursor\=id%3E8787892377160059793\&limit\=200
- No flakes in 600 builds https://ci.chromium.org/p/flutter/builders/staging/Mac%20flutter_packaging_test\?cursor\=id%3E8787870631931126673\&limit\=200
- No flakes in 600 builds https://ci.chromium.org/p/flutter/builders/staging/Mac_arm64%20flutter_packaging_test\?cursor\=id%3E8787809799161254193\&limit\=200

#124699 is the only closed bug that was linked for a test that is no longer flakey. Closed status helps the case that flakes are no longer an issue.
2023-05-15 18:12:52 +00:00
Zachary Anderson 468512e057
Increase test timeout for analyzer_benchmark to 60 minutes (#126586)
Fixes https://github.com/flutter/flutter/issues/126451
2023-05-12 08:45:14 -07:00
godofredoc fa117aad28
Add dimensions to release builds. (#126117)
Release builds running in dart internal require the dimensions to be set explicitly at the target level in ci.yaml.

Bug: https://github.com/flutter/flutter/issues/126116
2023-05-11 17:28:08 +00:00
keyonghan a7a1ddfb5a
Add cpu: x86 dimension and xcode dependency for Mac_build_test targets (#126482)
This is a follow up of https://github.com/flutter/flutter/pull/111164.

My local LED was based on `cpu: x86` dimension and `xcode` dependency. But these are missed in the above PR, which caused test failure: https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_build_test%20flutter_gallery__transition_perf_e2e_ios/19/overview
2023-05-11 16:41:26 +00:00
keyonghan 8ab782d08a
Migrate gallery ios tests to build+test (#111164)
Part of https://github.com/flutter/flutter/issues/103542
2023-05-10 00:50:11 +00:00
Reid Baker 0fdddd67f9
Test AGP 8.0 using java 17 (#125323)
- Update Gradle/AGP version and add namespace plus dependencies. 

https://github.com/flutter/flutter/issues/125181
2023-05-08 15:02:31 +00:00
Drew Roen 9de9a81b80
Bring back the failing build_test's (#126014)
https://github.com/flutter/flutter/issues/125897

The issue has been resolved, so bringing the tests out of bringup
2023-05-05 15:54:43 +00:00
Jenn Magder 4f3c989bd3
Remove complex_layout_*__compile benchmarks (#126023)
Fixes https://github.com/flutter/flutter/issues/126020.
2023-05-03 23:49:01 +00:00
Flutter GitHub Bot 15bd29c8e2
Marks Linux_android slider_perf_android to be unflaky (#125968)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_android slider_perf_android"
}
-->
The test has been passing for [50 consecutive
runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_android%20slider_perf_android%22).
This test can be marked as unflaky.
2023-05-03 09:22:21 -07:00
Chris Yang 9fcb42d0dd
Update xcode to 14e222b (#125716)
Framework part of https://github.com/flutter/flutter/issues/125814
2023-05-02 19:56:17 +00:00
Jenn Magder cc9a3d9d95
Mark microbenchmarks_ios unflaky (#125332)
This hasn't flaked for > 50 commits which is our threshhold.  Marking as unflaky.
2023-05-02 19:34:40 +00:00
godofredoc d0069bec94
Mark build_test builds as bringup. (#125898)
They require an additional dimension to be passed as part of the
.ci.yaml configuration.

Bug: https://github.com/flutter/flutter/issues/125897

## Pre-launch Checklist

- [X] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [X] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [X] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [X] I signed the [CLA].
- [X] I listed at least one issue that this PR fixes in the description
above.
- [X] I updated/added relevant documentation (doc comments with `///`).
- [X] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [X] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2023-05-02 12:22:56 -07:00
Zachary Anderson fef41cfce0
Remove bringup from new_gallery_skia_ios__transition_perf (#125715)
Final step of https://github.com/flutter/flutter/pull/125663
2023-04-28 15:31:19 -07:00
Zachary Anderson 50f83fc277
Add back one Skia test on iOS (#125663)
Fixes https://github.com/flutter/flutter/issues/125648
2023-04-28 10:41:33 -07:00
Taha Tesser 746cacc501
Add slider_perf_android task to ci.yaml (#125540)
This adds `devicelab` task `slider_perf_android` to `.ci.yaml`, which was added in https://github.com/flutter/flutter/pull/125296
2023-04-27 21:32:39 +00:00
Jenn Magder 28e17177fd
Mark module_test_ios tests unflaky (#125498)
[Dart workaround](https://dart-review.googlesource.com/c/sdk/+/297821 ) to prevent the flake has rolled into the framework.  Mark the tests unflaky so we can detect if it happens again when a fix for the underlying issue lands.

https://github.com/flutter/flutter/issues/125425#issuecomment-1522142360

Reverts https://github.com/flutter/flutter/pull/125426
2023-04-27 18:09:25 +00:00
Zachary Anderson 1d30a1fc5a
Remove flavors_test_macos (#125581)
This test has been failing for the past >1000 runs, and is wedging a bot
for 30 minutes as the failure mode is a timeout.

Fixes https://github.com/flutter/flutter/issues/119782
2023-04-26 14:02:49 -07:00
Zachary Anderson 66fa4c5d30
Remove web compile benchmarks that specify an attached device (#125559)
Fixes https://github.com/flutter/flutter/issues/125492
2023-04-26 09:37:56 -07:00
Zachary Anderson 53cba8fd49
Add hostonly web compile size benchmarks (#125518)
First part of https://github.com/flutter/flutter/issues/125492
2023-04-25 19:11:12 -07:00
keyonghan 624bdd38b0
Mark module_test_ios flaky (#125426)
https://github.com/flutter/flutter/issues/125425
2023-04-24 18:46:50 +00:00
Jenn Magder 9e2e64c7c0
Do not run windows_home_scroll_perf__timeline_summary benchmark in presubmit (#125343)
Introduced in https://github.com/flutter/flutter/pull/99564.  Checked with @jonahwilliams and this should not be running in presubmit.
2023-04-24 18:33:21 +00:00
Jenn Magder baafa1991d
Combine flutter_tool_startup devicelab tests (#125226)
`flutter_tool_startup__windows/linux/macos` were identical. Use the same task for all three.
2023-04-21 22:33:07 +00:00
keyonghan d186792c00
Update xcode to 14c18 for host only bots (#125211)
With https://flutter-review.googlesource.com/c/recipes/+/42160, we are
ready to make all tests consistent with xcode 14c18.

> auto label is removed for flutter/flutter, pr: 125211, due to This PR has not met approval requirements for merging. Changes were requested by {jmagman}, please make the needed changes and resubmit this PR. You have project association MEMBER and need 1 more review(s) in order to merge this PR.
> 
> * Merge guidelines: You need at least one approved review if you are already a MEMBER or two member reviews if you are not a MEMBER before re-applying the autosubmit label. **Reviewers**: If you left a comment approving, please use the "approve" review action instead.

Sounds an auto-submit bug. Manually submit.
2023-04-20 20:50:59 -07:00
Casey Hillers 7d2669b8e7
[ci.yaml] Remove unnecessary use of { (#125058)
In yaml, indented blocks indicate the following fields are children of the parent indent. The use of curly brackets is unnecessary here
2023-04-21 00:32:31 +00:00
Zachary Anderson c31467f18c
Stop running "_impeller_" benchmark variants (#125044)
Impeller is now the default on iOS, so these are redundant.

In a subsequent change, I'll clean up the source files for these.
2023-04-18 09:54:02 -07:00
Casey Hillers b1f8d5df64
[ci.yaml] Disable packaging_test on release candidates (#124712)
[ci.yaml] Disable packaging_test on release candidates
2023-04-12 18:42:23 +00:00
Flutter GitHub Bot 2a5ecb6c48
Marks Linux firebase_abstract_method_smoke_test to be flaky (#124692)
Marks Linux firebase_abstract_method_smoke_test to be flaky
2023-04-12 18:29:07 +00:00
Flutter GitHub Bot d15325407e
Marks Linux_samsung_s10 new_gallery_impeller__transition_perf to be flaky (#124694)
Marks Linux_samsung_s10 new_gallery_impeller__transition_perf to be flaky
2023-04-12 17:52:26 +00:00
Flutter GitHub Bot df48b922ca
Marks Mac_ios animated_blur_backdrop_filter_perf_ios__timeline_summary to be unflaky (#124706)
Marks Mac_ios animated_blur_backdrop_filter_perf_ios__timeline_summary to be unflaky
2023-04-12 16:59:43 +00:00
Flutter GitHub Bot 2dfb03fdac
Marks Linux_android animated_blur_backdrop_filter_perf__timeline_summary to be unflaky (#124705)
Marks Linux_android animated_blur_backdrop_filter_perf__timeline_summary to be unflaky
2023-04-12 16:58:25 +00:00
Flutter GitHub Bot e68086e910
Marks Linux_android new_gallery_impeller__transition_perf to be flaky (#124700)
Marks Linux_android new_gallery_impeller__transition_perf to be flaky
2023-04-12 16:55:44 +00:00
keyonghan cf1891218d
Mark android_semantics_integration_test as flaky (#124638)
Mark android_semantics_integration_test as flaky
2023-04-11 23:49:05 +00:00
godofredoc be2aa1c268
Set multiple tests as non flaky. (#124604)
Set multiple tests as non flaky.
2023-04-11 20:05:21 +00:00