Commit graph

967 commits

Author SHA1 Message Date
Victoria Ashworth d1d9605974
Remove force Xcode debug workflow (#144185)
Now that all tests are on Xcode 15 and iOS 17, we no longer need to force test Xcode debug workflow.

Related https://github.com/flutter/flutter/issues/144020.
2024-02-27 18:18:07 +00:00
keyonghan d319625329
Mark two other firebase targets as bringup: true (#144234)
https://github.com/flutter/flutter/issues/144183
2024-02-27 17:51:24 +00:00
Jonah Williams b2a3075fa5
Re-enable Impeller goldens blocking. (#144210)
The issues in flutter/flutter#144115 should be fixed with the removal of the incorrect render pass caching behavior in https://github.com/flutter/engine/pull/50976
2024-02-27 16:20:19 +00:00
keyonghan c82ca469a3
Mark firebase_release_smoke_test as bringup: true (#144186)
https://github.com/flutter/flutter/issues/144183
2024-02-26 15:47:17 -08:00
Jason Simmons f0e92a2bdc
Revert "Re-enable Impeller goldens blocking. (#144023)" (#144115)
On the current head some tests (such as test/widgets/shadow_test.dart) are segfaulting in the Impeller Vulkan back end.

This reverts commit 39585e66c1.

See https://github.com/flutter/flutter/issues/144116
2024-02-26 01:50:33 +00:00
Jonah Williams 39585e66c1
Re-enable Impeller goldens blocking. (#144023)
Part of https://github.com/flutter/flutter/issues/143616

Without this these tests only run on postsubmit which means we can't get gold results on PRs.

This was reverted because it landed after an engine change that broke flutter tester. That change has since been reverted.
2024-02-23 18:30:08 +00:00
Victoria Ashworth bb856079c4
Run tests on iOS 17 exclusively (#144022)
Almost all of our devicelab devices are on iOS 17 now, so let's use them exclusively so we can verify that tests pass consistently on iOS 17.

Fixes https://github.com/flutter/flutter/issues/144020.
2024-02-23 17:15:16 +00:00
keyonghan 91a6ec9a06
Remove unused drone_dimension field (#143984)
The drone_dimensions is expected to be defined under property only. The field level definition has never been used.

part of https://github.com/flutter/flutter/issues/143671
2024-02-23 00:13:51 +00:00
Zachary Anderson d13bc10aca
Shift Moto G4 staging tests to A02s (#143957)
https://github.com/flutter/flutter/issues/143915

cc @yusuf-goog
2024-02-22 11:26:09 -08:00
auto-submit[bot] c3e786f3f3
Reverts "[Impeller] Make impeller goldens test blocking. (#143864)" (#143896)
Reverts flutter/flutter#143864

Initiated by: goderbauer

Reason for reverting: The test is consistently failing in post-submit, e.g. https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8755443299639610049/+/u/run_test.dart_for_framework_tests_shard_and_subshard_impeller/stdout

Original PR Author: jonahwilliams

Reviewed By: {matanlurey}

This change reverts the following previous change:
Original Description:
Part of https://github.com/flutter/flutter/issues/143616

Without this these tests only run on postsubmit which means we can't get gold results on PRs.
2024-02-21 23:23:22 +00:00
Jonah Williams 0be6668418
[Impeller] Make impeller goldens test blocking. (#143864)
Part of https://github.com/flutter/flutter/issues/143616

Without this these tests only run on postsubmit which means we can't get gold results on PRs.
2024-02-21 21:08:07 +00:00
godofredoc 77c6a86ebf
Remove bringup from win arm64 builds. (#143548)
The Windows arm64 builds were all moved to bringup because new test beds not properly provisioned were added to the production pool.

Those new test beds are now fully validated but still running on staging pool to ensure they are not introducing noise in the prod pools.

Bug: https://github.com/flutter/flutter/issues/143180
2024-02-15 23:34:31 +00:00
Matan Lurey 7082ae1517
Swap the tasks that have been running fine for a while. (#143544)
Closes https://github.com/flutter/flutter/issues/142178.
2024-02-15 22:21:35 +00:00
godofredoc c530276f78
Remove certs dependency. (#143495)
The dependency was added as temporary workaround and was never removed.

Bug: https://github.com/flutter/flutter/issues/143489
Bug: https://github.com/flutter/flutter/issues/143490
2024-02-15 18:23:07 +00:00
godofredoc dde76f4961
Remove certs installation from win_arm builds. (#143487)
The certs dependency was added in 2020 as a workaround for missing certificates in win 10 images. These dependency is not required for windows arm64 builds because they are running on win 11.
2024-02-15 04:39:14 +00:00
Flutter GitHub Bot dc1f9f183f
Marks Linux_pixel_7pro integration_ui_keyboard_resize to be unflaky (#143440)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_pixel_7pro integration_ui_keyboard_resize"
}
-->
The issue https://github.com/flutter/flutter/issues/140645 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_pixel_7pro%20integration_ui_keyboard_resize%22).
This test can be marked as unflaky.
2024-02-14 16:20:09 +00:00
Matan Lurey 56387c0111
Rename external_ui to external_textures. (#143406)
My understanding is I can do this safely because the task is marked `bringup: true`?

Next PR will remove `bringup` and take down the old tasks.

Partial work towards https://github.com/flutter/flutter/issues/142178.
2024-02-13 23:22:23 +00:00
Zachary Anderson 9f832218da
Move Windows arm64 tests to bringup true (#143272)
For https://github.com/flutter/flutter/issues/143271
2024-02-10 21:21:09 +00:00
Victoria Ashworth d60b681ef7
Move Mac_build_test flutter_gallery__transition_perf_e2e_ios out of staging (#142987)
Move test out of staging now that https://github.com/flutter/flutter/issues/142917 is fixed.
2024-02-07 19:24:44 +00:00
Jenn Magder fee1868d89
Run Mac x64 build tests in postsubmit only (#142334)
It shouldn't be very common for x64 example projects to not build if they built on arm64.  We have other tool tests that would probably catch those issues in presubmit.

Instead run them on postsubmit only to free up limited x64 staging resources.

Follow up from #141206
2024-02-07 06:41:25 +00:00
Andrew Kolos bf9aa556d2
Mark Windows_android hot_mode_dev_cycle_win__benchmark as no longer flaky (#143016)
Marked flaky in flutter/flutter#142609

Per the reasons I discuss in [this comment](https://github.com/flutter/flutter/issues/142608#issuecomment-1930868003) on the corresponding issue, I believe this test is probably not actually flaky.

Fixes https://github.com/flutter/flutter/issues/142608
2024-02-07 03:36:02 +00:00
Victoria Ashworth 809a27ea67
Move Mac_build_test flutter_gallery__transition_perf_e2e_ios to staging (#142918)
Temporary solution for https://github.com/flutter/flutter/issues/142917.
2024-02-05 17:29:22 +00:00
Ricardo Amador 3f30dc2e0a
Add a link the different possible Android virtual device configs (#142765)
*Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.*

*List which issues are fixed by this PR. You must list at least one issue. An issue is not required if the PR fixes something trivial like a typo.*

*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
2024-02-02 16:13:40 +00:00
Victoria Ashworth 822ebc48e0
Allow all iOS tests to use either iOS 16 or 17 (#142714)
Undoing https://github.com/flutter/flutter/pull/142323 to allow all iOS tests to use iOS 16 or 17.
2024-02-02 16:02:08 +00:00
Ricardo Amador c13ebf1e78
Use proto name for emulator version and show cipd package version (#142262)
Changes to use the proto.textpb config file as the version of the emulator device and show the cipd dependency version in the dependencies so devs know it can be updated.

*List which issues are fixed by this PR. You must list at least one issue. An issue is not required if the PR fixes something trivial like a typo.*
Fixes https://github.com/flutter/flutter/issues/142261

*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
2024-02-01 23:48:19 +00:00
Flutter GitHub Bot 2c0cf448bd
Marks Linux_android_emu android views to be unflaky (#142590)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_android_emu android views"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_android_emu%20android%20views%22).
This test can be marked as unflaky.
2024-02-01 22:42:55 +00:00
Polina Cherkasova 05cd4fd042
Make leak_tracking bots sticked to the left even if bot thinks they are non-flacky. (#142744) 2024-02-01 21:47:54 +00:00
Flutter GitHub Bot 071830663a
Marks Mac_arm64_ios basic_material_app_ios__compile to be unflaky (#142594)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_arm64_ios basic_material_app_ios__compile"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_arm64_ios%20basic_material_app_ios__compile%22).
This test can be marked as unflaky.
2024-01-31 19:24:05 +00:00
Flutter GitHub Bot 9c107801ed
Marks Mac_arm64 tool_tests_commands to be unflaky (#142593)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_arm64 tool_tests_commands"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_arm64%20tool_tests_commands%22).
This test can be marked as unflaky.
2024-01-31 18:55:25 +00:00
Flutter GitHub Bot 1d48b60d48
Marks Mac_x64 tool_tests_commands to be unflaky (#142592)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_x64 tool_tests_commands"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_x64%20tool_tests_commands%22).
This test can be marked as unflaky.
2024-01-31 18:45:34 +00:00
Flutter GitHub Bot 0e274f5c7b
Marks Mac_x64_ios integration_test_test_ios to be unflaky (#142595)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_x64_ios integration_test_test_ios"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_x64_ios%20integration_test_test_ios%22).
This test can be marked as unflaky.
2024-01-31 18:42:28 +00:00
Flutter GitHub Bot 62fcc15887
Marks Mac_x64 native_ui_tests_macos to be unflaky (#142598)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_x64 native_ui_tests_macos"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_x64%20native_ui_tests_macos%22).
This test can be marked as unflaky.
2024-01-31 18:34:01 +00:00
Flutter GitHub Bot 2dc7c92205
Marks Mac_x64_ios hot_mode_dev_cycle_ios__benchmark to be unflaky (#142597)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_x64_ios hot_mode_dev_cycle_ios__benchmark"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_x64_ios%20hot_mode_dev_cycle_ios__benchmark%22).
This test can be marked as unflaky.
2024-01-31 18:30:47 +00:00
Flutter GitHub Bot 5efcfba66c
Marks Mac_arm64 native_ui_tests_macos to be unflaky (#142599)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_arm64 native_ui_tests_macos"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_arm64%20native_ui_tests_macos%22).
This test can be marked as unflaky.
2024-01-31 18:27:46 +00:00
Flutter GitHub Bot 8629dfa283
Marks Windows_android hot_mode_dev_cycle_win__benchmark to be flaky (#142609)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_android hot_mode_dev_cycle_win__benchmark"
}
-->
Issue link: https://github.com/flutter/flutter/issues/142608
2024-01-31 18:27:44 +00:00
Flutter GitHub Bot 5f2961d2dd
Marks Mac_arm64_ios integration_test_test_ios to be unflaky (#142596)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_arm64_ios integration_test_test_ios"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_arm64_ios%20integration_test_test_ios%22).
This test can be marked as unflaky.
2024-01-31 18:01:22 +00:00
Flutter GitHub Bot 347403f2ea
Marks Linux_android_emu android_defines_test to be unflaky (#142591)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_android_emu android_defines_test"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_android_emu%20android_defines_test%22).
This test can be marked as unflaky.
2024-01-31 17:08:54 +00:00
Flutter GitHub Bot 4af2051f8e
Marks Windows_android native_assets_android to be unflaky (#140042)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_android native_assets_android"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_android%20native_assets_android%22).
This test can be marked as unflaky.
2024-01-31 04:13:30 +00:00
Flutter GitHub Bot 3a442ab4e5
Marks Linux_pixel_7pro complex_layout_scroll_perf_impeller_gles__timeline_summary to be unflaky (#140039)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_pixel_7pro complex_layout_scroll_perf_impeller_gles__timeline_summary"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_pixel_7pro%20complex_layout_scroll_perf_impeller_gles__timeline_summary%22).
This test can be marked as unflaky.
2024-01-31 04:01:29 +00:00
Flutter GitHub Bot 6379378c88
Marks Mac flavors_test_macos to be unflaky (#140647)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac flavors_test_macos"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac%20flavors_test_macos%22).
This test can be marked as unflaky.
2024-01-31 03:08:36 +00:00
Ian Hickson cd4445345c
Be less verbose in CI for customer_testing, and revert old timeout hack (#139611)
This PR does two things. First, it reduces the verbosity in the customer_testing shard. We want to be less verbose because it's nigh on impossible to find anything when there's a failure in the verbose logs. Also, some additional comments are added to make following the breadcrumbs easier for the next person.

Second, it reverts a timeout that had been set to 90 minutes a while ago. It's no longer needed.

Fixes https://github.com/flutter/flutter/issues/120901.
2024-01-30 23:37:23 +00:00
Flutter GitHub Bot bd715036b5
Marks Windows_arm64 plugin_test_windows to be unflaky (#137675)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_arm64 plugin_test_windows"
}
-->
The issue https://github.com/flutter/flutter/issues/134083 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_arm64%20plugin_test_windows%22).
This test can be marked as unflaky.
2024-01-30 18:02:52 +00:00
Flutter GitHub Bot 9207b85de8
Marks Mac_pixel_7pro native_assets_android to be unflaky (#141675)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_pixel_7pro native_assets_android"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_pixel_7pro%20native_assets_android%22).
This test can be marked as unflaky.
2024-01-29 22:16:06 +00:00
Flutter GitHub Bot 1af09dde12
Marks Mac_x64 framework_tests_misc to be unflaky (#142118)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_x64 framework_tests_misc"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_x64%20framework_tests_misc%22).
This test can be marked as unflaky.
2024-01-29 21:02:25 +00:00
Flutter GitHub Bot 259c72a82d
Marks Mac_arm64 framework_tests_misc to be unflaky (#142119)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_arm64 framework_tests_misc"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_arm64%20framework_tests_misc%22).
This test can be marked as unflaky.
2024-01-29 19:53:21 +00:00
Loïc Sharma f727948f6d
[Windows Arm64] Run plugin test post-submit (#141987)
The Flutter try pool does not have any Windows Arm64 machines. Thus, any Windows Arm64 pre-submit tests will fail as there's no machines to run them.

This converts the only Windows Arm64 pre-submit test, the plugin test, to run post-submit.

See: https://github.com/flutter/flutter/issues/141986
2024-01-29 19:10:45 +00:00
Jenn Magder aaa3eba12d
Limit fuchsia_precache in presubmit to engine rolls (#142333)
This doesn't need to run on every PR in presubmit.  Limit to engine rolls to validate the artifacts have been published.

Fixes https://github.com/flutter/flutter/issues/142330
2024-01-26 22:34:20 +00:00
Victoria Ashworth e1785b315b
Only use iOS 17 physical devices in staging tests (#142323)
Don't use iOS 17 devices in pre and post submit tests yet.
2024-01-26 21:51:43 +00:00
Jenn Magder 91f0878fed
Move iOS content validation devicelab test into tool integration test (#142272)
The archiving was running in devicelab because certs are needed to codesign (see #73577).  However now the certs are available in chromium bots.  Move the archiving test into the existing tool integration test, and delete the devicelab variant.

arm64:
https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8757886514651624673/+/u/run_test.dart_for_tool_host_cross_arch_tests_shard_and_subshard_None/test_stdout#L6074_4
x64:
https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8757886514651624689/+/u/run_test.dart_for_tool_host_cross_arch_tests_shard_and_subshard_None/test_stdout#L6389_2

Part of https://github.com/flutter/flutter/issues/142070
2024-01-26 18:04:09 +00:00
Matan Lurey 2f6fdf2650
Start renaming by adding a new bringup: true as an Android emulator. (#142257)
Part 2 of https://github.com/flutter/flutter/issues/142178.

Supersedes https://github.com/flutter/flutter/pull/142078.

Feedback welcome to avoid reverts!
2024-01-26 01:07:08 +00:00