Commit graph

915 commits

Author SHA1 Message Date
godofredoc 3165fa30be
Migrate android_view to linux_android_emu platform. (#142184)
A new top level platform configuration was created for android emulator tests running on linux vms.

This is also a preparation step to remove adhoc recipes created to run emulator tests.
2024-01-25 01:15:12 +00:00
auto-submit[bot] 54e9f2dbe6
Reverts "Refactor external_uiexternal_textures" (#142173)
Reverts flutter/flutter#142062
Initiated by: eliasyishak
This change reverts the following previous change:
Original Description:
This PR makes no _behavioral_ changes to executed code, and instead focuses on organization and naming:

1. Almost[^1] anything named `external_ui` is renamed `external_textures`
1. Extended the README to explain the intent of the test, as well as how to run it
1. Renamed `main.dart` and `main_test.dart` to `frame_rate_main.dart` and `frame_rate_test.dart` (we'll add more)
1. Did some refactoring of the test to make it more obvious what is being asserted (i.e. `widgetBuilds` and friends)

Given how complex (and in-flux) this directory is, I'm also requesting either John, Jonah or I review any changes.

[^1]: Except the name of the `.ci.yaml` task, i.e. `name: Linux_pixel_7pro external_ui_integration_test` because I'm apparently not able to change that without creating a new task as `bringup: true` and playing a bit of a dance. Maybe that's worth doing though (in future PRs)?
2024-01-24 21:41:17 +00:00
Matan Lurey 2e2042ff8e
Refactor external_uiexternal_textures (#142062)
This PR makes no _behavioral_ changes to executed code, and instead
focuses on organization and naming:

1. Almost[^1] anything named `external_ui` is renamed
`external_textures`
1. Extended the README to explain the intent of the test, as well as how
to run it
1. Renamed `main.dart` and `main_test.dart` to `frame_rate_main.dart`
and `frame_rate_test.dart` (we'll add more)
1. Did some refactoring of the test to make it more obvious what is
being asserted (i.e. `widgetBuilds` and friends)

Given how complex (and in-flux) this directory is, I'm also requesting
either John, Jonah or I review any changes.

[^1]: Except the name of the `.ci.yaml` task, i.e. `name:
Linux_pixel_7pro external_ui_integration_test` because I'm apparently
not able to change that without creating a new task as `bringup: true`
and playing a bit of a dance. Maybe that's worth doing though (in future
PRs)?
2024-01-24 12:44:31 -08:00
Flutter GitHub Bot a0df2c4573
Marks Mac_arm64 build_tests_2_4 to be unflaky (#142115)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_arm64 build_tests_2_4"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_arm64%20build_tests_2_4%22).
This test can be marked as unflaky.
2024-01-24 18:11:53 +00:00
Flutter GitHub Bot 07ece1aad2
Marks Mac_x64 build_tests_2_4 to be unflaky (#142111)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_x64 build_tests_2_4"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_x64%20build_tests_2_4%22).
This test can be marked as unflaky.
2024-01-24 17:56:38 +00:00
Flutter GitHub Bot fdda3b5ac0
Marks Mac_x64 build_tests_1_4 to be unflaky (#142110)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_x64 build_tests_1_4"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_x64%20build_tests_1_4%22).
This test can be marked as unflaky.
2024-01-24 17:56:35 +00:00
Flutter GitHub Bot a0e0387a80
Marks Mac_x64 build_tests_3_4 to be unflaky (#142112)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_x64 build_tests_3_4"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_x64%20build_tests_3_4%22).
This test can be marked as unflaky.
2024-01-24 17:55:43 +00:00
godofredoc 5d48b5c8b5
Run some tests explicitly in both arm and x64. (#141910)
This is part of the process to start running tests on either arm64 or x64 if the host architecture is not important for the test.
2024-01-24 17:24:12 +00:00
Flutter GitHub Bot 161fd82c7f
Marks Mac_arm64 build_tests_1_4 to be unflaky (#142114)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_arm64 build_tests_1_4"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_arm64%20build_tests_1_4%22).
This test can be marked as unflaky.
2024-01-24 15:50:11 +00:00
Flutter GitHub Bot 277366d2cc
Marks Mac_arm64 build_tests_3_4 to be unflaky (#142116)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_arm64 build_tests_3_4"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_arm64%20build_tests_3_4%22).
This test can be marked as unflaky.
2024-01-24 15:39:25 +00:00
Flutter GitHub Bot db0ce8a498
Marks Mac_x64 build_tests_4_4 to be unflaky (#142113)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_x64 build_tests_4_4"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_x64%20build_tests_4_4%22).
This test can be marked as unflaky.
2024-01-24 15:09:03 +00:00
Flutter GitHub Bot e7ff0e0e8f
Marks Mac_arm64 build_tests_4_4 to be unflaky (#142117)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_arm64 build_tests_4_4"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_arm64%20build_tests_4_4%22).
This test can be marked as unflaky.
2024-01-24 14:59:41 +00:00
godofredoc 6f7aed593d
Run module_test_ios on arm and x64 (#141815)
This will ensure the test runs on the supported platforms.
2024-01-19 22:05:12 +00:00
godofredoc c95bb6fbb3
Add mac_x64_ios configuration. (#141828)
This is in preparation to allow tests to run on x64 and arm64 if they rely only on the ios phone.
2024-01-19 03:42:23 +00:00
godofredoc 9d9bd884ea
Run framework_tests_misc in arm64 and x64. (#141797)
This is to ensure coverage in both platforms.
2024-01-18 22:40:16 +00:00
Victoria Ashworth c92b20ba97
Run flutter_gallery_ios__start_up test on Mac-14 in staging (#141795)
Attempting to debug https://github.com/flutter/flutter/issues/141383.
2024-01-18 20:05:10 +00:00
Zachary Anderson a9d94be0da
Move mac pixel 7 pro test to presubmit: false (#141747)
The only bot with this configuration in the try pool just died.
2024-01-17 19:54:51 -08:00
Polina Cherkasova b90a3e4710
Remove non-needed bot and increase time out for leak_tracking. (#141712) 2024-01-17 15:31:38 -08:00
godofredoc 66f5e88e6b
Run build tests on both x64 and arm64. (#141206)
Most of the builds for mac_x64 can actually run on any available architecture. This is required to use capacity more efficiently.
2024-01-17 21:20:34 +00:00
Victoria Ashworth 8de818dd2a
Update tests to Xcode 15 (#141706)
Framework part of https://github.com/flutter/flutter/issues/132237.
2024-01-17 20:32:12 +00:00
Flutter GitHub Bot 8286f4993d
Marks Windows framework_tests_misc_leak_tracking to be unflaky (#141676)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows framework_tests_misc_leak_tracking"
}
-->
The issue https://github.com/flutter/flutter/issues/140414 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows%20framework_tests_misc_leak_tracking%22).
This test can be marked as unflaky.
2024-01-17 17:37:36 +00:00
godofredoc b9aba43b44
Add contexts to mac_ios targets. (#141494)
This is required to remove explicit contexts from recipes.
2024-01-16 19:45:04 +00:00
Qun Cheng 7dc856ab84
Revert "Reverts "Run iOS staging tests with Xcode 15.2"" (#141420)
Relands "Run iOS staging tests with Xcode 15.2""

Reverts flutter/flutter#141412
2024-01-12 16:11:22 +00:00
godofredoc e20e95100d
Add osx_sdk context for mac builds. (#141422)
This is to give ci.yaml the flexibility to select the contexts to execute and remove complexity from recipes.
2024-01-12 03:00:06 +00:00
auto-submit[bot] 66759d7522
Reverts "Run iOS staging tests with Xcode 15.2" (#141412)
Reverts flutter/flutter#141392
Initiated by: QuncCccccc
This change reverts the following previous change:
Original Description:
Potential fix for https://github.com/flutter/flutter/issues/141383.
2024-01-11 22:06:25 +00:00
Victoria Ashworth ef3de359de
Run iOS staging tests with Xcode 15.2 (#141392)
Potential fix for https://github.com/flutter/flutter/issues/141383.
2024-01-11 19:17:38 +00:00
Jonah Williams 35903620c8
Add impeller key to skia gold client, Turn on a framework test shard that will run unit tests with --enable-impeller (#141341)
Redo of https://github.com/flutter/flutter/pull/140985 due to CLA issues.
2024-01-11 17:57:00 +00:00
Polina Cherkasova 988d1a0679
Add environment variable to leak tracking bots. (#141137) 2024-01-09 14:14:53 -08:00
Casey Hillers 51af82d3db
[ci.yaml] Do not run packaging test on presubmit (#141192)
This test doesn't support presubmit as runs a separate git clone, which won't contain the presubmit hash.

Fixes https://github.com/flutter/flutter/issues/141188
2024-01-09 20:02:15 +00:00
Victoria Ashworth 1ec9daa9c2
Run tests on iOS 16 or iOS 17 (#141178)
Fixes https://github.com/flutter/flutter/issues/141177
2024-01-09 19:49:21 +00:00
Zachary Anderson e94a2c04c1
Shift mac Pixel 7 Pro tests to prod (#141142)
Related https://github.com/flutter/flutter/issues/140893
2024-01-08 18:33:47 -08:00
Zachary Anderson f111459632
Shift some macOS Android tests from Moto G4 to Pixel 7 Pro (#140912)
Related https://github.com/flutter/flutter/issues/140893
2024-01-05 08:20:49 -08:00
Polina Cherkasova b2f53e0cde
Temporarily remove env variable for leak tracking bots. (#140978)
For some reasons there are [failures](https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8759809744202981089/+/u/run_test.dart_for_framework_tests_shard_and_subshard_misc/test_stdout) in leak tracking bots, that do not relate to leak tracking. 

There are two differences between leak tracking bots and 'normal' windows framework bots: 

1. The flag "LEAK_TRACKING": "true". First failures happened before consumption of the flag "LEAK_TRACKING": "true" by testWidgets. Hypothesis: the failures may be caused by mechanisms that sets the env variable. This mechanism is used only by leak tracking bots.
2. bringup: true. 

This PR removes the leak tracking env variable to isolate the first difference and verify if the bots will start passing.
2024-01-04 19:32:17 +00:00
Victoria Ashworth c9f23d056c
Run half of iOS devicelab tests with Xcode 15 (#140927)
Since 50% of devicelab bots are on macOS 13, we can upgrade 50% of devicelab tests to Xcode 15.

There are 52 Mac_ios non-bringup tests, this PR updates 26 of them.

There are 4 Mac_arm64_ios non-bringup tests, this PR updates 2 of them.

Part of https://github.com/flutter/flutter/issues/140975
2024-01-04 18:11:07 +00:00
keyonghan fedf9ece4c
Add Windows leak tracking targets (#140423)
https://github.com/flutter/flutter/issues/140414
2024-01-03 23:09:02 +00:00
Flutter GitHub Bot bfcf0c5893
Marks Linux_pixel_7pro native_assets_android to be unflaky (#140866)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_pixel_7pro native_assets_android"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_pixel_7pro%20native_assets_android%22).
This test can be marked as unflaky.
2024-01-03 18:25:15 +00:00
Flutter GitHub Bot 70386a613b
Marks Linux_android hybrid_android_views_integration_test to be unflaky (#140865)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_android hybrid_android_views_integration_test"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_android%20hybrid_android_views_integration_test%22).
This test can be marked as unflaky.
2024-01-03 13:35:20 +00:00
Flutter GitHub Bot 30c2346806
Marks Linux_pixel_7pro integration_ui_keyboard_resize to be flaky (#140646)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_pixel_7pro integration_ui_keyboard_resize"
}
-->
Issue link: https://github.com/flutter/flutter/issues/140645
2023-12-27 20:25:09 +00:00
godofredoc c73bffe747
Migrate verify_codesigned. (#139328)
This is part of the migration of adhoc tests to shard tests.

Bug: https://github.com/flutter/flutter/issues/139153
2023-12-22 01:46:11 +00:00
Victoria Ashworth 2da1ff2d93
Run mac staging tests with Xcode 15 (#140518)
Run macOS staging tests with Xcode 15 and macOS 13. Also, cleanup some comments.
2023-12-21 22:01:14 +00:00
Andrew Kolos 9e104eb7c8
Fix flavor conditional asset bundling for macos (#140433)
Fixes https://github.com/flutter/flutter/issues/140430
Fixes https://github.com/flutter/flutter/issues/140432 while we are at it
2023-12-21 20:30:21 +00:00
Gray Mackall a3e0fa56a3
Enable android_preview_tool_integration_tests check (#140470)
Enables the check that was added in https://github.com/flutter/flutter/pull/131901.

Has been passing since https://github.com/flutter/flutter/pull/140099 (30 runs). Not sure what the normal number of successful runs we wait for is before enabling, let me know if we should wait for more data.
2023-12-21 19:20:43 +00:00
Zachary Anderson 76c8e7387b
Move tests shifted to Pixel 7 from staging to prod (#140438) 2023-12-20 18:57:40 +00:00
Zachary Anderson ea6017ded8
Move hybrid_android_views_integration_test back to Moto G4 (#140421)
Related https://github.com/flutter/flutter/issues/140420
2023-12-19 23:44:10 +00:00
Zachary Anderson 0eb7881340
Shift some Linux-hosted tests on moto g4 to pixel 7 (#140389) 2023-12-19 09:30:03 -08:00
Andrew Kolos f8b9748661
Restore and fix flavors_test_macos (#139841)
Partially resolves[^1] https://github.com/flutter/flutter/issues/139774.

Effectively reverts https://github.com/flutter/flutter/pull/125581.

The main change here is that I deleted and recreated the macos Xcode project for this integration test (hence the large diff). I tried fixing the existing project first, but it was set up quite differently, and—for whatever reason—the integration test would get stuck trying to load `dev/integration_tests/flavors/integration_test/integration_test.dart`.

I verified that this works locally, but I don't know if it's possible to run this on the devicelab try pool to verify that it works on devicelab hardware.

[^1]: I would not close the issue until 1) this PR lands, 2) the integration test consistently passes on CI, and 3) macOS support for flavors is publicly documented.
2023-12-18 08:52:22 +00:00
Daco Harkes 3b370c8539
[deps] update Android SDK to 34 (#138183)
Update Android SDK from 33 to 34.

Following the engine update:

* https://github.com/flutter/engine/pull/47839
2023-12-14 00:55:57 +00:00
Victoria Ashworth 0c280d4788
Unpin mac_toolchain version (#139938)
mac_toolchain has been updated to no longer have 2 minute slowdown (https://github.com/flutter/flutter/issues/138109), so unpin to use the latest version.
2023-12-14 00:11:32 +00:00
Gray Mackall 9a72e1c699
Allow plugins to use compileSdkPreview (#131901)
Fixes https://github.com/flutter/flutter/issues/124748

Based (heavily) off https://github.com/flutter/flutter/pull/104662
2023-12-12 20:06:04 +00:00
Casey Hillers 815dc96e9b
[ci.yaml] Add missing ci.yaml to runIf of android hot reload tests (#139932)
https://github.com/flutter/cocoon/pull/3333
2023-12-11 23:55:52 +00:00