Commit Graph

863 Commits

Author SHA1 Message Date
Christopher Fujino
e5c5e15a5b
re-enable windows build_tests (#139599)
These have all passed since being introduced >20 commits ago:

![Screenshot from 2023-12-05 14-37-41](https://github.com/flutter/flutter/assets/7856010/997f39a1-377c-4e85-a7a1-02bb7e687506)
2023-12-06 22:46:34 +00:00
Flutter GitHub Bot
32bddf2fe9
Marks Linux_pixel_7pro new_gallery_impeller__transition_perf to be flaky (#139644)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_pixel_7pro new_gallery_impeller__transition_perf"
}
-->
Issue link: https://github.com/flutter/flutter/issues/139643
2023-12-06 10:17:15 -08:00
godofredoc
ff881badb7
Move docs to use their own recipe. (#139521)
This is part of the deprecation of adhoc tests.

Bug: https://github.com/flutter/flutter/issues/139153
2023-12-05 17:37:56 +00:00
Christopher Fujino
79d6781b4f
increase sharding of windows build tests (#139503)
Fixes https://github.com/flutter/flutter/issues/139502
2023-12-04 23:45:07 +00:00
Jonah Williams
c421f65b3b
Add Impeller complex layout impeller benchmarks. (#139490)
These benchmarks do a nice job of showing CPU overhead, which is useful for many of the microoptimizations we've been doing.
2023-12-04 19:23:26 +00:00
godofredoc
8412adb981
Migrate docs_test to shard. (#139282)
This is part of the deprecation of adhoc tests.

Bug: https://github.com/flutter/flutter/issues/139153
2023-11-30 21:29:41 +00:00
Zachary Anderson
918e336014
Move Impeller tests on Pixel 7 Pro from staging to prod (#139280)
These tests are now expected to be reasonably stable, and increases in
flakiness should be reported as part of the regular flaky test triage
process.
2023-11-30 06:47:12 -08:00
godofredoc
cae47c4641
Use the correct recipe on fuchsia_precache. (#139279)
To run shard tests we need to use flutter_drone recipe.


## Pre-launch Checklist

- [X] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [X] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [X] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [X] I signed the [CLA].
- [X] I listed at least one issue that this PR fixes in the description
above.
- [X] I updated/added relevant documentation (doc comments with `///`).
- [X] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [X] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2023-11-29 18:19:30 -08:00
godofredoc
c8101b56a9
Migrate fuchsia_precache to shard tests. (#139202)
Adhoc tests are being deprecated and existing tests are being migrated to shard tests.

Bug: https://github.com/flutter/flutter/issues/139153
2023-11-30 00:03:23 +00:00
godofredoc
8b6277e638
Move analysis test to shard tests. (#139161)
This is part of the effort to deprecated adhoc tests.

Bug: https://github.com/flutter/flutter/issues/139153
2023-11-28 23:58:55 +00:00
Victoria Ashworth
358e39137f
Renable macOS 13 tests (#139083)
A workaround was added for https://github.com/flutter/flutter/issues/138238, so let's re-enable macOS 13 tests
2023-11-28 16:13:08 +00:00
Zachary Anderson
e49cc81222
Run platform_channels_benchmarks on Pixel 7 Pro (#139092)
Related to https://github.com/flutter/flutter/issues/138689 for 64-bit
coverage for Dart VM changes in this area

cc @aam
2023-11-27 19:43:09 -08:00
godofredoc
fe7299d318
Migrate customer_testing to sharded tests. (#138659)
This will allow Dart Team to run customer tests as part of monorepo and will be a step forward to remove ad_hoc tests.

Bug: https://github.com/dart-lang/sdk/issues/51042
Bug: https://github.com/flutter/flutter/issues/115476
2023-11-28 00:41:53 +00:00
Flutter GitHub Bot
29c95cda78
Marks Mac_build_test flutter_gallery__transition_perf_e2e_ios to be unflaky (#138886)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_build_test flutter_gallery__transition_perf_e2e_ios"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_build_test%20flutter_gallery__transition_perf_e2e_ios%22).
This test can be marked as unflaky.
2023-11-22 15:43:15 +00:00
Flutter GitHub Bot
defa2b7443
Marks Mac_android hot_mode_dev_cycle__benchmark to be unflaky (#138472)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_android hot_mode_dev_cycle__benchmark"
}
-->
The issue https://github.com/flutter/flutter/issues/138072 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Mac_android%20hot_mode_dev_cycle__benchmark%22).
This test can be marked as unflaky.
2023-11-15 11:32:08 +00:00
Victoria Ashworth
c0ca261e07
Only run tests on macOS 12 (#138260)
There's an https://github.com/flutter/flutter/issues/138238 with mac_toolchain that makes Xcode installs flakey and an https://github.com/flutter/flutter/issues/138246 that makes Xcode installs more frequent on macOS 13, which is causing presubmit tests to fall frequently. In the meantime, we'll only have tests run on macOS 12.
2023-11-10 23:53:23 +00:00
Victoria Ashworth
4eda3b96c5
Move Mac_build_test flutter_gallery__transition_perf_e2e_ios to staging (#138196)
Seems that bots recently upgraded to macOS 13 are missing certs (https://github.com/flutter/flutter/issues/138194). Temporarily move `Mac_build_test flutter_gallery__transition_perf_e2e_ios` to staging.
2023-11-10 00:12:06 +00:00
Victoria Ashworth
46dcb32c23
Use specific version of mac_toolchain (#138115)
Newer versions of mac_toolchain increase Xcode install times by ~2 minutes (https://github.com/flutter/flutter/issues/138109). As a temporary solution, we're using an older version of mac_toolchain.
2023-11-08 23:37:54 +00:00
Flutter GitHub Bot
35ca279ce9
Marks Mac_android hot_mode_dev_cycle__benchmark to be flaky (#138073)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_android hot_mode_dev_cycle__benchmark"
}
-->
Issue link: https://github.com/flutter/flutter/issues/138072
2023-11-08 20:24:04 +00:00
Victoria Ashworth
076846c7fe
Run a couple of iOS tests in presubmit (#138089)
Run a couple of iOS tests in presubmit to validate macOS 13 bots in the try pool are working successfully and consistently.
2023-11-08 17:57:15 +00:00
Zachary Anderson
2e50a54128
Move Skia new_gallery_transition_perf on a02 from staging to prod (#138013)
cc @yusuf-goog
2023-11-07 19:07:00 +00:00
godofredoc
90c73055b2
Migrate api docs from master to main. (#137726)
This is the first of a series of changes to migrate to remove master-docs-flutter-dev GCP project.

Bug: https://github.com/flutter/flutter/issues/133877
2023-11-02 18:56:47 +00:00
Flutter GitHub Bot
87c8e2417a
Marks Windows build_tests_1_6 to be unflaky (#137673)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows build_tests_1_6"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows%20build_tests_1_6%22).
This test can be marked as unflaky.
2023-11-02 15:53:53 +00:00
Flutter GitHub Bot
41e9235181
Marks Windows build_tests_4_6 to be unflaky (#137674)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows build_tests_4_6"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows%20build_tests_4_6%22).
This test can be marked as unflaky.
2023-11-02 15:03:55 +00:00
yusuf-goog
c617937009
Bumping version tag of provisioning profile cipd package. (#137710)
Bug:https://github.com/flutter/flutter/issues/137166

*Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.*

*List which issues are fixed by this PR. You must list at least one issue. An issue is not required if the PR fixes something trivial like a typo.*

*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
2023-11-01 19:44:02 +00:00
Kate Lovett
fbc145167a
Bump goldctl in .ci.yaml (#137441)
Related to https://github.com/flutter/flutter/issues/137177
2023-10-27 19:27:09 +00:00
Victoria Ashworth
e75f85bffd
Run tests on either macOS 12 or 13 (#137365)
In preparation for migrating our fleet to macOS 13, we're updating tests to run on either macOS 12 or macOS 13.

I have run all tests on macOS 13: https://docs.google.com/spreadsheets/d/1-KOSOSF7uVA4KuqSMn9xz3IIJW78u8SXLYK2OAfe_Qg/edit?usp=sharing&resourcekey=0-df8Bj5PRS1IPBccPDdvRCQ
2023-10-27 15:19:05 +00:00
Flutter GitHub Bot
f13196e3f6
Marks Windows_arm64 windows_startup_test to be unflaky (#137228)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_arm64 windows_startup_test"
}
-->
The issue https://github.com/flutter/flutter/issues/134083 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_arm64%20windows_startup_test%22).
This test can be marked as unflaky.
2023-10-26 22:35:22 +00:00
Flutter GitHub Bot
4a97a57831
Marks Windows_arm64 flutter_tool_startup__windows to be unflaky (#137229)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_arm64 flutter_tool_startup__windows"
}
-->
The issue https://github.com/flutter/flutter/issues/134083 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_arm64%20flutter_tool_startup__windows%22).
This test can be marked as unflaky.
2023-10-26 22:09:12 +00:00
Flutter GitHub Bot
e262926881
Marks Windows_arm64 windows_home_scroll_perf__timeline_summary to be unflaky (#137221)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_arm64 windows_home_scroll_perf__timeline_summary"
}
-->
The issue https://github.com/flutter/flutter/issues/134083 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_arm64%20windows_home_scroll_perf__timeline_summary%22).
This test can be marked as unflaky.
2023-10-26 22:05:50 +00:00
Flutter GitHub Bot
6f093b1db8
Marks Windows_arm64 complex_layout_win_desktop__start_up to be unflaky (#137225)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_arm64 complex_layout_win_desktop__start_up"
}
-->
The issue https://github.com/flutter/flutter/issues/134083 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_arm64%20complex_layout_win_desktop__start_up%22).
This test can be marked as unflaky.
2023-10-26 22:05:44 +00:00
Flutter GitHub Bot
6f3c3a3f2a
Marks Windows_arm64 hot_mode_dev_cycle_win_target__benchmark to be unflaky (#137217)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_arm64 hot_mode_dev_cycle_win_target__benchmark"
}
-->
The issue https://github.com/flutter/flutter/issues/134083 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_arm64%20hot_mode_dev_cycle_win_target__benchmark%22).
This test can be marked as unflaky.
2023-10-26 21:34:06 +00:00
Flutter GitHub Bot
b6cde8f720
Marks Windows_arm64 flutter_view_win_desktop__start_up to be unflaky (#137226)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_arm64 flutter_view_win_desktop__start_up"
}
-->
The issue https://github.com/flutter/flutter/issues/134083 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_arm64%20flutter_view_win_desktop__start_up%22).
This test can be marked as unflaky.
2023-10-26 21:34:03 +00:00
Flutter GitHub Bot
dfd003abb0
Marks Windows_arm64 platform_view_win_desktop__start_up to be unflaky (#137227)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_arm64 platform_view_win_desktop__start_up"
}
-->
The issue https://github.com/flutter/flutter/issues/134083 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_arm64%20platform_view_win_desktop__start_up%22).
This test can be marked as unflaky.
2023-10-26 21:34:00 +00:00
Flutter GitHub Bot
71bc3b02f2
Marks Windows_arm64 flutter_gallery_win_desktop__start_up to be unflaky (#137224)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_arm64 flutter_gallery_win_desktop__start_up"
}
-->
The issue https://github.com/flutter/flutter/issues/134083 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_arm64%20flutter_gallery_win_desktop__start_up%22).
This test can be marked as unflaky.
2023-10-26 21:11:52 +00:00
Flutter GitHub Bot
dc16104b35
Marks Windows_arm64 hello_world_win_desktop__compile to be unflaky (#137222)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_arm64 hello_world_win_desktop__compile"
}
-->
The issue https://github.com/flutter/flutter/issues/134083 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_arm64%20hello_world_win_desktop__compile%22).
This test can be marked as unflaky.
2023-10-26 21:08:01 +00:00
Flutter GitHub Bot
3836988980
Marks Windows_arm64 flutter_gallery_win_desktop__compile to be unflaky (#137223)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_arm64 flutter_gallery_win_desktop__compile"
}
-->
The issue https://github.com/flutter/flutter/issues/134083 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_arm64%20flutter_gallery_win_desktop__compile%22).
This test can be marked as unflaky.
2023-10-26 21:07:59 +00:00
Flutter GitHub Bot
fdd38d30d7
Marks Windows_arm64 run_release_test_windows to be unflaky (#137220)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_arm64 run_release_test_windows"
}
-->
The issue https://github.com/flutter/flutter/issues/134083 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_arm64%20run_release_test_windows%22).
This test can be marked as unflaky.
2023-10-26 21:06:19 +00:00
Flutter GitHub Bot
d737038f39
Marks Windows_arm64 platform_channel_sample_test_windows to be unflaky (#137218)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_arm64 platform_channel_sample_test_windows"
}
-->
The issue https://github.com/flutter/flutter/issues/134083 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_arm64%20platform_channel_sample_test_windows%22).
This test can be marked as unflaky.
2023-10-26 20:55:14 +00:00
Flutter GitHub Bot
17797ba50b
Marks Windows_arm64 run_debug_test_windows to be unflaky (#137219)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_arm64 run_debug_test_windows"
}
-->
The issue https://github.com/flutter/flutter/issues/134083 has been closed, and the test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows_arm64%20run_debug_test_windows%22).
This test can be marked as unflaky.
2023-10-26 18:15:56 +00:00
Victoria Ashworth
341af51391
Run test with older version of Xcode (#137293)
Xcode Debug tests require an older version of Xcode to run in CI.

New test is failing due to incompatible Xcode version: https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_arm64_ios%20flavors_test_ios_xcode_debug/1/overview
2023-10-26 14:42:17 +00:00
Victoria Ashworth
5dd2a4e0aa
Ensure Xcode project is setup to start debugger (#136977)
Some users have their Xcode settings set to not debug (see example here https://github.com/flutter/flutter/issues/136197#issuecomment-1766834195). This will cause the [engine check for a debugger](22ce5c6a45/runtime/ptrace_check.cc (L56-L71)) to fail, which will cause an error and cause the app to crash.

This PR parses the scheme file to ensure the scheme is set to start a debugger and warn the user if it's not.

Fixes https://github.com/flutter/flutter/issues/136197.
2023-10-25 17:08:57 +00:00
Flutter GitHub Bot
9366170e84
Marks Windows build_tests_6_6 to be unflaky (#137216)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows build_tests_6_6"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows%20build_tests_6_6%22).
This test can be marked as unflaky.
2023-10-25 17:07:19 +00:00
Flutter GitHub Bot
cd2c08ee51
Marks Windows build_tests_3_6 to be unflaky (#137214)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows build_tests_3_6"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows%20build_tests_3_6%22).
This test can be marked as unflaky.
2023-10-25 14:49:32 +00:00
Flutter GitHub Bot
7db948cfdb
Marks Windows build_tests_5_6 to be unflaky (#137215)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows build_tests_5_6"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows%20build_tests_5_6%22).
This test can be marked as unflaky.
2023-10-25 14:40:12 +00:00
Flutter GitHub Bot
1001a0a3d0
Marks Windows build_tests_2_6 to be unflaky (#137213)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows build_tests_2_6"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Windows%20build_tests_2_6%22).
This test can be marked as unflaky.
2023-10-25 14:37:57 +00:00
Jackson Gardner
4afdfca264
Remove bringup: true from realm_checker and remove the redundant tool test. (#137186)
Now that the realm checker is no longer in bringup, we can remove the tool test that checks for the empty realm file.
2023-10-25 01:09:18 +00:00
Jackson Gardner
cdd1ae994d
Check the realm file in its own shard. (#137160)
This way, it is easier to differentiate between a real failure in the tools tests vs. just the realm check that blocks the merge.
2023-10-24 18:35:13 +00:00
godofredoc
69d0ca60ef
Migrate mac builds to ruby dep. (#136929)
The ruby dependency has been automated to include cocoapods.

Bug: https://github.com/flutter/flutter/issues/136377
2023-10-24 17:10:02 +00:00
Casey Hillers
347ae937cf
[ci_yaml] Remove back_fill=false on roller (#137076)
Follow up from https://flutter-review.googlesource.com/c/50200 which removed the issue with inconsistencies

https://github.com/flutter/flutter/issues/112127
2023-10-23 20:39:01 +00:00