Commit Graph

701 Commits

Author SHA1 Message Date
Brandon DeRosier
b898d1d270
[devicelab] Add platform view scroll benchmarks for Impeller on Android (#129455)
For tracking perf improvements with changes like
https://github.com/flutter/engine/pull/43152.
2023-06-23 18:40:17 -07:00
godofredoc
929468e171
Add a devicelab test to recipes cq. (#129411)
This is to ensure the devicelab_drone recipe is tested on presubmit.

Bug: https://github.com/flutter/flutter/issues/126111
2023-06-23 17:38:38 +00:00
Kate Lovett
309e50173e
Run misc test shard when examples/api/** change (#129433)
Fixes https://github.com/flutter/flutter/issues/129390

Found in https://github.com/flutter/flutter/pull/129381, changing sample code in `examples/api` did not trigger the misc test shard that executes the tests for `examples/api`.
2023-06-23 16:46:04 +00:00
godofredoc
f582c0c573
Add comments to .ci.yaml that were removed from recipes. (#129321)
Firebaselab recipe was cleaned up and comments removed. This is adding the still valid comments to the .ci.yaml file.
2023-06-23 05:10:54 +00:00
gmackall
51f659825e
Unpin path_provider_android (#129205)
Unpins path_provider_android where it is pinned. Follows the same steps as https://github.com/flutter/flutter/pull/128898.

Fixes https://github.com/flutter/flutter/issues/116376
2023-06-22 21:22:49 +00:00
godofredoc
dffef2715e
Move all the firebase lab device configs to .ci.yaml. (#129219)
Bug: https://github.com/flutter/flutter/issues/129211
2023-06-21 01:10:07 +00:00
Jackson Gardner
54e27e3ed5
Update Chrome version for testing (#128447)
Most of the tests are using Chrome 96.2, which is quite old. Update to Chrome 114.
2023-06-08 15:37:06 +00:00
Reid Baker
b4796915b3
i127682 android api 33 test support (#127689)
- try to set emulator to version 33 to see what happens

#127682
2023-05-31 16:38:16 +00:00
godofredoc
7d310e3b1b
Add oriol_33 firebaselab configuration. (#127894)
This allows to run the api 33 as a separate build.

Bug: https://github.com/flutter/flutter/issues/127884
Bug: https://github.com/flutter/flutter/issues/118736
2023-05-31 00:19:58 +00:00
keyonghan
2b4e303b5a
Remove legacy xcode properties/dependencies (#127724)
Now only $flutter/osx_sdk property is being used, and it's safe to remove deprecated entries from ci.yaml.

Part of https://github.com/flutter/flutter/issues/127534
2023-05-30 23:14:02 +00:00
keyonghan
3eecc217cf
Append $flutter/osx sdk property to existing platforms/targets relying on xcode (#127537)
Part of https://github.com/flutter/flutter/issues/127534
2023-05-25 22:47:31 +00:00
keyonghan
784ebdf2a3
Run Mac intel only targets on both intel and arm (#127230)
The slowness issue was resolved with latest xcode: https://github.com/flutter/flutter/issues/119750, and all these targets should resume on both platforms as before. This will help mitigate high queue time on intel bots.

This PR updates these targets:

- Mac plugin_test_ios: https://github.com/flutter/flutter/issues/119764
- Mac build_tests: https://github.com/flutter/flutter/pull/120620
- Mac plugin_test: https://github.com/flutter/flutter/pull/120714
- Mac plugin_test_macos: https://github.com/flutter/flutter/pull/122212
- Mac framework_tests_misc: https://github.com/flutter/flutter/pull/122618
2023-05-19 23:22:25 +00:00
Zachary Anderson
31a16f342e
Roll goldctl to f808dcff91b221ae313e540c09d79696cd08b8de (#127218)
goldctl hasn't been updated in about a year, and an update may be needed to reland https://github.com/flutter/flutter/pull/127110.
2023-05-19 18:52:50 +00:00
Christopher Fujino
344c9c6af2
mark windows build tests non-bringup (#127059)
This is a follow-up to aa0a1a7ae1. I was forced to mark these targets bringup in that PR because the targets were technically new (since each name changed 3 -> 4), so this is marking them non-bringup.
2023-05-18 17:08:21 +00:00
Christopher Fujino
aa0a1a7ae1
shard windows build tests 3 -> 4 (#127057)
Time outs are currently making the tree red.

Fixes https://github.com/flutter/flutter/issues/127004
2023-05-17 15:49:35 -07:00
godofredoc
25a9efe994
Fix drone_dimensions. (#126953)
Use drone_dimensions instead of dimensions for builds running on dart-internal. This is required for those builders to select the correct drone type.
2023-05-16 22:11:21 +00:00
keyonghan
3f01c7e019
Move Mac_build_test flutter_gallery__transition_perf_e2e_ios to prod (#126941)
Target `Mac_build_test flutter_gallery__transition_perf_e2e_ios` was enabled in staging: https://github.com/flutter/flutter/pull/111164, and it has passed more than 50 runs: https://ci.chromium.org/p/flutter/builders/staging/Mac_build_test%20flutter_gallery__transition_perf_e2e_ios?limit=50. 

Manually enabling it in prod and removing the old `Mac_ios flutter_gallery__transition_perf_e2e_ios`.

The `Mac_build_test` one does the same thing as `Mac_ios` one, but separating build and test steps in separate targets.

Context: https://github.com/flutter/flutter/issues/103542
2023-05-16 21:46:06 +00:00
Jonah Williams
41abe998ee
[devicelab] add drawPoints benchmark (#126728)
Add a benchmark that measures the improvements from https://github.com/flutter/engine/pull/41803
2023-05-15 19:14:58 +00:00
Zachary Anderson
674b1ad0ae
Remove obsolete iOS benchmark (#126862)
Fixes https://github.com/flutter/flutter/issues/126861
2023-05-15 19:12:56 +00:00
Reid Baker
159ebf736a
Mark no longer flakey tests as bringup:false (#126281)
This is a result of auditing the ci.yaml tests marked as bringup true and determining which ones were still flakey. 
Each commit markes one tests as non flakey along with a link of proof of the lack of flakes. 

- Mark new_gallery_impeller__transition_perf as not flakey, #124699 closed, not recent flakes
- No failures in 500 runs for Staging_build_linux analyze https://ci.chromium.org/p/flutter/builders/staging/Staging_build_linux%20analyze\?cursor\=id%3E8782771663962116337\&limit\=200
- No flakes in 600 builds https://ci.chromium.org/p/flutter/builders/staging/Linux%20flutter_packaging_test\?cursor\=id%3E8787892377160059793\&limit\=200
- No flakes in 600 builds https://ci.chromium.org/p/flutter/builders/staging/Mac%20flutter_packaging_test\?cursor\=id%3E8787870631931126673\&limit\=200
- No flakes in 600 builds https://ci.chromium.org/p/flutter/builders/staging/Mac_arm64%20flutter_packaging_test\?cursor\=id%3E8787809799161254193\&limit\=200

#124699 is the only closed bug that was linked for a test that is no longer flakey. Closed status helps the case that flakes are no longer an issue.
2023-05-15 18:12:52 +00:00
Zachary Anderson
468512e057
Increase test timeout for analyzer_benchmark to 60 minutes (#126586)
Fixes https://github.com/flutter/flutter/issues/126451
2023-05-12 08:45:14 -07:00
godofredoc
fa117aad28
Add dimensions to release builds. (#126117)
Release builds running in dart internal require the dimensions to be set explicitly at the target level in ci.yaml.

Bug: https://github.com/flutter/flutter/issues/126116
2023-05-11 17:28:08 +00:00
keyonghan
a7a1ddfb5a
Add cpu: x86 dimension and xcode dependency for Mac_build_test targets (#126482)
This is a follow up of https://github.com/flutter/flutter/pull/111164.

My local LED was based on `cpu: x86` dimension and `xcode` dependency. But these are missed in the above PR, which caused test failure: https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_build_test%20flutter_gallery__transition_perf_e2e_ios/19/overview
2023-05-11 16:41:26 +00:00
keyonghan
8ab782d08a
Migrate gallery ios tests to build+test (#111164)
Part of https://github.com/flutter/flutter/issues/103542
2023-05-10 00:50:11 +00:00
Reid Baker
0fdddd67f9
Test AGP 8.0 using java 17 (#125323)
- Update Gradle/AGP version and add namespace plus dependencies. 

https://github.com/flutter/flutter/issues/125181
2023-05-08 15:02:31 +00:00
Drew Roen
9de9a81b80
Bring back the failing build_test's (#126014)
https://github.com/flutter/flutter/issues/125897

The issue has been resolved, so bringing the tests out of bringup
2023-05-05 15:54:43 +00:00
Jenn Magder
4f3c989bd3
Remove complex_layout_*__compile benchmarks (#126023)
Fixes https://github.com/flutter/flutter/issues/126020.
2023-05-03 23:49:01 +00:00
Flutter GitHub Bot
15bd29c8e2
Marks Linux_android slider_perf_android to be unflaky (#125968)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_android slider_perf_android"
}
-->
The test has been passing for [50 consecutive
runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_android%20slider_perf_android%22).
This test can be marked as unflaky.
2023-05-03 09:22:21 -07:00
Chris Yang
9fcb42d0dd
Update xcode to 14e222b (#125716)
Framework part of https://github.com/flutter/flutter/issues/125814
2023-05-02 19:56:17 +00:00
Jenn Magder
cc9a3d9d95
Mark microbenchmarks_ios unflaky (#125332)
This hasn't flaked for > 50 commits which is our threshhold.  Marking as unflaky.
2023-05-02 19:34:40 +00:00
godofredoc
d0069bec94
Mark build_test builds as bringup. (#125898)
They require an additional dimension to be passed as part of the
.ci.yaml configuration.

Bug: https://github.com/flutter/flutter/issues/125897

## Pre-launch Checklist

- [X] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [X] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [X] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [X] I signed the [CLA].
- [X] I listed at least one issue that this PR fixes in the description
above.
- [X] I updated/added relevant documentation (doc comments with `///`).
- [X] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [X] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2023-05-02 12:22:56 -07:00
Zachary Anderson
fef41cfce0
Remove bringup from new_gallery_skia_ios__transition_perf (#125715)
Final step of https://github.com/flutter/flutter/pull/125663
2023-04-28 15:31:19 -07:00
Zachary Anderson
50f83fc277
Add back one Skia test on iOS (#125663)
Fixes https://github.com/flutter/flutter/issues/125648
2023-04-28 10:41:33 -07:00
Taha Tesser
746cacc501
Add slider_perf_android task to ci.yaml (#125540)
This adds `devicelab` task `slider_perf_android` to `.ci.yaml`, which was added in https://github.com/flutter/flutter/pull/125296
2023-04-27 21:32:39 +00:00
Jenn Magder
28e17177fd
Mark module_test_ios tests unflaky (#125498)
[Dart workaround](https://dart-review.googlesource.com/c/sdk/+/297821 ) to prevent the flake has rolled into the framework.  Mark the tests unflaky so we can detect if it happens again when a fix for the underlying issue lands.

https://github.com/flutter/flutter/issues/125425#issuecomment-1522142360

Reverts https://github.com/flutter/flutter/pull/125426
2023-04-27 18:09:25 +00:00
Zachary Anderson
1d30a1fc5a
Remove flavors_test_macos (#125581)
This test has been failing for the past >1000 runs, and is wedging a bot
for 30 minutes as the failure mode is a timeout.

Fixes https://github.com/flutter/flutter/issues/119782
2023-04-26 14:02:49 -07:00
Zachary Anderson
66fa4c5d30
Remove web compile benchmarks that specify an attached device (#125559)
Fixes https://github.com/flutter/flutter/issues/125492
2023-04-26 09:37:56 -07:00
Zachary Anderson
53cba8fd49
Add hostonly web compile size benchmarks (#125518)
First part of https://github.com/flutter/flutter/issues/125492
2023-04-25 19:11:12 -07:00
keyonghan
624bdd38b0
Mark module_test_ios flaky (#125426)
https://github.com/flutter/flutter/issues/125425
2023-04-24 18:46:50 +00:00
Jenn Magder
9e2e64c7c0
Do not run windows_home_scroll_perf__timeline_summary benchmark in presubmit (#125343)
Introduced in https://github.com/flutter/flutter/pull/99564.  Checked with @jonahwilliams and this should not be running in presubmit.
2023-04-24 18:33:21 +00:00
Jenn Magder
baafa1991d
Combine flutter_tool_startup devicelab tests (#125226)
`flutter_tool_startup__windows/linux/macos` were identical. Use the same task for all three.
2023-04-21 22:33:07 +00:00
keyonghan
d186792c00
Update xcode to 14c18 for host only bots (#125211)
With https://flutter-review.googlesource.com/c/recipes/+/42160, we are
ready to make all tests consistent with xcode 14c18.

> auto label is removed for flutter/flutter, pr: 125211, due to This PR has not met approval requirements for merging. Changes were requested by {jmagman}, please make the needed changes and resubmit this PR. You have project association MEMBER and need 1 more review(s) in order to merge this PR.
> 
> * Merge guidelines: You need at least one approved review if you are already a MEMBER or two member reviews if you are not a MEMBER before re-applying the autosubmit label. **Reviewers**: If you left a comment approving, please use the "approve" review action instead.

Sounds an auto-submit bug. Manually submit.
2023-04-20 20:50:59 -07:00
Casey Hillers
7d2669b8e7
[ci.yaml] Remove unnecessary use of { (#125058)
In yaml, indented blocks indicate the following fields are children of the parent indent. The use of curly brackets is unnecessary here
2023-04-21 00:32:31 +00:00
Zachary Anderson
c31467f18c
Stop running "_impeller_" benchmark variants (#125044)
Impeller is now the default on iOS, so these are redundant.

In a subsequent change, I'll clean up the source files for these.
2023-04-18 09:54:02 -07:00
Casey Hillers
b1f8d5df64
[ci.yaml] Disable packaging_test on release candidates (#124712)
[ci.yaml] Disable packaging_test on release candidates
2023-04-12 18:42:23 +00:00
Flutter GitHub Bot
2a5ecb6c48
Marks Linux firebase_abstract_method_smoke_test to be flaky (#124692)
Marks Linux firebase_abstract_method_smoke_test to be flaky
2023-04-12 18:29:07 +00:00
Flutter GitHub Bot
d15325407e
Marks Linux_samsung_s10 new_gallery_impeller__transition_perf to be flaky (#124694)
Marks Linux_samsung_s10 new_gallery_impeller__transition_perf to be flaky
2023-04-12 17:52:26 +00:00
Flutter GitHub Bot
df48b922ca
Marks Mac_ios animated_blur_backdrop_filter_perf_ios__timeline_summary to be unflaky (#124706)
Marks Mac_ios animated_blur_backdrop_filter_perf_ios__timeline_summary to be unflaky
2023-04-12 16:59:43 +00:00
Flutter GitHub Bot
2dfb03fdac
Marks Linux_android animated_blur_backdrop_filter_perf__timeline_summary to be unflaky (#124705)
Marks Linux_android animated_blur_backdrop_filter_perf__timeline_summary to be unflaky
2023-04-12 16:58:25 +00:00
Flutter GitHub Bot
e68086e910
Marks Linux_android new_gallery_impeller__transition_perf to be flaky (#124700)
Marks Linux_android new_gallery_impeller__transition_perf to be flaky
2023-04-12 16:55:44 +00:00