Commit graph

710 commits

Author SHA1 Message Date
Dan Field 13283f86cb
Revert "Marks Linux firebase_oriol33_abstract_method_smoke_test to be unflaky" (#130497)
Reverts flutter/flutter#128398

See e.g.
https://ci.chromium.org/ui/p/flutter/builders/try/Linux%20firebase_oriol33_abstract_method_smoke_test/350/overview
2023-07-13 08:55:36 -07:00
Christopher Fujino b89bfd8e37
re-enable "Linux packages_autoroller" (#130088)
Fixes https://github.com/flutter/flutter/issues/129744

This change:

1. re-enables the Linux packages_autoroller
2. ensures we redact the token from appearing in any logs (in local testing I realized some failure logs might still expose the token)

What actually fixed authentication however was creating and uploading a new GitHub personal access token, not this change.

It's currently failing post-submit because being marked `bringup` it is running in the try pool, which does not have permissions to access the cloud KMS. However, I ran a LED build in the prod pool that succeeded:

3a8f128c35/+/build.proto?server=chromium-swarm.appspot.com
2023-07-10 19:32:56 +00:00
Flutter GitHub Bot c8265f81cf
Marks Linux firebase_oriol33_abstract_method_smoke_test to be unflaky (#128398)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux firebase_oriol33_abstract_method_smoke_test"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux%20firebase_oriol33_abstract_method_smoke_test%22).
This test can be marked as unflaky.
2023-07-06 19:42:24 +00:00
Christopher Fujino 97fce9aa19
mark packages-autoroller bringup again (#129859)
Even after updating the account (and the cloudKMS GitHub token), when
the bot tries to push to
https://github.com/flutter-pub-roller-bot/flutter, it gets a 403, which
probably means authentication failed. I will have to investigate next
week.

```
Exception: GitException: Exception on command "push https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git packages-autoroller-branch-1:packages-autoroller-branch-1": Command "git push https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git packages-autoroller-branch-1:packages-autoroller-branch-1" failed in directory "/b/s/w/ir/x/w/flutter_conductor_checkouts/framework" to update the release branch with the commit. Git exited with error code 128.
stderr from git:
16:33:21.199510 exec-cmd.c:90           trace: resolved executable path from procfs: /b/s/w/ir/cipd_bin_packages/bin/git
16:33:21.199587 exec-cmd.c:237          trace: resolved executable dir: /b/s/w/ir/cipd_bin_packages/bin
16:33:21.200155 git.c:460               trace: built-in: git push https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git packages-autoroller-branch-1:packages-autoroller-branch-1
16:33:21.200542 run-command.c:655       trace: run_command: GIT_DIR=.git git remote-https https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git
16:33:21.201787 exec-cmd.c:90           trace: resolved executable path from procfs: /b/s/w/ir/cipd_bin_packages/libexec/git-core/git
16:33:21.201845 exec-cmd.c:237          trace: resolved executable dir: /b/s/w/ir/cipd_bin_packages/libexec/git-core
16:33:21.202175 git.c:750               trace: exec: git-remote-https https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git
16:33:21.202195 run-command.c:655       trace: run_command: git-remote-https https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git https://[GitHub TOKEN]@github.com/flutter-pub-roller-bot/flutter.git
16:33:21.203510 exec-cmd.c:90           trace: resolved executable path from procfs: /b/s/w/ir/cipd_bin_packages/libexec/git-core/git-remote-http
16:33:21.203567 exec-cmd.c:237          trace: resolved executable dir: /b/s/w/ir/cipd_bin_packages/libexec/git-core
remote: Permission to flutter-pub-roller-bot/flutter.git denied to flutter-pub-roller-bot.
fatal: unable to access 'https://github.com/flutter-pub-roller-bot/flutter.git/': The requested URL returned error: 403
```


https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20packages_autoroller/6031/overview

This is still being tracked in
https://github.com/flutter/flutter/issues/129744
2023-06-30 16:54:55 -07:00
Christopher Fujino 67b7aa22c2
Change pub roller bot to push to flutter-pub-roller-bot/flutter.git (#129844)
Fixes https://github.com/flutter/flutter/issues/129744
Part of https://github.com/flutter/flutter/issues/129847

Migrate the packages_autoroller script to push to
flutter-pub-roller-bot/flutter.git, rather than
fluttergithubbot/flutter.git.
2023-06-30 16:28:00 -07:00
gmackall d1eb1f0a57
Upgrade integration tests to use AGP 7.3/Gradle 7.4 (#129642)
Upgrades tests under `dev/integration_tests` to use the latest values [in the templates](944d6c8fef/dev/tools/bin/generate_gradle_lockfiles.dart (L135)). Most of this PR was auto-generated, by running `find dev/integration_tests/ -type d -name 'android' | dart dev/tools/bin/generate_gradle_lockfiles.dart` from the root of the flutter directory.

The pieces that were not are:
1. Upgrading the Gradle versions used in integration tests to be >=7.4, in places where it was currently lower.
2. Upgrading the mac, windows, and linux build_tests .ci.yaml configuration to use jdk 17 on all shards. It currently was using a mix of 17 and 11. This isn't desirable, because some of the tests require 17, and the distribution is [random across shards](cef9a8c54b/dev/bots/test.dart (L553)) (so they were only passing because they were getting randomly placed on shards using jdk 17).
3. Adding a dependency on jdk 11 for the tests based on the `firebaselab/firebaselab` recipe. Previously they had no java dependency, and were therefore defaulting to java 1.8. The newer AGP versions necessitated an upgrade to 11 to run.

Note that it also ended up downgrading the AGP version in two places (in the [hybrid_android_views](https://github.com/flutter/flutter/pull/129642/files#diff-d7cfaa1c96d5c2383e7fc253ac8c9df062b544c24f7d80aa8a02290c01d17205L17) and [platform_interaction](https://github.com/flutter/flutter/pull/129642/files#diff-b2fba811efadd1cfb097bebd93be99fe4b6d6f17aefa863267290fc704290766L17) tests), because those had been manually edited to a newer version than the template.

Related to: https://github.com/flutter/flutter/issues/123636, https://github.com/flutter/flutter/pull/123910
2023-06-30 19:34:20 +00:00
Jackson Gardner 1b887c72b5
Skwasm benchmarks. (#129681)
This enables benchmarks for the Skwasm renderer, compiled with
dart2wasm.

Platform views aren't supported in Skwasm yet, so we are skipping those
benchmarks for now.
2023-06-28 17:16:19 -07:00
Christopher Fujino c1b764a80f
mark autoroller flaky (#129745)
Unblock tree while I investigate: https://github.com/flutter/flutter/issues/129744
2023-06-28 23:26:56 +00:00
Victoria Ashworth 9531cd4000
Update Xcode to 14.3.1 (#129024)
Framework part of https://github.com/flutter/flutter/issues/129019
2023-06-27 18:21:33 +00:00
Brandon DeRosier b898d1d270
[devicelab] Add platform view scroll benchmarks for Impeller on Android (#129455)
For tracking perf improvements with changes like
https://github.com/flutter/engine/pull/43152.
2023-06-23 18:40:17 -07:00
godofredoc 929468e171
Add a devicelab test to recipes cq. (#129411)
This is to ensure the devicelab_drone recipe is tested on presubmit.

Bug: https://github.com/flutter/flutter/issues/126111
2023-06-23 17:38:38 +00:00
Kate Lovett 309e50173e
Run misc test shard when examples/api/** change (#129433)
Fixes https://github.com/flutter/flutter/issues/129390

Found in https://github.com/flutter/flutter/pull/129381, changing sample code in `examples/api` did not trigger the misc test shard that executes the tests for `examples/api`.
2023-06-23 16:46:04 +00:00
godofredoc f582c0c573
Add comments to .ci.yaml that were removed from recipes. (#129321)
Firebaselab recipe was cleaned up and comments removed. This is adding the still valid comments to the .ci.yaml file.
2023-06-23 05:10:54 +00:00
gmackall 51f659825e
Unpin path_provider_android (#129205)
Unpins path_provider_android where it is pinned. Follows the same steps as https://github.com/flutter/flutter/pull/128898.

Fixes https://github.com/flutter/flutter/issues/116376
2023-06-22 21:22:49 +00:00
godofredoc dffef2715e
Move all the firebase lab device configs to .ci.yaml. (#129219)
Bug: https://github.com/flutter/flutter/issues/129211
2023-06-21 01:10:07 +00:00
Jackson Gardner 54e27e3ed5
Update Chrome version for testing (#128447)
Most of the tests are using Chrome 96.2, which is quite old. Update to Chrome 114.
2023-06-08 15:37:06 +00:00
Reid Baker b4796915b3
i127682 android api 33 test support (#127689)
- try to set emulator to version 33 to see what happens

#127682
2023-05-31 16:38:16 +00:00
godofredoc 7d310e3b1b
Add oriol_33 firebaselab configuration. (#127894)
This allows to run the api 33 as a separate build.

Bug: https://github.com/flutter/flutter/issues/127884
Bug: https://github.com/flutter/flutter/issues/118736
2023-05-31 00:19:58 +00:00
keyonghan 2b4e303b5a
Remove legacy xcode properties/dependencies (#127724)
Now only $flutter/osx_sdk property is being used, and it's safe to remove deprecated entries from ci.yaml.

Part of https://github.com/flutter/flutter/issues/127534
2023-05-30 23:14:02 +00:00
keyonghan 3eecc217cf
Append $flutter/osx sdk property to existing platforms/targets relying on xcode (#127537)
Part of https://github.com/flutter/flutter/issues/127534
2023-05-25 22:47:31 +00:00
keyonghan 784ebdf2a3
Run Mac intel only targets on both intel and arm (#127230)
The slowness issue was resolved with latest xcode: https://github.com/flutter/flutter/issues/119750, and all these targets should resume on both platforms as before. This will help mitigate high queue time on intel bots.

This PR updates these targets:

- Mac plugin_test_ios: https://github.com/flutter/flutter/issues/119764
- Mac build_tests: https://github.com/flutter/flutter/pull/120620
- Mac plugin_test: https://github.com/flutter/flutter/pull/120714
- Mac plugin_test_macos: https://github.com/flutter/flutter/pull/122212
- Mac framework_tests_misc: https://github.com/flutter/flutter/pull/122618
2023-05-19 23:22:25 +00:00
Zachary Anderson 31a16f342e
Roll goldctl to f808dcff91b221ae313e540c09d79696cd08b8de (#127218)
goldctl hasn't been updated in about a year, and an update may be needed to reland https://github.com/flutter/flutter/pull/127110.
2023-05-19 18:52:50 +00:00
Christopher Fujino 344c9c6af2
mark windows build tests non-bringup (#127059)
This is a follow-up to aa0a1a7ae1. I was forced to mark these targets bringup in that PR because the targets were technically new (since each name changed 3 -> 4), so this is marking them non-bringup.
2023-05-18 17:08:21 +00:00
Christopher Fujino aa0a1a7ae1
shard windows build tests 3 -> 4 (#127057)
Time outs are currently making the tree red.

Fixes https://github.com/flutter/flutter/issues/127004
2023-05-17 15:49:35 -07:00
godofredoc 25a9efe994
Fix drone_dimensions. (#126953)
Use drone_dimensions instead of dimensions for builds running on dart-internal. This is required for those builders to select the correct drone type.
2023-05-16 22:11:21 +00:00
keyonghan 3f01c7e019
Move Mac_build_test flutter_gallery__transition_perf_e2e_ios to prod (#126941)
Target `Mac_build_test flutter_gallery__transition_perf_e2e_ios` was enabled in staging: https://github.com/flutter/flutter/pull/111164, and it has passed more than 50 runs: https://ci.chromium.org/p/flutter/builders/staging/Mac_build_test%20flutter_gallery__transition_perf_e2e_ios?limit=50. 

Manually enabling it in prod and removing the old `Mac_ios flutter_gallery__transition_perf_e2e_ios`.

The `Mac_build_test` one does the same thing as `Mac_ios` one, but separating build and test steps in separate targets.

Context: https://github.com/flutter/flutter/issues/103542
2023-05-16 21:46:06 +00:00
Jonah Williams 41abe998ee
[devicelab] add drawPoints benchmark (#126728)
Add a benchmark that measures the improvements from https://github.com/flutter/engine/pull/41803
2023-05-15 19:14:58 +00:00
Zachary Anderson 674b1ad0ae
Remove obsolete iOS benchmark (#126862)
Fixes https://github.com/flutter/flutter/issues/126861
2023-05-15 19:12:56 +00:00
Reid Baker 159ebf736a
Mark no longer flakey tests as bringup:false (#126281)
This is a result of auditing the ci.yaml tests marked as bringup true and determining which ones were still flakey. 
Each commit markes one tests as non flakey along with a link of proof of the lack of flakes. 

- Mark new_gallery_impeller__transition_perf as not flakey, #124699 closed, not recent flakes
- No failures in 500 runs for Staging_build_linux analyze https://ci.chromium.org/p/flutter/builders/staging/Staging_build_linux%20analyze\?cursor\=id%3E8782771663962116337\&limit\=200
- No flakes in 600 builds https://ci.chromium.org/p/flutter/builders/staging/Linux%20flutter_packaging_test\?cursor\=id%3E8787892377160059793\&limit\=200
- No flakes in 600 builds https://ci.chromium.org/p/flutter/builders/staging/Mac%20flutter_packaging_test\?cursor\=id%3E8787870631931126673\&limit\=200
- No flakes in 600 builds https://ci.chromium.org/p/flutter/builders/staging/Mac_arm64%20flutter_packaging_test\?cursor\=id%3E8787809799161254193\&limit\=200

#124699 is the only closed bug that was linked for a test that is no longer flakey. Closed status helps the case that flakes are no longer an issue.
2023-05-15 18:12:52 +00:00
Zachary Anderson 468512e057
Increase test timeout for analyzer_benchmark to 60 minutes (#126586)
Fixes https://github.com/flutter/flutter/issues/126451
2023-05-12 08:45:14 -07:00
godofredoc fa117aad28
Add dimensions to release builds. (#126117)
Release builds running in dart internal require the dimensions to be set explicitly at the target level in ci.yaml.

Bug: https://github.com/flutter/flutter/issues/126116
2023-05-11 17:28:08 +00:00
keyonghan a7a1ddfb5a
Add cpu: x86 dimension and xcode dependency for Mac_build_test targets (#126482)
This is a follow up of https://github.com/flutter/flutter/pull/111164.

My local LED was based on `cpu: x86` dimension and `xcode` dependency. But these are missed in the above PR, which caused test failure: https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_build_test%20flutter_gallery__transition_perf_e2e_ios/19/overview
2023-05-11 16:41:26 +00:00
keyonghan 8ab782d08a
Migrate gallery ios tests to build+test (#111164)
Part of https://github.com/flutter/flutter/issues/103542
2023-05-10 00:50:11 +00:00
Reid Baker 0fdddd67f9
Test AGP 8.0 using java 17 (#125323)
- Update Gradle/AGP version and add namespace plus dependencies. 

https://github.com/flutter/flutter/issues/125181
2023-05-08 15:02:31 +00:00
Drew Roen 9de9a81b80
Bring back the failing build_test's (#126014)
https://github.com/flutter/flutter/issues/125897

The issue has been resolved, so bringing the tests out of bringup
2023-05-05 15:54:43 +00:00
Jenn Magder 4f3c989bd3
Remove complex_layout_*__compile benchmarks (#126023)
Fixes https://github.com/flutter/flutter/issues/126020.
2023-05-03 23:49:01 +00:00
Flutter GitHub Bot 15bd29c8e2
Marks Linux_android slider_perf_android to be unflaky (#125968)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux_android slider_perf_android"
}
-->
The test has been passing for [50 consecutive
runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux_android%20slider_perf_android%22).
This test can be marked as unflaky.
2023-05-03 09:22:21 -07:00
Chris Yang 9fcb42d0dd
Update xcode to 14e222b (#125716)
Framework part of https://github.com/flutter/flutter/issues/125814
2023-05-02 19:56:17 +00:00
Jenn Magder cc9a3d9d95
Mark microbenchmarks_ios unflaky (#125332)
This hasn't flaked for > 50 commits which is our threshhold.  Marking as unflaky.
2023-05-02 19:34:40 +00:00
godofredoc d0069bec94
Mark build_test builds as bringup. (#125898)
They require an additional dimension to be passed as part of the
.ci.yaml configuration.

Bug: https://github.com/flutter/flutter/issues/125897

## Pre-launch Checklist

- [X] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [X] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [X] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [X] I signed the [CLA].
- [X] I listed at least one issue that this PR fixes in the description
above.
- [X] I updated/added relevant documentation (doc comments with `///`).
- [X] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [X] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2023-05-02 12:22:56 -07:00
Zachary Anderson fef41cfce0
Remove bringup from new_gallery_skia_ios__transition_perf (#125715)
Final step of https://github.com/flutter/flutter/pull/125663
2023-04-28 15:31:19 -07:00
Zachary Anderson 50f83fc277
Add back one Skia test on iOS (#125663)
Fixes https://github.com/flutter/flutter/issues/125648
2023-04-28 10:41:33 -07:00
Taha Tesser 746cacc501
Add slider_perf_android task to ci.yaml (#125540)
This adds `devicelab` task `slider_perf_android` to `.ci.yaml`, which was added in https://github.com/flutter/flutter/pull/125296
2023-04-27 21:32:39 +00:00
Jenn Magder 28e17177fd
Mark module_test_ios tests unflaky (#125498)
[Dart workaround](https://dart-review.googlesource.com/c/sdk/+/297821 ) to prevent the flake has rolled into the framework.  Mark the tests unflaky so we can detect if it happens again when a fix for the underlying issue lands.

https://github.com/flutter/flutter/issues/125425#issuecomment-1522142360

Reverts https://github.com/flutter/flutter/pull/125426
2023-04-27 18:09:25 +00:00
Zachary Anderson 1d30a1fc5a
Remove flavors_test_macos (#125581)
This test has been failing for the past >1000 runs, and is wedging a bot
for 30 minutes as the failure mode is a timeout.

Fixes https://github.com/flutter/flutter/issues/119782
2023-04-26 14:02:49 -07:00
Zachary Anderson 66fa4c5d30
Remove web compile benchmarks that specify an attached device (#125559)
Fixes https://github.com/flutter/flutter/issues/125492
2023-04-26 09:37:56 -07:00
Zachary Anderson 53cba8fd49
Add hostonly web compile size benchmarks (#125518)
First part of https://github.com/flutter/flutter/issues/125492
2023-04-25 19:11:12 -07:00
keyonghan 624bdd38b0
Mark module_test_ios flaky (#125426)
https://github.com/flutter/flutter/issues/125425
2023-04-24 18:46:50 +00:00
Jenn Magder 9e2e64c7c0
Do not run windows_home_scroll_perf__timeline_summary benchmark in presubmit (#125343)
Introduced in https://github.com/flutter/flutter/pull/99564.  Checked with @jonahwilliams and this should not be running in presubmit.
2023-04-24 18:33:21 +00:00
Jenn Magder baafa1991d
Combine flutter_tool_startup devicelab tests (#125226)
`flutter_tool_startup__windows/linux/macos` were identical. Use the same task for all three.
2023-04-21 22:33:07 +00:00