mirror of
https://github.com/flutter/flutter
synced 2024-08-25 02:46:31 +00:00
Update Devicelab README with new dashboard information (#48908)
This commit is contained in:
parent
cad08149ca
commit
781cd4d87c
|
@ -1023,18 +1023,6 @@ final Set<Hash256> _grandfatheredBinaries = <Hash256>{
|
|||
Hash256(0x4C67221E25626CB2, 0x3F94E1F49D34E4CF, 0x3A9787A514924FC5, 0x9EF1E143E5BC5690),
|
||||
|
||||
|
||||
// HISTORICAL DEVICE LAB IMAGES
|
||||
|
||||
// dev/devicelab/images/agent-statuses.png
|
||||
Hash256(0x0A43016C84095771, 0x1C610E1C01B1C3B2, 0x681DA0B2062C02AA, 0x31CC7FB3FDC298FD),
|
||||
|
||||
// dev/devicelab/images/broken-test.png
|
||||
Hash256(0x319459F42967A888, 0x90B20063544D6707, 0x849E1E3447CC56A5, 0xE226C47DE34F13AD),
|
||||
|
||||
// dev/devicelab/images/legend.png
|
||||
Hash256(0x92A98975AF5F076C, 0xE7BFAB86B0DD7A3D, 0xC45287B706D10456, 0x4E512BA3C41B2749),
|
||||
|
||||
|
||||
// MISCELLANEOUS
|
||||
|
||||
// dev/bots/serviceaccount.enc
|
||||
|
@ -1060,7 +1048,7 @@ Future<void> verifyNoBinaries(String workingDirectory, { Set<Hash256> grandfathe
|
|||
assert(
|
||||
_grandfatheredBinaries
|
||||
.expand<int>((Hash256 hash) => <int>[hash.a, hash.b, hash.c, hash.d])
|
||||
.reduce((int value, int element) => value ^ element) == 0x17138C817203D1A7 // Please do not modify this line.
|
||||
.reduce((int value, int element) => value ^ element) == 0x606B51C908B40BFA // Please do not modify this line.
|
||||
);
|
||||
grandfatheredBinaries ??= _grandfatheredBinaries;
|
||||
if (!Platform.isWindows) { // TODO(ianh): Port this to Windows
|
||||
|
|
|
@ -1,41 +1,30 @@
|
|||
# Flutter devicelab
|
||||
|
||||
"Devicelab" (a.k.a. "cocoon") is a physical lab that tests Flutter on real
|
||||
Android and iOS devices.
|
||||
"Devicelab" (a.k.a. [Cocoon](https://github.com/flutter/cocoon)) is a physical
|
||||
lab that tests Flutter on real Android and iOS devices.
|
||||
|
||||
This package contains the code for test framework and the tests. More generally
|
||||
the tests are referred to as "tasks" in the API, but since we primarily use it
|
||||
for testing, this document refers to them as "tests".
|
||||
|
||||
Build results are available at https://flutter-dashboard.appspot.com.
|
||||
Current statuses for the devicelab are available at
|
||||
https://flutter-dashboard.appspot.com.
|
||||
|
||||
# Reading the dashboard
|
||||
# Dashboards
|
||||
|
||||
## The build page
|
||||
## Build dashboard
|
||||
|
||||
The build page is accessible at https://flutter-dashboard.appspot.com/build.html.
|
||||
This page reports the health of build servers, called _agents_, and the statuses
|
||||
of build tasks.
|
||||
|
||||
### Agents
|
||||
|
||||
A green agent is considered healthy and ready to receive new tasks to build. A
|
||||
red agent is broken and does not receive new tasks.
|
||||
|
||||
In the example below, the dashboard shows that the `linux2` agent is broken and
|
||||
requires attention. All other agents are healthy.
|
||||
|
||||
![Agent statuses](images/agent-statuses.png)
|
||||
The build page is accessible at https://flutter-dashboard.appspot.com/#/build.
|
||||
This page reports the build statuses of commits to the flutter/flutter repo.
|
||||
|
||||
### Tasks
|
||||
|
||||
The table below the agent statuses displays the statuses of build tasks. Task
|
||||
statuses are color-coded. The following statuses are available:
|
||||
Task statuses are color-coded in the following manner:
|
||||
|
||||
**New task** (light blue): the task is waiting for an agent to pick it up and
|
||||
**New task** (blue): the task is waiting for an agent to pick it up and
|
||||
start the build.
|
||||
|
||||
**Task is running** (spinning blue): an agent is currently building the task.
|
||||
**Task is running** (blue with clock): an agent is currently building the task.
|
||||
|
||||
**Task succeeded** (green): an agent reported a successful completion of the
|
||||
task.
|
||||
|
@ -45,13 +34,11 @@ latest attempt succeeded (we currently only try twice).
|
|||
|
||||
**Task failed** (red): the task failed all of the attempts.
|
||||
|
||||
**Task underperformed** (orange): currently not used.
|
||||
**Task is rerunning** (orange): the task is being rerun.
|
||||
|
||||
**Task was skipped** (transparent): the task is not scheduled for a build. This
|
||||
usually happens when a task is removed from `manifest.yaml` file.
|
||||
|
||||
**Task status unknown** (purple): currently not used.
|
||||
|
||||
In addition to color-coding, a task may display a question mark. This means
|
||||
that the task was marked as flaky manually. The status of such task is ignored
|
||||
when considering whether the build is broken or not. For example, if a flaky
|
||||
|
@ -59,18 +46,11 @@ task fails, GitHub will not prevent PR submissions. However, if the latest
|
|||
status of a non-flaky task is red, all pending PRs will contain a warning about
|
||||
the broken build and recommend caution when submitting.
|
||||
|
||||
Legend:
|
||||
|
||||
![Task status legend](images/legend.png)
|
||||
|
||||
The example below shows that commit `e122d5d` caused a wide-spread breakage,
|
||||
which was fixed by `bdc6f10`. It also shows that Cirrus and Chrome
|
||||
Infra (left-most tasks) decided to skip building these commits. Hovering over
|
||||
a cell will pop up a tooltip containing the name of the broken task. Clicking
|
||||
on the cell will open the log file in a new browser tab (only visible to core
|
||||
contributors as of today).
|
||||
|
||||
![Broken Test](images/broken-test.png)
|
||||
Clicking a cell will pop up an overlay with information about that task. It
|
||||
includes information such as the task name, number of attempts, run time,
|
||||
queue time, whether it is manually marked flaky, and the agent it was run on.
|
||||
It has actions to download the log, rerun the task, and view the agent on
|
||||
the agent dashboard.
|
||||
|
||||
## Why is a task stuck on "new task" status?
|
||||
|
||||
|
@ -81,7 +61,19 @@ may skip some commits. For example, Cirrus will only test the
|
|||
_last_ commit of a PR that's merged into the `master` branch. Chrome Infra may
|
||||
skip commits when they come in too fast.
|
||||
|
||||
## How the devicelab runs the tasks
|
||||
## Agent dashboard
|
||||
|
||||
Agent statuses are available at https://flutter-dashboard.appspot.com/#/agents.
|
||||
|
||||
A green agent is considered healthy and ready to receive new tasks to build. A
|
||||
red agent is broken and does not receive new tasks.
|
||||
|
||||
## Performance dashboard
|
||||
|
||||
Flutter benchmarks are available at
|
||||
https://flutter-dashboard.appspot.com/benchmarks.html.
|
||||
|
||||
# How the devicelab runs tasks
|
||||
|
||||
The devicelab agents have a small script installed on them that continuously
|
||||
asks the CI server for tasks to run. When the server finds a suitable task for
|
||||
|
@ -238,5 +230,10 @@ Where:
|
|||
- `{STAGE}` is `devicelab` if you want to run on Android, or `devicelab_ios` if
|
||||
you want to run on iOS.
|
||||
- `{CAPABILITIES}` is an array that lists the capabilities required of
|
||||
the test agent (the computer that runs the test) to run your test. Available
|
||||
capabilities are: `has-android-device`, `has-ios-device`.
|
||||
the test agent (the computer that runs the test) to run your test. As of writing,
|
||||
the available capabilities are: `linux`, `linux/android`, `linux-vm`,
|
||||
`mac`, `mac/ios`, `mac/iphonexs`, `mac/ios32`, `mac-catalina/ios`,
|
||||
`mac-catalina/android`, `ios/gl-render-image`, `windows`, `windows/android`.
|
||||
|
||||
If your test needs to run on multiple operating systems, create a separate test
|
||||
for each operating system.
|
||||
|
|
Binary file not shown.
Before Width: | Height: | Size: 14 KiB |
Binary file not shown.
Before Width: | Height: | Size: 56 KiB |
Binary file not shown.
Before Width: | Height: | Size: 30 KiB |
Loading…
Reference in a new issue