flutter/dev/bots
Greg Spencer 797b39e9b2
Creates a new flutter command 'ide-config' and removes *.iml and .idea from tree. (#12958)
Creates a new (hidden) flutter command 'ide-config' that will create and/or update
existing .iml files and some files under the .idea directory, as well as
removing existing *.iml files and the .idea directory.

It also:

 * Adds *.iml to the .gitignore
 * Removes existing .iml files from the repo, and moves them to the
   packages/flutter_tools/ide_templates/intellij directory.
 * Adds a flag to ide-config ('--update-templates') that will take any new .iml
   files in the flutter tree and add them to the existing templates.
     - If --overwrite is also specified, then all existing templates will also
       be overwritten with the contents from the flutter tree, and any that have
       been deleted from the flutter tree will also be removed from the
       templates.
 * Added new run configurations for all existing app targets that will now also
   be automatically added to IntelliJ.
 * Setting up the environment also includes setting the coding style guidelines
   and the git VCS.
 * Note that after this PR lands, Flutter developers will need to run it once to
   re-create the .iml files and configuration files that have been removed.

After this PR lands, .iml files will no longer appear in the untracked files
section for git.
2017-11-13 10:55:22 -08:00
..
.gitignore Git ignore new temporary files in dev/bots (#8033) 2017-02-09 14:29:19 -08:00
analyze-sample-code.dart Make analysis of sample code work in OS-X (#11139) 2017-07-10 14:42:45 -07:00
docs.sh Upgrade dartdoc to v0.14.1 (#12420) 2017-10-05 11:26:49 -07:00
download_android_tools.py make download_android_tools fail when the download fails (#8261) 2017-02-17 16:00:12 -08:00
find_depot_tools.py Try to simplify our bot setup. (#3986) 2016-05-18 11:38:45 -07:00
pubspec.yaml Reapply #12700 but keep linter pinned at 0.1.35 (#12735) 2017-10-26 09:55:36 -07:00
README.md Update Chromium recipe instructions to reflect new flow (#9919) 2017-05-09 17:30:05 +02:00
test.dart Enable lint prefer asserts in initializer lists (#12903) 2017-11-08 22:59:49 +01:00
travis_install.sh Upgrade firebase tools version (#9548) 2017-04-24 11:14:54 +02:00
travis_setup.sh Don't pollute Travis log with Google Cloud SDK installation (#11159) 2017-07-11 20:47:58 -07:00
travis_upload.sh Retry deploy when docs deployment fails. (#12077) 2017-09-13 15:26:41 -07:00
upload_android_tools.py make upload_android_tool.py executable (#8176) 2017-02-15 11:08:16 -08:00
VERSION_LINUX_NDK Try to simplify our bot setup. (#3986) 2016-05-18 11:38:45 -07:00
VERSION_LINUX_SDK Revert "Revert "Replace jarsigner with apksigner" (#8164)" (#8320) 2017-02-21 16:26:30 -08:00
VERSION_MACOSX_NDK Try to simplify our bot setup. (#3986) 2016-05-18 11:38:45 -07:00
VERSION_MACOSX_SDK Revert "Revert "Replace jarsigner with apksigner" (#8164)" (#8320) 2017-02-21 16:26:30 -08:00
VERSION_WIN_SDK Revert "Revert "Replace jarsigner with apksigner" (#8164)" (#8320) 2017-02-21 16:26:30 -08:00

Flutter's Build Infrastructure

This directory exists to support building Flutter on our build infrastructure.

The results of such builds are viewable at:

The external master pages for the chromium infra bots do not allow forcing new builds. Contact @eseidelGoogle or another member of Google's Flutter team if you need to do that.

The Travis-based bots are trivial, and just run a couple of shell scripts. The rest of this document discusses only the chromium infra bots.

This infrastructure is broken into two parts. A buildbot master specified by our builders.pyl file, and a set of recipes which we run on that master. Both of these technologies are highly specific to Google's Chromium project. We're just borrowing some of their infrastructure.

Prerequisites

  • install depot_tools
  • Python package installer: sudo apt-get install python-pip
  • Python coverage package (only needed for training_simulation): sudo pip install coverage

Getting the code

The following will get way more than just recipe code, but it will get the recipe code:

mkdir chrome_infra
cd chrome_infra
fetch infra

More detailed instructions can be found here.

Most of the functionality for recipes comes from recipe_modules, which are unfortunately spread to many separate repositories. After checking out the code search for files named api.py or example.py under infra/build.

Editing a recipe

Flutter has one recipe per repository. Currently flutter/flutter and flutter/engine:

  • build/scripts/slave/recipes/flutter/flutter.py
  • build/scripts/slave/recipes/flutter/engine.py

Recipes are just Python. They are documented by the luci/recipes-py github project.

The typical cycle for editing a recipe is:

  1. Make your edits (probably to files in //chrome_infra/build/scripts/slave/recipes/flutter).
  2. Run build/scripts/slave/recipes.py --use-bootstrap test train to update expected files
  3. Run build/scripts/tools/run_recipe.py flutter/<repo> slavename=<slavename> mastername=client.flutter buildername=<buildername> where <repo> is one of flutter or engine, and slavename and buildername can be looked up from the Build Properties section of a recent build.
  4. Upload the patch (git commit, git cl upload) and send it to someone in the recipes/flutter/OWNERS file for review.

Editing the client.flutter buildbot master

Flutter uses Chromium's fancy builders.pyl master generation system. Chromium hosts 100s (if not 1000s) of buildbot masters and thus has lots of infrastructure for turning them up and down. Eventually all of buildbot is planned to be replaced by other infrastructure, but for now flutter has its own client.flutter master.

You would need to edit client.flutter's master in order to add slaves (talk to @eseidelGoogle), add builder groups, or to change the html layout of https://build.chromium.org/p/client.flutter. Carefully follow the builders.pyl docs to do so.

Future Directions

We would like to host our own recipes instead of storing them in build. Support for cross-repository recipes is in-progress. If you view the git log of this directory, you'll see we initially tried, but it's not quite ready.

Android Tools

Instructions to update the Android Tools version that the bots download by executing download_android_tools.py.

How to update Android SDK on Google Cloud Storage

  1. Run Android SDK Manager and update packages $ dev/bots/android_tools/sdk/tools/android update sdk Use android.bat on Windows.

  2. Choose/Update packages The following packages are currently installed:

    • Android SDK Tools
    • Android SDK platform-tools
    • Android SDK Build-tools 24.0.3
    • Android 6.0 (API 23)
      • SDK Platform 23
    • Extras
      • Android Support Repository
      • Google Play services
  3. Run upload_android_tools.py -t sdk $ dev/bots/upload_android_tools.py -t sdk

How to update Android NDK on Google Cloud Storage

  1. Download a new NDK binary (e.g. android-ndk-r10e-linux-x86_64.bin)

  2. cd dev/bots/android_tools $ cd dev/bots/android_tools

  3. Remove the old ndk directory $ rm -rf ndk

  4. Run the new NDK binary file $ ./android-ndk-r10e-linux-x86_64.bin

  5. Rename the extracted directory to ndk $ mv android-ndk-r10e ndk

  6. Run upload_android_tools.py -t ndk $ cd ../.. $ dev/bots/upload_android_tools.py -t ndk