Commit graph

1168 commits

Author SHA1 Message Date
Alexander Markov 9ee73fe322 [Kernel] Remove outdated and duplicated front-end Targets
This CL replaces outdated VmTarget and FlutterTarget with VmFastaTarget
and FlutterFastaTarget. 'Fasta' suffix is droped from target names.

The new FlutterTarget extends VmTarget, so they share more code.

Change-Id: Id79956698a889c9a49b8a67914f1f96a731407ab
Reviewed-on: https://dart-review.googlesource.com/9423
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
2017-10-04 18:02:49 +00:00
Florian Loitsch 38bf70d7ac Use generic functions in zones.
Migrated from https://chromiumcodereview.appspot.com/2893893002/

Change-Id: I0bd6dc1438eb1e6762e7760a08b5a760b07d4b10
Reviewed-on: https://dart-review.googlesource.com/4942
Reviewed-by: Florian Loitsch <floitsch@google.com>
Commit-Queue: Florian Loitsch <floitsch@google.com>
2017-09-20 15:46:06 +00:00
Jens Johansen d854c77e58 [kernel] Only serialize non-external libaries and sources from those.
Previously the VM couldn't handle external libraries, but that was
fixed in 2f49198520.

Part of the CL was reverted though because the compilatin was changed
to using an outline instead of the platform which doesn't work.
What does work though, is not including the external libraries in the
output.

This CL makes the following changes:

* Don't include external libraries in the output (by not setting all
  libraries to be non-external).

* Only writes the sources actually used to the binary (i.e. whatever
  libraries left out because they were external will not contribute
  source code either).

* Cleanup of now unused code.


Timings (only run once though):


Without this CL (but with the CL it's based on):

$ time python tools/test.py -m release -cdartk language -j6
Test configuration: dartk_vm_release_x64
[05:43 | 100% | + 3504 | -    0]

real    5m43.597s
user    33m48.152s
sys     9m34.140s


Only the "utils/kernel-service/kernel-service.dart" part of this CL:

$ time python tools/test.py -m release -cdartk language -j6
Test configuration: dartk_vm_release_x64
[04:55 | 100% | + 3504 | -    0]

real    4m55.684s
user    29m54.360s
sys     8m7.408s


Entire CL:

$ time python tools/test.py -m release -cdartk language -j6
Test configuration: dartk_vm_release_x64
[04:20 | 100% | + 3504 | -    0]

real    4m20.416s
user    27m17.320s
sys     6m53.472s

Change-Id: Ie9c5bfa958e558a5007784e821a0b58d417bae55
Reviewed-on: https://dart-review.googlesource.com/3161
Reviewed-by: Samir Jindel <sjindel@google.com>
2017-09-06 11:56:24 +00:00
Peter von der Ahé a5bcf58b06 Move tools to tool folder.
R=sigmund@google.com

Review-Url: https://codereview.chromium.org/3003743002 .
2017-08-29 14:30:51 +02:00
Alexander Aprelev ba543167ec Pass path to platform kernel binary to kernel-service.
Currently kernel-service derives path to platform binary from path to dart executable. It doesn't always work, so dartk tests fail.

R=asiva@google.com, sigmund@google.com

BUG:http://dartbug.com/30456
Review-Url: https://codereview.chromium.org/3001013002 .
2017-08-17 09:49:48 -07:00
Alexander Aprelev e25898d963 Allow caller of application_snapshot to override .packages.
This will be used in flutter engine to build snapshot of frontend server dart app.

BUG=
R=zra@google.com

Review-Url: https://codereview.chromium.org/2995053002 .
2017-08-15 16:10:20 -07:00
Alexander Aprelev 747348caec Remove unused import statement.
This is a follow-up to a9afb54d6d to fix the tests.

BUG=
R=sigmund@google.com

Review-Url: https://codereview.chromium.org/2999983002 .
2017-08-15 10:06:57 -07:00
Alexander Aprelev a9afb54d6d Revert "Revert "Introduce IKG into kernel-service to support incremental compilation.""
This reverts commit c35510d835
with fix for the build - remove implicit temporary array in TestCase::LoadTestScript.

BUG=
R=asiva@google.com

Review-Url: https://codereview.chromium.org/2998963002 .
2017-08-15 09:03:48 -07:00
Alexander Aprelev c35510d835 Revert "Introduce IKG into kernel-service to support incremental compilation."
This reverts commit f596d440f5.

BUG=

Review-Url: https://codereview.chromium.org/3002553003 .
2017-08-14 14:15:41 -07:00
Alexander Aprelev f596d440f5 Introduce IKG into kernel-service to support incremental compilation.
This CL also adds few hot-reload incremental kernel files tests(some pass, one is failing), which are useful to track progress of hot reload functionality in Flutter setting.

BUG=
R=asiva@google.com, sigmund@google.com

Review-Url: https://codereview.chromium.org/2993013002 .
2017-08-14 14:06:15 -07:00
Zach Anderson 0394d3a0cf [infra] Begin removing gyp
This CL begins removing gyp from Dart's tree.

Removing gyp will go in three stages:
1. Remove unused .gyp and .gypi files.
2. Translate *_sources.gypi files to *_sources.gni files
3. Remove //third_party/gyp from DEPS

This CL is stage 1.

R=asiva@google.com

Review-Url: https://codereview.chromium.org/2992593002 .
2017-08-04 22:19:35 -07:00
Sigmund Cherem de7e14436c Include platform.dill files in sdk builds
R=zra@google.com

Review-Url: https://codereview.chromium.org/2984243002 .
2017-07-26 14:50:49 -07:00
Siva Annamalai 33c50e9374 1. Correctly set root_library based on the application script URI instead of looking for the library that has the 'main' method.
2. Adjust the Dart front end to not require a 'main' method when the in memory filesystem is used, this allows execution of VM unit tests which all do not necessarily have a 'main' method (integrated CL from siggi)

3. Change kernel reader to not look for a 'main' methos when Dart_LoadKernel is called.

4. Change Dart_LoadScript to also support loading of dart programs when Dart front end is used

R=aam@google.com, sigmund@google.com

Review-Url: https://codereview.chromium.org/2948273002 .
2017-07-19 18:09:19 -07:00
Sigmund Cherem 1aa139bc94 Unifying compiler context
Changes in this CL:
 - Updated CompilerContext:
    - it now contains a ProcessedOptions object

    - it no longer depends on CompilerCommandLine/CommandLine

    - it delegates to ProcessedOptions.report so all error reporting
      goes to one single place.

    - use "withContext" term instead of "withGlobalOptions" to be
      more clear about the intent

 - Changes in public API
    - added more options that correspond to flags in command-line
      fasta tools

    - default onError is different: we now use the
      command_line_reporting report, which prints and throws
      on fatal messages, but doesn't throw eagerly on all messages
      as before.

    - introduced "printMessages" option: make it easy to have
      both onError + command_line_reporting  (kernel-service.dart
      is the main use case at this time, other ideas welcome!)

    - renamed CompilationError to CompilationMessage

 - Other changes

    - set exit code is done on report, not on format
    - fixed corner cases not covered in previous CL
        - error reporting with missing-main needs to happen with
          a context
        - missing error cases when inferring .packages and input
          URIs are not file:* URIs

Ideas for follow up after this CL:
 - combine ProcessedOptions and CompilerContext into a single class
   (or extend one from the other)
 - switch onError to a stream

R=ahe@google.com

Review-Url: https://codereview.chromium.org/2982093003 .
2017-07-18 17:02:59 -07:00
Bob Nystrom 1d8c3af663 Delete old tests that aren't used and/or useful any more.
R=efortuna@google.com

Review-Url: https://codereview.chromium.org/2976313002 .
2017-07-18 14:12:57 -07:00
Sigmund Cherem 2465864241 simplify and remove unnecessary checks in kernel service
Checks here should also be handled internally by the front-end, so no need to do
them twice.

BUG=
R=asiva@google.com

Review-Url: https://codereview.chromium.org/2979053002 .
2017-07-17 15:31:33 -07:00
Sigmund Cherem ebd9440b18 support resolving .packages in FE
BUG=
R=paulberry@google.com

Review-Url: https://codereview.chromium.org/2979003002 .
2017-07-14 12:54:35 -07:00
Sigmund Cherem cc567226a8 Fix remaining failures from earlier CL (that switched kernel-service and dart2js
to use the public frontend apis.)

 * As part of the refactor I made kernel-service truncate the .dill file it
   sends to the VM so it excludes the SDK code. This works in most cases, but
   caused 26 failures in debug mode. The error matches what I saw in Issue
   #30111.

 * Some Dart2js tests using --use-sdk failed to run because they tried to find
   platform.dill incorrectly. Our bots do have it available, we just have
   to make sure to resolve it based on the build configuraiton and not based on
   the SDK executable.

Review-Url: https://codereview.chromium.org/2974683003 .
2017-07-08 10:26:49 -07:00
Sigmund Cherem b71bd543a9 Reapply "Tweak public APIs and use them in patch_sdk, dart2js, and kernel-service.""
Original CL had a bug that wasn't visible unless you delete your
out/ReleaseX64/patched_sdk folder.

Patchset #1 is the original CL, patchset #2 shows the fix.

This reverts commit 4aadfe09df.

BUG=

Review-Url: https://codereview.chromium.org/2976543002 .
2017-07-07 15:32:24 -07:00
Sigmund Cherem 4aadfe09df Revert "Tweak public APIs and use them in patch_sdk, dart2js, and kernel-service."
Reason: broke VM bots, still investigating.

This reverts commit 610d081947.

TBR= paulberry@google.com

Review-Url: https://codereview.chromium.org/2979463002 .
2017-07-07 15:21:37 -07:00
Sigmund Cherem 610d081947 Tweak public APIs and use them in patch_sdk, dart2js, and kernel-service.
This CL tweaks the public APIs in package:front_end, and
starts using those APIs outside the package. For example, this
removes 9 uses of DillTarget, so it is not longer mentioned
outside pkg/front_end and the analyzer_target.

Actual changes:
 - in package:front_end

   * added kernel_generator_impl: new file contains code that
     used to be in kernel_generator. Code has some modifications:
     it uses a single canonical-root when loading summaries, and
     it supports generating both outlines and kernel in one go.

   * removed code that didn't belong here:
     a. most of calculating deps for .GN moved to patch_sdk
     b. vm-specific outcomes moved to kernel-service

   * updated how `native` is implemented, so we can more easily
     support dart2js and ddc

   * updated how we check where `int`, `bool`, etc can be implemented.

   * added support "hermetic mode" in modular builds
     ('chaseDependencies = false' option)

   * moved `trim` step out of fasta, and for now call it only within
     the public API. This is not yet exposed, and I stopped covering it in
     most tests (now only covered in shaker tests). The plan is to add
     tests for the public API covering this in the future.

   * removed `uriToSource` when serializing outlines

   * added unit tests for public APIs

 - patch_sdk
   * use the public API to craete platform.dill, outline.dill (now
     500K insted of 3Mb because it excludes sources), and vmservice_io.dill
   * moved here logic internal to .GN

 - kernel service
   * use the public API
   * moved here logic that depends on VM internals (e.g. status enum,
     compilation results)

 - package:compiler
   * use the public API in tools and unit tests
   * simplified patched-sdk generation: no more extending fasta's internals

 - package:kernel
   * fix bug in deserialization: initializers and other lists were
     overwritten accidentally with external definitions.
   * updated unit tests, moved shared logic to frontend/src/fasta/testing

R=johnniwinther@google.com, paulberry@google.com

Review-Url: https://codereview.chromium.org/2953703002 .
2017-07-07 15:14:22 -07:00
Bob Nystrom 0b2c9b048d Dynamically load packages for dartdevc tests in test.dart.
This involves a few pieces:

- Add support to DDC for specifying the module name associated with a
  given summary. This lets test.dart invoke DDC using summaries in the
  build directory outside of the directory containing the test itself.

- Add support to the build scripts for building the packages. This adds
  a new GN target that builds everything needed to run test.dart with
  dartdevc. In particular, it invokes build_pkgs.dart to compile the
  relevant packages to JS+summary so that the tests can use them.

  This requires some changes to build_pkgs.dart so it can output to a
  given directory.

- In test.dart, when tests are compiled with dartdevc, pass in the
  summaries for the packages so they don't get compiled in. Then, when
  the test is run, configure require.js with the right paths to their
  JS files so they can be loaded.

  I also removed a bunch of unneeded buildDir parameters being passed
  around the various CompilerConfiguration class methods now that they
  have direct access to the configuration.

Fix #29923.

R=vsm@google.com, whesse@google.com, zra@google.com

Review-Url: https://codereview.chromium.org/2955513002 .
2017-06-29 13:45:57 -07:00
Florian Loitsch 978213d427 Revert "Make Zone API strong mode clean."
This reverts commit 0b35711d48.

BUG=

Review-Url: https://codereview.chromium.org/2963743002 .
2017-06-28 13:31:32 +02:00
Florian Loitsch 0b35711d48 Make Zone API strong mode clean.
Review-Url: https://codereview.chromium.org/2959163002 .
2017-06-28 13:16:39 +02:00
Erik Corry 029b1cb948 Spelling fixes e to i.
R=kevmoo@google.com
BUG=

Review-Url: https://codereview.chromium.org/2957593002 .
2017-06-24 13:41:39 +02:00
Alexander Aprelev 4f9b25e410 Fix previous CL where dart_bootstrap was accidently changed to dart_bootstrap_host_arch.
Also fix two missed places where host_toolchain should be replaced with dart_host_toolchain.

R=rmacnak@google.com

BUG:flutter/flutter#10841
Review-Url: https://codereview.chromium.org/2953873002 .
2017-06-22 13:17:53 -07:00
Alexander Aprelev 8cd713bed3 Introduce dart_host_toolchain argument for generate_patch_sdk.gni. This allows for Flutter build to make sure that host_toolchain used for patched_sdk generation word size matches target platform.
Flutter Dart in product configuration doesn't allow --checked option. So remove '--checked' from invocation.

R=rmacnak@google.com

BUG:https://github.com/flutter/flutter/issues/10841
Review-Url: https://codereview.chromium.org/2944103003 .
2017-06-22 12:41:31 -07:00
Zachary Anderson ac16656161 Remove MIPS support
R=asiva@google.com

Review-Url: https://codereview.chromium.org/2858623002 .
2017-06-22 08:49:22 -07:00
Ryan Macnak 1eee6d8af3 Mostly revert "[fuchsia] Prepare for setting the Dart target architecture appropriately in the host binaries."
This reverts commit 9ab39ae099.

Keep removal of bad assert in loader.dart.  Keep using script snapshot in cross builds.

We expect to use a checked-in SDK in the Fuschia build instead.

R=zra@google.com

Review-Url: https://codereview.chromium.org/2951133002 .
2017-06-21 15:19:21 -07:00
Ryan Macnak 9ab39ae099 Reapply "[fuchsia] Prepare for setting the Dart target architecture appropriately in the host binaries."
Add missing config to run_vm_tests.

R=zra@google.com

Review-Url: https://codereview.chromium.org/2937503003 .
2017-06-13 08:49:00 -07:00
Ryan Macnak 3383c4ac25 Revert "[fuchsia] Prepare for setting the Dart target architecture appropriately in the host binaries."
This reverts commit 54e80a8c32.

Review-Url: https://codereview.chromium.org/2940513002 .
2017-06-12 15:19:23 -07:00
Ryan Macnak 54e80a8c32 [fuchsia] Prepare for setting the Dart target architecture appropriately in the host binaries.
Build another varient of dart_bootstrap targeting the host architecture to ensure reasonable Observatory build times.

Remove assert assuming a root script, which does not exist when gen_snapshot builds core snapshots.

R=zra@google.com

Review-Url: https://codereview.chromium.org/2924173005 .
2017-06-12 14:33:10 -07:00
Sigmund Cherem ebdb0af64f Add integration test: hot reload + incremental compiler
R=scheglov@google.com

Review-Url: https://codereview.chromium.org/2925953002 .
2017-06-07 12:47:27 -07:00
Sigmund Cherem edd835c5bb Fix vm tests: introduce a hybrid file system so that the vm tests can find the
sdk files.

The reason the tests broke is that they use a memory-file-system which only
contains the test file and a mock .packages file.

BUG= https://github.com/dart-lang/sdk/issues/29723
R=paulberry@google.com

Review-Url: https://codereview.chromium.org/2910473003 .
2017-05-25 12:25:15 -07:00
Jacob Richman 728f71f740 Ship dart2js output for dev_compiler/web in the sdk.
These tools are used to support applying source maps to stack traces
and running the Dart dev compiler in the browser as part of an extension.

This CL adds the following files
dart-sdk/lib/dev_compiler/web/ddc_web_compiler.js
dart-sdk/lib/dev_compiler/web/dart_stack_trace_mapper

BUG=
R=rmacnak@google.com, vsm@google.com

Review-Url: https://codereview.chromium.org/2889253002 .
2017-05-19 13:21:53 -07:00
Alexander Aprelev b0c6a59731 Fix analyzer tests in checked mode.
BUG=
R=paulberry@google.com

Review-Url: https://codereview.chromium.org/2884023002 .
2017-05-15 10:25:59 -07:00
Alexander Aprelev 089a355766 Build up and pass MemoryFileSystem to fasta frontend.
Allow kernel service request to provide source files(filename, content), which will be used to build up MemoryFileSystem instance. MemoryFileSystem instance will be used instead of PhysicalFileSystem instance.

Immediate need for this is to support VM unit testing, where small Dart scripts that have to be parsed are hardcoded in the unit tests.

BUG=https://github.com/dart-lang/sdk/issues/28264
R=scheglov@google.com

Review-Url: https://codereview.chromium.org/2880453005 .
2017-05-15 08:44:34 -07:00
Vijay Menon 96f9287c0b Regenerate DDC artifacts
I had to put --unsafe-force-compile back to get the snapshot to build.  Investigating...

TBR=jmesserly@google.com

Review-Url: https://codereview.chromium.org/2873013003 .
2017-05-09 14:08:47 -07:00
Ryan Macnak 98d31056a7 Change build.py to be like ninja.py and only use a single GN/Ninja invocation to cross-build the SDK. This step alone would fix #29580. This breaks cross-building the SDK with GYP.
Remove hack in bin/snapshot_utils.cc that avoided building a true app-jit snapshot for ARM, ARM64 and MIPS because of the multiple invocations above.

Add GN flags to choose to build script snapshots instead of app-jit snapshots, avoiding a slow training run inside a simulator.

Fixes #29580.

R=zra@google.com

Review-Url: https://codereview.chromium.org/2871713002 .
2017-05-08 13:06:09 -07:00
Sigmund Cherem de36d11817 Add support for building patched_sdk and platform.dill for dart2js:
- defines a .gn target for patched_dart2js_sdk
 - changes patch_sdk.dart to handle a dart2js target
 - adds support in fasta to understand the `native` clause still used by dart2js
 - add option to fasta to know whether it is generating kernel for the vm or dart2js

I'm not excited about how I added the target distinction in fasta, the reason I
did so is that it affects the behavior in several ways: in how we handle
`native`, in the libraries that are included by default in the platform.dill file,
and in the transformations that need to be done.  I'm happy to change the
approach.

BUG=
R=ahe@google.com, vegorov@google.com, zra@google.com

Committed: 714523b103
Review-Url: https://codereview.chromium.org/2832353002 .
2017-05-01 17:23:02 -07:00
Zachary Anderson c5751b27e0 [GN] Change a filesystem absolute path deps to be source absolute
R=rmacnak@google.com

Review-Url: https://codereview.chromium.org/2856713003 .
2017-05-01 15:58:35 -07:00
Sigmund Cherem e9cd0b0fa5 Revert "Add support for building patched_sdk and platform.dill for dart2js:"
This reverts commit 714523b103.

TBR=ahe@google.com,vegorov@google.com

Review-Url: https://codereview.chromium.org/2849183002 .
2017-05-01 10:29:26 -07:00
Sigmund Cherem 714523b103 Add support for building patched_sdk and platform.dill for dart2js:
- defines a .gn target for patched_dart2js_sdk
 - changes patch_sdk.dart to handle a dart2js target
 - adds support in fasta to understand the `native` clause still used by dart2js
 - add option to fasta to know whether it is generating kernel for the vm or dart2js

I'm not excited about how I added the target distinction in fasta, the reason I
did so is that it affects the behavior in several ways: in how we handle
`native`, in the libraries that are included by default in the platform.dill file,
and in the transformations that need to be done.  I'm happy to change the
approach.

BUG=
R=ahe@google.com, vegorov@google.com

Review-Url: https://codereview.chromium.org/2832353002 .
2017-05-01 10:11:42 -07:00
Vijay Menon 75b7bea014 Snapshot DDC trained on itself
Note: path.canonicalize appears to normalize windows paths (c:/ vs c:\).

Fixes #29119

R=jmesserly@google.com, rnystrom@google.com

Review-Url: https://codereview.chromium.org/2836483002 .
2017-04-21 16:19:57 -07:00
Jacob Richman 119b2d58ce Format all files under tools and utils directory.
BUG=
R=terry@google.com

Review-Url: https://codereview.chromium.org/2827793002 .
2017-04-20 09:08:31 -07:00
Peter von der Ahé 2d9956a93d Move kernel baseline tests to front_end.
R=paulberry@google.com

Review-Url: https://codereview.chromium.org/2825063002 .
2017-04-19 10:57:58 +02:00
William Hesse 011a5a6a66 Remove ie10 and opera sections from status files
BUG=
R=kustermann@google.com

Review-Url: https://codereview.chromium.org/2783653002 .
2017-03-28 16:38:59 +02:00
Vijay Menon 6854dec8a7 Revert "Train ddc snapshot on itself"
This reverts commit 1bd92d4f04.

Breaking on windows.

TBR=jakemac@google.com
BUG=

Review-Url: https://codereview.chromium.org/2762043002 .
2017-03-20 18:57:22 -07:00
Vijay Menon 1bd92d4f04 Train ddc snapshot on itself
This speeds up dartdevc on hello world from about 600ms to about 150ms.

OTOH, it also increases the snapshot size from ~4.6M to ~31M (the
latter is just under the size of the dart2js snapshot).

I tried this before.  This may break again on windows - if so, i'll revert.

R=jakemac@google.com

Review-Url: https://codereview.chromium.org/2760043002 .
2017-03-20 18:51:37 -07:00
Peter von der Ahé d1bdc9596d Improve detection of compile-time errors.
R=karlklose@google.com

Review-Url: https://codereview.chromium.org/2739213004 .
2017-03-10 14:14:49 +01:00