Use depfiles for the SDK's application snapshots.
Add check that we don't try to create a script snapshot after loading from an app snapshot.
Bug: TO-596
Change-Id: Ib4209c3062dbe0f3c37b8f65e5a50ca64544a39b
Reviewed-on: https://dart-review.googlesource.com/26043
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Change-Id: Id13cc838b67785b12f95d19b480a6a830ad5c003
Reviewed-on: https://dart-review.googlesource.com/24120
Commit-Queue: Alexander Thomas <athom@google.com>
Commit-Queue: Karl Klose <karlklose@google.com>
Reviewed-by: Vijay Menon <vsm@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
- Remove random build-id.
- Replace build time in embedded version string with commit time.
- Remove timestamps from Observatory tarball.
- Zero-initialize skipped bytes in snapshot streams.
- Fix uninitialized fields in PatchClass, Script and Library.
- Disable (under flag) random identity hashes and concurrent GC.
Bug: https://github.com/dart-lang/sdk/issues/31427
Change-Id: I3e95de679c8372841cd27ca60df78d9b00ffbfe1
Reviewed-on: https://dart-review.googlesource.com/22901
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
This should, I hope, fix the debianpackage bot.
Change-Id: I39347cf64aba1d99c882d78bf7cf60483b3e85c5
Reviewed-on: https://dart-review.googlesource.com/22301
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
This involves a few main pieces:
- Add code to the GN scripts to generate DDC's patched SDK and then
compile it to summaries and JS in the build output directory.
- Add support to the underlying DDC build scripts to support controlling
which files are built where.
- Update test.dart to use the DDC SDK from the build directory.
- Update create_sdk to use the built SDK instead of the checked in one.
- Fix various internal DDC tools to build their own copy of the SDK
(since they can't easily find the one in the build directory because
it's path if config-specific) and use those.
- Delete the checked DDC SDK JS and summaries.
I think I got everything working. The built Dart SDK looks fine -- it's
identical to one built using the old build scripts.
The various tools and DDC's little test runner I *think* work, but there
may be a bug or two in there. I tried the various things I could and it
seems like they work but it's hard to tell since they may be kind of
broken right now anyway.
Bug:
Change-Id: Iea77915a5c1cc8450f60ebfbdf8c725c7ea2f32c
Reviewed-on: https://dart-review.googlesource.com/18144
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Vijay Menon <vsm@google.com>
* kernel-service.dart is moved to pkg/vm/bin and renamed to
kernel_service.dart
* LICENSE, pubspec.yaml and analysis_options.yaml added to pkg/vm.
* Kernel isolate name is corrected to be independent of URI passed
in --dfe VM option.
Change-Id: I5b7b860297314e240e557af80913e7ac3e6324ad
Reviewed-on: https://dart-review.googlesource.com/21260
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Setting this build argument to something other than host_toolchain will
cause the build target create_host_sdk to be exposed. That target will
build the Dart SDK with the toolchain specified by
dart_host_sdk_toolchain.
Relanding to fix bad toolchain conditions in dart_os_config and
dart_arch_config.
TO-549
Change-Id: I276f47d9071233acb98f6029a4666da0cee33d7d
Reviewed-on: https://dart-review.googlesource.com/18361
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
This reverts commit 8fc90bd3bc.
Broken precomp bots due to wrong toolchain condition in dart_arch_config and dart_os_config.
Change-Id: Ic5c75f9ca3a0d4f316678354da44d1fcca10499d
Reviewed-on: https://dart-review.googlesource.com/18360
Reviewed-by: Zach Anderson <zra@google.com>
Setting this build argument to something other than host_toolchain will
cause the build target create_host_sdk to be exposed. That target will
build the Dart SDK with the toolchain specified by
dart_host_sdk_toolchain.
TO-549
Change-Id: Ida92521724cf7af34b34fb8c6d3514a226c48dd4
Reviewed-on: https://dart-review.googlesource.com/18145
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
This allows dartdevk to compile tests that import packages like expect.
There are a few pieces to this:
- Add support to build_pkgs.dart to build the kernel summaries for each
test package (in addition to the analyzer summaries it already
builds).
- Plumb that through the dartdevc_test target in the GN build as well.
- While we're at it, use GN to build the ddc_sdk.dill file and have
test.dart load that one instead of the manually built one from calling
./tool/kernel_sdk.dart.
- Add command-line arguments to dartdevk for passing in the path to the
SDK summary and the other summaries to compile against.
- Fix a little typo in processed_options.dart that was preventing it
from resolving "package:" URIs.
- In test.dart, when compiling a test, link in the summaries for all of
the test packages.
At runtime, it still uses the JS for those packages generated from the
old analyzer-based front end since the kernel-based compiler isn't
complete enough to compile any of those packages yet.
With all of this, if I change a test to:
import "package:expect/expect.dart";
main() {
Expect.equals("a", "b");
}
Then it compiles but fails at runtime. The compiler is completing, but
the generated code has some bugs. I don't know enough to fix them
myself, but here's what I've found out:
- In _libraryToModule(), the Library we get from kernel has a null
fileUri, so this returns an empty string. That in turn means the
generated JS tries to use "$" as the module name.
Using this works around it temporarily:
if (moduleName.isEmpty) moduleName = library.name;
- In _emitTopLevelNameNoInterop(), it doesn't handle the case where the
NamedNode is a static method on a class. It just generates the library
and method name, skipping the class, so "Expect.equals(1, 2)" gets
compiled to "expect.equals(1, 2)" instead of
"expect.Expect.equals(1, 2)".
Change-Id: I6bd9d98bc9706965160d8fb7cf70b20eeebab3a8
Reviewed-on: https://dart-review.googlesource.com/16687
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Vijay Menon <vsm@google.com>
Change-Id: Ie4e06ec5badd7f06cf8c38096b0ee87c917b6d8a
Reviewed-on: https://dart-review.googlesource.com/11980
Commit-Queue: Peter von der Ahé <ahe@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Change-Id: Ib3338e616b05b9b1a7462c9f484402faca0fc13b
Reviewed-on: https://dart-review.googlesource.com/13983
Commit-Queue: Peter von der Ahé <ahe@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Change-Id: I6e6960c471b2fab3f0cd8a672da05d7699217410
Reviewed-on: https://dart-review.googlesource.com/12291
Commit-Queue: Peter von der Ahé <ahe@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Change-Id: If5a1ea7082eae52e109f9b7272b2fcd0a866ba97
Reviewed-on: https://dart-review.googlesource.com/12000
Reviewed-by: Peter von der Ahé <ahe@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
This separates compiling platform.dill files from the patch_sdk.dart
script. The motivation for that is that I'm working on reading patch
files directly from Fasta, so we can completely remove the build step
for generating patched_sdk and dart2js_patched_sdk.
Short-term this should allow Paul to add a strong-mode version of
platform.dill without causing to many conflicts with my work on
patches.
Change-Id: I1150845b2986348d4fffe27092701d8a9b57ea54
Reviewed-on: https://dart-review.googlesource.com/11506
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
This CL replaces outdated VmTarget and FlutterTarget with VmFastaTarget
and FlutterFastaTarget. 'Fasta' suffix is droped from target names.
The new FlutterTarget extends VmTarget, so they share more code.
Change-Id: Id79956698a889c9a49b8a67914f1f96a731407ab
Reviewed-on: https://dart-review.googlesource.com/9423
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Previously the VM couldn't handle external libraries, but that was
fixed in 2f49198520.
Part of the CL was reverted though because the compilatin was changed
to using an outline instead of the platform which doesn't work.
What does work though, is not including the external libraries in the
output.
This CL makes the following changes:
* Don't include external libraries in the output (by not setting all
libraries to be non-external).
* Only writes the sources actually used to the binary (i.e. whatever
libraries left out because they were external will not contribute
source code either).
* Cleanup of now unused code.
Timings (only run once though):
Without this CL (but with the CL it's based on):
$ time python tools/test.py -m release -cdartk language -j6
Test configuration: dartk_vm_release_x64
[05:43 | 100% | + 3504 | - 0]
real 5m43.597s
user 33m48.152s
sys 9m34.140s
Only the "utils/kernel-service/kernel-service.dart" part of this CL:
$ time python tools/test.py -m release -cdartk language -j6
Test configuration: dartk_vm_release_x64
[04:55 | 100% | + 3504 | - 0]
real 4m55.684s
user 29m54.360s
sys 8m7.408s
Entire CL:
$ time python tools/test.py -m release -cdartk language -j6
Test configuration: dartk_vm_release_x64
[04:20 | 100% | + 3504 | - 0]
real 4m20.416s
user 27m17.320s
sys 6m53.472s
Change-Id: Ie9c5bfa958e558a5007784e821a0b58d417bae55
Reviewed-on: https://dart-review.googlesource.com/3161
Reviewed-by: Samir Jindel <sjindel@google.com>
This CL begins removing gyp from Dart's tree.
Removing gyp will go in three stages:
1. Remove unused .gyp and .gypi files.
2. Translate *_sources.gypi files to *_sources.gni files
3. Remove //third_party/gyp from DEPS
This CL is stage 1.
R=asiva@google.com
Review-Url: https://codereview.chromium.org/2992593002 .
2. Adjust the Dart front end to not require a 'main' method when the in memory filesystem is used, this allows execution of VM unit tests which all do not necessarily have a 'main' method (integrated CL from siggi)
3. Change kernel reader to not look for a 'main' methos when Dart_LoadKernel is called.
4. Change Dart_LoadScript to also support loading of dart programs when Dart front end is used
R=aam@google.com, sigmund@google.com
Review-Url: https://codereview.chromium.org/2948273002 .
Changes in this CL:
- Updated CompilerContext:
- it now contains a ProcessedOptions object
- it no longer depends on CompilerCommandLine/CommandLine
- it delegates to ProcessedOptions.report so all error reporting
goes to one single place.
- use "withContext" term instead of "withGlobalOptions" to be
more clear about the intent
- Changes in public API
- added more options that correspond to flags in command-line
fasta tools
- default onError is different: we now use the
command_line_reporting report, which prints and throws
on fatal messages, but doesn't throw eagerly on all messages
as before.
- introduced "printMessages" option: make it easy to have
both onError + command_line_reporting (kernel-service.dart
is the main use case at this time, other ideas welcome!)
- renamed CompilationError to CompilationMessage
- Other changes
- set exit code is done on report, not on format
- fixed corner cases not covered in previous CL
- error reporting with missing-main needs to happen with
a context
- missing error cases when inferring .packages and input
URIs are not file:* URIs
Ideas for follow up after this CL:
- combine ProcessedOptions and CompilerContext into a single class
(or extend one from the other)
- switch onError to a stream
R=ahe@google.com
Review-Url: https://codereview.chromium.org/2982093003 .
to use the public frontend apis.)
* As part of the refactor I made kernel-service truncate the .dill file it
sends to the VM so it excludes the SDK code. This works in most cases, but
caused 26 failures in debug mode. The error matches what I saw in Issue
#30111.
* Some Dart2js tests using --use-sdk failed to run because they tried to find
platform.dill incorrectly. Our bots do have it available, we just have
to make sure to resolve it based on the build configuraiton and not based on
the SDK executable.
Review-Url: https://codereview.chromium.org/2974683003 .
Original CL had a bug that wasn't visible unless you delete your
out/ReleaseX64/patched_sdk folder.
Patchset #1 is the original CL, patchset #2 shows the fix.
This reverts commit 4aadfe09df.
BUG=
Review-Url: https://codereview.chromium.org/2976543002 .
This CL tweaks the public APIs in package:front_end, and
starts using those APIs outside the package. For example, this
removes 9 uses of DillTarget, so it is not longer mentioned
outside pkg/front_end and the analyzer_target.
Actual changes:
- in package:front_end
* added kernel_generator_impl: new file contains code that
used to be in kernel_generator. Code has some modifications:
it uses a single canonical-root when loading summaries, and
it supports generating both outlines and kernel in one go.
* removed code that didn't belong here:
a. most of calculating deps for .GN moved to patch_sdk
b. vm-specific outcomes moved to kernel-service
* updated how `native` is implemented, so we can more easily
support dart2js and ddc
* updated how we check where `int`, `bool`, etc can be implemented.
* added support "hermetic mode" in modular builds
('chaseDependencies = false' option)
* moved `trim` step out of fasta, and for now call it only within
the public API. This is not yet exposed, and I stopped covering it in
most tests (now only covered in shaker tests). The plan is to add
tests for the public API covering this in the future.
* removed `uriToSource` when serializing outlines
* added unit tests for public APIs
- patch_sdk
* use the public API to craete platform.dill, outline.dill (now
500K insted of 3Mb because it excludes sources), and vmservice_io.dill
* moved here logic internal to .GN
- kernel service
* use the public API
* moved here logic that depends on VM internals (e.g. status enum,
compilation results)
- package:compiler
* use the public API in tools and unit tests
* simplified patched-sdk generation: no more extending fasta's internals
- package:kernel
* fix bug in deserialization: initializers and other lists were
overwritten accidentally with external definitions.
* updated unit tests, moved shared logic to frontend/src/fasta/testing
R=johnniwinther@google.com, paulberry@google.com
Review-Url: https://codereview.chromium.org/2953703002 .
This involves a few pieces:
- Add support to DDC for specifying the module name associated with a
given summary. This lets test.dart invoke DDC using summaries in the
build directory outside of the directory containing the test itself.
- Add support to the build scripts for building the packages. This adds
a new GN target that builds everything needed to run test.dart with
dartdevc. In particular, it invokes build_pkgs.dart to compile the
relevant packages to JS+summary so that the tests can use them.
This requires some changes to build_pkgs.dart so it can output to a
given directory.
- In test.dart, when tests are compiled with dartdevc, pass in the
summaries for the packages so they don't get compiled in. Then, when
the test is run, configure require.js with the right paths to their
JS files so they can be loaded.
I also removed a bunch of unneeded buildDir parameters being passed
around the various CompilerConfiguration class methods now that they
have direct access to the configuration.
Fix#29923.
R=vsm@google.com, whesse@google.com, zra@google.com
Review-Url: https://codereview.chromium.org/2955513002 .
Build another varient of dart_bootstrap targeting the host architecture to ensure reasonable Observatory build times.
Remove assert assuming a root script, which does not exist when gen_snapshot builds core snapshots.
R=zra@google.com
Review-Url: https://codereview.chromium.org/2924173005 .
These tools are used to support applying source maps to stack traces
and running the Dart dev compiler in the browser as part of an extension.
This CL adds the following files
dart-sdk/lib/dev_compiler/web/ddc_web_compiler.js
dart-sdk/lib/dev_compiler/web/dart_stack_trace_mapper
BUG=
R=rmacnak@google.com, vsm@google.com
Review-Url: https://codereview.chromium.org/2889253002 .
Allow kernel service request to provide source files(filename, content), which will be used to build up MemoryFileSystem instance. MemoryFileSystem instance will be used instead of PhysicalFileSystem instance.
Immediate need for this is to support VM unit testing, where small Dart scripts that have to be parsed are hardcoded in the unit tests.
BUG=https://github.com/dart-lang/sdk/issues/28264R=scheglov@google.com
Review-Url: https://codereview.chromium.org/2880453005 .
Remove hack in bin/snapshot_utils.cc that avoided building a true app-jit snapshot for ARM, ARM64 and MIPS because of the multiple invocations above.
Add GN flags to choose to build script snapshots instead of app-jit snapshots, avoiding a slow training run inside a simulator.
Fixes#29580.
R=zra@google.com
Review-Url: https://codereview.chromium.org/2871713002 .
- defines a .gn target for patched_dart2js_sdk
- changes patch_sdk.dart to handle a dart2js target
- adds support in fasta to understand the `native` clause still used by dart2js
- add option to fasta to know whether it is generating kernel for the vm or dart2js
I'm not excited about how I added the target distinction in fasta, the reason I
did so is that it affects the behavior in several ways: in how we handle
`native`, in the libraries that are included by default in the platform.dill file,
and in the transformations that need to be done. I'm happy to change the
approach.
BUG=
R=ahe@google.com, vegorov@google.com, zra@google.com
Committed: 714523b103
Review-Url: https://codereview.chromium.org/2832353002 .
- defines a .gn target for patched_dart2js_sdk
- changes patch_sdk.dart to handle a dart2js target
- adds support in fasta to understand the `native` clause still used by dart2js
- add option to fasta to know whether it is generating kernel for the vm or dart2js
I'm not excited about how I added the target distinction in fasta, the reason I
did so is that it affects the behavior in several ways: in how we handle
`native`, in the libraries that are included by default in the platform.dill file,
and in the transformations that need to be done. I'm happy to change the
approach.
BUG=
R=ahe@google.com, vegorov@google.com
Review-Url: https://codereview.chromium.org/2832353002 .
This speeds up dartdevc on hello world from about 600ms to about 150ms.
OTOH, it also increases the snapshot size from ~4.6M to ~31M (the
latter is just under the size of the dart2js snapshot).
I tried this before. This may break again on windows - if so, i'll revert.
R=jakemac@google.com
Review-Url: https://codereview.chromium.org/2760043002 .
Without this, we risk creating two different libraries at runtime. One
named file:///something/ and one named package:front_end/something.
These libraries are different and if enums from one gets mixed with
the other, you get very confusing errors (to apparently identical enums
aren't equal).
R=johnniwinther@google.com
Review-Url: https://codereview.chromium.org/2726793003 .
This adds a class CanonicalName that can represent a library, class,
or member. All references now go through a Reference object, which is
linked to both the AST node and its CanonicalName, so either can be
created first.
dartk now accepts multiple input files:
- If multiple dart files are given, they are all compiled.
- If multiple binaries are given, they are linked together.
Mixed dart and binary input is not supported by dartk.
dartk now has a flag --include-sdk which includes the entire SDK in
the output. This is so the SDK can be compiled alone and then linked.
Example of compiling separately and then linking:
dartk foo.dart -o foo.dill
dartk main.dart -o main.dill
dartk --include-sdk -o sdk.dill
dartk main.dill foo.dill sdk.dill --target=vm --link -o program.dill
dartk still has incredibly slow cold start due to the analyzer loading
the dart sdk, so this does not actually speed things up at the moment.
BUG=
R=ahe@google.com, kmillikin@google.com, kustermann@google.com, sigmund@google.com
Review-Url: https://codereview.chromium.org/2665723002 .
This speeds up dartdevc on hello world from about 600ms to about 150ms.
OTOH, it also increases the snapshot size from ~4.6M to ~31M (the
latter is just under the size of the dart2js snapshot).
R=jacobr@google.com
Review-Url: https://codereview.chromium.org/2707023002 .
We are not switching yet, but now everybody can try fasta by doing:
$ dart -DDFE_USE_FASTA=true --dfe=utils/kernel-service.dart hello.dart
Note that when using fasta we expect that patched_sdk contains platform.dill
Kernel binary containing compiled patched sdk. This file can be obtained by
doing:
$ export DART_AOT_SDK=<path-to-patched_sdk>
$ dart pkg/front_end/lib/src/fasta/bin/compile_platform.dart \
${DART_AOT_SDK}/platform.dill
We are also adding --use-fasta to testing script to allow end-to-end testing
of fasta (though platform.dill file needs to be generated manually prior to
running tests):
$ tools/test.py -c dartk -m release -a x64 --nobatch --use-fasta
Current test status:
$ tools/test.py -c dartk -m release -a x64 --nobatch --use-fasta
[27:34 | 100% | +12566 | - 1941]
BUG=
R=ahe@google.com, kustermann@google.com
Review-Url: https://codereview.chromium.org/2684943003 .
Target-specific modular transformations have to be able to cope with
external libraries. Target-specific global transformations should be
optimizations, and not required for correctness, since a simple linker
may choose not to perform them.
Make mixin resolution modular by making it fail when a mixed-in class
comes from an external library. (We cannot mix in such a class because
we do not necessarily have all class members.)
R=asgerf@google.com
Review-Url: https://codereview.chromium.org/2671653003 .
- Don't mark core_isolate_snapshot_buffer as a const pointer.
- Update app snapshots without code to not rewrite the VM isolate snapshot, as already done by app snapshots with code.
Fixes#28368R=asiva@google.com
Review-Url: https://codereview.chromium.org/2637193002 .
This makes doing an SDK build in the Fuchsia tree easier.
The previous attempt at this change failed in gyp builds targeting xcode.
They failed because input lists for actions of the same target (e.g. the
actions for dartanalyzer) are merged and de-duped when the lists contain
relative paths, but merged and *not* de-duped when the lists contain
absolute paths. gyp rejects xcode files that it generated itself
when they contain duplicate entries.
This CL works around that bug, by using absolute paths in the GN build
and keeping relative paths in the gyp build.
R=rmacnak@google.com
Review-Url: https://codereview.chromium.org/2567213002 .
GN is unhappy when a rule that consumes a file generated by
another rule doesn't depend directly on that rule.
This change also quiets output from invoking Dart to follow the
rule of GN builds that a successful build generates no output.
Review URL: https://codereview.chromium.org/2446473002 .
Also update DEPS to bring the latest dev_compiler into the SDK.
This is a second attempt at
https://codereview.chromium.org/1777173003, with the following fixes:
- Added .bat files for Windows.
- Added missing dependency: create_sdk_internal depends on
dartdevc.dart.snapshot.
- Used pkg_files.stamp to note the source code dependency for
dartdevc, so that we don't go beyond xcodebuild's input length
limit.
R=jmesserly@google.com
Review URL: https://codereview.chromium.org/1818543002 .
Paul thinks that there is a bug in gyp or make, and we should generate
only one output. So, we generate the whole bundle with all summaries
and index information. And then we have separate actions that take
this bundle and extract one output each - spec.sum and strong.sum files.
R=paulberry@google.com, whesse@google.com
BUG=
Review URL: https://codereview.chromium.org/1775973003 .
We cannot commit this CL right now, because we will attempt to use
these summaries by default, and they are incomplete yet. But I wanted
to see how to add something to the build process, and we might do
changes to make using SDK summaries optional initiallyand then commit
this CL.
R=paulberry@google.com, whesse@google.com
BUG=
Review URL: https://codereview.chromium.org/1588093005 .
Once issue 6943 is fixed, the http package will start using pub's
cross-platform library support to work with both dart:io and
dart:html. However, this package is also used *within* pub, so we need
a way for pub to import it that doesn't require it being run through
pub.
This change supports this by stripping out and dart:html imports from
the http package and creating a special packages directory only used
by pub that uses this modified version.
BUG=20068
R=floitsch@google.com
Review URL: https://codereview.chromium.org//397773002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@38255 260f80e4-7a28-3924-810f-c04153c831b5
