Commit graph

161 commits

Author SHA1 Message Date
Ryan Macnak f9b7ed74dc [vm] Remove dead library references from the object store.
- dart:_builtin is only available in the standalone embedder, and should have never been referenced by the VM proper.
 - dart:profiler was long ago renamed to dart:developer.

TEST=build
Change-Id: I0743f52efc3331a8500494b2d87f621190566b62
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212680
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-09-08 16:29:19 +00:00
Martin Kustermann d1b829f791 [vm] Move main ObjectStore initialization right after bootstrapping but before loading user classes
This CL fixes AOT failures that happened after [0] landed. That code
removed lookup code for the `future_class()` in the
`ObjectStore::LazyInit*()` methods and replaced it with an assert instead.
The change was correct, since the lazy members should be initialized
on-demand - after the core, non-lazy members.
=> Though this seemed to not be the case in AOT, which hit the assert.

The reason [0] only caused failures in AOT is because in JIT the VM is
bootstrapped only from the vm_platform.dill file and as a secondary
embedder call the application.dill file is loaded.
Yet in AOT we have a combined full_application.dill which will all be
loaded during bootstrapping. That has caused the application to be
loaded before `ObjectStore::InitKnownObjects()` was invoked.

The initialization of [ObjectStore] which contains common core library
classes/functions/fields should happen before we use the kernel loader
to load user classes (since general kernel loading depends on the
[ObjectStore] being populated).

This also ensures that `ObjectStore::InitKnownObjects()` will be called
before `ObjectStore::LazyInit*()` - since they depend on the former having
run (e.g. that the non-lazy `future_class()` member of `ObjectStore`
was set).

[0] https://dart-review.googlesource.com/c/sdk/+/206782

TEST=Fixes AOT builders.

Change-Id: I0861d1a2f39effbcea08d7796742845b874a0084
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/207002
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-07-15 08:39:50 +00:00
Ryan Macnak b1c09ecd8f [vm] Make naming more consistent when converting between handles, tagged and untagged pointers.
Currently we have things called XPtr which are not what you get from ptr().

Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)

After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*

New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*

TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-01-15 23:32:02 +00:00
Martin Kustermann 0f76981bb1 [vm/concurrency] Change references to class_table/object_store/heap from Isolate to IsolateGroup
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.

The class_table and object_store were already moved to `IsolateGroup`.

This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Pure refactoring - relying on existing test coverage.

Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-06 15:22:11 +00:00
Martin Kustermann 18971e409f [vm/concurrency] Move obfuscation related state from Isolate to IsolateGroup
As part of making the compiler independent of `Isolate` we have to move
various state from `Isolate` to `IsolateGroup`.

This CL moves obfuscation related state.

TEST=Pure refactoring - relying on existing test coverage.

Issue https://github.com/dart-lang/sdk/issues/36097

Change-Id: I4b0a74bff9abc200cf8f4ce76b3e1d3d688e664a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177122
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-06 11:12:40 +00:00
Alexander Aprelev 37abbb2e3d [vm/concurrency] Introduce program_lock to guard program structure changes.
This CL adds acquisiton of write lock and check for whether write lock is held when updating class functions, primarily populated during class finalization. Read locks will be added in successive CLs, so is extending of the locks coverage to all aspects of program structure changes.

Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I3dba6bc23db4e45599a20717226210f12e7fd2b3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168140
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-10-22 13:08:10 +00:00
Ryan Macnak d4ffb92ed8 [vm] Reify loading units.
Loading unit metadata is expanded to create an object representing each loading unit, and each library to reference the loading unit it is a part of.

To be used at compile-time to help assign objects to snapshots.

To be used at runtime to track which units have been loading and the base objects for cross-snapshot references.

Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Icd3fa95ffd5949bd8f7f84a6f750c4c515de0dba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151865
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-07-09 00:15:07 +00:00
asiva a782dd82db [VM/runtime] - detect null safety before isolate initialization
- Detect null safety when not specified before isolate
      initialization
      - for source files by having CFE parse the source file
        for @dart annotations
      - for kernel files by sniffing the kernel file for
        compilation mode
      - for appJIT files by sniffing the feature string
      - for AOT snapshots by sniffing the feature string

    - Remove workaround of returning null safety to false during
      bootstrapping

    - Add a new Dart C API call for detecting null safety

Bug: 41766
Change-Id: Ia8cf264323a2d0d58c2855ce6491456aa6f1da07
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150089
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-06-11 01:22:35 +00:00
Alexander Markov fef83c96fc [vm] Report compile-time errors when loading bootstrap libraries from kernel
Previously, we were crashing when trying to report a compile-time
error via H.ReportError when loading a bootstrap library from kernel.
That happened because we didn't have LongJumpScope set up.

This change sets up LongJumpScope and makes sure compile-time
errors during kernel loading can be reported and propagated up.

This also removes the need to use FATAL error when detecting
a dill file which is incompatible to the current runtime mode,
so we can use less severe compile-time error for that.

Fixes https://github.com/dart-lang/sdk/issues/41870

Change-Id: Ib3690f7812c20e5529f3066ed37fdb9052dd559b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148683
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2020-05-20 02:14:21 +00:00
Alexander Markov 85cb6eddf4 [vm/nnbd] Auto-detect null safety mode when bootstrapping from kernel
This change also enables auto-detection in gen_snapshot.

Issue: https://github.com/dart-lang/sdk/issues/41206
Change-Id: If047ab8457e8fa15dc24ec10c0dcfceaf836508a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148152
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2020-05-14 20:49:06 +00:00
Ryan Macnak 6fe15f6df9 [vm] Represent tagged pointers as C++ value types instead of C++ pointer types.
This works around bugs in UndefinedBehaviorSanitizer and Clang.

Bug: b/28638298
Change-Id: I6be595f9664516019d28017d24559583a1ae3a21
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144354
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2020-04-25 05:21:27 +00:00
Regis Crelier cfc006a8f4 [VM/nnbd] Fix sharing of type argument vectors (fixes #40451).
Apply the same fix as in https://dart-review.googlesource.com/c/sdk/+/136709
to TypeArguments::CanShareInstantiatorTypeArguments and to TypeArguments::CanShareFunctionTypeArguments.

Move the inlined code performing an instantiation cache lookup to stubs in order to reduce code size.
Add stubs checking nullability of type arguments to share instantiator (or function) type arguments before looking up instantiation cache, and possibly calling into the runtime for instantiation.
Modify constant propagator to instantiate constant type arguments at compile time.

Change-Id: I72fee1a6881ac3bec55fae7d0ef5a3361544a141
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138009
Commit-Queue: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-03-13 22:29:47 +00:00
Victor Lima b68d95ec9e [vm] Reland support for real unboxed floating point fields in AOT
Issue https://github.com/dart-lang/sdk/issues/40404

Change-Id: Icfa801ff0640a6b27bb3c13d0b737c40452cbf7d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/133983
Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-01-31 12:56:31 +00:00
Zichang Guo 162d6c5634 Revert "[vm] Add support for real unboxed floating point fields in AOT"
This reverts commit 9eb531bde4.

Reason for revert: Bots are red. Some tests are failing.

https://ci.chromium.org/p/dart/builders/ci.sandbox/vm-kernel-precomp-obfuscate-linux-release-x64/6039

https://ci.chromium.org/p/dart/builders/ci.sandbox/vm-kernel-precomp-android-release-arm_x64/957

Original change's description:
> [vm] Add support for real unboxed floating point fields in AOT
> 
> Non-nullable floating point fields (double, Float32x4, Float64x2)
> are fully unboxed in their classes.
> 
> A bitmap for each class was added to the shared class table in order to keep
> track of the pointers of the classes. Since all classes in Flutter Gallery
> have less than 64 fields, the bitmap is represented by a 64 bit integer and
> fields whose offset is more than 64 words are not unboxed.
> 
> The instance sizes and field offsets might change between target and host
> in cross-compilation, since the number of words used to store unboxed fields
> may differ.
> 
> dart-aot Xeon
> 
>   SplayLatency               -4.62%
>   SplayHarderLatency         -4.17%
>   NavierStokes               -2.20%
>   Tracer                      8.72%
>   ParticleSystemPaint         2.90%
>   NBodySIMD                   8.35%
>   NBody                      25.59%
> 
> With hack TFA to make doubles in Rect/Offset/Size classes in flutter non-nullable:
> 
> flutter arm-v8:
> 
>   gallery total size: -1%
> 
>   matrix_utils_transform_rect_perspective   -16.70% (less is better)
>   matrix_utils_transform_rect_affine        -31.82% (less is better)
>   matrix_utils_transform_point_perspective  -24.90% (less is better)
>   matrix_utils_transform_point_affine)      -27.26% (less is better)
>   rrect_contains_bench                      -4.719% (less is better)
> 
> Change-Id: I9ae09c9c3167d99f9efd071a92937aa51093fd1d
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131824
> Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Samir Jindel <sjindel@google.com>

TBR=kustermann@google.com,rmacnak@google.com,sjindel@google.com,victoragnez@google.com

Change-Id: Ic73858f6adb7f55c4129d4f46ff4731b378cb634
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134020
Reviewed-by: Zichang Guo <zichangguo@google.com>
Commit-Queue: Zichang Guo <zichangguo@google.com>
2020-01-30 18:13:28 +00:00
Victor Lima 9eb531bde4 [vm] Add support for real unboxed floating point fields in AOT
Non-nullable floating point fields (double, Float32x4, Float64x2)
are fully unboxed in their classes.

A bitmap for each class was added to the shared class table in order to keep
track of the pointers of the classes. Since all classes in Flutter Gallery
have less than 64 fields, the bitmap is represented by a 64 bit integer and
fields whose offset is more than 64 words are not unboxed.

The instance sizes and field offsets might change between target and host
in cross-compilation, since the number of words used to store unboxed fields
may differ.

dart-aot Xeon

  SplayLatency               -4.62%
  SplayHarderLatency         -4.17%
  NavierStokes               -2.20%
  Tracer                      8.72%
  ParticleSystemPaint         2.90%
  NBodySIMD                   8.35%
  NBody                      25.59%

With hack TFA to make doubles in Rect/Offset/Size classes in flutter non-nullable:

flutter arm-v8:

  gallery total size: -1%

  matrix_utils_transform_rect_perspective   -16.70% (less is better)
  matrix_utils_transform_rect_affine        -31.82% (less is better)
  matrix_utils_transform_point_perspective  -24.90% (less is better)
  matrix_utils_transform_point_affine)      -27.26% (less is better)
  rrect_contains_bench                      -4.719% (less is better)

Change-Id: I9ae09c9c3167d99f9efd071a92937aa51093fd1d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131824
Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
2020-01-30 13:45:08 +00:00
Alexander Markov 8e63413fff [vm/bytecode] Explicitly mark _Closure._hash as nullable.
_Closure objects are created by the VM "magically", without invoking
constructor. Usually nullability of fields is calculated when compiling
a constructor, but constructor of _Closure is not compiled. With AST
nullability is also derived from field initializers, but this is not
the case in bytecode mode (as there are no initializers for instance
fields unless they are needed for hot reload).

Fixes language_2/bound_closure_equality_test with bytecode
and --optimization-counter-threshold=5.

Change-Id: If86a276c875f798af8511038326714e66e212cc1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/115300
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2019-09-04 00:08:58 +00:00
Alexander Markov 7b50b7a8e6 [vm/bytecode] Bootstrapping VM from bytecode
Previously, core snapshot was generated from AST
(because --enable-interpreter/--use-bytecode-compiler was not specified
when building core snapshot).

As the result, CL
https://dart.googlesource.com/sdk/+/da8cb470cc94830a98d49532e8d5d1a5b3d80f8b
which declared libraries in bytecode also removed bytecode entirely from
core snapshot in Dart SDK.

This CL enables bytecode by default if --bytecode argument is
specified for gn.py. This enables JIT compiler from bytecode
(interpreter is still disabled by default but can be enabled
using --enable-interpreter). Core snapshot and other
snapshots now have bytecode.

This change revealed a bunch of bugs which are fixed in this CL:

* _Closure fields were treated as unboxing candidates which triggered
  assertion in LoadFieldTOS in interpreter.

* Several places should load class declarations if they are not loaded yet.

* Canonicalization of TypeRef objects which are not fully initialized
  may cause duplicate entries in the hash table of canonical
  TypeArguments. This triggers assertions when hash table is rehashed.
  The solution is to avoid canonicalization of non-root recursive types
  and recursive type arguments. Also, TypeRef::Canonicalize and
  TypeRef::Hash are reverted to assert and work only if type was set.

* Native wrapper classes are eagerly stamped as type-finalized
  which caused assertion failures when reading their class declarations
  from bytecode.

* When building flow graph for FFI trampolines kernel offset of library
  (which is now declared in bytecode) was queried. Added special case
  to Function::KernelDataProgramOffset().

* In interpreter-only mode with simulator (e.g. SIMARM64) if simulator
  is not called before code is interrupted with stack overflow check,
  simulator returns get_sp() = 0, which was treated as stack overflow.

* test standalone_2/io/platform_resolved_executable_test.dart
  spawns sub-process but it didn't pass VM options.

Change-Id: I81bc4f1a4c6725cfa246a435ebe5d8abe43abc67
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107199
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2019-06-26 18:25:26 +00:00
Matthew Dempsky 929c45afc7 [vm] Use std::unique_ptr for kernel::Program
Updates #37244.

Change-Id: I994298162966eca9cf5217cad2ae0aae4f438b4c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/106950
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Matthew Dempsky <mdempsky@google.com>
2019-06-21 19:41:08 +00:00
Alexander Markov da8cb470cc [vm/bytecode] Represent classes, libraries, scripts and recursive types in bytecode
Size of a large application:
Before: 26628600
After: 21480120 (-19.3%)

Size of snapshots:

isolate_snapshot_framework.bin
Before: 9322496
After: 6782976 (-27.2%)

isolate_snapshot_product_framework.bin
Before: 9166848
After: 6602752 (-27.9%)

Regressions in tests:

1) Test language_2/type_alias_equality_test/04 fails similarly to default mode, as VM
does not implement comparison of function types according to the specification.
Previously this test was passing as function types were canonicalized in bytecode,
which was not always correct. This CL fixes the problem with canonicalization of
function types in bytecode and the test starts failing again.

2) Tests standalone_2/entrypoints_verification_test, standalone_2/io/test_extension_test,
standalone_2/io/test_extension_fail_test fail as native extensions are not supported
in bytecode yet. These tests start passing after df5e7aac17,
which switched bytecode tests to kernel service (on x64), because kernel service doesn't
drop ASTs. This CL switches from reading AST library declarations to bytecode even if
AST is not removed, so tests fail again.

Change-Id: I8b7ba44bfa49d0b1599b2509553ff7c831a4e244
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/104700
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2019-06-20 18:27:21 +00:00
Alexander Markov 0e6b74543c Revert "[vm/kernel] Use GC-tracked ExternalTypedData/TypedDataView for kernel buffers"
This reverts commit ab6aeaa106.

Revert "[vm/compiler] Speed up the compiler part which deals with kernel reading up in DEBUG mode"

This reverts commit b316210d94.

Reason for revert: regression of snapshot sizes (DNO-599).

Original change's description:
> [vm/kernel] Use GC-tracked ExternalTypedData/TypedDataView for kernel buffers
>
> Until now we often leaked kernel buffers (e.g. hot reload buffers) because various
> objects were referencing ExternalTypedData objects pointing into the middle of
> c-allocated memory. This made it impossible for the GC to determine when the last
> reference is gone.
>
> This CL ensures that the actual buffers are *always* made available via
> ExternalTypedData and any inner pointers into it are created via TypedDataViews.
>
> The embedder guarantees to the free kernel buffers it has provided to:
>     - Dart_CreateIsolateFromKernel
>     - Dart_LoadScriptFromKernel
>     - Dart_LoadLibraryFromKernel
>     - Dart_SetDartLibrarySourcesKernel
> on isolate shutdown.
>
> All other kernel buffers will get a finalizer attached, which ensures the
> kernel buffers get freed by the GC once they are no longer referenced:
>     - Kernel blobs for expression evaluation
>     - Kernel blobs for Hot-Reload
>     - Kernel blobs for cc tests
>
> Fixes https://github.com/dart-lang/sdk/issues/33973
> Fixes https://github.com/dart-lang/sdk/issues/36857
> Issue https://github.com/dart-lang/sdk/issues/37030
>
> Change-Id: I1cc410c94c0f4b229413e793728a261afcb10aaf
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103130
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Martin Kustermann <kustermann@google.com>

TBR=kustermann@google.com,rmacnak@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: I49715d2400f4a5c8806b7d6a2912b7258f671a0a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/104343
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
2019-05-31 22:15:51 +00:00
Martin Kustermann ab6aeaa106 [vm/kernel] Use GC-tracked ExternalTypedData/TypedDataView for kernel buffers
Until now we often leaked kernel buffers (e.g. hot reload buffers) because various
objects were referencing ExternalTypedData objects pointing into the middle of
c-allocated memory. This made it impossible for the GC to determine when the last
reference is gone.

This CL ensures that the actual buffers are *always* made available via
ExternalTypedData and any inner pointers into it are created via TypedDataViews.

The embedder guarantees to the free kernel buffers it has provided to:
    - Dart_CreateIsolateFromKernel
    - Dart_LoadScriptFromKernel
    - Dart_LoadLibraryFromKernel
    - Dart_SetDartLibrarySourcesKernel
on isolate shutdown.

All other kernel buffers will get a finalizer attached, which ensures the
kernel buffers get freed by the GC once they are no longer referenced:
    - Kernel blobs for expression evaluation
    - Kernel blobs for Hot-Reload
    - Kernel blobs for cc tests

Fixes https://github.com/dart-lang/sdk/issues/33973
Fixes https://github.com/dart-lang/sdk/issues/36857
Issue https://github.com/dart-lang/sdk/issues/37030

Change-Id: I1cc410c94c0f4b229413e793728a261afcb10aaf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103130
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2019-05-22 01:23:46 +00:00
Jens Johansen ef168de000 [vm/kernel] Index all source and line starts before using them (2nd try)
By indexing all sources in a concatenated dill file and then using that
index to find the sources and line starts we can fix missing information
(leading to crashes when collecting coverage) in for instance circular
instances.

This reverts commit 57321c1590 and adds a
fix.

Change-Id: I7a91801fb318cad0218cf3101d6a15f1ec929175
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98006
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2019-03-28 08:36:51 +00:00
Keerti Parthasarathy 57321c1590 Revert "[vm/kernel] Index all source and line starts before using them"
This reverts commit a7ab99dd84.

Reason for revert: <INSERT REASONING HERE>

Original change's description:
> [vm/kernel] Index all source and line starts before using them
> 
> By indexing all sources in a concatenated dill file and then using that
> index to find the sources and line starts we can fix missing information
> (leading to crashes when collecting coverage) in for instance circular
> instances.
> 
> Change-Id: I454cff57f84f21de72f6486e1cecf15d1389a498
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96901
> Commit-Queue: Jens Johansen <jensj@google.com>
> Reviewed-by: Vyacheslav Egorov <vegorov@google.com>

TBR=vegorov@google.com,jensj@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: Iea75c9dd56e74fa0e98c90ea6841ed66b39d40cc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97941
Reviewed-by: Keerti Parthasarathy <keertip@google.com>
Commit-Queue: Keerti Parthasarathy <keertip@google.com>
2019-03-26 22:13:10 +00:00
Jens Johansen a7ab99dd84 [vm/kernel] Index all source and line starts before using them
By indexing all sources in a concatenated dill file and then using that
index to find the sources and line starts we can fix missing information
(leading to crashes when collecting coverage) in for instance circular
instances.

Change-Id: I454cff57f84f21de72f6486e1cecf15d1389a498
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96901
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2019-03-21 12:05:15 +00:00
asiva 2427a69e14 [VM/Runtime] Refactoring sources to enable interpreter only builds.
The interpreter only build of the Dart VM should not be including any
sources from the 'compiler' directory.
First part of restructuring sources to enable an interpreter only build
of the VM
- Move CompileClass from compiler.cc file to class_finalizer.cc

Change-Id: I8743c6537ed2d63e6991d78949f858a1ae506121
Reviewed-on: https://dart-review.googlesource.com/c/87987
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2019-01-07 18:49:17 +00:00
Samir Jindel e23acaaf90 [vm/aot] Fix obfuscation.
Test Plan:

Make vm-kernel-precomp-obfuscate-linux-release-x64-try green, ensuring status file changes
reflect design gaps and not bugs.

Tested locally that obfuscated Flutter Gallery runs.

Change-Id: Ifcdc334de58f43c310e15b58dbcf6fe1597206f2
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-obfuscate-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/81009
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-10-26 13:00:33 +00:00
asiva 30aa464f77 [VM] Remove 'corelib' and 'dart-io' sources generation into C++ arrays
- Remove all code that generates the corelib and dart:io library sources into C++ arrays and link them into dart_bootstrap
- Remove the executable dart_bootstrap and all uses of it
- Remove bootstrap_nolib.cc and builtin_nocore.cc
- Remove the Dart 1 code in core lib bootstrap path

Change-Id: Ifd33496204285a08b42fe09e39428e7a92b416b6
Reviewed-on: https://dart-review.googlesource.com/c/77241
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2018-10-02 23:27:05 +00:00
Jens Johansen f67308a983 Handle invalid dill files more gracefully
Instead of bailing out with FATAL et al (which shuts down the VM),
give an ApiError when the dill file is invalid.

Bug: #33577

Change-Id: I8354b4e68ee95e36584284fd15b3abab3c3bf980
Reviewed-on: https://dart-review.googlesource.com/61937
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-06-26 11:16:35 +00:00
Ryan Macnak fa7c38dec5 [vm, standalone] Do not copy when loading kernel.
In a topaz release build with JIT apps,
mondrian(flutter_runner):      PSS 44.4M -> 35.8M (-19.4%)
ledger_dashboard(dart_runner): PSS 15.2M -> 12.0M (-21.1%)

Change-Id: I501c24de56a5a7c65a2f03b87709cb09bb95d465
Reviewed-on: https://dart-review.googlesource.com/60202
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2018-06-21 23:27:53 +00:00
Samir Jindel 39c93a2b19 Revert "Revert "[vm/kernel] Expression execution through Kernel."
This reverts commit b5154dd01e.

Original revision is in patchset 1.

Change-Id: Ic0776011c6fdbd64c025086535351474a9a88b6e
Reviewed-on: https://dart-review.googlesource.com/54407
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2018-05-15 12:03:22 +00:00
Ryan Macnak a5c11d7d03 [vm] Remove Dart_ReadKernelBinary, take 2.
Update status for simulators, which the test harness invokes with DIL files instead of source files.

Change-Id: I8444ad7e17a0a71a1ce3c0021487397baa1c3e65
Reviewed-on: https://dart-review.googlesource.com/54622
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2018-05-14 20:58:08 +00:00
Ryan Macnak 407872b9e5 Revert "[vm] Remove Dart_ReadKernelBinary."
This reverts commit 2998c32951.

Reason for revert: Failures with kernel + simulators

Original change's description:
> [vm] Remove Dart_ReadKernelBinary.
> 
>  - Fix const-ness of Dart_FileReadCallback and callers.
>  - Fix leak on read error in DartUtils::FileRead.
> 
> This is progress towards sharing kernel memory on Fuchsia.
> 
> Bug: https://github.com/dart-lang/sdk/issues/32618
> Change-Id: I47f8b224905d6a105a5ca0ab2ee4ab6a42b5e342
> Reviewed-on: https://dart-review.googlesource.com/47102
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>

TBR=aam@google.com,rmacnak@google.com,asiva@google.com

Change-Id: Ibb636b817de1629a062fe0a6ef9462a2fef669ef
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/32618
Reviewed-on: https://dart-review.googlesource.com/54560
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2018-05-10 01:36:39 +00:00
Ryan Macnak 2998c32951 [vm] Remove Dart_ReadKernelBinary.
- Fix const-ness of Dart_FileReadCallback and callers.
 - Fix leak on read error in DartUtils::FileRead.

This is progress towards sharing kernel memory on Fuchsia.

Bug: https://github.com/dart-lang/sdk/issues/32618
Change-Id: I47f8b224905d6a105a5ca0ab2ee4ab6a42b5e342
Reviewed-on: https://dart-review.googlesource.com/47102
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-05-10 00:35:37 +00:00
Samir Jindel b5154dd01e Revert "[vm/kernel] Expression execution through Kernel.
This reverts commit afd0a52a5a.

Reason for revert: Breaks kernel-a32 and kernel-rbc bots

Original change's description:
> [vm/kernel] Expression execution through Kernel.
> 
> Change-Id: Id90c91cb3b71f9ce703e21fe48147b86a7764d34
> Reviewed-on: https://dart-review.googlesource.com/42562
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Alexander Aprelev <aam@google.com>
> Commit-Queue: Samir Jindel <sjindel@google.com>

TBR=aam@google.com,rmacnak@google.com,sjindel@google.com

Change-Id: If1ecad6145aeab81d5d3581798a628ead0265c38
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/54420
Reviewed-by: Samir Jindel <sjindel@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
2018-05-09 12:41:00 +00:00
Samir Jindel afd0a52a5a [vm/kernel] Expression execution through Kernel.
Change-Id: Id90c91cb3b71f9ce703e21fe48147b86a7764d34
Reviewed-on: https://dart-review.googlesource.com/42562
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
2018-05-09 10:27:52 +00:00
Samir Jindel 8aa36b75c5 Re-land "[vm/kernel] Fix non-canoncialized type arguments in partial instantiation."
Original revision is in Patchset 1.

Run against failing kernel-precomp tests in "cl-linux-try".

Change-Id: I997de294150ef7dd0874eeccb8b6187ae64ea813
Reviewed-on: https://dart-review.googlesource.com/51220
Reviewed-by: Erik Corry <erikcorry@google.com>
2018-04-18 13:44:25 +00:00
Martin Kustermann b52931ff30 Revert "[vm/kernel] Fix non-canoncialized type arguments in partial instantiation."
This reverts commit bd12c869ae.

Reason for revert: Made many dartkp builders result in crashes.

Change-Id: I908f3929bf3a0227fcc3979b8ba134b1904c8e1c
Reviewed-on: https://dart-review.googlesource.com/51140
Reviewed-by: Régis Crelier <regis@google.com>
2018-04-13 16:39:20 +00:00
Samir Jindel bd12c869ae [vm/kernel] Fix non-canoncialized type arguments in partial instantiation.
Also lay the foundation for partial instantiation of regular closures.

Change-Id: If1c0afd9291d2f9202d8287ebc3ec563679eb06f
Reviewed-on: https://dart-review.googlesource.com/50182
Reviewed-by: Régis Crelier <regis@google.com>
2018-04-13 13:24:17 +00:00
Vyacheslav Egorov 73ecd4c7b0 [vm] Don't disregard KernelLoader::LoadProgram result when bootstrapping
It might be an error, in which case Isolate is left in half initialized
state.

Change-Id: I011ce2b4bbbe5a84f6579eb6d4d22ea5c1afbd3a
Reviewed-on: https://dart-review.googlesource.com/27801
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2017-12-08 21:51:40 +00:00
Siva Annamalai c29b19fb34 - Do not generate a separate vmservice kernel file, include it in vm_platform file
- Load vmservice_io library only in the service isolate or when an AOT snapshot is being generated

This should account for the following issues:
- https://github.com/dart-lang/sdk/issues/31203 - there should be a vmservice_io_strong.dill file
- https://github.com/dart-lang/sdk/issues/30158 - Consider extracting flutter vmservice_sky into
                                                  separate .dill from platform.dill
- https://github.com/dart-lang/sdk/issues/30111 - support compiling vmservice with an external
                                                  outline of the sdk

Bug:
Change-Id: If945ec05afe8127e288e8d3b55172c0f7c3d6e7a
Reviewed-on: https://dart-review.googlesource.com/18147
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Siva Chandra <sivachandra@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2017-11-09 19:05:15 +00:00
Siva Chandra 9f0175059e Load class members lazily when loading from kernel.
This change brings down core snapshot size by ~750KB, and brings
down app-jit snapshot size of simple "Hello, World" dart script by
~650KB. The bot cycle times will also come down by around ~20%.

Change-Id: I2a01c98bedc7ebfa2a653983995486a71504daf3
Reviewed-on: https://dart-review.googlesource.com/16323
Commit-Queue: Siva Chandra <sivachandra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2017-10-31 19:19:25 +00:00
Alexander Aprelev d822bdb797 Prefix scripts from core builtin libraries so the URIs are unique.
After https://dart-review.googlesource.com/c/sdk/+/7588 we ended up with two crypto.dart files in the core prebuilt libraries(io and _http), which caused problem with coverage tool that relied on the fact that script uris can uniquely identify files. https://github.com/dart-lang/coverage/issues/194 was filed to handle non-unique uris.
This CL ensures that core builtin libraries URIs stays unique by prefixing them with 'dart:' library prefix. This makes core builtin libraries scripts have names similarly structured to script names for other core libraries.

Bug:
Change-Id: I79960f4f24e6e958836df866365355584c28df27
Reviewed-on: https://dart-review.googlesource.com/11140
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2017-10-05 15:47:43 +00:00
Jens Johansen dc5df933e4 Use kernel indexes to jump past procedure bodies.
Before this CL we skipped procedure bodies in kernel_loader.cc by
parsing the body (but not storing anything).
With this CL we now skip them directly (i.e. don't read them at all)
in kernel_loader.cc by using the newly available extra indexes in kernel.

Change-Id: I48cf0599b2a85102c9008ff7c455785151ef3c9c
Reviewed-on: https://dart-review.googlesource.com/5764
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
2017-09-27 06:19:35 +00:00
Vyacheslav Egorov 8a179fb953 [VM, Compiler] Move compiler to a separate folder.
New folder structure (nested under vm/):

- compiler/
-   jit/         - JIT specific code
-   aot/         - AOT specific code
-   backend/     - all middle-end and back-end code (IL, flow graph)
-   assembler/   - assemblers and disassemblers
-   frontend/    - front ends (AST -> IL, Kernel -> IL)

compiler/README.md would be the documentation root for the compiler
pipeline

Bug: https://github.com/dart-lang/sdk/issues/30575
Change-Id: I2dfd9688793bff737f7632ddc77fca766875ce36
Reviewed-on: https://dart-review.googlesource.com/2940
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2017-09-04 15:15:18 +00:00
Beeravolu Siva Chandra Reddy 245be6c74c Rename the class "KernelReader" to "KernelLoader".
There already exists a class named "Reader" in the "kernel" namespace
which reads bytes from a kernel binary. The class KernelLoader
(KernelReader before this change) creates VM heap objects and populates
them.

R=kmillikin@google.com

Review-Url: https://codereview.chromium.org/3010543002 .
2017-08-29 10:59:18 +02:00
Kevin Millikin a2712fc099 Do not qualify with dart:: except where necessary
The Kernel C++ implementation no longer has classes like Library,
Class, Field, or String, so it's no longer necessary to explicitly
write `dart::` in most places.

BUG=
R=jensj@google.com

Review-Url: https://codereview.chromium.org/3001103002 .
2017-08-18 11:28:34 +02:00
Alexander Markov 203955bc48 Reapply "Improve hashCode for closure objects" with fixes.
This CL includes the following fixes:
* Fix for incorrect non-nullable assumption about _Closure._hash field.
* Add error handling into BecomeMapTraits::Hash.
* Correct assertions for validating layout of Closure objects.
* Add identityHashCode to the list of VM entry points in precompiler.

Closes #30211.

Original code review:

https://codereview.chromium.org/2983823002/

Original CL description:

This performance improvement is inspired by Flutter listeners stored in
the HashSet (see ObserverList) and frequently checked using
HashSet.contains(). If there are many such listeners and they are
implicit instance closures (for example, created by
'new Listenable.merge(...)'), HashSet.contains() becomes very slow.
It spends a lot of time in Closure_equals native method due to hash
collisions between closure objects with same function
but different receivers.

This CL improves hashCode() calculation for implicit instance closures
by mixing function hashcode with identity hashcode of the receiver.
For explicit closures and static implicit closures hashCode() is
improved by using identityHashCode() of a closure object.

Also, hashcode is calculated once and cached in each closure instance.
The size of a closure instance doesn't grow up because there was unused
word-size padding both on 32-bit and 64-bit architectures.

The execution time of the following micro-benchmark is reduced from
47665ms to 135ms on my Linux/x64 box.

-------------------------------------

import "dart:collection";

class Foo {
  int _a;
  Foo(this._a);
  void bar() {}
}

main() {
  HashSet hs = new HashSet();
  for (int i = 0; i < 1000; ++i) {
    hs.add(new Foo(i).bar);
  }

  var watch = new Stopwatch()..start();

  for (int i = 0; i < 1000; ++i) {
    for (var c in hs) {
      hs.contains(c);
    }
  }

  int time = watch.elapsedMilliseconds;
  print("Time: ${time}ms\n");
}

-------------------------------------

R=zra@google.com

Review-Url: https://codereview.chromium.org/2988493002 .
2017-07-20 15:22:18 -07:00
Alexander Markov 51d580d8bf Revert "Improve hashCode for closure objects"
This reverts commit b1197eb714.

Revert "Correct assertions for validating layout of Closure objects"

This reverts commit b1215faa59.

Revert "Add identityHashCode to the list of VM entry points in precompiler"

This reverts commit e1d04cc242.

R=asiva@google.com

Review-Url: https://codereview.chromium.org/2983973002 .
2017-07-20 08:34:28 -07:00
Alexander Markov b1197eb714 Improve hashCode for closure objects
This performance improvement is inspired by Flutter listeners stored in
the HashSet (see ObserverList) and frequently checked using
HashSet.contains(). If there are many such listeners and they are
implicit instance closures (for example, created by
'new Listenable.merge(...)'), HashSet.contains() becomes very slow.
It spends a lot of time in Closure_equals native method due to hash
collisions between closure objects with same function
but different receivers.

This CL improves hashCode() calculation for implicit instance closures
by mixing function hashcode with identity hashcode of the receiver.
For explicit closures and static implicit closures hashCode() is
improved by using identityHashCode() of a closure object.

Also, hashcode is calculated once and cached in each closure instance.
The size of a closure instance doesn't grow up because there was unused
word-size padding both on 32-bit and 64-bit architectures.

The execution time of the following micro-benchmark is reduced from
47665ms to 135ms on my Linux/x64 box.

-------------------------------------

import "dart:collection";

class Foo {
  int _a;
  Foo(this._a);
  void bar() {}
}

main() {
  HashSet hs = new HashSet();
  for (int i = 0; i < 1000; ++i) {
    hs.add(new Foo(i).bar);
  }

  var watch = new Stopwatch()..start();

  for (int i = 0; i < 1000; ++i) {
    for (var c in hs) {
      hs.contains(c);
    }
  }

  int time = watch.elapsedMilliseconds;
  print("Time: ${time}ms\n");
}

-------------------------------------

R=rmacnak@google.com, zra@google.com

Review-Url: https://codereview.chromium.org/2983823002 .
2017-07-19 14:36:42 -07:00
Zachary Anderson 6cd8a79078 VM: Re-format to use at most one newline between functions
R=asiva@google.com

Review-Url: https://codereview.chromium.org/2974233002 .
2017-07-13 08:08:37 -07:00