Commit graph

117 commits

Author SHA1 Message Date
Vyacheslav Egorov c704e11ee2 [vm/api] Introduce Dart_WriteHeapSnapshot API
Allows embedders to programmatically request heap snapshot from a
running isolate group instead of relying on a vm-service or hidden
internal Dart APIs

Additionally we allow to include snapshot writing functionality into
PRODUCT builds using DART_ENABLE_HEAP_SNAPSHOT_WRITER define.

TEST=vm/cc/DartAPI_WriteHeapSnapshot

Bug: b/259115846
Change-Id: Ic3ef76e5fb9adcf8f23a1959f5238742b64ecde2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/273181
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2022-12-01 14:38:44 +00:00
Ryan Macnak c29f994023 [vm] Removed dead parameter from VisitPointersPrecise.
TEST=ci
Change-Id: I7ca38b2ebb659cdf1b20cc7ab7bed9b0958e6330
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/269980
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-11-15 23:47:45 +00:00
Ryan Macnak a2de36e708 [vm] Rename the default implementation classes of Map and Set.
_InternalLinkedHashMap => _Map
_InternalImmutableLinkedHashMap => _ConstMap
_InternalLinkedHashSet => _Set
_InternalImmutableLinkedHashSet => _ConstSet

This makes things nicer to read in places that display implementation names, such as stack traces, debuggers, profilers and inspectors.

TEST=ci
Change-Id: Iec851c80ea2086cbe79934565dbf35f04809a836
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/266303
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
2022-11-15 17:15:58 +00:00
Ryan Macnak 4ee2a6bc82 [vm, service] Include "names" for type vectors in heap snapshots.
TEST=ci
Change-Id: I19ac8b75eab48e99c57e6e1c3655ce0e3a2430c0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/265522
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2022-11-11 21:45:05 +00:00
Tess Strickland 9ad1e36b17 [vm] Visit slots in objects with unboxed fields in Pass1Visitor.
In the heap snapshot writer, Pass1Visitor assigns object ids both to
heap objects and to Smis, and Pass2Visitor uses that information when
writing the heap snapshot. However, their logic for visiting slots in an
object with unboxed fields varies slightly. Pass1Visitor does not visit
each individual slot, but Pass2Visitor does. This means that SmiPtrs
that are only reachable via an object with unboxed fields are not added
to the writer in Pass1Visitor, and thus have an object id of 0 during
Pass2Visitor. This CL fixes this difference in visiting behavior.

TEST=vm/dart_2/heap_snapshot_regress_49711_test
     vm/dart_2/heap_snapshot_test

Fixes: https://github.com/dart-lang/sdk/issues/50341
Change-Id: Id238771af18ee098f39e6d1cc80e021dc8e117fe
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/267580
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-11-03 17:50:58 +00:00
Ryan Macnak a99eb47cf6 [vm, service] Fix compressed pointers build.
TEST=ci
Change-Id: I86c7e478f2ea1b1607516dc1a23f7321d3ed66dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/265861
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-10-26 18:48:07 +00:00
Ryan Macnak 2d6037a144 [vm, service] Include Smis in heap snapshots.
TEST=ci
Change-Id: Iadb3bb749da0d065c46bc723c7f5a8a8951b68bd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/265562
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-10-26 16:04:29 +00:00
Ryan Macnak efbf68f306 [vm, gc] Common page structure for both generations.
- Prep work for non-moving promotion.
 - The generations still have different alignment offsets.
 - Old-space pages now participate in the page cache.
 - Fix boolean/null bit tricks to assert the right requirements on the alignment of a page's first object.

TEST=ci
Change-Id: I4369d8c6af73228e162c226d411914868bafed33
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/260401
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-09-27 21:34:05 +00:00
Vyacheslav Egorov 83ab5d5ca3 [vm] Remove dynamic field unboxing in JIT
Instead apply the same approach as we do in AOT: unbox based on the
static type information. There are no TFA results available in JIT,
but we could still unbox fields when running in sound null-safety.

TEST=ci

Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try
Change-Id: Ide2e78c6659261ef8d245a4586cf699ea0fbb459
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256211
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2022-09-20 13:42:45 +00:00
Alexander Aprelev 992672b549 [gardening] Fix gcc bot - add suggested parenthesis around '&&' within '||'
Broken gcc bot https://ci.chromium.org/ui/p/dart/builders/ci.sandbox/vm-kernel-gcc-linux/366/overview

TEST=gcc bot

Change-Id: If93047686597d1e822b213a765ee3679f9d0d8b2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256219
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2022-08-24 17:34:31 +00:00
Martin Kustermann 3669086a40 [vm] Fix field index information in heap snapshots in AOT
In AOT we can have unboxed fields that will not be visited by GC
visitors and as such are also not reported as <obj>.references when
producing heap snapshots.

Though the heapsnapshot's <class>.fields will contain entries
for unboxed fields, which causes the fields index calculation to be
incorrect.

This can cause various confusing things, e.g. incorrectly caluclated
retaining paths, since the <class>.fields.index doesn't match the
index in <obj>.references.

To fix this this CL will change the visiting code to emit dummy
references for unboxed fields.

The written test uncovered a few other issues:

  - report type_arguments and native_fields in <class>.fields
    => the visitors already visit those fields

  - fix `Class::next_{host,target}_field_offset` for classes
    with native fields
    => this will also shrink size of `NativeWrapperClass` subclasses

  - ensure classes with vm-defined layout are finalized
    => we assume objects should only exist in heap if their class
       is finalized

  - fix `FfiNativeFunction` type_argument offset

  - fixes for object fields list
    => add missing entries, fix existing entries

Fixes https://github.com/dart-lang/sdk/issues/49711

TEST=vm/dart{,_2}/heap_snapshot_regress_49711_test

Change-Id: I8e25680b2c0c8c49caafbb8da57b0b6419e89c4f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/255814
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2022-08-23 05:17:49 +00:00
Martin Kustermann d68ca2cc57 [vm] Avoid repeating fields in heapsnapshot class descriptions
Fixes https://github.com/dart-lang/sdk/issues/49710

TEST=tests/vm/dart{,_2}/heap_snapshot_regress_49710_test.dart

Change-Id: Ib2c31a582381dcfaf12739b9d0c7e28d98063791
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/255813
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2022-08-19 16:51:45 +00:00
Martin Kustermann 4539bf6584 [vm] Make heap snapshot writer also visit isolate stacks as roots
Heap snapshots currently produced don't visit isolate stacks. As such
analyzing such snapshots may lead one to conclude there is a lot of
garbage while objects are actually reachable.

=> This CL makes us visit isolate stacks when building heap snapshots.

Furthermore we add a new `VMInternals.writeHeapSnapshotToFile` helper
that can be used to programmatically write snapshots and can be handy
for internal use at times. (We also use this helper in a test)

TEST=vm/dart{,_2}/heap_snapshot_test

Change-Id: I976544b7f6d20863764af9a40bf1ffb3c319bbce
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/253785
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2022-08-06 11:24:28 +00:00
Ryan Macnak 1e4a611ca4 [vm, service] Create edges from the root to objects on read-only pages in heap snapshots.
This prevents objects, such as InstructionSection, that have no ordinary references from being labeled as "Uncollected Garbage".

TEST=examine heap snapshot in an AOT build with bare-instructions enabled
Bug: b/200288075
Change-Id: If576d7f9eabb80afaca1b600507c3a9629beeafd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/215006
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-09-30 16:50:58 +00:00
Daco Harkes e8e9e1d152 [vm] Introduce immutable maps and sets in backend
This CL introduces immutable maps and sets in the VM backend but does
not yet target them from the frontend. The changes are tested by unit
tests constructing these immutable maps and sets.

This CL introduces immutable variants of the hash map and set in
compact_hash.dart and recognizes them in the VM. The immutable ones
use a different mixin with a different recognized method for accessing
members.
* Data list is an immutable list with a different cid. (Otherwise the
  optimizer notices that immutable and mutable lists cannot be equal.)
* Index is a nullable mutable typed data. (Otherwise optimizer removes
  necessary null checks.)
* Index should use a store-release barrier when written to.

Multiple isolates might lazily compute the index for const sets and
maps. This is fine because all identityHashCodes and hashCodes are
guaranteed to be race-free. The later isolates will override the index
pointer with an identical index.

This CL does not introduce support for using these immutable maps and
sets in AOT (clustered_snapshot) and in messages to other isolates
(message_snapshot) because that is harder to test with unit tests. That
will be added in the follow-up CL.

Design doc: go/dart-vm-const-maps

TEST=runtime/vm/object_test.cc

Bug: https://github.com/dart-lang/sdk/issues/45908

Change-Id: I4042179c15e8b37692d3255655351c01c7124991
Cq-Include-Trybots: luci.dart.try:analyzer-nnbd-linux-release-try,app-kernel-linux-debug-x64-try,dart-sdk-linux-try,front-end-nnbd-linux-release-x64-try,pkg-linux-debug-try,vm-canary-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-checked-linux-release-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-simarm64c-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210860
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-09-06 06:28:59 +00:00
Ryan Macnak d9048340ba [vm, service] Include each isolate a separate synthetic object in heap snapshots.
Fix labels for sentinel, transition_sentinel, ImmutableArray, LinkedHashMap and LinkedHashSet.

Display unlimited children when look at successors, as the worst case is the size of the field table. For predecessors and dominators, the worst case is the whole heap.

TEST=ci
Change-Id: I6fcfdfb0833d58c9ac4f586b823244d817aeba27
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/209841
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-08-17 16:49:32 +00:00
Tess Strickland cd6fb67103 [vm] Clean up changes around internal-only class ids.
* IsInternalOnlyId -> IsInternalOnlyClassId

* Removes constexpr predicate ClassComesBeforeOtherInternalOnlyClasses()
  used in only two locations and inlines the returned condition.

* Change IsErrorClassId to check <= kUnwindErrorCid instead of
  <= kLastInternalOnlyCid.

TEST=Refactoring, so existing tests.

Change-Id: Ib7dfc89aba16d52733de05687f0ca7055c16e7a3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210126
Auto-Submit: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-08-13 20:00:41 +00:00
Tess Strickland e8ddc0219f [vm] Abstract out internal-only cid ordering.
* Adds IsInternalOnlyCid, which replaces the old `cid < kInstanceCid`
  check.

* Adds kFirstInternalOnlyCid and kLastInternalOnlyCid, which should be
  used in places that loop over the internal-only cids.

* Adds ObjectComesBeforeOtherInternalOnlyClasses(), which can be used
  with COMPILE_ASSERT in places that assume kObjectCid immediately
  precedes the other internal-only cids.

* Replaces assumption in IsErrorId() that Instance immediately follows
  the internal-only cids with an assumption that the current last error
  cid is the same as kLastInternalOnlyCid.

* Separates out CLASS_LIST_NO_OBJECT_NOR_STRING_... into two pieces to
  make it clearer where to add new internal-only cids and where to add
  new Instance cids:

    * CLASS_LIST_INTERNAL_ONLY, which contains class ids for real
      classes that are VM internal, except for Object.

    * CLASS_LIST_INSTANCE_SINGLETONS, which contains class ids for
      subclasses of Instance that are not part of a separately handled
      group of cids.

TEST=Refactoring, so existing tests.

Change-Id: I9905d5928a592164d58de1f121b9ce669f2b3ce1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210122
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-08-13 13:27:41 +00:00
Ryan Macnak 2642c7c9ce [vm] Fix gcc build.
TEST=local build
Change-Id: I9503dfbb1b9ba6ef2b5fb8423c13d5066e5af98b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206140
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-07-13 01:55:45 +00:00
Daco Harkes 483179c934 [vm] Recognize non-const Set in the VM
`_CompactLinkedHashSet` now extends `_HashVMBase`. The class hierarchy
is organized as mixins similar to LinkedHashMap to accomodate for the
other Sets not extending `_HashVMBase`.

Also, rearranges some code so that introducing ImmutableHashMap and
ImmutableHashSet is easier.
1) snapshot.h and snapshot.cc now have a MapReadFrom, MapWriteTo,
   SetReadFrom, and SetWriteTo to facilitate code sharing between
   mutable and immutable implementations similar to ArrayReadFrom and
   ArrayWriteTo.
2) Macros for CLASS_LIST_MAPS and CLASS_LIST_SETS to facilitate
   treating mutable and immutable implementations with the same handle.
   Also similar to Array.

Clustered snapshots for HashMaps is currently dead code. This CL makes
it explicit by marking these as unreachable. Immutable maps and sets
will end up in the clustered snapshot in follow up CLs.

Bug: https://github.com/dart-lang/sdk/issues/36077
Bug: https://github.com/dart-lang/sdk/issues/45908

TEST=runtime/vm/object_test.cc
TEST=tests/**_test.dart on many bots

Change-Id: If3cc5ebb3138535aeb0d5e06d9da3d1c9fb2deb2
Cq-Include-Trybots: luci.dart.try:analyzer-nnbd-linux-release-try,app-kernel-linux-debug-x64-try,dart-sdk-linux-try,front-end-nnbd-linux-release-x64-try,pkg-linux-debug-try,vm-canary-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-checked-linux-release-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-simarm64c-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206222
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2021-07-12 09:36:19 +00:00
Ryan Macnak dfbb0acbc7 [vm, service] Fix retaining path, inbound references, and heap snapshot tools for compressed pointers.
TEST=ci
Change-Id: Ib67f4ff1a9968cc97c0af91af62a03b0e64068ac
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/200681
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-07-07 23:12:53 +00:00
Ryan Macnak 9bd0eebf0c [vm, service] Fix over-count of HeapSnapshot.referenceCount.
Adjusts the counting phase to use the same visitor as the writing phase.

TEST=deliberately omitting checks from reader for backward compatibility
Change-Id: I8a08c1e9549fd4137714097a868541b09f160157
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205245
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2021-06-29 01:37:32 +00:00
Liam Appelbe 155ac440f7 [vm] Migrate Arrays and Instances to compressed pointers
Change-Id: I8264b340ee91d883afc52a52e83cda504f552e37
TEST=CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191302
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-06-24 17:31:12 +00:00
Alexander Markov 00d6b8972a [vm] Separate class for sentinel objects
This change adds a new VM-internal class for sentinel objects.
Previously sentinel objects used class Never.

TEST=existing tests

Issue: https://github.com/dart-lang/sdk/issues/46141
Change-Id: Ibb3361092967132f4f1952d64fe0168659f3075e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/202870
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-06-11 18:44:56 +00:00
Martin Kustermann 2505b6c64e [vm/concurrency] Ensure no races can occur when setting identity hash code
Usually the identity hashcode is implemented by returning if present and
otherwise generating a random number, setting the hash code and
returning it.

This works fine in single-mutator environments, though when multiple
mutators might race to install identity hash codes we should ensure only
one of them wins and all others will obtain whatever hash code has been
set by the winner.

This will matter once we start sharing more objects across isolates.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=vm/cc/AsmIntrinsifier_SetHashIfNotSetYet

Change-Id: Ie760ca9658e6ec0640255361544d6822b07574e2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/201827
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2021-06-03 14:45:44 +00:00
Vyacheslav Egorov a11d9f17fb [vm] Fix race when setting identity hash codes.
We might interrupt Object._objectHashCode(obj) helper to serve a
vm-service request between checking for unset hash code and setting
a new hash code. While serving this request we might set the hash
code of [obj] (e.g. when generating heap snapshot or in
Instance::PrintJSONImpl). Later Object._objectHashCode will call
Object_setHash, which (on 64-bit platforms) assumes that
hash field in the object header is set to 0 and uses bitwise-or
to initialize it. This leads to a mismatch between hash code that
the first invocation of _objectHashCode will return and the
value stored in the header (because we OR non-zero value set
by vm-service with a value which _objectHashCode intended to
use as a hash).

This CL changes Object_setHash to avoid overwriting or mangling
the hash value if it was already set.

We also fix hash code generation in vm-service to ensure that we
only generate values which are valid Smis because the rest of the
code expects that.

TEST=pkg/front_end/test/incremental_compiler_leak_test.dart

Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm64-try
Change-Id: Ica913af8bc1cfef0ad60a9e7504531ee4de53015
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/197400
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2021-05-03 11:47:08 +00:00
Alexander Markov 5129cff930 [vm/aot] Discard Code objects from the heap
While deserializing AOT snapshot, Code objects which do not contain
valuable information besides entry point and stack maps are discarded
and not allocated on the heap (they are replaced with
StubCode::UnknownDartCode()).

PC -> Code/CompressedStackMaps lookup is implemented using a separate
table (InstructionsTable).

Flutter gallery in release-sizeopt mode:
Heap size of snapshot objects: arm -26.89%, arm64 -27.68%

Large Flutter application in release mode with --dwarf-stack-traces:
Heap size of snapshot objects: -24.3%.
Discarded Code objects: 72.5% of all Code objects.

Issue: https://github.com/dart-lang/sdk/issues/44852.

TEST=existing tests; "--dwarf_stack_traces --no-retain_function_objects
--no-retain_code_objects" mode is enabled for a few tests.

Change-Id: I5fe3e283630c8e8f4442319d5dcae38d174dd0d8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189560
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-04-13 02:36:16 +00:00
Ryan Macnak a6ffc74a4a [vm] Switch most Instances whose layout is determined in C++ to compressed pointers.
Instances whose layout must agree with a layout determined by Dart code are not switched: Array, GrowableObjectArray, Closure, LinkedHashMap.

TEST=ci
Change-Id: I19baf6473a29dd14f84aa1e087eee16b0e6e9d77
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193442
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-04-05 17:43:16 +00:00
Ryan Macnak 19e5899a02 [vm] Switch most non-Instances to compressed pointers.
The following are not compressed: Code, ObjectPool, Context, CallSiteData and subclasses.

Code::instructions_ is uncompressible because the VM doesn't control the address of Instructions when they are loaded via dynamic library.

Loads from the ObjectPool are a frequent occurrence in generated code, and shrinking ObjectPool entries would be offset by longer code sequences to decompress them. Also, some ObjectPool entries contain uncompressible pointers to C functions.

Context is not compressed based on reports from earlier work on compressed pointers that compressing them is disproporately bad for performance. The earlier work used a different, more-expensive compression scheme, so we should reinvestigate this.

CallSiteData et al. are on the hot path for unoptimized code but make up a comparatively small portion of the heap size.

TEST=ci
Change-Id: I9a425fa4e12113c9fd0b51999525d32a7e589a90
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190600
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-03-18 22:52:04 +00:00
Alexander Markov ddfc98337b Reland "[vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces"
This is a reland of b6dc4dad4d

TEST=ci

Original change's description:
> [vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces
>
> The following changes are done in preparation for the removal of Code
> objects in AOT with --dwarf-stack-traces:
>
> * Stack trace objects are extended to hold uword PCs (which may not
>   fit into Smi range).
>
> * Scanning stack frames in GC (StackFrame::VisitObjectPointers)
>   now avoids using Code objects.
>   In order to find CompressedStackMaps it now calls
>   ReversePc::FindCompressedStackMaps.
>
> * Singleton Code object (StubCode::UnknownDartCode()) is prepared as
>   a replacement for Code objects in stack traces. It has
>   PayloadStart() == 0 and Size() == kUwordMax so it includes
>   arbitrary PCs.
>
> * In --dwarf-stack-traces mode, most Code objects obtained from stack
>   frames are replaced with StubCode::UnknownDartCode().
>   This simulates future behavior of ReversePc::Lookup when Code objects
>   will be removed.
>
> Issue: https://github.com/dart-lang/sdk/issues/44852
> Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
> TEST=ci
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

Change-Id: Ia2fc4672a085cd963b7fc103369851df3603590c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186202
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-02-22 10:46:28 +00:00
Alexander Markov fde0ed2e93 Revert "[vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces"
This reverts commit b6dc4dad4d.

Reason for revert: broke package:vm_snapshot_analysis in Flutter
(https://github.com/flutter/flutter/issues/76313).

Original change's description:
> [vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces
>
> The following changes are done in preparation for the removal of Code
> objects in AOT with --dwarf-stack-traces:
>
> * Stack trace objects are extended to hold uword PCs (which may not
>   fit into Smi range).
>
> * Scanning stack frames in GC (StackFrame::VisitObjectPointers)
>   now avoids using Code objects.
>   In order to find CompressedStackMaps it now calls
>   ReversePc::FindCompressedStackMaps.
>
> * Singleton Code object (StubCode::UnknownDartCode()) is prepared as
>   a replacement for Code objects in stack traces. It has
>   PayloadStart() == 0 and Size() == kUwordMax so it includes
>   arbitrary PCs.
>
> * In --dwarf-stack-traces mode, most Code objects obtained from stack
>   frames are replaced with StubCode::UnknownDartCode().
>   This simulates future behavior of ReversePc::Lookup when Code objects
>   will be removed.
>
> Issue: https://github.com/dart-lang/sdk/issues/44852
> Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
> TEST=ci
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: I6f66171eecf1133363a7ce56193e782e43a20baf
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185488
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2021-02-18 18:04:29 +00:00
Alexander Markov b6dc4dad4d [vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces
The following changes are done in preparation for the removal of Code
objects in AOT with --dwarf-stack-traces:

* Stack trace objects are extended to hold uword PCs (which may not
  fit into Smi range).

* Scanning stack frames in GC (StackFrame::VisitObjectPointers)
  now avoids using Code objects.
  In order to find CompressedStackMaps it now calls
  ReversePc::FindCompressedStackMaps.

* Singleton Code object (StubCode::UnknownDartCode()) is prepared as
  a replacement for Code objects in stack traces. It has
  PayloadStart() == 0 and Size() == kUwordMax so it includes
  arbitrary PCs.

* In --dwarf-stack-traces mode, most Code objects obtained from stack
  frames are replaced with StubCode::UnknownDartCode().
  This simulates future behavior of ReversePc::Lookup when Code objects
  will be removed.

Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
TEST=ci
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-02-17 20:12:32 +00:00
Martin Kustermann 5b4349de54 [vm/concurrency] Make main hot-reload implementation independent of current isolate
Since hot-reloading will affect all isolates within a group, there
should be nothing isolate-specific inside it. We therefore add a
NoActiveIsolateScope to the main part of the reloading
implementation.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing implementation.

Change-Id: I1c8db00e6a016c77c68c2c04448de056a76d42c5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184782
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-02-17 09:38:29 +00:00
Ryan Macnak dd83f15826 [vm] Account for compressed pointers in pointer visitors.
TEST=ci
Change-Id: I033bb30e86dfc66ddeb7316a625e3f8925528a19
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183784
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-02-16 18:27:39 +00:00
Ben Konyi 3e83400e0f [ VM / Service ] Added getAllocationTraces and setTraceClassAllocation RPCs and identity hash support
This change adds two new RPCs and various new properties:
  - getAllocationTraces
  - setTraceClassAllocation
  - classId and identityHashCode properties in CpuSample
  - traceAllocations property in Class

TEST=get_allocation_traces_test.dart

package:vm_service has been regenerated for 3.43 of the service protocol
and is ready for a 6.1.0 release.
Change-Id: Ia8ed055423798d7d17fe9f5fd74efb4239b875fc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182666
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-02-12 21:15:35 +00:00
Ben Konyi 565dc5b6a2 Reland "[ VM / Service ] Report identity hash codes of objects at end of heap snapshot"
This reverts commit 7083692624 in order to
reland commit 8d99d295da.

TEST=N/A

Change-Id: I06c80a666a9307caf0313fef1fde212ec801203c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182600
Reviewed-by: Nate Bosch <nbosch@google.com>
2021-02-03 19:13:40 +00:00
Ben Konyi 7083692624 [ package:vm_service ] Prepare for 6.0.1 stable null-safe release
There is no diff between this version and 6.0.1-nullsafety.1.

This temporarily reverts commit 8d99d295da.

TEST=None

Change-Id: I3597b3543c9ea9122865604b9ba07c99683a5355
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182580
Reviewed-by: Nate Bosch <nbosch@google.com>
2021-02-03 18:16:27 +00:00
Ben Konyi 8d99d295da [ VM / Service ] Report identity hash codes of objects at end of heap
snapshot

TEST=pkg/vm_service/test/object_graph_identity_hash_test.dart

Change-Id: I71eff4e8bb2153d8ef1117bc90d078f678f358ed
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/180484
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2021-02-01 22:37:21 +00:00
Ryan Macnak b1c09ecd8f [vm] Make naming more consistent when converting between handles, tagged and untagged pointers.
Currently we have things called XPtr which are not what you get from ptr().

Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)

After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*

New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*

TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-01-15 23:32:02 +00:00
Martin Kustermann 0f76981bb1 [vm/concurrency] Change references to class_table/object_store/heap from Isolate to IsolateGroup
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.

The class_table and object_store were already moved to `IsolateGroup`.

This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Pure refactoring - relying on existing test coverage.

Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-06 15:22:11 +00:00
Daco Harkes 5278383736 [vm] Native API: Make Dart_NewWeakPersistentHandle not auto delete
Changes Dart_NewWeakPersistentHandle to no longer auto delete the
weak persistent handle.

Changes the signatures of WeakPersistentHandleFinalizers to no longer
have access to the handle.

Flutter PR: https://github.com/flutter/engine/pull/19843
g3 presubmit: cl/318028238

Issue: https://github.com/dart-lang/sdk/issues/42312

TEST=runtime/vm/dart_api_impl_test.cc

Change-Id: I3f77db9954d9486759f903b78c03a494f73c68ba
Cq-Include-Trybots:dart/try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151525
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-11-03 10:27:44 +00:00
Alexander Aprelev 083ecee7f6 [vm/fieldtable] Fix retaining paths for static fields.
This is a follow-up to https://dart.googlesource.com/sdk/+/e2aadc7a075da3b52957c93dc39d3b7fc86a5d1b which regressed retaining path reporting for static field values.

Fixes https://github.com/dart-lang/sdk/issues/43025

Change-Id: Iee9f289652e41fafd075a9e93b225099aac8e01c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/162900
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2020-09-15 18:32:31 +00:00
Tess Strickland fa03ee7782 [vm] Create a common base class for TextBuffer/ZoneTextBuffer.
Generally, methods that take a *TextBuffer pointer do not care how the
internal buffer is allocated, and so they could be used for either if
both were subclasses of a base class that contained the printing
methods.  This CL makes that base class, and now TextBuffer and
ZoneTextBuffer now share the exact same set of methods for printing to
the internal buffer.

Since the base class is in platform, this does mean dropping the
overload of AddString for Dart String objects that was part of
ZoneTextBuffer.  Instead, this CL just adds an intermediate call to
ToCString() for the small number of callers that used the overload,
keeping the printing interface the same for both.

In addition, one use of TextBuffer that then re-allocated the buffer
contents into the zone manually has been replaced with a ZoneTextBuffer
instead.

Change-Id: I438a085e7e20d55d93987fd7f36afd636f95955f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/157741
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2020-08-07 18:09:35 +00:00
Daco Harkes 202e32afeb [vm] Native API: Introduce Dart_FinalizableHandle
Introduces Dart_NewFinalizableHandle which does auto delete itself,
but does not allow accessing the weak referenced object.

Issue: https://github.com/dart-lang/sdk/issues/42312

Change-Id: I24ea732925122c453213c4fa3f629761c352f838
Cq-Include-Trybots:dart/try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154695
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-07-23 11:30:39 +00:00
Ryan Macnak f124f0fcfc [vm, service] Add getProcessMemory, which reports a tree of major ownership of memory known to the VM.
Removes the private extension to heap snapshots, which did not represent nesting.

Change-Id: I62e60922f094db68cf63bac037140e1a8b2b9fd7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152424
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2020-06-25 19:51:12 +00:00
Ryan Macnak 41caebb39e [vm] Fix overly broad HeapIterationScope in getInstances.
Fix nested HeapIterationScopes.

Fix inadvertently disabled UI for getInstances.

Bug: https://github.com/dart-lang/sdk/issues/41943
Change-Id: I09fa385246dda7427a4f51a02144f945ec638557
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148502
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-05-19 17:37:10 +00:00
Ryan Macnak 5241586246 [vm, gc] Rename HeapPage to OldPage for symmetry.
Change-Id: I282eaf894fad17a731dbfc76873468b04117cebd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147324
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-05-08 18:28:06 +00:00
Ryan Macnak 6fe15f6df9 [vm] Represent tagged pointers as C++ value types instead of C++ pointer types.
This works around bugs in UndefinedBehaviorSanitizer and Clang.

Bug: b/28638298
Change-Id: I6be595f9664516019d28017d24559583a1ae3a21
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144354
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2020-04-25 05:21:27 +00:00
Ryan Macnak 160046f44c [vm] Move CID predicates out of RawObject.
When using C++ value types to represent tagged pointers, we cannot use forward declarations for tagged pointers. This helps to break include cycles when attempting to keep IsXYZ predicates as tagged pointer member functions.

class_id.h also seems like a more natural place for these predicates, which were written before there was a class_id.h.

Change-Id: I0677560a794ed084d10f844606e202feb0c3820a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144321
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-04-21 20:09:55 +00:00
Teagan Strickland 34742535d6 [vm] Look up dynamic symbols in DWARF stack traces.
If lookup succeeds, then the DWARF stack frame uses the symbol
name and offset instead of the dynamic library name. The
native_stack_traces package (once updated) can translate these
stack frames into symbolic stack frames even if given a partial
stack trace without the header.

Also, consistently use uword instead of a mix of uintptr_t and uword.

Change-Id: Ida9ba4726a306fb77b141cb9c6ad635a12a320ec
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/140282
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-03-23 10:13:43 +00:00