The JIT support for dynamically unboxing has been removed in [0]. As
such all double/float32x4/float64x2/int32x4 objects are immutable and
can therefore be shared across isolates.
[0] https://dart-review.googlesource.com/c/sdk/+/256211
TEST=ci
Change-Id: Ifd4e7c2444415b2e3b5269d9fbeb6570cc5d6768
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/273680
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
RegExp code objects keep backtracking and registers stacks in object pools, can't be shared between isolates.
BUG=https://github.com/dart-lang/sdk/issues/50082
TEST=one_regexp_many_workers
Change-Id: Ic7db8d7a75a0951178b2f4800f96224d52506545
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/273480
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
_InternalLinkedHashMap => _Map
_InternalImmutableLinkedHashMap => _ConstMap
_InternalLinkedHashSet => _Set
_InternalImmutableLinkedHashSet => _ConstSet
This makes things nicer to read in places that display implementation names, such as stack traces, debuggers, profilers and inspectors.
TEST=ci
Change-Id: Iec851c80ea2086cbe79934565dbf35f04809a836
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/266303
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
'is' tests against record types are split into series of 'is' tests of
record fields. This is more efficient as record instances cannot be
added to subtype test caches (because type of a record instance
depends on types of its fields).
This change also adds canonicalization and constant evaluation of
LoadFieldInstr for record fields.
Performance on a trivial micro-benchmark (on x64):
JIT (RunTime): 4519104.5 -> 20031.5
AOT (RunTime): 4352583.0 -> 26281.6
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I2ed464cd3b31f365b17805f4e7debe1d6d1051fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/268080
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This avoids expensive promotion of large messages.
Cf. 04659de9f05af63d5fe4b8d67d70bb3cdf1c9a7a
TEST=ci
Change-Id: I89d5e560063f7d0c1f791ef299bb74c971f3dccf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/257801
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Instead apply the same approach as we do in AOT: unbox based on the
static type information. There are no TFA results available in JIT,
but we could still unbox fields when running in sound null-safety.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try
Change-Id: Ide2e78c6659261ef8d245a4586cf699ea0fbb459
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256211
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
* Merge ClassTable and SharedClassTable back together;
* Simplify handling of multiple arrays growing in sync;
* Refactor how reload deals with ClassTable.
The last change is the most important because it makes it
much easier to reason about the code. We move away from
copying bits and pieces of the class table and shared
class table into reload contexts.
Having two class table fields in the isolate group makes
it easier to reason about. One field contains program
class table (one modified by kernel loader and accessed
by various program structure cid lookups) and heap
walk class table (used by GC visitors). Normally these
two fields point to the same class table, but during
hot reload we temporary split them apart: original
class table is kept as a heap walk class table, while
program class table is replaced by a clone and updated
by reload.
If reload succeeds we drop original class table and
set program class table as heap walk one.
If reload fails we drop the program class table and
restore original one from heap walk table.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-asan-linux-release-x64-try
Change-Id: I8b66259fcc474dea7dd2af063e4772df99be06c4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/258361
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This is a reland of commit b8d4e24338
How the failures were fixed:
1. My ExternalSizeLimit test crashed on msvc because I was using a
0-sized array. I have now changed that array to have size 1.
2. My ExternalSizeLimit test crashed on x64c because
ExternalTypedData::MaxElements(kExternalTypedDataUint8ArrayCid) is much
smaller than kMaxAddrSpaceMB/4 on x64c. I now call
ExternalTypedData::New() with a length argument of 1, and just pretend
that the external allocations are larger when calling
FinalizablePersistentHandle::New().
Original change's description:
> [VM - Runtime] Return nullptr when allocating a
> FinalizablePersistentHandle fails
>
> This CL adds checks to ensure that the tracked total size of
> externally allocated objects never exceeds the amount of memory on the
> system. When the limit is exceeded, then
> FinalizablePersistentHandle::New() will return nullptr.
>
> Resolves https://github.com/dart-lang/sdk/issues/49332
>
> TEST=ci
>
> Change-Id: Ib6cc92325b1d5efcb2965098fa45cfecc90995e3
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256201
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Derek Xu <derekx@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>
TEST=I ran the tryjobs for the configurations that broke CI.
Change-Id: I813aa74667c59a4dbec7f53440ca8d0bf21256ce
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256973
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
This reverts commit b8d4e24338.
Reason for revert: breaks msvc, msan, x64c builds
Original change's description:
> [VM - Runtime] Return nullptr when allocating a FinalizablePersistentHandle fails
>
> This CL adds checks to ensure that the tracked total size of externally
> allocated objects never exceeds the amount of memory on the system. When
> the limit is exceeded, then FinalizablePersistentHandle::New() will
> return nullptr.
>
> Resolves https://github.com/dart-lang/sdk/issues/49332
>
> TEST=ci
>
> Change-Id: Ib6cc92325b1d5efcb2965098fa45cfecc90995e3
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256201
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Derek Xu <derekx@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>
TBR=bkonyi@google.com,asiva@google.com,dnfield@google.com,derekx@google.com
Change-Id: I934bfbf5dc2e8e2ead5c74fe6b1d84e7b311788c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256972
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This CL adds checks to ensure that the tracked total size of externally
allocated objects never exceeds the amount of memory on the system. When
the limit is exceeded, then FinalizablePersistentHandle::New() will
return nullptr.
Resolves https://github.com/dart-lang/sdk/issues/49332
TEST=ci
Change-Id: Ib6cc92325b1d5efcb2965098fa45cfecc90995e3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256201
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
This foregoes the optimization of removing CheckWritable if the unmodifiable views are not used from Dart code.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/49784
Change-Id: I18f3c36437ef136daf875358278caca4e3e0faa0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/255816
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This reverts commit d0a60d67c9.
Reason for revert: Slow path transition seems to occur earlier than I expected
Original change's description:
> [vm] Allocate old in object-graph-copy's slow path.
>
> If we have reached the slow path, the message's subgraph is very likely to be large.
>
> TEST=ci
> Change-Id: Ie8a18c78936ae8a53f30dd61da3650e684c09dfa
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249081
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
TBR=kustermann@google.com,rmacnak@google.com
Change-Id: I61faddd37e424a742427ae7c270cdb8b0a13c418
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/255300
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
If we have reached the slow path, the message's subgraph is very likely to be large.
TEST=ci
Change-Id: Ie8a18c78936ae8a53f30dd61da3650e684c09dfa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249081
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
These types now work with Dart_TypedDataAcquireData.
The presence of these types no longer degrades the performance of typed data indexed loads.
The presence of these types degrades the performance of typed data indexed stores much less. The performance of indexed stores is somewhat regressed if these types were not used.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/32028
Bug: https://github.com/dart-lang/sdk/issues/40924
Bug: https://github.com/dart-lang/sdk/issues/42785
Change-Id: I05ac5c9543f6f61ac37533b9efe511254778caed
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/253700
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This gives starting or finalizing concurrent marking a chance to run not-immediately-after-a-scavenge.
TEST=ci
Change-Id: Iec1ba7b7440045cc18e01637af1f865d588b24c2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/246163
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
- Greatly reduces the size of the root set, thus reducing pauses for scavenges or marking that occur during a copy.
- Reduces the memory overhead of the from_to table from 6 words per object to 2 words per object.
TEST=ci
Change-Id: Icf81f4b25adff22590a9c84c40068e35dd4d502b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/246305
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
To make responsiveness to safepointing requests better when a thread
does a transitive object copy we add safepoint checkins.
In the fast object copy implementation - which runs inside
NoSafepointScope - we bail out to the slow path if there's a safepoint
requested.
The slow path is safe to GC at any point due to having all objects
inside handles / GC-visible objects.
TEST=Existing test suite.
Change-Id: Ie2c37e2f618506ab62a592aa9ff9ab266a02b64b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/245166
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The new implementation moves away from desugaring of async
functions on kernel AST, state machine generated in the flow graph and
capturing all local variables in the context.
Instead, async/await is implemented using a few stubs
(InitSuspendableFunction, Suspend, Resume, Return and
AsyncExceptionHandler). The stubs are implemented in a
platform-independent way using (macro-)assembler helpers.
When suspending a function, its frame is copied into a SuspendState
object, and when resuming a function it is copied back onto the stack.
No extra code is generated for accessing local variables.
Callback closures are created lazily on the first await.
Design doc: go/compact-async-await.
Part 1 (kernel): https://dart-review.googlesource.com/c/sdk/+/241842
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: Ibad757035b7cc438ebdff80b460728b1d3eff1f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242000
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
This CL implements `NativeFinalizer` in the GC.
`FinalizerEntry`s are extended to track `external_size` and in which
`Heap::Space` the finalizable value is.
On attaching a native finalizer, the external size is added to the
relevant heap. When the finalizable value is promoted from new to old
space, the external size is promoted as well. And when a native
finalizer is run or is detached, the external size is removed from the
relevant heap again.
In contrast to Dart `Finalizer`s, `NativeFinalizer`s are run on isolate
shutdown.
When the `NativeFinalizer`s themselves are collected, the finalizers are
not run. Users should stick the native finalizer in a global variable to
ensure finalization. We will revisit this design when we add send and
exit support, because there is a design space to explore what to do in
that case. This current solution promises the least to users.
In this implementation native finalizers have a Dart entry to clean up
the entries from the `all_entries` field of the finalizer. We should
consider using another data structure that avoids the need for this Dart
entry. See the TODO left in the code.
Bug: https://github.com/dart-lang/sdk/issues/47777
TEST=runtime/tests/vm/dart(_2)/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
TEST=tests/ffi(_2)/vmspecific_native_finalizer_*
Change-Id: I8f594c80c3c344ad83e1f2de10de028eb8456121
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/236320
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Original CL in patchset 1.
Split-off https://dart-review.googlesource.com/c/sdk/+/238341
And pulled in fix https://dart-review.googlesource.com/c/sdk/+/238582
(Should merge cleanly when this lands later.)
This CL implements the `Finalizer` in the GC.
The GC is specially aware of two types of objects for the purposes of
running finalizers.
1) `FinalizerEntry`
2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
A `FinalizerEntry` contains the `value`, the optional `detach` key, and
the `token`, and a reference to the `finalizer`.
An entry only holds on weakly to the value, detach key, and finalizer.
(Similar to how `WeakReference` only holds on weakly to target).
A `Finalizer` contains all entries, a list of entries of which the value
is collected, and a reference to the isolate.
When a the value of an entry is GCed, the enry is added over to the
collected list.
If any entry is moved to the collected list, a message is sent that
invokes the finalizer to call the callback on all entries in that list.
When a finalizer is detached by the user, the entry token is set to the
entry itself and is removed from the all entries set.
This ensures that if the entry was already moved to the collected list,
the finalizer is not executed.
To speed up detaching, we use a weak map from detach keys to list of
entries. This ensures entries can be GCed.
Both the scavenger and marker tasks process finalizer entries in
parallel.
Parallel tasks use an atomic exchange on the head of the collected
entries list, ensuring no entries get lost.
The mutator thread is guaranteed to be stopped when processing entries.
This ensures that we do not need barriers for moving entries into the
finalizers collected list.
Dart reads and replaces the collected entries list also with an atomic
exchange, ensuring the GC doesn't run in between a load/store.
When a finalizer gets posted a message to process finalized objects, it
is being kept alive by the message.
An alternative design would be to pre-allocate a `WeakReference` in the
finalizer pointing to the finalizer, and send that itself.
This would be at the cost of an extra object.
Send and exit is not supported in this CL, support will be added in a
follow up CL. Trying to send will throw.
Bug: https://github.com/dart-lang/sdk/issues/47777
TEST=runtime/tests/vm/dart/finalizer/*
TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
Change-Id: Ibdfeadc16d5d69ade50aae5b9f794284c4c4dbab
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238086
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This refactors the delayed lists in the marker and scavenger, but does
not have any behavior changes.
Split off https://dart-review.googlesource.com/c/sdk/+/238086
TEST=running test suites will trigger GC.
Change-Id: I82ca4c3dca3351b6543bafaee1318b8e0da75f1f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238341
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This reverts commit 7dca34c235.
Reason for revert: b/226085355 dart_vm_test crashing. Unclear what
the cause is. Reverting so we can triage the issue.
TEST=This is a revert.
Original change's description:
> [vm] Implement `Finalizer`
>
> This CL implements the `Finalizer` in the GC.
>
> (This CL does not yet implement `NativeFinalizer`.)
>
> The GC is specially aware of two types of objects for the purposes of
> running finalizers.
>
> 1) `FinalizerEntry`
> 2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
>
> A `FinalizerEntry` contains the `value`, the optional `detach` key, and
> the `token`, and a reference to the `finalizer`.
> An entry only holds on weakly to the value, detach key, and finalizer.
> (Similar to how `WeakReference` only holds on weakly to target).
>
> A `Finalizer` contains all entries, a list of entries of which the value
> is collected, and a reference to the isolate.
>
> When a the value of an entry is GCed, the enry is added over to the
> collected list.
> If any entry is moved to the collected list, a message is sent that
> invokes the finalizer to call the callback on all entries in that list.
>
> When a finalizer is detached by the user, the entry token is set to the
> entry itself and is removed from the all entries set.
> This ensures that if the entry was already moved to the collected list,
> the finalizer is not executed.
>
> To speed up detaching, we use a weak map from detach keys to list of
> entries. This ensures entries can be GCed.
>
> Both the scavenger and marker tasks process finalizer entries in
> parallel.
> Parallel tasks use an atomic exchange on the head of the collected
> entries list, ensuring no entries get lost.
> The mutator thread is guaranteed to be stopped when processing entries.
> This ensures that we do not need barriers for moving entries into the
> finalizers collected list.
> Dart reads and replaces the collected entries list also with an atomic
> exchange, ensuring the GC doesn't run in between a load/store.
>
> When a finalizer gets posted a message to process finalized objects, it
> is being kept alive by the message.
> An alternative design would be to pre-allocate a `WeakReference` in the
> finalizer pointing to the finalizer, and send that itself.
> This would be at the cost of an extra object.
>
> Send and exit is not supported in this CL, support will be added in a
> follow up CL. Trying to send will throw.
>
> Bug: https://github.com/dart-lang/sdk/issues/47777
>
> TEST=runtime/tests/vm/dart/finalizer/*
> TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
> TEST=runtime/vm/object_test.cc
>
> Change-Id: I03e6b4a46212316254bf46ba3f2df333abaa686c
> Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229544
> Reviewed-by: Lasse Nielsen <lrn@google.com>
> Reviewed-by: Slava Egorov <vegorov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Daco Harkes <dacoharkes@google.com>
TBR=lrn@google.com,vegorov@google.com,kustermann@google.com,rmacnak@google.com,dacoharkes@google.com
Change-Id: I991f6e49896d18a8d70210cf315d858b462d66c9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/47777
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238080
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This CL implements the `Finalizer` in the GC.
(This CL does not yet implement `NativeFinalizer`.)
The GC is specially aware of two types of objects for the purposes of
running finalizers.
1) `FinalizerEntry`
2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
A `FinalizerEntry` contains the `value`, the optional `detach` key, and
the `token`, and a reference to the `finalizer`.
An entry only holds on weakly to the value, detach key, and finalizer.
(Similar to how `WeakReference` only holds on weakly to target).
A `Finalizer` contains all entries, a list of entries of which the value
is collected, and a reference to the isolate.
When a the value of an entry is GCed, the enry is added over to the
collected list.
If any entry is moved to the collected list, a message is sent that
invokes the finalizer to call the callback on all entries in that list.
When a finalizer is detached by the user, the entry token is set to the
entry itself and is removed from the all entries set.
This ensures that if the entry was already moved to the collected list,
the finalizer is not executed.
To speed up detaching, we use a weak map from detach keys to list of
entries. This ensures entries can be GCed.
Both the scavenger and marker tasks process finalizer entries in
parallel.
Parallel tasks use an atomic exchange on the head of the collected
entries list, ensuring no entries get lost.
The mutator thread is guaranteed to be stopped when processing entries.
This ensures that we do not need barriers for moving entries into the
finalizers collected list.
Dart reads and replaces the collected entries list also with an atomic
exchange, ensuring the GC doesn't run in between a load/store.
When a finalizer gets posted a message to process finalized objects, it
is being kept alive by the message.
An alternative design would be to pre-allocate a `WeakReference` in the
finalizer pointing to the finalizer, and send that itself.
This would be at the cost of an extra object.
Send and exit is not supported in this CL, support will be added in a
follow up CL. Trying to send will throw.
Bug: https://github.com/dart-lang/sdk/issues/47777
TEST=runtime/tests/vm/dart/finalizer/*
TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
Change-Id: I03e6b4a46212316254bf46ba3f2df333abaa686c
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229544
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
These GCs usually did not free very much memory but did consume a lot of CPU. They would on low-powered, low-memory devices often take ~1s, during which time the OS might decide the OOM signal wasn't working and kill us before the compaction can complete and free pages. Instead, only release pooled memory.
Also use more appropriate GCReasons in calls of CollectMost/AllGarbage.
TEST=ci
Bug: b/216333343
Change-Id: Ia56b9ca409410f17d40508c69fb1bc9df0ce4028
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/235300
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This CL implements `WeakReference` in the VM.
* This reduces the size of weak references from 2 objects using 8 words
to 1 object using 4 words.
* This makes loads of weak reference targets a single load instead of
two.
* This avoids the fix-point in the GC and message object copying for
weak references. (N.b. Weak references need to be processed _after_
the fix-point for weak properties.)
The semantics of weak references in messages is that their target gets
set to `null` if the target is not included in the message by a strong
reference.
The tests take particular care to exercise the case where a weak
reference's target is only kept alive because a weak property key is
alive and it refers to the target in its value. This exercises the fact
that weak references need to be processed last.
Does not add support for weak references in the app snapshot. It would
be dead code until we start using weak references in for example the
CFE.
This CL does not try to unify weak references and weak properties in
the GC or messaging (as proposed in go/dart-vm-weakreference), because
their semantics differ enough.
Closes: https://github.com/dart-lang/sdk/issues/48162
TEST=runtime/tests/vm/dart/finalizer/weak_reference_run_gc_test.dart
TEST=runtime/tests/vm/dart/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
TEST=tests/lib/isolate/weak_reference_message_1_test.dart
TEST=tests/lib/isolate/weak_reference_message_2_test.dart
Change-Id: I3810e919a5866f3ae8a95bd9aa23a880a0b0921c
Cq-Include-Trybots: luci.dart.try:app-kernel-linux-debug-x64-try,dart-sdk-mac-arm64-try,vm-canary-linux-debug-try,vm-fuchsia-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-mac-debug-arm64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-win-release-ia32-try,vm-kernel-nnbd-win-release-x64-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-win-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/232087
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
When we rehash a linked hash map, we rebuild the index but keep the
existing data array. The [_deletedKeys] specifies the number of entries in the
[_index] which were marked as deleted. The Dart code that triggers
rehashing and initializes a new [_index] didn't initialize
[_deletedKeys].
=> We'll fix this in the Dart code and as a precaution also in the
transitive object copy code.
Furthermore we add a missing optimization: In the example from the bug
report we shouldn't even have attempted to re-hash, because the maps
have only primitive keys (which do not rely on identity or user-defined
hashcodes).
=> We'll change the existing optimization to ignore deleted entries when
determining whether a rehash is needed.
The rehashing code for Sets works differently and takes this already
into account.
Closes https://github.com/dart-lang/sdk/issues/47598
TEST=vm/dart{,_2}/isolates/fast_object_copy_test
Change-Id: I3529ecbf500009ab0c72b98e6c427b8840a69371
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/219000
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The existing lib/isolate/stacktrace_message_test was failing and also
incorrectly written due to "!" in front of the expected stringification
of the stacktrace.
Issue https://github.com/dart-lang/sdk/issues/46623
TEST=vm/dart{,_2}/isolates/fast_object_copy2_test, lib{,_2}/isolate/stacktrace_message_test
Change-Id: I169d8fd7a7c3cb3b8c57e00287565e3a5c622acf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/216041
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Add a GCReason for store buffer overflow instead of using a separate print.
This is a pure refactoring that doesn't change GC policy.
TEST=ci
Change-Id: Idcd95b0701a36f45ebaa02db2c2fac9a9081d7c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/215102
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
There have been duplicate cids
* kPointerCid / kFfiPointerCid
* kDynamicLibraryCid / kFfiDynamicLibraryCid
Only one of each made sense. This CL ensures we have only the former.
TEST=ci
Change-Id: Icdcef40c80ed09815da6074a774a02c707f6be37
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212592
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
The new space allocated objects created by transitive object copy are
themselves always GC safe. If an invalid object is encountered and the
copy is aborted, there will be unreachable, allocated new space objects
left, that the next scavenge will reclaim.
If an object in the heap is unreachable it shouldn't really be touched
(not even by GC). Only heap iterations can possibly look at them.
Turns out that hot reload performs heap iterations without collecting
all garbage first. That means it can encounter such an unreachable
object.
Hot reload assumes that it can (for instances of generic classes) always
get a valid TypeArguments vector (or null) - though it encountered 0
instead.
So there's several solutions:
* Make Hot-Reload only work on actually alive objects.
* Make the unreachable objects have null instead of 0 in them
We'll choose the latter and use the existing Object::InitializeObject()
helper method (although it will be slower than the existing
implementation).
Fixes https://github.com/dart-lang/sdk/issues/47131
TEST=Fixes vm/dart_2/isolates/closures_without_captured_variables_test in hot-reload test mode.
Change-Id: I15d48473f0ce4a856316fe24166e81ebc87ccdbd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212585
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
We already allow sending closures (if immutable via sharing
otherwise via copying). This CL makes use of this to allow the argument
to `Isolate.spawn()` to be any closure.
Similar to normal message sending, the spawn will fail if the closure
cannot be sent (or causes an error on the new isolate, e.g. rehashing
error).
Issue https://github.com/dart-lang/sdk/issues/46623
TEST=vm/dart{_2,}/isolates/closure_entrypoint_test
Change-Id: Iab342267d87bd87bc8c0c82d16aec58a69a3df44
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212295
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The allocation routine for allocating context objects (used for copying
closures) has to use uncompressed mode - since [Context] objects aren't
compressed yet.
This is a follow-up to https://dart-review.googlesource.com/c/sdk/+/209110
TEST=Fixes closure tests in vm/dart_2/isolates/fast_object_copy2_test/<num> in compressed mode.
Change-Id: I3a16701fe662ff7ea501a238653c7efb376cda55
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212465
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Closures can be shared if they have no state they capture (i.e. a null
context). Otherwise we copy them by also transitively copying the
parent context chain.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Added tests to vm/dart{,_2}/isolates/fast_object_copy{,2}_test
Change-Id: Ie641b97806edd0c21f0a8d5c514f8e850823e165
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/209110
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This is a reland of 824bec596f
Original change's description:
> [vm] Hide internal implementation List types and expose them as List
>
> When taking a type of an instance with x.runtimeType we can map
> internal classes _List, _ImmutableList and _GrowableList to a
> user-visible List class. This is similar to what we do for
> implementation classes of int, String and Type.
> After that, result of x.runtimeType for built-in lists would be
> compatible with List<T> type literals.
>
> Also, both intrinsic and native implementations of _haveSameRuntimeType
> are updated to agree with new semantic of runtimeType.
>
> TEST=co19/LanguageFeatures/Constructor-tear-offs/type_literal_A01_t01
> TEST=runtime/tests/vm/dart/have_same_runtime_type_test
>
> Fixes https://github.com/dart-lang/sdk/issues/46893
> Issue https://github.com/dart-lang/sdk/issues/46231
>
> Change-Id: Ie24a9f527f66a06118427b7a09e49c03dff93d8e
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210066
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>
TEST=co19/LanguageFeatures/Constructor-tear-offs/type_literal_A01_t01
TEST=runtime/tests/vm/dart/have_same_runtime_type_test
TEST=lib/mirrors/regress_b196606044_test
Fixes https://github.com/dart-lang/sdk/issues/46893
Issue https://github.com/dart-lang/sdk/issues/46231
Change-Id: I79b587540338808bd73a6554f00a5eed042f4c26
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210201
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
This reverts commit 824bec596f.
Reason for revert: b/196606044
Original change's description:
> [vm] Hide internal implementation List types and expose them as List
>
> When taking a type of an instance with x.runtimeType we can map
> internal classes _List, _ImmutableList and _GrowableList to a
> user-visible List class. This is similar to what we do for
> implementation classes of int, String and Type.
> After that, result of x.runtimeType for built-in lists would be
> compatible with List<T> type literals.
>
> Also, both intrinsic and native implementations of _haveSameRuntimeType
> are updated to agree with new semantic of runtimeType.
>
> TEST=co19/LanguageFeatures/Constructor-tear-offs/type_literal_A01_t01
> TEST=runtime/tests/vm/dart/have_same_runtime_type_test
>
> Fixes https://github.com/dart-lang/sdk/issues/46893
> Issue https://github.com/dart-lang/sdk/issues/46231
>
> Change-Id: Ie24a9f527f66a06118427b7a09e49c03dff93d8e
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210066
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>
TBR=rmacnak@google.com,alexmarkov@google.com,sstrickl@google.com
Change-Id: I4c3dddbc358d6ad3b14081706f7aec110a2e0562
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210200
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
When taking a type of an instance with x.runtimeType we can map
internal classes _List, _ImmutableList and _GrowableList to a
user-visible List class. This is similar to what we do for
implementation classes of int, String and Type.
After that, result of x.runtimeType for built-in lists would be
compatible with List<T> type literals.
Also, both intrinsic and native implementations of _haveSameRuntimeType
are updated to agree with new semantic of runtimeType.
TEST=co19/LanguageFeatures/Constructor-tear-offs/type_literal_A01_t01
TEST=runtime/tests/vm/dart/have_same_runtime_type_test
Fixes https://github.com/dart-lang/sdk/issues/46893
Issue https://github.com/dart-lang/sdk/issues/46231
Change-Id: Ie24a9f527f66a06118427b7a09e49c03dff93d8e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210066
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Add missing checks for FFI types disallowed in isolate messages.
Align error messages with the other serializer and test expectations.
TEST=ci
Change-Id: I3813dd8f26e5122524bdf41f9ce6e0785fdd260b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/209840
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
The transitive copy algorithm was missing support for [WeakProperty]s,
thereby ensuring that we only copy the values if the keys are reachable.
Furthermore we might need to re-hash [Expando]s - since the copied
objects start with no identity hash codes.
The CL also makes us avoid calling to Dart for each [Expando]
separately and instead use a list - just as we do in the re-hashing of
maps/sets.
We also move the C++ code to invoke rehashing logic into
DartLibraryCalls::*.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=vm/dart{,_2}/isolates/fast_object_copy{,2}_test
Change-Id: I836745feef8a6d7573faa94e29a19c1eca0c39f2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/209106
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Even though all canonical objects can be shared, they might cause
re-hashing on the receiver side due to user-defined get:hashCode
implementation.
This is why we do not consider the canonical bit (of a key in a map) when
determining whether rehashing is needed.
Though some well-known classes will have a properly behaved
get:hashCode, the most important being for kNullCid.
=> So we'll allow kNullCid, kBoolCid.
This makes receiving json data much faster due to not needing to re-hash
anymore (was done due to `null` values in the map backing store).
TEST=Existing test suite.
Change-Id: Ie6d39da7fe27ce8925644cf2c17a3944a9e936b2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/207080
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
We use message passing as comunication mechanism between isolates.
The transitive closure of an object to be sent is currently serialized
into a snapshot form and deserialized on the receiver side. Furthermore
the receiver side will re-hash any linked hashmaps in that graph.
If isolate gropus are enabled we have all isolates in a group work on
the same heap. That removes the need to use an intermediate
serialization format. It also removes the need for an O(n) step on the
receiver side.
This CL implements a fast transitive object copy implementation and
makes use of it a message that is to be passed to another isolate stays
within the same isolate group.
In the common case the object graph will fit into new space. So the
copy algorithm will try to take advantage of it by having a fast path
and a fallback path. Both of them effectively copy the graph in BFS
order.
The algorithm works effectively like a scavenge operation, but instead
of first copying the from-object to the to-space and then re-writing the
object in to-space to forward the pointers (which requires us writing to
the to-space memory twice), we only reserve space for to-objects and
then initialize the to-objects to it's final contents, including
forwarded pointers (i.e. write the to-space object only once).
Compared with a scavenge operation (which stores forwarding pointers in
the objects themselves), we use a [WeakTable] to store them. This is the
only remaining expensive part of the algorithm and could be further
optimized. To avoid relying on iterating the to-space, we'll remember
[from, to] addresses.
=> All of this works inside a [NoSafepointOperationScope] and avoids
usages of handles as well as write barriers.
While doing the transitive object copy, we'll share any object we can
safely share (canonical objects, strings, sendports, ...) instead of
copying it.
If the fast path fails (due to allocation failure or hitting) we'll
handlify any raw pointers and continue almost the same algorithm in a
safe way, where GC is possible at every object allocation site and
normal barriers are used for any stores of object pointers.
The copy algorithm uses templates to share the copy logic between the
fast and slow case (same copy routines can work on raw pointers as well
as handles).
There's a few special things to take into consideration:
* If we copy a view on external typed data we need to know the
external typed data address to compute the inner pointer of the
view, so we'll eagerly initialize external typed data.
* All external typed data needs to get a finalizer attached
(irrespective if the object copy suceeds or not) to ensure the
`malloc()`ed data is freed again.
* Transferables will only be transferred on successful transitive
copies. Also they need to attach finalizers to objects (which
requires all objects be in handles).
* We copy linked hashmaps as they are - instead of compressing the
data by removing deleted entries. We may need to re-hash those
hashmaps on the receiver side (similar to the snapshot-based copy
approach) since new object graph will have no identity hash codes
assigned to them. Though if the hashmaps only has sharable objects
as keys (very common, e.g. json) there is no need for re-hashing.
It changes the SendPort.* benchmarks as follows:
```
Benchmark | default | IG | IG + FOC
----------------------------------------------------------------------------------------------------------------------------
SendPort.Send.Nop(RunTimeRaw): | 0.25 us (1 x) | 0.26 us (0.96 x) | 0.25 us (1.00 x)
SendPort.Send.Json.400B(RunTimeRaw): | 4.15 us (1 x) | 1.45 us (2.86 x) | 1.05 us (3.95 x)
SendPort.Send.Json.5KB(RunTimeRaw): | 82.16 us (1 x) | 27.17 us (3.02 x) | 18.32 us (4.48 x)
SendPort.Send.Json.50KB(RunTimeRaw): | 784.70 us (1 x) | 242.10 us (3.24 x) | 165.50 us (4.74 x)
SendPort.Send.Json.500KB(RunTimeRaw): | 8510.4 us (1 x) | 3083.80 us (2.76 x) | 2311.29 us (3.68 x)
SendPort.Send.Json.5MB(RunTimeRaw): | 122381.33 us (1 x) | 62959.40 us (1.94 x) | 55492.10 us (2.21 x)
SendPort.Send.BinaryTree.2(RunTimeRaw): | 1.91 us (1 x) | 0.92 us (2.08 x) | 0.72 us (2.65 x)
SendPort.Send.BinaryTree.4(RunTimeRaw): | 6.32 us (1 x) | 2.70 us (2.34 x) | 2.10 us (3.01 x)
SendPort.Send.BinaryTree.6(RunTimeRaw): | 25.24 us (1 x) | 10.47 us (2.41 x) | 8.61 us (2.93 x)
SendPort.Send.BinaryTree.8(RunTimeRaw): | 104.08 us (1 x) | 41.08 us (2.53 x) | 33.51 us (3.11 x)
SendPort.Send.BinaryTree.10(RunTimeRaw): | 373.39 us (1 x) | 174.11 us (2.14 x) | 134.75 us (2.77 x)
SendPort.Send.BinaryTree.12(RunTimeRaw): | 1588.64 us (1 x) | 893.18 us (1.78 x) | 532.05 us (2.99 x)
SendPort.Send.BinaryTree.14(RunTimeRaw): | 6849.55 us (1 x) | 3705.19 us (1.85 x) | 2507.90 us (2.73 x)
SendPort.Receive.Nop(RunTimeRaw): | 0.67 us (1 x) | 0.69 us (0.97 x) | 0.68 us (0.99 x)
SendPort.Receive.Json.400B(RunTimeRaw): | 4.37 us (1 x) | 0.78 us (5.60 x) | 0.77 us (5.68 x)
SendPort.Receive.Json.5KB(RunTimeRaw): | 45.67 us (1 x) | 0.90 us (50.74 x) | 0.87 us (52.49 x)
SendPort.Receive.Json.50KB(RunTimeRaw): | 498.81 us (1 x) | 1.24 us (402.27 x) | 1.06 us (470.58 x)
SendPort.Receive.Json.500KB(RunTimeRaw): | 5366.02 us (1 x) | 4.22 us (1271.57 x) | 4.65 us (1153.98 x)
SendPort.Receive.Json.5MB(RunTimeRaw): | 101050.88 us (1 x) | 20.81 us (4855.88 x) | 21.0 us (4811.95 x)
SendPort.Receive.BinaryTree.2(RunTimeRaw): | 3.91 us (1 x) | 0.76 us (5.14 x) | 0.74 us (5.28 x)
SendPort.Receive.BinaryTree.4(RunTimeRaw): | 9.90 us (1 x) | 0.79 us (12.53 x) | 0.76 us (13.03 x)
SendPort.Receive.BinaryTree.6(RunTimeRaw): | 33.09 us (1 x) | 0.87 us (38.03 x) | 0.84 us (39.39 x)
SendPort.Receive.BinaryTree.8(RunTimeRaw): | 126.77 us (1 x) | 0.92 us (137.79 x) | 0.88 us (144.06 x)
SendPort.Receive.BinaryTree.10(RunTimeRaw): | 533.09 us (1 x) | 0.94 us (567.12 x) | 0.92 us (579.45 x)
SendPort.Receive.BinaryTree.12(RunTimeRaw): | 2223.23 us (1 x) | 3.03 us (733.74 x) | 3.04 us (731.33 x)
SendPort.Receive.BinaryTree.14(RunTimeRaw): | 8945.66 us (1 x) | 4.03 us (2219.77 x) | 4.30 us (2080.39 x)
```
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=vm/dart{,_2}/isolates/fast_object_copy{,2}_test
Change-Id: I835c59dab573d365b8a4b9d7c5359a6ea8d8b0a7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203776
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
