This CL implements `NativeFinalizer` in the GC.
`FinalizerEntry`s are extended to track `external_size` and in which
`Heap::Space` the finalizable value is.
On attaching a native finalizer, the external size is added to the
relevant heap. When the finalizable value is promoted from new to old
space, the external size is promoted as well. And when a native
finalizer is run or is detached, the external size is removed from the
relevant heap again.
In contrast to Dart `Finalizer`s, `NativeFinalizer`s are run on isolate
shutdown.
When the `NativeFinalizer`s themselves are collected, the finalizers are
not run. Users should stick the native finalizer in a global variable to
ensure finalization. We will revisit this design when we add send and
exit support, because there is a design space to explore what to do in
that case. This current solution promises the least to users.
In this implementation native finalizers have a Dart entry to clean up
the entries from the `all_entries` field of the finalizer. We should
consider using another data structure that avoids the need for this Dart
entry. See the TODO left in the code.
Bug: https://github.com/dart-lang/sdk/issues/47777
TEST=runtime/tests/vm/dart(_2)/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
TEST=tests/ffi(_2)/vmspecific_native_finalizer_*
Change-Id: I8f594c80c3c344ad83e1f2de10de028eb8456121
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/236320
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Original CL in patchset 1.
Split-off https://dart-review.googlesource.com/c/sdk/+/238341
And pulled in fix https://dart-review.googlesource.com/c/sdk/+/238582
(Should merge cleanly when this lands later.)
This CL implements the `Finalizer` in the GC.
The GC is specially aware of two types of objects for the purposes of
running finalizers.
1) `FinalizerEntry`
2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
A `FinalizerEntry` contains the `value`, the optional `detach` key, and
the `token`, and a reference to the `finalizer`.
An entry only holds on weakly to the value, detach key, and finalizer.
(Similar to how `WeakReference` only holds on weakly to target).
A `Finalizer` contains all entries, a list of entries of which the value
is collected, and a reference to the isolate.
When a the value of an entry is GCed, the enry is added over to the
collected list.
If any entry is moved to the collected list, a message is sent that
invokes the finalizer to call the callback on all entries in that list.
When a finalizer is detached by the user, the entry token is set to the
entry itself and is removed from the all entries set.
This ensures that if the entry was already moved to the collected list,
the finalizer is not executed.
To speed up detaching, we use a weak map from detach keys to list of
entries. This ensures entries can be GCed.
Both the scavenger and marker tasks process finalizer entries in
parallel.
Parallel tasks use an atomic exchange on the head of the collected
entries list, ensuring no entries get lost.
The mutator thread is guaranteed to be stopped when processing entries.
This ensures that we do not need barriers for moving entries into the
finalizers collected list.
Dart reads and replaces the collected entries list also with an atomic
exchange, ensuring the GC doesn't run in between a load/store.
When a finalizer gets posted a message to process finalized objects, it
is being kept alive by the message.
An alternative design would be to pre-allocate a `WeakReference` in the
finalizer pointing to the finalizer, and send that itself.
This would be at the cost of an extra object.
Send and exit is not supported in this CL, support will be added in a
follow up CL. Trying to send will throw.
Bug: https://github.com/dart-lang/sdk/issues/47777
TEST=runtime/tests/vm/dart/finalizer/*
TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
Change-Id: Ibdfeadc16d5d69ade50aae5b9f794284c4c4dbab
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238086
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This reverts commit 7dca34c235.
Reason for revert: b/226085355 dart_vm_test crashing. Unclear what
the cause is. Reverting so we can triage the issue.
TEST=This is a revert.
Original change's description:
> [vm] Implement `Finalizer`
>
> This CL implements the `Finalizer` in the GC.
>
> (This CL does not yet implement `NativeFinalizer`.)
>
> The GC is specially aware of two types of objects for the purposes of
> running finalizers.
>
> 1) `FinalizerEntry`
> 2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
>
> A `FinalizerEntry` contains the `value`, the optional `detach` key, and
> the `token`, and a reference to the `finalizer`.
> An entry only holds on weakly to the value, detach key, and finalizer.
> (Similar to how `WeakReference` only holds on weakly to target).
>
> A `Finalizer` contains all entries, a list of entries of which the value
> is collected, and a reference to the isolate.
>
> When a the value of an entry is GCed, the enry is added over to the
> collected list.
> If any entry is moved to the collected list, a message is sent that
> invokes the finalizer to call the callback on all entries in that list.
>
> When a finalizer is detached by the user, the entry token is set to the
> entry itself and is removed from the all entries set.
> This ensures that if the entry was already moved to the collected list,
> the finalizer is not executed.
>
> To speed up detaching, we use a weak map from detach keys to list of
> entries. This ensures entries can be GCed.
>
> Both the scavenger and marker tasks process finalizer entries in
> parallel.
> Parallel tasks use an atomic exchange on the head of the collected
> entries list, ensuring no entries get lost.
> The mutator thread is guaranteed to be stopped when processing entries.
> This ensures that we do not need barriers for moving entries into the
> finalizers collected list.
> Dart reads and replaces the collected entries list also with an atomic
> exchange, ensuring the GC doesn't run in between a load/store.
>
> When a finalizer gets posted a message to process finalized objects, it
> is being kept alive by the message.
> An alternative design would be to pre-allocate a `WeakReference` in the
> finalizer pointing to the finalizer, and send that itself.
> This would be at the cost of an extra object.
>
> Send and exit is not supported in this CL, support will be added in a
> follow up CL. Trying to send will throw.
>
> Bug: https://github.com/dart-lang/sdk/issues/47777
>
> TEST=runtime/tests/vm/dart/finalizer/*
> TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
> TEST=runtime/vm/object_test.cc
>
> Change-Id: I03e6b4a46212316254bf46ba3f2df333abaa686c
> Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229544
> Reviewed-by: Lasse Nielsen <lrn@google.com>
> Reviewed-by: Slava Egorov <vegorov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Daco Harkes <dacoharkes@google.com>
TBR=lrn@google.com,vegorov@google.com,kustermann@google.com,rmacnak@google.com,dacoharkes@google.com
Change-Id: I991f6e49896d18a8d70210cf315d858b462d66c9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/47777
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238080
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This CL implements the `Finalizer` in the GC.
(This CL does not yet implement `NativeFinalizer`.)
The GC is specially aware of two types of objects for the purposes of
running finalizers.
1) `FinalizerEntry`
2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
A `FinalizerEntry` contains the `value`, the optional `detach` key, and
the `token`, and a reference to the `finalizer`.
An entry only holds on weakly to the value, detach key, and finalizer.
(Similar to how `WeakReference` only holds on weakly to target).
A `Finalizer` contains all entries, a list of entries of which the value
is collected, and a reference to the isolate.
When a the value of an entry is GCed, the enry is added over to the
collected list.
If any entry is moved to the collected list, a message is sent that
invokes the finalizer to call the callback on all entries in that list.
When a finalizer is detached by the user, the entry token is set to the
entry itself and is removed from the all entries set.
This ensures that if the entry was already moved to the collected list,
the finalizer is not executed.
To speed up detaching, we use a weak map from detach keys to list of
entries. This ensures entries can be GCed.
Both the scavenger and marker tasks process finalizer entries in
parallel.
Parallel tasks use an atomic exchange on the head of the collected
entries list, ensuring no entries get lost.
The mutator thread is guaranteed to be stopped when processing entries.
This ensures that we do not need barriers for moving entries into the
finalizers collected list.
Dart reads and replaces the collected entries list also with an atomic
exchange, ensuring the GC doesn't run in between a load/store.
When a finalizer gets posted a message to process finalized objects, it
is being kept alive by the message.
An alternative design would be to pre-allocate a `WeakReference` in the
finalizer pointing to the finalizer, and send that itself.
This would be at the cost of an extra object.
Send and exit is not supported in this CL, support will be added in a
follow up CL. Trying to send will throw.
Bug: https://github.com/dart-lang/sdk/issues/47777
TEST=runtime/tests/vm/dart/finalizer/*
TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
Change-Id: I03e6b4a46212316254bf46ba3f2df333abaa686c
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229544
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Fixes#47190
TEST=None, only markdown files where edited.
Change-Id: Ife204f9c792b6bce30d0cd7bf2260ced11c8f2b4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/213049
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Revert the optimization that stores function entry point addresses
directly in these caches. It doesn't work in compressed mode, and
should be an unimportant optimization these days as most cases are
handled by the dispatch table.
Change-Id: I2ebbf6549aadb49c767553574a3bafe5d867d194
TEST=CI
Bug: https://github.com/dart-lang/sdk/issues/46468
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205636
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
This includes support for calling Dart_PropagateError in native code
when doing FFI calls, and catching uncaught exceptions with Dart_IsError
when doing FFI callbacks.
The support for Dart_PropagateError adds a catch entry to the FFI
trampoline, which prevents inlining these trampolines in AOT. This
regresses the FfiCall benchmarks by 1-2% in AOT.
In addition, Dart_PropagateError requires maintaining a bit whether we
entered native/VM code from generated code through FFI or not. That way
we can do the proper transition on the exception path. When entering
generated code, we store this bit on the stack, right after the entry
frame.
Design: http://go/dart-ffi-handles
Issue: https://github.com/dart-lang/sdk/issues/36858
Issue: https://github.com/dart-lang/sdk/issues/41319
Change-Id: Idfd7ff69132fb29cc730931a4113d914d4437396
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145591
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This reverts commit 898b080d09.
Reason for revert: Seems to cause crashes in g3, see b/153524644.
Original change's description:
> Reland "[vm, gc] Parallel scavenge."
>
> Remove header access from typed data case in RawObject::HeapSizeFromClass.
>
> Address data race between visiting remembered cards and allocating new large pages for promotion.
>
> Change-Id: I92cc4604ed38ab8f42b87b140a26a8118e7030d4
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142565
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
TBR=kustermann@google.com,aam@google.com,rmacnak@google.com,asiva@google.com
Change-Id: I3ef7bd5c3d0ca575f33cf5f712f58476037d2d90
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142861
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Remove header access from typed data case in RawObject::HeapSizeFromClass.
Address data race between visiting remembered cards and allocating new large pages for promotion.
Change-Id: I92cc4604ed38ab8f42b87b140a26a8118e7030d4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142565
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This reverts commit 9209f522d6.
Reason for revert: Flaky analyzer crashes
Original change's description:
> [vm, gc] Parallel scavenge.
>
> Run N tasks that compete to copy or promote survivors. Installation of the forwarding pointer uses a CAS.
> Tasks do not share the list of copied objects to be processed.
> Tasks do share the list of promoted objects to be processed.
> Divide the freelist in N copies. Each scavenger task uses one without locking. The sweeper inserts into the freelists round-robin.
>
> Change-Id: I54c7839ac5f15829462b3be7cf8c9635c01b785f
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135962
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
TBR=kustermann@google.com,aam@google.com,rmacnak@google.com,asiva@google.com
Change-Id: I0d97b8d9148b2a81cabafde4602512e423eba223
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142554
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Run N tasks that compete to copy or promote survivors. Installation of the forwarding pointer uses a CAS.
Tasks do not share the list of copied objects to be processed.
Tasks do share the list of promoted objects to be processed.
Divide the freelist in N copies. Each scavenger task uses one without locking. The sweeper inserts into the freelists round-robin.
Change-Id: I54c7839ac5f15829462b3be7cf8c9635c01b785f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135962
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
