[vm/concurrency] Implement a fast transitive object copy for isolate message passing

We use message passing as comunication mechanism between isolates.
The transitive closure of an object to be sent is currently serialized
into a snapshot form and deserialized on the receiver side. Furthermore
the receiver side will re-hash any linked hashmaps in that graph.

If isolate gropus are enabled we have all isolates in a group work on
the same heap. That removes the need to use an intermediate
serialization format. It also removes the need for an O(n) step on the
receiver side.

This CL implements a fast transitive object copy implementation and
makes use of it a message that is to be passed to another isolate stays
within the same isolate group.

In the common case the object graph will fit into new space. So the
copy algorithm will try to take advantage of it by having a fast path
and a fallback path. Both of them effectively copy the graph in BFS
order.

The algorithm works effectively like a scavenge operation, but instead
of first copying the from-object to the to-space and then re-writing the
object in to-space to forward the pointers (which requires us writing to
the to-space memory twice), we only reserve space for to-objects and
then initialize the to-objects to it's final contents, including
forwarded pointers (i.e. write the to-space object only once).

Compared with a scavenge operation (which stores forwarding pointers in
the objects themselves), we use a [WeakTable] to store them. This is the
only remaining expensive part of the algorithm and could be further
optimized. To avoid relying on iterating the to-space, we'll remember
[from, to] addresses.

=> All of this works inside a [NoSafepointOperationScope] and avoids
   usages of handles as well as write barriers.

While doing the transitive object copy, we'll share any object we can
safely share (canonical objects, strings, sendports, ...) instead of
copying it.

If the fast path fails (due to allocation failure or hitting) we'll
handlify any raw pointers and continue almost the same algorithm in a
safe way, where GC is possible at every object allocation site and
normal barriers are used for any stores of object pointers.

The copy algorithm uses templates to share the copy logic between the
fast and slow case (same copy routines can work on raw pointers as well
as handles).

There's a few special things to take into consideration:

  * If we copy a view on external typed data we need to know the
    external typed data address to compute the inner pointer of the
    view, so we'll eagerly initialize external typed data.

  * All external typed data needs to get a finalizer attached
    (irrespective if the object copy suceeds or not) to ensure the
    `malloc()`ed data is freed again.

  * Transferables will only be transferred on successful transitive
    copies. Also they need to attach finalizers to objects (which
    requires all objects be in handles).

  * We copy linked hashmaps as they are - instead of compressing the
    data by removing deleted entries. We may need to re-hash those
    hashmaps on the receiver side (similar to the snapshot-based copy
    approach) since new object graph will have no identity hash codes
    assigned to them. Though if the hashmaps only has sharable objects
    as keys (very common, e.g. json) there is no need for re-hashing.


It changes the SendPort.* benchmarks as follows:

```
Benchmark                                     |              default |                        IG |                  IG + FOC
----------------------------------------------------------------------------------------------------------------------------
SendPort.Send.Nop(RunTimeRaw):                |        0.25 us (1 x) |       0.26 us    (0.96 x) |       0.25 us    (1.00 x)
SendPort.Send.Json.400B(RunTimeRaw):          |        4.15 us (1 x) |       1.45 us    (2.86 x) |       1.05 us    (3.95 x)
SendPort.Send.Json.5KB(RunTimeRaw):           |       82.16 us (1 x) |      27.17 us    (3.02 x) |      18.32 us    (4.48 x)
SendPort.Send.Json.50KB(RunTimeRaw):          |      784.70 us (1 x) |     242.10 us    (3.24 x) |     165.50 us    (4.74 x)
SendPort.Send.Json.500KB(RunTimeRaw):         |     8510.4  us (1 x) |    3083.80 us    (2.76 x) |    2311.29 us    (3.68 x)
SendPort.Send.Json.5MB(RunTimeRaw):           |   122381.33 us (1 x) |   62959.40 us    (1.94 x) |   55492.10 us    (2.21 x)
SendPort.Send.BinaryTree.2(RunTimeRaw):       |        1.91 us (1 x) |       0.92 us    (2.08 x) |       0.72 us    (2.65 x)
SendPort.Send.BinaryTree.4(RunTimeRaw):       |        6.32 us (1 x) |       2.70 us    (2.34 x) |       2.10 us    (3.01 x)
SendPort.Send.BinaryTree.6(RunTimeRaw):       |       25.24 us (1 x) |      10.47 us    (2.41 x) |       8.61 us    (2.93 x)
SendPort.Send.BinaryTree.8(RunTimeRaw):       |      104.08 us (1 x) |      41.08 us    (2.53 x) |      33.51 us    (3.11 x)
SendPort.Send.BinaryTree.10(RunTimeRaw):      |      373.39 us (1 x) |     174.11 us    (2.14 x) |     134.75 us    (2.77 x)
SendPort.Send.BinaryTree.12(RunTimeRaw):      |     1588.64 us (1 x) |     893.18 us    (1.78 x) |     532.05 us    (2.99 x)
SendPort.Send.BinaryTree.14(RunTimeRaw):      |     6849.55 us (1 x) |    3705.19 us    (1.85 x) |    2507.90 us    (2.73 x)
SendPort.Receive.Nop(RunTimeRaw):             |        0.67 us (1 x) |       0.69 us    (0.97 x) |       0.68 us    (0.99 x)
SendPort.Receive.Json.400B(RunTimeRaw):       |        4.37 us (1 x) |       0.78 us    (5.60 x) |       0.77 us    (5.68 x)
SendPort.Receive.Json.5KB(RunTimeRaw):        |       45.67 us (1 x) |       0.90 us   (50.74 x) |       0.87 us   (52.49 x)
SendPort.Receive.Json.50KB(RunTimeRaw):       |      498.81 us (1 x) |       1.24 us  (402.27 x) |       1.06 us  (470.58 x)
SendPort.Receive.Json.500KB(RunTimeRaw):      |     5366.02 us (1 x) |       4.22 us (1271.57 x) |       4.65 us (1153.98 x)
SendPort.Receive.Json.5MB(RunTimeRaw):        |   101050.88 us (1 x) |      20.81 us (4855.88 x) |      21.0  us (4811.95 x)
SendPort.Receive.BinaryTree.2(RunTimeRaw):    |        3.91 us (1 x) |       0.76 us    (5.14 x) |       0.74 us    (5.28 x)
SendPort.Receive.BinaryTree.4(RunTimeRaw):    |        9.90 us (1 x) |       0.79 us   (12.53 x) |       0.76 us   (13.03 x)
SendPort.Receive.BinaryTree.6(RunTimeRaw):    |       33.09 us (1 x) |       0.87 us   (38.03 x) |       0.84 us   (39.39 x)
SendPort.Receive.BinaryTree.8(RunTimeRaw):    |      126.77 us (1 x) |       0.92 us  (137.79 x) |       0.88 us  (144.06 x)
SendPort.Receive.BinaryTree.10(RunTimeRaw):   |      533.09 us (1 x) |       0.94 us  (567.12 x) |       0.92 us  (579.45 x)
SendPort.Receive.BinaryTree.12(RunTimeRaw):   |     2223.23 us (1 x) |       3.03 us  (733.74 x) |       3.04 us  (731.33 x)
SendPort.Receive.BinaryTree.14(RunTimeRaw):   |     8945.66 us (1 x) |       4.03 us (2219.77 x) |       4.30 us (2080.39 x)
```

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=vm/dart{,_2}/isolates/fast_object_copy{,2}_test

Change-Id: I835c59dab573d365b8a4b9d7c5359a6ea8d8b0a7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203776
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
This commit is contained in:
Martin Kustermann 2021-07-13 19:04:20 +00:00 committed by commit-bot@chromium.org
parent 0fc2d929e3
commit 430aa20a1f
35 changed files with 2198 additions and 66 deletions

View file

@ -20,6 +20,7 @@
#include "vm/longjump.h"
#include "vm/message_handler.h"
#include "vm/object.h"
#include "vm/object_graph_copy.h"
#include "vm/object_store.h"
#include "vm/port.h"
#include "vm/resolver.h"
@ -111,10 +112,22 @@ DEFINE_NATIVE_ENTRY(SendPortImpl_sendInternal_, 0, 2) {
PortMap::PostMessage(
Message::New(destination_port_id, obj.ptr(), Message::kNormalPriority));
} else {
MessageWriter writer(can_send_any_object);
// TODO(turnidge): Throw an exception when the return value is false?
PortMap::PostMessage(writer.WriteMessage(obj, destination_port_id,
Message::kNormalPriority));
const bool same_group = IsolateGroup::AreIsolateGroupsEnabled() &&
PortMap::IsReceiverInThisIsolateGroup(
destination_port_id, isolate->group());
if (same_group) {
const auto& copy = Object::Handle(CopyMutableObjectGraph(obj));
auto handle = isolate->group()->api_state()->AllocatePersistentHandle();
handle->set_ptr(copy.ptr());
std::unique_ptr<Message> message(
new Message(destination_port_id, handle, Message::kNormalPriority));
PortMap::PostMessage(std::move(message));
} else {
MessageWriter writer(can_send_any_object);
// TODO(turnidge): Throw an exception when the return value is false?
PortMap::PostMessage(writer.WriteMessage(obj, destination_port_id,
Message::kNormalPriority));
}
}
return Object::null();
}

View file

@ -188,6 +188,10 @@ class BaseGrowableArray : public B {
// The content is uninitialized after calling it.
void SetLength(intptr_t new_length);
// The content (if expanded) is uninitialized after calling it.
// The backing store (if expanded) will grow with by a power-of-2.
void Resize(intptr_t new_length);
// Sort the array in place.
inline void Sort(int compare(const T*, const T*));
@ -211,9 +215,6 @@ class BaseGrowableArray : public B {
T* data_;
Allocator* allocator_; // Used to (re)allocate the array.
// Used for growing the array.
void Resize(intptr_t new_length);
DISALLOW_COPY_AND_ASSIGN(BaseGrowableArray);
};

View file

@ -0,0 +1,75 @@
// Copyright (c) 2021, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy --gc-on-foc-slow-path --force-evacuation
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy --gc-on-foc-slow-path --force-evacuation
// The tests in this file will only succeed when isolate groups are enabled
// (hence the VMOptions above).
import 'dart:async';
import 'dart:isolate';
import 'dart:typed_data';
import 'package:expect/expect.dart';
import 'fast_object_copy_test.dart'
show UserObject, SendReceiveTestBase, notAllocatableInTLAB;
// When running with isolate groups enabled, we can share all of the following
// objects.
final sharableObjects = [
1,
0xffffffffffffffff,
'foobar',
const UserObject(1, 1.2, ''),
(() {
final rp = ReceivePort();
final sp = rp.sendPort;
rp.close();
return sp;
})(),
const [1, 2, 3],
const {1: 1, 2: 2, 3: 2},
const {1, 2, 3},
RegExp('a'),
Isolate.current.pauseCapability,
Int32x4(1, 2, 3, 4),
];
class SendReceiveTest extends SendReceiveTestBase {
Future runTests() async {
await testSharable();
await testSharable2();
}
Future testSharable() async {
final sharableObjectsCopy = await sendReceive([
...sharableObjects,
]);
Expect.notIdentical(sharableObjects, sharableObjectsCopy);
for (int i = 0; i < sharableObjects.length; ++i) {
Expect.identical(sharableObjects[i], sharableObjectsCopy[i]);
}
}
Future testSharable2() async {
final sharableObjectsCopy = await sendReceive([
notAllocatableInTLAB,
...sharableObjects,
]);
Expect.notIdentical(sharableObjects, sharableObjectsCopy);
Expect.equals(
notAllocatableInTLAB[0], (sharableObjectsCopy[0] as Uint8List)[0]);
for (int i = 0; i < sharableObjects.length; ++i) {
Expect.identical(sharableObjects[i], sharableObjectsCopy[i + 1]);
}
}
}
main() async {
await SendReceiveTest().run();
}

View file

@ -3,7 +3,10 @@
// BSD-style license that can be found in the LICENSE file.
// VMOptions=
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy --gc-on-foc-slow-path --force-evacuation --verify-store-buffer
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy --gc-on-foc-slow-path --force-evacuation --verify-store-buffer
// The tests in this file are particularly for an implementation that tries to
// allocate the entire graph in BFS order using a fast new space allocation
@ -28,7 +31,7 @@ final Uint8List largeExternalTypedData =
final Uint8List largeInternalTypedData = Uint8List(20 * 1024 * 1024)..[0] = 42;
final Uint8List smallExternalTypedData =
File(Platform.script.toFilePath()).readAsBytesSync();
File(Platform.script.toFilePath()).readAsBytesSync()..[0] = 21;
final Uint8List smallExternalTypedDataView =
Uint8List.view(smallExternalTypedData.buffer, 1, 1);
@ -110,6 +113,17 @@ void expectGraphsMatch(dynamic a, dynamic b) {
}
return;
}
if (a is Set) {
final cb = b as Set;
Expect.equals(a.length, cb.length);
final aKeys = a.toList();
final cbKeys = cb.toList();
for (int i = 0; i < a.length; ++i) {
expectGraphsMatch(aKeys[i], cbKeys[i]);
}
return;
}
throw 'Unexpected object encountered when matching object graphs $a / $b';
}
@ -126,6 +140,8 @@ void expectViewOf(Uint8List view, Uint8List backing) {
class HashIncrementer {
static int counter = 1;
const HashIncrementer();
int get hashCode => counter++;
bool operator ==(other) => identical(this, other);
}
@ -135,21 +151,39 @@ class UserObject {
final double unboxedDouble;
final dynamic slot;
UserObject(this.unboxedInt, this.unboxedDouble, this.slot);
const UserObject(this.unboxedInt, this.unboxedDouble, this.slot);
}
class SendReceiveTest {
abstract class SendReceiveTestBase {
late final ReceivePort receivePort;
late final SendPort sendPort;
late final StreamIterator si;
SendReceiveTest();
SendReceiveTestBase();
Future run() async {
receivePort = ReceivePort();
sendPort = receivePort.sendPort;
si = StreamIterator(receivePort);
await runTests();
si.cancel();
receivePort.close();
print('done');
}
Future runTests();
Future<T> sendReceive<T>(T graph) async {
sendPort.send(graph);
Expect.isTrue(await si.moveNext());
return si.current as T;
}
}
class SendReceiveTest extends SendReceiveTestBase {
Future runTests() async {
await testTransferrable();
await testTransferrable2();
await testTransferrable3();
@ -161,6 +195,7 @@ class SendReceiveTest {
await testExternalTypedData3();
await testExternalTypedData4();
await testExternalTypedData5();
await testExternalTypedData6();
await testInternalTypedDataView();
await testInternalTypedDataView2();
@ -172,16 +207,18 @@ class SendReceiveTest {
await testExternalTypedDataView3();
await testExternalTypedDataView4();
await testArray();
await testMapRehash();
await testMapRehash2();
await testMapRehash3();
await testSetRehash();
await testSetRehash2();
await testSetRehash3();
await testFastOnly();
await testSlowOnly();
si.cancel();
receivePort.close();
print('done');
}
Future testTransferrable() async {
@ -293,6 +330,16 @@ class SendReceiveTest {
}
}
Future testExternalTypedData6() async {
print('testExternalTypedData6');
final etd = await sendReceive([
smallExternalTypedData,
largeExternalTypedData,
]);
Expect.equals(21, etd[0][0]);
Expect.equals(42, etd[1][0]);
}
Future testInternalTypedDataView() async {
print('testInternalTypedDataView');
final graph = [
@ -401,6 +448,18 @@ class SendReceiveTest {
expectGraphsMatch(graph, copiedGraph);
}
Future testArray() async {
print('testArray');
final oldSpace = List<dynamic>.filled(1024 * 1024, null);
final newSpace = UserObject(1, 1.1, 'foobar');
oldSpace[0] = newSpace;
final oldSpaceCopy = await sendReceive(oldSpace);
final newSpaceCopy = oldSpaceCopy[0] as UserObject;
Expect.equals(newSpaceCopy.unboxedInt, 1);
Expect.equals(newSpaceCopy.unboxedDouble, 1.1);
Expect.equals(newSpaceCopy.slot, 'foobar');
}
Future testMapRehash() async {
print('testMapRehash');
final obj = Object();
@ -437,7 +496,7 @@ class SendReceiveTest {
Future testMapRehash3() async {
print('testMapRehash3');
final obj = HashIncrementer();
final obj = const HashIncrementer();
final graph = [
{obj: 42},
notAllocatableInTLAB,
@ -448,6 +507,57 @@ class SendReceiveTest {
Expect.equals(before + 1, after);
}
Future testSetRehash() async {
print('testSetRehash');
final obj = Object();
final graph = <dynamic>[
<dynamic>{42, obj},
notAllocatableInTLAB,
];
final result = await sendReceive(graph);
final setCopy = result[0] as Set<dynamic>;
Expect.equals(2, setCopy.length);
Expect.equals(42, setCopy.toList()[0]);
Expect.equals(obj.runtimeType, setCopy.toList()[1].runtimeType);
Expect.notIdentical(obj, setCopy.toList()[1]);
Expect.notEquals(
identityHashCode(obj), identityHashCode(setCopy.toList()[1]));
Expect.isFalse(setCopy.contains(obj));
Expect.isTrue(setCopy.contains(setCopy.toList()[1]));
}
Future testSetRehash2() async {
print('testSetRehash2');
final obj = Object();
final graph = <dynamic>[
notAllocatableInTLAB,
<dynamic>{42, obj},
];
final result = await sendReceive(graph);
final setCopy = result[1] as Set<dynamic>;
Expect.equals(2, setCopy.length);
Expect.equals(42, setCopy.toList()[0]);
Expect.equals(obj.runtimeType, setCopy.toList()[1].runtimeType);
Expect.notIdentical(obj, setCopy.toList()[1]);
Expect.notEquals(
identityHashCode(obj), identityHashCode(setCopy.toList()[1]));
Expect.isFalse(setCopy.contains(obj));
Expect.isTrue(setCopy.contains(setCopy.toList()[1]));
}
Future testSetRehash3() async {
print('testSetRehash3');
final obj = const HashIncrementer();
final graph = [
{42, obj},
notAllocatableInTLAB,
];
final int before = HashIncrementer.counter;
await sendReceive(graph);
final int after = HashIncrementer.counter;
Expect.equals(before + 1, after);
}
Future testFastOnly() async {
print('testFastOnly');
for (final smallPrimitive in smallPrimitives) {
@ -469,12 +579,6 @@ class SendReceiveTest {
await sendReceive([notAllocatableInTLAB, smallContainer]));
}
}
Future<T> sendReceive<T>(T graph) async {
sendPort.send(graph);
Expect.isTrue(await si.moveNext());
return si.current as T;
}
}
main() async {

View file

@ -0,0 +1,75 @@
// Copyright (c) 2021, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy --gc-on-foc-slow-path --force-evacuation
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy --gc-on-foc-slow-path --force-evacuation
// The tests in this file will only succeed when isolate groups are enabled
// (hence the VMOptions above).
import 'dart:async';
import 'dart:isolate';
import 'dart:typed_data';
import 'package:expect/expect.dart';
import 'fast_object_copy_test.dart'
show UserObject, SendReceiveTestBase, notAllocatableInTLAB;
// When running with isolate groups enabled, we can share all of the following
// objects.
final sharableObjects = [
1,
0xffffffffffffffff,
'foobar',
const UserObject(1, 1.2, ''),
(() {
final rp = ReceivePort();
final sp = rp.sendPort;
rp.close();
return sp;
})(),
const [1, 2, 3],
const {1: 1, 2: 2, 3: 2},
const {1, 2, 3},
RegExp('a'),
Isolate.current.pauseCapability,
Int32x4(1, 2, 3, 4),
];
class SendReceiveTest extends SendReceiveTestBase {
Future runTests() async {
await testSharable();
await testSharable2();
}
Future testSharable() async {
final sharableObjectsCopy = await sendReceive([
...sharableObjects,
]);
Expect.notIdentical(sharableObjects, sharableObjectsCopy);
for (int i = 0; i < sharableObjects.length; ++i) {
Expect.identical(sharableObjects[i], sharableObjectsCopy[i]);
}
}
Future testSharable2() async {
final sharableObjectsCopy = await sendReceive([
notAllocatableInTLAB,
...sharableObjects,
]);
Expect.notIdentical(sharableObjects, sharableObjectsCopy);
Expect.equals(
notAllocatableInTLAB[0], (sharableObjectsCopy[0] as Uint8List)[0]);
for (int i = 0; i < sharableObjects.length; ++i) {
Expect.identical(sharableObjects[i], sharableObjectsCopy[i + 1]);
}
}
}
main() async {
await SendReceiveTest().run();
}

View file

@ -3,7 +3,10 @@
// BSD-style license that can be found in the LICENSE file.
// VMOptions=
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy --gc-on-foc-slow-path --force-evacuation --verify-store-buffer
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy --gc-on-foc-slow-path --force-evacuation --verify-store-buffer
// The tests in this file are particularly for an implementation that tries to
// allocate the entire graph in BFS order using a fast new space allocation
@ -28,7 +31,7 @@ final Uint8List largeExternalTypedData =
final Uint8List largeInternalTypedData = Uint8List(20 * 1024 * 1024)..[0] = 42;
final Uint8List smallExternalTypedData =
File(Platform.script.toFilePath()).readAsBytesSync();
File(Platform.script.toFilePath()).readAsBytesSync()..[0] = 21;
final Uint8List smallExternalTypedDataView =
Uint8List.view(smallExternalTypedData.buffer, 1, 1);
@ -110,6 +113,17 @@ void expectGraphsMatch(dynamic a, dynamic b) {
}
return;
}
if (a is Set) {
final cb = b as Set;
Expect.equals(a.length, cb.length);
final aKeys = a.toList();
final cbKeys = cb.toList();
for (int i = 0; i < a.length; ++i) {
expectGraphsMatch(aKeys[i], cbKeys[i]);
}
return;
}
throw 'Unexpected object encountered when matching object graphs $a / $b';
}
@ -126,6 +140,8 @@ void expectViewOf(Uint8List view, Uint8List backing) {
class HashIncrementer {
static int counter = 1;
const HashIncrementer();
int get hashCode => counter++;
bool operator ==(other) => identical(this, other);
}
@ -135,21 +151,39 @@ class UserObject {
final double unboxedDouble;
final dynamic slot;
UserObject(this.unboxedInt, this.unboxedDouble, this.slot);
const UserObject(this.unboxedInt, this.unboxedDouble, this.slot);
}
class SendReceiveTest {
abstract class SendReceiveTestBase {
ReceivePort receivePort;
SendPort sendPort;
StreamIterator si;
SendReceiveTest();
SendReceiveTestBase();
Future run() async {
receivePort = ReceivePort();
sendPort = receivePort.sendPort;
si = StreamIterator(receivePort);
await runTests();
si.cancel();
receivePort.close();
print('done');
}
Future runTests();
Future<T> sendReceive<T>(T graph) async {
sendPort.send(graph);
Expect.isTrue(await si.moveNext());
return si.current as T;
}
}
class SendReceiveTest extends SendReceiveTestBase {
Future runTests() async {
await testTransferrable();
await testTransferrable2();
await testTransferrable3();
@ -161,6 +195,7 @@ class SendReceiveTest {
await testExternalTypedData3();
await testExternalTypedData4();
await testExternalTypedData5();
await testExternalTypedData6();
await testInternalTypedDataView();
await testInternalTypedDataView2();
@ -172,16 +207,18 @@ class SendReceiveTest {
await testExternalTypedDataView3();
await testExternalTypedDataView4();
await testArray();
await testMapRehash();
await testMapRehash2();
await testMapRehash3();
await testSetRehash();
await testSetRehash2();
await testSetRehash3();
await testFastOnly();
await testSlowOnly();
si.cancel();
receivePort.close();
print('done');
}
Future testTransferrable() async {
@ -293,6 +330,16 @@ class SendReceiveTest {
}
}
Future testExternalTypedData6() async {
print('testExternalTypedData6');
final etd = await sendReceive([
smallExternalTypedData,
largeExternalTypedData,
]);
Expect.equals(21, etd[0][0]);
Expect.equals(42, etd[1][0]);
}
Future testInternalTypedDataView() async {
print('testInternalTypedDataView');
final graph = [
@ -401,6 +448,18 @@ class SendReceiveTest {
expectGraphsMatch(graph, copiedGraph);
}
Future testArray() async {
print('testArray');
final oldSpace = List<dynamic>.filled(1024 * 1024, null);
final newSpace = UserObject(1, 1.1, 'foobar');
oldSpace[0] = newSpace;
final oldSpaceCopy = await sendReceive(oldSpace);
final newSpaceCopy = oldSpaceCopy[0] as UserObject;
Expect.equals(newSpaceCopy.unboxedInt, 1);
Expect.equals(newSpaceCopy.unboxedDouble, 1.1);
Expect.equals(newSpaceCopy.slot, 'foobar');
}
Future testMapRehash() async {
print('testMapRehash');
final obj = Object();
@ -437,7 +496,7 @@ class SendReceiveTest {
Future testMapRehash3() async {
print('testMapRehash3');
final obj = HashIncrementer();
final obj = const HashIncrementer();
final graph = [
{obj: 42},
notAllocatableInTLAB,
@ -448,6 +507,57 @@ class SendReceiveTest {
Expect.equals(before + 1, after);
}
Future testSetRehash() async {
print('testSetRehash');
final obj = Object();
final graph = <dynamic>[
<dynamic>{42, obj},
notAllocatableInTLAB,
];
final result = await sendReceive(graph);
final setCopy = result[0] as Set<dynamic>;
Expect.equals(2, setCopy.length);
Expect.equals(42, setCopy.toList()[0]);
Expect.equals(obj.runtimeType, setCopy.toList()[1].runtimeType);
Expect.notIdentical(obj, setCopy.toList()[1]);
Expect.notEquals(
identityHashCode(obj), identityHashCode(setCopy.toList()[1]));
Expect.isFalse(setCopy.contains(obj));
Expect.isTrue(setCopy.contains(setCopy.toList()[1]));
}
Future testSetRehash2() async {
print('testSetRehash2');
final obj = Object();
final graph = <dynamic>[
notAllocatableInTLAB,
<dynamic>{42, obj},
];
final result = await sendReceive(graph);
final setCopy = result[1] as Set<dynamic>;
Expect.equals(2, setCopy.length);
Expect.equals(42, setCopy.toList()[0]);
Expect.equals(obj.runtimeType, setCopy.toList()[1].runtimeType);
Expect.notIdentical(obj, setCopy.toList()[1]);
Expect.notEquals(
identityHashCode(obj), identityHashCode(setCopy.toList()[1]));
Expect.isFalse(setCopy.contains(obj));
Expect.isTrue(setCopy.contains(setCopy.toList()[1]));
}
Future testSetRehash3() async {
print('testSetRehash3');
final obj = const HashIncrementer();
final graph = [
{42, obj},
notAllocatableInTLAB,
];
final int before = HashIncrementer.counter;
await sendReceive(graph);
final int after = HashIncrementer.counter;
Expect.equals(before + 1, after);
}
Future testFastOnly() async {
print('testFastOnly');
for (final smallPrimitive in smallPrimitives) {
@ -469,12 +579,6 @@ class SendReceiveTest {
await sendReceive([notAllocatableInTLAB, smallContainer]));
}
}
Future<T> sendReceive<T>(T graph) async {
sendPort.send(graph);
Expect.isTrue(await si.moveNext());
return si.current as T;
}
}
main() async {

View file

@ -620,6 +620,8 @@ class Object {
};
protected:
friend ObjectPtr AllocateObject(intptr_t, intptr_t);
// Used for extracting the C++ vtable during bringup.
Object() : ptr_(null_) {}
@ -1511,6 +1513,9 @@ class Class : public Object {
void set_num_native_fields(uint16_t value) const {
StoreNonPointer(&untag()->num_native_fields_, value);
}
static uint16_t NumNativeFieldsOf(ClassPtr clazz) {
return clazz->untag()->num_native_fields_;
}
#if !defined(DART_PRECOMPILED_RUNTIME)
CodePtr allocation_stub() const { return untag()->allocation_stub(); }
@ -8978,6 +8983,11 @@ class Smi : public Integer {
static ClassPtr Class();
static intptr_t Value(const SmiPtr raw_smi) { return RawSmiValue(raw_smi); }
#if defined(DART_COMPRESSED_POINTERS)
static intptr_t Value(const CompressedSmiPtr raw_smi) {
return Smi::Value(static_cast<SmiPtr>(raw_smi.DecompressSmi()));
}
#endif
static intptr_t RawValue(intptr_t value) {
return static_cast<intptr_t>(New(value));
@ -11221,6 +11231,9 @@ class Closure : public Instance {
static intptr_t function_offset() {
return OFFSET_OF(UntaggedClosure, function_);
}
static FunctionPtr FunctionOf(ClosurePtr closure) {
return closure.untag()->function();
}
#if defined(DART_PRECOMPILER)
FunctionTypePtr signature() const {
@ -11237,6 +11250,9 @@ class Closure : public Instance {
static intptr_t context_offset() {
return OFFSET_OF(UntaggedClosure, context_);
}
static ContextPtr ContextOf(ClosurePtr closure) {
return closure.untag()->context();
}
bool IsGeneric(Thread* thread) const { return NumTypeParameters(thread) > 0; }
intptr_t NumTypeParameters(Thread* thread) const;

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,26 @@
// Copyright (c) 2021, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
#ifndef RUNTIME_VM_OBJECT_GRAPH_COPY_H_
#define RUNTIME_VM_OBJECT_GRAPH_COPY_H_
namespace dart {
class Object;
class ObjectPtr;
// Makes a transitive copy of the object graph referenced by [object]. Will not
// copy objects that can be safely shared - due to being immutable.
//
// The result will be an array of length 2 of the format
//
// [<copy-of-root>, <array-of-objects-to-rehash / null>]
//
// If the array of objects to rehash is not `null` the receiver should re-hash
// those objects.
ObjectPtr CopyMutableObjectGraph(const Object& root);
} // namespace dart
#endif // RUNTIME_VM_OBJECT_GRAPH_COPY_H_

View file

@ -243,7 +243,9 @@ bool PortMap::IsReceiverInThisIsolateGroup(Dart_Port receiver,
MutexLocker ml(mutex_);
auto it = ports_->TryLookup(receiver);
if (it == ports_->end()) return false;
return (*it).handler->isolate()->group() == group;
auto isolate = (*it).handler->isolate();
if (isolate == nullptr) return false;
return isolate->group() == group;
}
void PortMap::Init() {

View file

@ -63,6 +63,7 @@ class PortMap : public AllStatic {
// Returns the owning Isolate for port 'id'.
static Isolate* GetIsolate(Dart_Port id);
// Whether the destination port's isolate is a member of [isolate_group].
static bool IsReceiverInThisIsolateGroup(Dart_Port receiver,
IsolateGroup* group);

View file

@ -150,6 +150,8 @@ enum TypedDataElementType {
friend class object##DeserializationCluster; \
friend class Serializer; \
friend class Deserializer; \
template <typename Base> \
friend class ObjectCopy; \
friend class Pass2Visitor;
// RawObject is the base class of all raw objects; even though it carries the
@ -540,7 +542,7 @@ class UntaggedObject {
void StorePointer(type const* addr, type value) {
reinterpret_cast<std::atomic<type>*>(const_cast<type*>(addr))
->store(value, order);
if (value->IsHeapObject()) {
if (value.IsHeapObject()) {
CheckHeapPointerStore(value, Thread::Current());
}
}
@ -552,7 +554,7 @@ class UntaggedObject {
reinterpret_cast<std::atomic<compressed_type>*>(
const_cast<compressed_type*>(addr))
->store(static_cast<compressed_type>(value), order);
if (value->IsHeapObject()) {
if (value.IsHeapObject()) {
CheckHeapPointerStore(value, Thread::Current());
}
}
@ -560,7 +562,7 @@ class UntaggedObject {
template <typename type>
void StorePointer(type const* addr, type value, Thread* thread) {
*const_cast<type*>(addr) = value;
if (value->IsHeapObject()) {
if (value.IsHeapObject()) {
CheckHeapPointerStore(value, thread);
}
}
@ -570,7 +572,7 @@ class UntaggedObject {
type value,
Thread* thread) {
*const_cast<compressed_type*>(addr) = value;
if (value->IsHeapObject()) {
if (value.IsHeapObject()) {
CheckHeapPointerStore(value, thread);
}
}
@ -772,6 +774,9 @@ class UntaggedObject {
friend class WriteBarrierUpdateVisitor; // CheckHeapPointerStore
friend class OffsetsTable;
friend class Object;
friend uword TagsFromUntaggedObject(UntaggedObject*); // tags_
friend void SetNewSpaceTaggingWord(ObjectPtr, classid_t, uint32_t); // tags_
friend class ObjectCopyBase; // LoadPointer/StorePointer
friend void ReportImpossibleNullError(intptr_t cid,
StackFrame* caller_frame,
Thread* thread);
@ -2808,6 +2813,12 @@ class UntaggedTypedDataBase : public UntaggedPointerBase {
private:
friend class UntaggedTypedDataView;
friend void UpdateLengthField(intptr_t, ObjectPtr, ObjectPtr); // length_
friend void InitializeExternalTypedData(
intptr_t,
ExternalTypedDataPtr,
ExternalTypedDataPtr); // initialize fields.
RAW_HEAP_OBJECT_IMPLEMENTATION(TypedDataBase);
};
@ -2899,6 +2910,7 @@ class UntaggedTypedDataView : public UntaggedTypedDataBase {
VISIT_TO(offset_in_bytes)
CompressedObjectPtr* to_snapshot(Snapshot::Kind kind) { return to(); }
friend void InitializeTypedDataView(TypedDataViewPtr);
friend class Api;
friend class Object;
friend class ObjectPoolDeserializationCluster;
@ -2964,6 +2976,8 @@ class UntaggedArray : public UntaggedInstance {
template <typename Table, bool kAllCanonicalObjectsAreIncludedIntoSet>
friend class CanonicalSetDeserializationCluster;
friend class OldPage;
friend class FastObjectCopy; // For initializing fields.
friend void UpdateLengthField(intptr_t, ObjectPtr, ObjectPtr); // length_
};
class UntaggedImmutableArray : public UntaggedArray {

View file

@ -186,6 +186,8 @@ vm_sources = [
"object.h",
"object_graph.cc",
"object_graph.h",
"object_graph_copy.cc",
"object_graph_copy.h",
"object_id_ring.cc",
"object_id_ring.h",
"object_reload.cc",

View file

@ -2,7 +2,8 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
library CountTest;

View file

@ -2,7 +2,8 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
import "dart:isolate";

View file

@ -2,7 +2,8 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
// Dart test program for testing isolate communication with

View file

@ -2,7 +2,8 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
library isolate_current_test;

View file

@ -2,7 +2,8 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
// Tests that sets of enums can be set through ports.

View file

@ -2,7 +2,8 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
import "dart:async";

View file

@ -2,7 +2,8 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
// Dart test program for testing serialization of messages.

View file

@ -2,7 +2,8 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
// Dart test program for testing serialization of messages.

View file

@ -2,7 +2,8 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
// Verifies that large BigInt can be passed through a message port and

View file

@ -2,7 +2,8 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
import "dart:io" show ServerSocket;

View file

@ -2,7 +2,8 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
import "dart:async";

View file

@ -4,7 +4,8 @@
// @dart = 2.9
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
library CountTest;

View file

@ -4,7 +4,8 @@
// @dart = 2.9
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
import "dart:isolate";

View file

@ -4,7 +4,8 @@
// @dart = 2.9
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
// Dart test program for testing isolate communication with

View file

@ -4,7 +4,8 @@
// @dart = 2.9
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
library isolate_current_test;

View file

@ -4,7 +4,8 @@
// @dart = 2.9
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
// Tests that sets of enums can be set through ports.

View file

@ -4,7 +4,8 @@
// @dart = 2.9
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
import "dart:async";

View file

@ -4,7 +4,8 @@
// @dart = 2.9
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
// Dart test program for testing serialization of messages.

View file

@ -4,7 +4,8 @@
// @dart = 2.9
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
// Dart test program for testing serialization of messages.

View file

@ -4,7 +4,8 @@
// @dart = 2.9
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
// Verifies that large BigInt can be passed through a message port and

View file

@ -4,7 +4,8 @@
// @dart = 2.9
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
import "dart:io" show ServerSocket;

View file

@ -4,7 +4,8 @@
// @dart = 2.9
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --no-enable-fast-object-copy
// VMOptions=--enable-isolate-groups --experimental-enable-isolate-groups-jit --enable-fast-object-copy
// VMOptions=--no-enable-isolate-groups
import "dart:async";