Commit graph

14816 commits

Author SHA1 Message Date
Liam Appelbe d6931e618e [vm] Make x86 disassembler able to handle invalid input
Arm, arm64, and riscv disassemblers already pass the new test.

Change-Id: Iaeb84d8db007d35225f77505503ca7614f82fb77
TEST=Disassembler_InvalidInput
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/303560
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Auto-Submit: Liam Appelbe <liama@google.com>
2023-05-16 06:31:21 +00:00
Derek Xu 0e3069ea23 [VM/Timeline] Support serializing duration events in Perfetto's format
TEST=Reported duration events using the embedder API, then retrieved the
trace using GetPerfettoVMTimeline and checked that the timestamps of
events were correct in the Perfetto trace viewer.

Change-Id: I077d7f545d9546727bcc3022c05dfc0b7b3826f3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302962
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
2023-05-15 20:51:38 +00:00
Derek Xu fe43431184 [VM/Timeline] Add perfetto_utils::SetTimestampAndMonotonicClockId
TEST=Retrieved a trace using GetPerfettoVMTimeline and checked that the
timestamps of events were correct in the Perfetto trace viewer.

Change-Id: Ie06e9e883a5d740022fa162bf7162dca6babcf58
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302961
Reviewed-by: Ben Konyi <bkonyi@google.com>
2023-05-15 20:51:38 +00:00
Derek Xu 61e280aa11 [VM/Timeline] Change return type of GetIsolate(Group)Id to unique_ptr
TEST=Retrieved a trace using GetPerfettoVMTimeline and checked that the
isolateIds and isolateGroupIds of events still showed up correctly in
the Perfetto trace viewer.

Change-Id: I13cf8dd2bb9af7fbe82da199b7f0d3c32a8cf1e7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302960
Reviewed-by: Ben Konyi <bkonyi@google.com>
2023-05-15 20:51:38 +00:00
Ryan Macnak a0a210a779 [vm] Initialize the is-zone-handle slot of read-only handles.
Cf. 715866aed3.

TEST=msan
Bug: https://github.com/dart-lang/sdk/issues/52351
Change-Id: I5d435f40b55a530cce0bb61ea2a9be232caf8e1f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/303145
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-05-15 15:41:28 +00:00
Ryan Macnak 20674eee48 [vm, gc] Force inline VisitPointersNonvirtual.
Avoids some spill overhead in a scavenger hot-path.

TEST=ci
Change-Id: Icdbec3d71f0a833c5005b76815e94091afe79c47
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/303004
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-05-12 21:26:07 +00:00
Ryan Macnak 4641a46077 [vm] Update comment on thread interrupters.
TEST=docs only
Change-Id: Ib2921c485defc232264d0c4f86b783ce33c6eadd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302848
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-05-11 21:41:01 +00:00
Ryan Macnak f4bd3590fc [vm, gc] Avoid double-checking when draining work lists.
TEST=ci
Change-Id: Icc1a8e42e58eee967e562e4e0429eaab33dd7fe0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302802
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-05-11 18:31:30 +00:00
Alexander Aprelev 3a668dcf03 [vm/win/aot] Provide unwinding information for Windows AOT snapshots.
Fixes https://github.com/dart-lang/sdk/issues/52045
TEST=ffi_induce_a_crash_test

Change-Id: Ic047df10732d6ff8cf695ce098d80ec3a098dbf9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302380
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2023-05-11 03:14:27 +00:00
Tess Strickland 4b8aa6a634 [vm] Update assembler to access AbstractType::flags_ as Uint32.
In d592882f, the old uint8_t `type_state_` and `nullability_` fields
in subclasses of `AbstractType` were merged into a single uint32_t
`flags_` field in `AbstractType` itself.

Accessing the nullability information was abstracted into two methods:

* `LoadAbstractTypeNullability`
* `CompareAbstractTypeNullabilityWith`

However, the code from these methods were based off the old
`CompareTypeNullabilityWith` and `CompareFunctionTypeNullabilityWith`
methods that used unsigned byte loads to access the old field, and kept
that even though the new field is now a 32-bit unsigned integer. This
works for now, because the nullability portion of the `flags_` field
happens to be the lowest two bits, but might lead to confusion if
someone attempts to write similar code for accessing the other parts of
the `flags_` field without realizing it's larger than 8 bits.

Thus, this CL does the following:

* Unifies the implementation of both methods across architectures.
* Adds a native slot definition for the `AbstractType::flags_` field
  and uses `LoadFromSlot` to load the `flags_` field instead. Other
  places where the `AbstractType::flags_` field is accessed from the
  assembler are also changed to use `LoadFromSlot` appropriately.

TEST=The existing suite of tests are sufficient.

Change-Id: Ie578e4dddce810223ca9e767403136cb40d340f3
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-release-x64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-linux-debug-x64c-try,vm-kernel-precomp-linux-release-simarm-try,vm-linux-release-x64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-linux-release-ia32-try,vm-linux-debug-simriscv64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302187
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-05-10 22:40:58 +00:00
Martin Kustermann 3ed8736bdd [vm] Make [ReloadOperationScope] a macro instead of a class
The [ReloadOperationScope] has [StackResource]s as fields and is itself
a [StackResource] which is problemantic if unwinding happens manually.

So we'll make it a macro that expands to the 3 fields instead.

TEST=ci

Change-Id: I3fb7bec7ca87193c83ec34908f9a43c5db005900
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302201
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2023-05-10 19:08:23 +00:00
Derek Xu 6c3c34560e [VM/Service] Add code for working with Perfetto protos to package:vm_service
TEST=get_perfetto_vm_timeline_rpc_test and
get_perfetto_cpu_samples_rpc_test

Change-Id: I23a30629f1d7a8cd7e2c9daad2d02fae4decb338
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302421
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
2023-05-10 19:06:10 +00:00
Derek Xu e26fe2817c [VM/Timeline] Start skipping over dart::TimelineEvents that provide no useful information when writing Perfetto traces
TEST=Checked that traces written using the Perfetto file recorder, and
traces retrieved through `getPerfettoVMTimeline` still looked correct.

Change-Id: I7e327ef525c99187fa8aea868d1cc48721af9f98
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302420
Reviewed-by: Ben Konyi <bkonyi@google.com>
2023-05-10 19:06:10 +00:00
Ryan Macnak 1c1e11b0e7 [vm, gc] Retry allocation under the safepoint before performing GC.
This avoids back-to-back GCs where multiple threads race to enter a safepoint operation to perform GC, and the losers of the race perform GC after they enter their own safepoint operation.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/29415
Change-Id: Ic9be830b0c4438f9b57ea9f54464b342872175f8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/300860
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-05-10 17:24:50 +00:00
Ryan Macnak 28c2491f8d [vm, service] Account for weakness in getRetainingPath and getRetainedSize.
We had been treating all references as strong for these RPCs. Now weak references are skipped, but ephemeron references are still treated as strong.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/49155
Change-Id: I6c4f344ce3c0df5bdbeb133a697bb26ff972f372
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302367
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-05-10 16:12:28 +00:00
Daco Harkes e5bb28bc4d [vm/ffi] Pointer.asTypedList finalizer
TEST=tests/ffi/external_typed_data_finalizer_test.dart

Closes: https://github.com/dart-lang/sdk/issues/50507
CoreLibraryReviewExempt: https://github.com/dart-lang/sdk/issues/52261
Change-Id: I1a82dcca15961b28c0de64637970fe38a39286e5
Cq-Include-Trybots: luci.dart.try:vm-asan-linux-release-x64-try,vm-aot-asan-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-qemu-linux-release-arm-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-aot-win-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-aot-mac-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/301001
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2023-05-10 11:38:57 +00:00
Liam Appelbe bd48058fe1 [vm] Switch offsets to hexadecimal
This makes them easier to match up with disassembly.

Change-Id: I9782365e104e807af34a06aee8697e8588583181
TEST=CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302221
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
2023-05-09 22:01:00 +00:00
Michael Thomsen 152296e642 Cleanup experiment flags formatting:
- Move down shipped 3.0 flags
- Sort all flags consistently

TEST=Manually verified

Change-Id: I21c209df4ff11322512e50a2be666293b5a9743f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302323
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Michael Thomsen <mit@google.com>
2023-05-09 21:08:41 +00:00
Tess Strickland a41d1b5813 [vm] Clear out specialized stubs for record types on reload.
There's also no need to visit function types on reload, as their
TTSes do not specialize.

TEST=vm/cc/TTS_Reload

Change-Id: Ibbc7bc220b9b75aab27b3e6815be06db461ec46f
Cq-Include-Trybots: luci.dart.try:vm-reload-linux-debug-x64-try,vm-reload-linux-release-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-reload-rollback-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302321
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-05-09 14:53:47 +00:00
Vyacheslav Egorov 54be4bc80e [vm/service] Remove Stack.awaiterFrames
This field was supposed[1] to replace Stack.asyncCausalFrames but IDEs
and other service protocol users never adopted it.

The mechanism for collecting awaiterFrames and asyncCausalFrames
was originally considerably different, but we have since unified
both[2] after we switched to lazy async stack traces[3].

Today the only differences between the two are:

- asyncCausalFrames represens async gaps explicitly, while
  awaiterFrames does not;
- when asynchronous function is running its synchronous part
  before the first suspension awaiterFrames will still represent
  the corresponding frame as an asynchronous activation, while
  asyncCausalFrames will treat the same frame as regular frame.

Consequently having this field does not add any value.

This CL removes it from the response and updates all tests to
test against asyncCausalFrames.

[1]: https://codereview.chromium.org/2692803006/
[2]: https://dart-review.googlesource.com/c/sdk/+/167561
[3]: https://github.com/dart-lang/sdk/issues/37668

Tested: pkg/vm_service, service, service_2
Bug: https://github.com/dart-lang/sdk/issues/51763
Change-Id: If2b035a8840047423b8ed8ce3b5d81155e9f38d0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/301062
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2023-05-09 13:46:56 +00:00
Martin Kustermann 7f38a7965e [vm] Ensure reload safepoint operation sends OOBs to all non-parked mutators
The reload safepoint operation mechanism will send out-of-band message
to all isolates that are not already at-reload-safepoint.

It identified all such threads by iterating the active threads list,
filtering those that are not at-reload-safepoint and those with an
active isolate (i.e. `thread->isolate() != nullptr`).

This `thread->isolate() != nullptr` condition isn't quite
correct. A mutator may temporarily run under a [NoActiveIsolateScope] to
make the `Isolate::Current()` unavailable to code that shouldn't depend
on isolates (e.g. GC and Compiler do that) or may even have the
incorrect isolate on it (e.g. Debugger Service notifications).

So if one thread triggers a [ReloadSafepointOperation] and another
thread is under a [NoActiveIsolateScope] it will not get the OOB message
and therefore not get interrupted to check-in.

This has caused flaky timeouts of `vm/dart/isolates/reload_active_stack_test`
- as this test runs isolates with active stack without going back
to event loop (which would check into reload operations).

Closes https://github.com/dart-lang/sdk/issues/52135

TEST=Updated vm/cc/Reload_NotAtSafepoint for regression test
TEST=Fixes flaky timeouts of vm/dart/isolates/reload_active_stack_test

Change-Id: Ib407c42aa97798ac994aff3bce263da79b83666a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302320
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2023-05-09 13:01:37 +00:00
Johnni Winther 13c4ad23c5 [kernel] Add SwitchStatement.expressionType
This adds the static type of the switch statement expression to
the AST to better support optimizations based on the possible
runtime values of the switch expression.

TEST=existing

Change-Id: Ief35b687150401100d8c96721a026e9f0911b3b4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/301063
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Reviewed-by: Mayank Patke <fishythefish@google.com>
2023-05-08 07:39:02 +00:00
Vyacheslav Egorov dd2f556730 [vm] Reoder CFI directives in generated assembly.
This should not actually matter, but we discovered
that LLVM code responsible for emitting compact unwind information
expects this specific ordering of CFI directives. If we don't
follow the order then LLVM fails to emit compact unwind info and emits
__eh_frame instead which is very large.

See https://github.com/llvm/llvm-project/issues/62574.

Fixes https://github.com/flutter/flutter/issues/126004.

Tested: manually via pkg/vm/tool/precompiler2 --build-assembly and objdump --unwind-info
Change-Id: Idb1f4f64afdaa7206bf43adf1685bb1f4086217f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/301740
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2023-05-05 18:39:09 +00:00
Derek Xu cd83aa8494 [VM/Service] Prepare to publish package:vm_service 11.5.0
TEST=`dart pub publish --dry-run`

Change-Id: I6ef1c8175f39a086716be859068518d124124718
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/301540
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
2023-05-04 20:20:58 +00:00
Alexander Markov c1f9930f5b [vm] Restore InstanceKind.TypeRef in the VM-service API for backwards compatibility
TEST=doc/comment change
Change-Id: I0fcb9d418a9c95e6ce57a03eb693aefb9e245d2a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/301501
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2023-05-04 18:04:13 +00:00
Derek Xu 1dd644f595 [VM/Service] Add getPerfettoCpuSamples RPC
TEST=Checked that Observatory's timeline view was still able to load CPU
samples correctly, and ensured that samples retrived using
getPerfettoCpuSamples were the same as the ones shown in Observatory.

Change-Id: I9b58cd32bc9a1c08848718f25f10db9fa6d4d241
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/297760
Commit-Queue: Derek Xu <derekx@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2023-05-03 19:17:11 +00:00
Derek Xu da971bbf13 [VM/Timeline] Add assertions to check that the recorder's lock is held where required
TEST=Checked that the timeline in DevTools still looked correct, and
checked that no assertions failed when fetching the timeline

Change-Id: Ia7ed4b8faf4e7c9e8871ecca3111a82e7f298318
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/300420
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-05-03 19:17:11 +00:00
Derek Xu 2dc4db8b57 [VM/Timeline] Ensure that ReclaimCachedBlocksFromThreads is called before events are serialized
TEST=Checked that the timeline in DevTools still looked correct, TSAN,
ASAN

Change-Id: Ibefd92f23728a5b7a7d9db4ae972a75576069ea0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299820
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
2023-05-01 14:52:30 +00:00
Derek Xu 0606c58fde [VM/Timeline] Correctly skip over blocks that are in use when serializing the timeline
TEST=TSAN, ASAN

Change-Id: I3b106c8f0bf3fef5fec8f9fd7146a13401ab9448
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299761
Reviewed-by: Ben Konyi <bkonyi@google.com>
2023-05-01 14:52:30 +00:00
Derek Xu c4f31a9108 [VM/Timeline] Lock accesses to the async track metadata map where needed
This CL also adds the `dart_support_perfetto` GN setting and enables it
by default, and adds a test to ensure that TSAN checks for races related
to reporting async events.

TEST=TSAN, ASAN

Change-Id: I5f43ceca0185287ca4d8e295a509088ea50fd9c3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299740
Reviewed-by: Ben Konyi <bkonyi@google.com>
2023-05-01 14:52:30 +00:00
asiva c4cb74cb9d [VM/GC] Ensure that all the GC verification flags do not produce trace
output unless there is an assertion or Fatal error.

(This is needed to ensure that we are able to run the flutter tests with
all these verifications turned on and not disturbing the output from the
tests with these extraneous trace messages)

TEST=ci

Change-Id: Ifb7211e340f1cf29f6d2bff412eb4107f9cb64a2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299440
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-04-28 23:25:43 +00:00
Alexander Markov 50e0e616c0 [vm] Fix gcc build
TEST=ci

Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-riscv64-try
Change-Id: Ib44701aa648c34f032456ee1dad6dc34fdc1584a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299940
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-04-28 21:26:41 +00:00
Alexander Markov 8d0677eca5 [vm] Simplify type finalization
After recursive types are gone, finalization of types can be
simplified: types can be finalized with a simple recursive traversal
and they no longer need to track "being finalized" state.

Also, finalization doesn't need to write back finalized types
because type finalization without canonicalization is always performed
in place and doesn't result in a new type (or type arguments vector).

TEST=ci

Change-Id: Ifc0a4e12aa410eb2be18c4a475c2f486cfa9ebf2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299300
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-04-28 19:37:10 +00:00
Alexander Markov 2f2af4cfae [vm] Cleanup unnecessary SetHash(0)
After recursive types are gone, there is no need to reset hash code
of types during canonicalization, as it cannot change.

TEST=ci

Change-Id: Ibc64f08b52a66829f72dbae5906771f134bbffb2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299260
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2023-04-28 19:37:10 +00:00
Alexander Markov e46b97e1b7 [vm] Cleanup UntaggedTypeParameter::parameterized_class_id_
For class type parameters 'parameterized_class_id' duplicates
a recently added 'owner' and can be removed.
'owner' is changed to hold class reference as a class id (Smi).

TEST=ci

Change-Id: I93fc11e6bcfcc00058a5281b497f59c5c5847ea6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299160
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2023-04-28 19:37:10 +00:00
Alexander Markov 2ee6fcf514 [vm] Remove TypeRef
TypeRef type wraps around another type and it was used to represent
a graph of recursive types. After [0], the only use of TypeRef is
for TypeParameter.bound which may indirectly reference the same
TypeParameter.

This change replaces TypeParameter.bound with TypeParameter.owner and
removes TypeRef entirely. Various parts of the VM no longer need to
handle and support TypeRefs.

TypeParameter.owner can reference a FunctionType, Class,
or, as an optimization, it can be set to null in order to share
class type parameters among different classes.

With the exception of the 'TypeParameter.owner' back pointer,
VM types are now not recursive and can be visited without
additional tracking.

Caveats:

* Generic FunctionType cannot be cloned in a shallow way:
  when copying a FunctionType, type parameters should be cloned too
  and their owners should be updated. For that reason,
  a mapping between 'from' and 'to' function types
  (FunctionTypeMapping) is maintained during type transformations
  such as InstantiateFrom.
  FunctionType::Clone is used instead of Object::Clone where
  appropriate.

* When testing types for subtyping and equivalence, mapping
  between function types is passed to make sure
  type parameters belong to the equivalent function types.

* IL serializer needs to serialize function types as a whole before
  serializing any types potentially pointing into the middle of a
  function type (such as return type 'List<Y0>' pointing into
  the middle of a function type 'List<Y0> Function<Y0>()').

[0] https://dart-review.googlesource.com/c/sdk/+/296300

TEST=ci

Change-Id: I67c2fd0117c6183a45e183919a7847fd1af70b3e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294165
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2023-04-28 19:29:36 +00:00
Ryan Macnak 709ba7aa24 [vm] Use PR_SET_VMA_ANON_NAME on Linux too.
This was upstreamed to Linux in 5.17.

TEST=ci
Change-Id: Idcc0c6a80a8f8e148bf13ed4a1d9d54122379ff9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/279206
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-04-28 18:29:58 +00:00
Alexander Aprelev 86d754690a [vm/isolates] Put limit on how many isolates are suspended.
Fixes https://github.com/dart-lang/sdk/issues/52154
TEST=ci

Change-Id: I6f620113232cf8f771dd73be334ace0e385784f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/297920
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2023-04-26 22:18:51 +00:00
Alexander Aprelev 4b7de54e03 [vm/regex] Pick a fix for unicode negated range RegExp.
Pick up v8 fix https://chromium-review.googlesource.com/c/v8/v8/+/3802690.

Fixes https://github.com/dart-lang/sdk/issues/52182
TEST=corelib/regexp/regexp_test

Change-Id: I01057ec5c8babdafc06ddeea3db48e14ae7c5357
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/298700
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
2023-04-26 19:46:06 +00:00
Alexander Markov 233a7298e0 [vm] Fix assertion failure when taking a tear-off of a constructor with nested function type
TEST=runtime/tests/vm/dart/regress_52179_test.dart
Fixes https://github.com/dart-lang/sdk/issues/52179

Change-Id: I449e2aebe3ffeb78f4f9a213ec40e6892f7a0447
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/298660
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-04-26 17:33:50 +00:00
Ryan Macnak cc3bf609f2 [vm, gc] Avoid suprious race between GC helper thread exit and GC visiting roots.
A sweeper helper thread may be exiting while the scavenge is visiting thread roots. GC helpers have no reusable handle state to clear, so the race is an uninteresting one between null and null.

TEST=tsan
Bug: https://github.com/dart-lang/sdk/issues/52125
Change-Id: Iae68f8a3a938ae20e540b11c172fbc4a50a90a9c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/298262
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-04-25 22:10:09 +00:00
Ben Konyi 8d97964e6e [ VM ] Move class name from heap profiler report callback to allocation callback
Moving the class name of the allocated object to the allocation callback
gives more flexibility to the embedder, which may decide to track all
sampled allocations, not just those that are still alive.

TEST=Updated DartAPI_HeapSampling_*

Change-Id: Iaae290e15b67ca3e47d72e1896fa6e32b2d4b081
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/297761
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2023-04-25 13:02:49 +00:00
Derek Xu 5978b607d2 [VM/Timeline] Make a header file for perfetto_utils
This is being done to allow us to access the utils from
profiler_service.cc.

TEST=Requested a trace using the getPerfettoVMTimeline RPC and confirmed
that it still looked correct.

Change-Id: Icfb7a5b41da0fc987a72098c4345e7e108b6566e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/297740
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
2023-04-24 18:29:23 +00:00
Ryan Macnak 78daab15b3 [vm] Remove FindObject.
Before the calling convention change during the introduction of AOT, this was used to get code metadata associated with a frame. Its remaining non-test use is forming service IDs for code object, which can  use the regular visitors.

TEST=ci
Change-Id: Ica8eafe988d19eba8d905d9dc5907afbfd559118
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296705
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-04-21 22:41:28 +00:00
Ryan Macnak 29472dbb73 [vm, gc] Avoid race between unregistering pages with LSAN and exit.
After the Page is unregistered, the only reference to its VirtualMemory is in memory not traced by LSAN. If exit happens here before the VirtualMemory is deleted or put into the global cache, it appears to be leaked.

TEST=lsan
Bug: https://github.com/dart-lang/sdk/issues/51560
Change-Id: I973e09bb19910d9cdcc3af6c8c9a188f095df36a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/297040
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-04-21 19:32:40 +00:00
Martin Kustermann 79d133ab3a [vm] Fix product build by if/defing out reload tests
TEST=Fixes product build of run_vm_tests

Change-Id: I62fae4aba74cca6ebe1b3ce965fc015db3277e02
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296803
Reviewed-by: Alexander Aprelev <aam@google.com>
2023-04-21 17:38:00 +00:00
Alexander Markov d83caf1c0c [vm/compiler] Support obfuscation of record field names
TEST=language/records/simple/* on vm-aot-obfuscate-linux-release-x64

Fixes https://github.com/dart-lang/sdk/issues/52124
Fixes https://github.com/flutter/flutter/issues/125289

Cq-Include-Trybots: luci.dart.try:vm-aot-obfuscate-linux-release-x64-try
Change-Id: Ic53ae5282ba1d30b395fa706840393fd467abf2e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296920
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2023-04-21 15:14:58 +00:00
Martin Kustermann 2ea92acba5 [vm] Make reloading of isolate groups use new safepoint-level mechanism
The current hot-reload implementation [0] will perform a reload by
first sending OOB messages to all isolates and waiting until those OOB
messages are being handled. The handler of the OOB message will block
the thread (and unschedule isolate) and notify the thread performing
reload it's ready.

This requires that all isolates within a group can actually run & block.
This is the case for the VM implementation of isolates (as they are
run an unlimited size thread pool).

Though flutter seems to multiplex several engine isolates on the same OS
thread. Reloading can then result in one engine isolate performing
reload waiting for another to act on the OOB message (which it will not
do as it's multiplexed on the same thread as the former).

Now that we have a more flexible safepointing mechanism (introduced in
[1]) we can utilize for hot reloading by introducing a new "reloading"
safepoint level.

Reload safepoints
-----------------------

We introduce a new safepoint level (SafepointLevel::kGCAndDeoptAndReload).

Being at a "reload safepoint" implies being at a "deopt safepoint"
which implies being at a "gc safepoint".

Code has to explicitly opt-into making safepoint checks participate /
check into "reload safepoints" using [ReloadParticipationScope]. We do
that at certain well-defined places where reload is possible (e.g. event
loop boundaries, descheduling of isolates, OOM message processing, ...).

While running under [NoReloadScope] we disable checking into "reload
safepoints".

Initiator of hot-reload
-----------------------

When a mutator initiates a reload operation (e.g. as part of a
`ReloadSources` `vm-service` API call) it will use a
[ReloadSafepointOperationScope] to get all other mutators to a
safepoint.

For mutators that aren't already at a "reload safepoint", we'll
notify them via an OOB message (instead of scheduling kVMInterrupt).

While waiting for all mutators to check into a "reload safepoint", the
thread is itself at a safepoint (as other mutators may perform lower
level safepoint operations - e.g. GC, Deopt, ...)

Once all mutators are at a "reload safepoint" the thread will take
ownership of all safepoint levels.

Other mutators
-----------------------

Mutators can be at a "reload safepoint" already (e.g. isolate is not
scheduled). If they try to exit safepoint they will block until the
reload operation is finished.

Mutators that are not at a "reload safepoint" (e.g. executing Dart or VM
code) will be sent an OOB message indicating it should check into a
"reload safepoint". We assume mutators make progress until they can
process OOB message.

Mutators may run under a [NoReloadScope] when handling the OOM message.
In that case they will not check into the "reload safepoint" and simply
ignore the message. To ensure the thread will eventually check-in,
we'll make the destructor of [~NoReloadScope] check & send itself a new OOB
message indicating reload should happen. Eventually getting the mutator
to process the OOM message (which is a well-defined place where we can
check into the reload safepoint).

Non-isolate mutators such as the background compiler do not react to OOB
messages. This means that either those mutators have to be stopped (e.g.
bg compiler) before initiating a reload safepoint operation, the
threads have to explicitly opt-into participating in reload safepoints
or the threads have to deschedule themselves eventually.

Misc
----

Owning a reload safepoint operation implies also owning the deopt &
gc safepoint operation. Yet some code would like to ensure it actually
runs under a [DeoptSafepointOperatoinScope]/[GCSafepointOperationScope].
=> The `Thread::OwnsGCSafepoint()` handles that.

While performing hot-reload we may exercise common code (e.g. kernel
loader, ...) that acquires safepoint locks. Normally it's disallows to
acquire safepoint locks while holding a safepoint operation (since
mutators may be stopped at places where they hold locks, creating
deadlock scenarios).
=> We explicitly opt code into participating in reload safepointing
requests. Those well-defined places aren't holding safepoint locks.
=> The `Thread::CanAcquireSafepointLocks()` will return `true` despite
owning a reload operation. (But if one also holds deopt/gc safepoint
operation it will return false)

Example where this matters: As part of hot-reload, we load kernel which
may create new symbols. The symbol creation code may acquire the symbol
lock and `InsertNewOrGet()` a symbol. This is safe as other mutators
don't hold the symbol lock at reload safepoints. The same cannot be said
for Deopt/GC safepoint operations - as they can interrupt code at many
more places where there's no guarantee that no locks are held.

[0] https://dart-review.googlesource.com/c/sdk/+/187461
[1] https://dart-review.googlesource.com/c/sdk/+/196927

Issue https://github.com/flutter/flutter/issues/124546

TEST=Newly added Reload_* tests.

Change-Id: I6842d7d2b284d043cc047fd702b7c5c7dd1fa3c5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296183
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2023-04-21 13:56:49 +00:00
Martin Kustermann 742612ddac [vm] Add Isolate::{Enter,Exit}AsNonMutator() APIs, use in sweeper
The sweeper threads are bypassing safepoints and have no need for normal
mutator state (e.g. storebuffer, marking stacks, .,..) on the [Thread]
object.

This fixes a TSAN report where sweeper would clear reusable handles
(which it didn't actually modify) and scavenger is reading those
handles (which arguably it doesn't have to either).

Issue https://github.com/dart-lang/sdk/issues/52125

TEST=ci

Change-Id: I03a36e8518b6c00eb7f3b57f65fd469dddba23e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296860
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2023-04-21 11:23:47 +00:00
Martin Kustermann 2e466f66db [vm] Refactor thread scheduling code to better handle exits with active stack
An embedder (or the VM) can exit an isolate via `Thread::ExitIsolate()`
at a point where there's still active state (e.g. dart frames).

Because of this the VM has so far conservatively retained the [Thread]
object of dart mutators throughout the isolate's lifetime. After which
is was manually `delete`ed. We'd never re-use those [Thread] objects (we
do re-use [Thread] objects of non-dart-mutator threads).

When exiting via `Thread::ExitIsolate()` with active state, the mutator
was assumed to be at-safepoint at all levels. It was removed from the
thread registry's active threads. This also means that when e.g. GC runs
it can't use the thread registry to find all active threads it may
need to scan, instead it uses [Isolate::mutator_thread_] of all isolates.

This causes a variety of subtle issues, but the main one that motivated
this change is the following:

If a thread obtains a safepoint operation it means all other mutators
are parked. The thread owning the safepoint can do whatever it likes.
When introducing reload operation safepoints, a thread may want to

    ReloadSafepointOperation reload(thread);
    ...

    // Compile sources.
    {
      TransitionVMToNative transition(thread);

      // Will temporarily exit & re-enter current isolate.
      response_port = Dart_NewNativePort();

      Dart_PostCObject(kernel_isolate_port, ...);

      // Wait on [response_port] for response.
    }

This will cause the reloading thread to own the reload safepoint
operation but still transition states and even exit/re-enter the
isolate. Though this is currently not possible in the way enter/exit is
implemented.

So we'll refactor this fragile code in the following way:

* Move thread enter/exit logic entirely to the [Thread] object.

* Keep used threads in the thread registry's active list.

  => This allows us to keep various state on the [Thread] and thereby
  avoids clearing it when suspending & re-initialing it when resuming

  => It makes nested `Thread::ExitIsolate()` faster as we mainly have
  to enter safepoint (avoid acquiring threads lock, avoid releasing
  storebuffers, ...)

  => It makes nested `Thread::EnterIsolate()` faster as we mainly have
  to leave the safepoint (avoid acquiring threads lock, avoid acquiring
  storebuffers, ...).

  => A mutator can now own a safepoint operation (e.g. reload safepoint
  operation) and still `ExitSafepoint()` / `EnterSafepoint()` safely -
  as those are based on the normal `EnterSafepoint()` and
  `LeaveSafepoint()` APIs.

* We separate

   - Suspend & Resume of a dart mutator (possibly with active stack)
   - Setup & Reset of state only relevant for dart mutators
   - Setup & Reset of state relevant for any mutator

* We unify how the [Thread] objects are freed between dart mutator and
  non-dart mutators: [Thread] objects without state can be given back to
  the [ThreadRegistry] and re-used (instead of being deleted in
  `Isolate::~Isolate`)

* We have capability to free [Thread] objects if a dart mutator has an
  empty stack & re-use for another isolate of the same group.
  (In future we may have N Thread objects for N cores and the threads
   would even maintain their TLABs when switching between isolates)

* Since we allow reusing of [Thread] objects also for dart mutators now,
  we have extensive asserts to ensure they are "clean" when they get
  into the free list and come out "clean" again.

TEST=ci

Change-Id: Id85e8e484efd98d28e323b33795716420e619986
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296585
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2023-04-21 08:06:49 +00:00