Commit graph

513 commits

Author SHA1 Message Date
Ryan Macnak a7e20dd2b0 Revert "[vm, gc] Mark through new-space."
This reverts commit 6194209b28.

Reason for revert: issues on arm32

Original change's description:
> [vm, gc] Mark through new-space.
>
>  - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
>  - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
>  - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
>  - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
>  - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I3668a2e56821f9eadf96e38c228dab27be656016
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309826
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>

Change-Id: I434eb595c9e7858efc8c9b07cbca954e5649f506
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319321
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-08-09 18:13:17 +00:00
Ryan Macnak 6194209b28 [vm, gc] Mark through new-space.
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
 - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
 - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
 - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
 - reachabilityBarrier is now just a count of major GCs.

TEST=ci
Change-Id: I3668a2e56821f9eadf96e38c228dab27be656016
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309826
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-08-09 15:57:17 +00:00
Martin Kustermann 389aab624d [vm] Allow non-TSAN gen_snapshot target TSAN AOT runtime
This is a revert of b6bbfff8c0c6c44cdab0d7d78 which temporarily disabled
this.

Turns out that a non-TSAN gen_snapshot would - when emitting code via
`Assembler::TsanLoadAcquire` - use an incorrect `Thread`-offset:

It calculated the offset via `Thread::OffsetFromThread`. That function
took a `dart::RuntimeEntry*` and tried to find its offset.

We happen to have the following leaf runtime entries:

  #define LEAF_RUNTIME_ENTRY_LIST(V)
      ...
      V(void, MsanUnpoison, void*, size_t)
      V(void, MsanUnpoisonParam, size_t)
      V(void, TsanLoadAcquire, void*)
      V(void, TsanStoreRelease, void*)
      ...

It loops over all runtime entries and finds the first one that has the
identical `dart::RuntimeEntry::function_` pointer.

Though all 4 of them are `nullptr` at `gen_snapshot` time, so when
searching for offset for

  `Thread::OffsetFromThread(kTsanLoadAcquireRuntimeEntry)`

it looked for the first runtime entry with `nullptr` function pointer -
which turned out to be `MsanUnpoison` (instead of `TsanLoadAcquire`).

=> The obvious fix is to use the `dart::RuntimeEntry*` pointer for
comparison instead of it's `function_` member.

TEST=ci
Issue b/287638965

Change-Id: I85c06674927978ef8561e9e7bdfab4823c0a8e1c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/312902
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2023-07-10 11:53:58 +00:00
Martin Kustermann 2cb0882a2d [vm] Unblock dart->g3 roll
The adjustments to g3 build rules that allow running non-TSAN
gen_snapshot targeting TSAN AOT runtime aren't quite fleshed out yet.

So we temporarily let `gen_snapshot` detect whether to target TSAN by
whether it is itself compiled in TSAN (instead of via
`-DTARGET_USES_THREAD_SANITIZER`)

TEST=ci
Issue b/287638965

Change-Id: I2cd744f959d8302ba81cb1feceedee2756e02656
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/312901
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2023-07-07 19:28:58 +00:00
Martin Kustermann 9d876d12b2 [vm] Rename kUsingThreadSanitizer->kTargetUsesThreadSanitizer, include tsan in snapshot flags
TEST=ci
Issue b/287638965

Change-Id: I649b28ac0181761d0927e80b8eff7c8af5fe305a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/312840
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2023-07-07 14:42:22 +00:00
Martin Kustermann 82577191ec [vm] Allow build rules to pass TARGET_USES_THREAD_SANITIZER in any configuration
This may simplify some g3 build rules as it allows passing this define
whenever we compile under TSAN - instead currently it can only be passed
if DART_PRECOPMILER is also passed.

TEST=ci

Change-Id: I73e51243716373cdeca7559b344408bb206e22ba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/312801
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2023-07-07 09:02:32 +00:00
Martin Kustermann 2287c76724 [vm] Make AOT compiler use TARGET_USES_THREAD_SANITIZER
The VM has two parts related to TSAN: compiler & runtime.

So far both parts used self-detection whether running with TSAN. This
implies that if AOT runtime runs under TSAN that the AOT compiler then
also has to run under TSAN (which is slow).

We separate the two by making the compiler use
`TARGET_USES_THREAD_SANITIZER`. It will be auto-detected by-default
except when `DART_PRECOPMILER` is set. In the precompiler case we
require the build rules to pass in whether the target runs under
TSAN or not.

Issue b/287638965

TEST=ci

Change-Id: Ib333a8a69ed2d7bdca7596014a0d25b103af25b6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/312480
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2023-07-06 08:44:33 +00:00
Ilya Yanok ac06671ac6 Revert "[vm] Mark Zone memory as unallocated/allocated/uninitialized for Address and Memory Sanitizer."
This reverts commit c7b288492a.

Reason for revert: breaks tests in G3, b/287581831

Original change's description:
> [vm] Mark Zone memory as unallocated/allocated/uninitialized for Address and Memory Sanitizer.
>
> TEST=ci
> Change-Id: Ia283d9aefec767e6ccc4f1c88abba73ce1c35e87
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212022
> Reviewed-by: Brian Quinlan <bquinlan@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>

Change-Id: I94ce2141e7b16c8e4be6641253e37dfa82698486
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309861
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Ilya Yanok <yanok@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2023-06-16 11:22:38 +00:00
Ryan Macnak c7b288492a [vm] Mark Zone memory as unallocated/allocated/uninitialized for Address and Memory Sanitizer.
TEST=ci
Change-Id: Ia283d9aefec767e6ccc4f1c88abba73ce1c35e87
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212022
Reviewed-by: Brian Quinlan <bquinlan@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-06-14 21:50:17 +00:00
Liam Appelbe 07f587504b Reland "[vm] Migrate FFI callbacks to the new metadata system."
This reverts https://dart-review.googlesource.com/c/sdk/+/306674

Patchset 1 is a pure rollback of the rollback.
Patchset 2 is https://dart-review.googlesource.com/c/sdk/+/306316
Patchset 4+ is the forward fix for the Fuchsia issues.

The Fuchsia bug that we're fixing (or working around), is that
VirtualMemory::DuplicateRX doesn't work on Fuchsia. A proper fix will
require special casing it, like on MacOS. In the mean time we can avoid
using this function by only allowing one page of trampolines on Fuchsia.
Unfortunately, when I removed the BSS stuff from the original CL, it
was necessary to duplicate even the first page, so I've had to add that
stuff back just for Fuchsia.

Change-Id: Id42de78ee5de126bcc83bfa4148f6efb4045f976
Bug: https://github.com/dart-lang/sdk/issues/52579
Bug: https://buganizer.corp.google.com/issues/284959841
Fixes: https://github.com/dart-lang/sdk/issues/52581
TEST=CI, especially vm-fuchsia-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306676
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2023-06-06 02:07:58 +00:00
Liam Appelbe 8ea3e9fd0d Revert "[vm] Migrate FFI callbacks to the new metadata system."
Roll back https://dart-review.googlesource.com/c/sdk/+/302903
and https://dart-review.googlesource.com/c/sdk/+/306124

Since the bug is in VirtualMemory::DuplicateRX, also roll back
https://dart-review.googlesource.com/c/sdk/+/303960
https://dart-review.googlesource.com/c/sdk/+/304400
https://dart-review.googlesource.com/c/sdk/+/304620

Bug: https://buganizer.corp.google.com/issues/284959841
Change-Id: Id212de209fb57b2c3395bb61064dc8f569832884
TEST=CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306674
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-06-01 01:00:58 +00:00
Liam Appelbe f0f732b1a6 [vm] Migrate FFI callbacks to the new metadata system.
This is the same metadata system that will support async callbacks. The
main thing this does is take the trampolines that used to be JIT only,
and use them in AOT too. This is partly a safety thing (there's some
extra checks that used to be skipped on AOT), but mostly just so that
the metadata system is unified between the sync and async callbacks.

More details about the design:
https://docs.google.com/document/d/1QDjyY_6wOTOgURwpeYMKU9qEz0gKxx2MUrdruC6Kp6c/edit?usp=sharing

I split this off the async CL. Some of the comments refer to async
stuff that doesn't exist yet, but it's coming immediately after this so
I didn't update them.

Change-Id: Icd5e86934ee9ae34c2c0e2ed2bbd1b928a7184ac
TEST=ffi_callback_metadata_test.cc and CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302903
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2023-05-24 22:22:04 +00:00
Martin Kustermann 2d230aa0b5 [vm/ffi] Move ffi-callback related state from Thread to ObjectStore, move jit trampolines from Isolate to IsolateGroup
The ffi-callback related information on the [Thread] object is metadata
corresponding to ffi-callback-trampoline [Function] objects. There is
nothing thread or isolate specific about it.

Moving it away from [Thread] is needed because an [Isolate] can
have different [Thread] objects across its lifetime (see [0]): When
the stack of an isolate is empty, we reserve now the right to
re-cycle the [Thread]. If the isolate later runs again, it may
get a new [Thread] object.

This CL moves this information from [Thread] to the [ObjectStore]. In
addition we make the compiler be responsible for populating this
metadata - instead of doing this per-call site of
`Pointer.fromFunction()`. It will be preserved across snapshot writing
& snapshot reading (for AppJIT as well as AppAOT).

Similarly the JIT trampolines that are on Isolate aren't isolate
specific and can go to [IsolateGroup]. This simplifies doing the above
as the compiler can allocate those as well.

The effect is that [Thread] object gets smaller, GC doesn't have to
visit the 2 slots per-thread. It comes at expense of 2 more loads
when invoking the callback.

[0] https://dart-review.googlesource.com/c/sdk/+/297920

TEST=Regression test is vm/ffi{,_2}/invoke_callback_after_suspension_test

Change-Id: Ifde46a9f6e79819b5c0e359c3d3998d1d93b9b1e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/303700
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2023-05-17 11:23:28 +00:00
Ryan Macnak 6250eeca95 [vm] Make clz/ctz inlinable.
TEST=ci
Change-Id: I0eb4480e165259763befd3e7ee519fda111faefe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/303040
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-05-15 17:52:49 +00:00
Tess Strickland 6b922021a1 [platform] Fix include of vm/globals.h instead of platform/globals.h.
TEST=build

Change-Id: If9e88f09268ff9aecd483734924c9c6f6170d4fe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302783
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2023-05-11 11:53:56 +00:00
Alexander Aprelev 3a668dcf03 [vm/win/aot] Provide unwinding information for Windows AOT snapshots.
Fixes https://github.com/dart-lang/sdk/issues/52045
TEST=ffi_induce_a_crash_test

Change-Id: Ic047df10732d6ff8cf695ce098d80ec3a098dbf9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302380
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2023-05-11 03:14:27 +00:00
asiva c4cb74cb9d [VM/GC] Ensure that all the GC verification flags do not produce trace
output unless there is an assertion or Fatal error.

(This is needed to ensure that we are able to run the flutter tests with
all these verifications turned on and not disturbing the output from the
tests with these extraneous trace messages)

TEST=ci

Change-Id: Ifb7211e340f1cf29f6d2bff412eb4107f9cb64a2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299440
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-04-28 23:25:43 +00:00
Derek Xu 6011ba67b3 Reland "Reland "[VM] Begin supporting Perfetto file recorder""
This is a reland of commit 4cd9c9c666

This will be relanded without changes because we discussed the .proto
definition package name clashing problem with the Dart in google3 team,
and we have decided that the best solution is to ignore the checked in
.proto files in google3 builds.

TEST=Recorded traces with the Perfetto file recorder and explored them
in Perfetto's trace viewer, CI

Original change's description:
> Reland "[VM] Begin supporting Perfetto file recorder"
>
> This is a reland of commit 7424295ce9
>
> The differences between this reland and the original CL are: now the
> Perfetto file recorder does not get built in PRODUCT builds, and it does
> not get built unless the SUPPORT_PERFETTO macro is defined.
>
> TEST=Recorded traces with the Perfetto file recorder and explored them
> in Perfetto's trace viewer, CI
>
> Original change's description:
> > [VM] Begin supporting Perfetto file recorder
> >
> > This CL adds the `TimelineEventPerfettoFileRecorder` class, which is a
> > timeline recorder that writes a trace to file in Perfetto's proto
> > format. This CL supports the recording of all types of timeline events
> > except flow events. Support for flow events will be added in a future
> > CL.
> >
> > TEST=Recorded traces with the Perfetto file recorder and explored them
> > in Perfetto's trace viewer, CI
> >
> > Change-Id: Iaa2051e536589a473c5e15f9de9bb9c251f13d38
> > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278942
> > Reviewed-by: Ben Konyi <bkonyi@google.com>
> > Commit-Queue: Derek Xu <derekx@google.com>
>
> Change-Id: I8713f704b5fbeed5f1231012bce8a32aaf566ae4
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286020
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Derek Xu <derekx@google.com>

Change-Id: Ia97ef1f0fe73e76c7022623b9ae7e21a4ea73a46
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/295804
Commit-Queue: Derek Xu <derekx@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2023-04-18 19:39:05 +00:00
Ilya Yanok 3c284a0c95 Revert "Reland "[VM] Begin supporting Perfetto file recorder""
This reverts commit 4cd9c9c666.

Reason for revert: reusing the same proto package for copied files breaks rolling it to G3.

Original change's description:
> Reland "[VM] Begin supporting Perfetto file recorder"
>
> This is a reland of commit 7424295ce9
>
> The differences between this reland and the original CL are: now the
> Perfetto file recorder does not get built in PRODUCT builds, and it does
> not get built unless the SUPPORT_PERFETTO macro is defined.
>
> TEST=Recorded traces with the Perfetto file recorder and explored them
> in Perfetto's trace viewer, CI
>
> Original change's description:
> > [VM] Begin supporting Perfetto file recorder
> >
> > This CL adds the `TimelineEventPerfettoFileRecorder` class, which is a
> > timeline recorder that writes a trace to file in Perfetto's proto
> > format. This CL supports the recording of all types of timeline events
> > except flow events. Support for flow events will be added in a future
> > CL.
> >
> > TEST=Recorded traces with the Perfetto file recorder and explored them
> > in Perfetto's trace viewer, CI
> >
> > Change-Id: Iaa2051e536589a473c5e15f9de9bb9c251f13d38
> > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278942
> > Reviewed-by: Ben Konyi <bkonyi@google.com>
> > Commit-Queue: Derek Xu <derekx@google.com>
>
> Change-Id: I8713f704b5fbeed5f1231012bce8a32aaf566ae4
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286020
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Derek Xu <derekx@google.com>

Change-Id: I2da67ffe5bff01d568f32f682455bb2c789499b0
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/295821
Commit-Queue: Ilya Yanok <yanok@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Oleh Prypin <oprypin@google.com>
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
2023-04-18 11:25:49 +00:00
Ryan Macnak c4795a7550 [vm, compiler] Don't do safepoint transitions in generated code under TSAN.
Go to the runtime so TSAN sees the instrumented access to Thread::safepoint_state_, avoiding false data race reports.

TEST=tsan
Bug: https://github.com/dart-lang/sdk/issues/52024
Change-Id: Ic686bac2221d8ac6b9865ce9c82a4c36711037a7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/295740
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-04-17 22:37:57 +00:00
Derek Xu b4aa83b0b7 [VM/Service] Rename JSONStream::AppendSerializedObject to JSONStream::AppendBytes
TEST=CI

Change-Id: I2bb950c608716156e0dfe357a0fd481463b8d0dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/287800
Reviewed-by: Ben Konyi <bkonyi@google.com>
2023-04-17 15:58:28 +00:00
Derek Xu 4cd9c9c666 Reland "[VM] Begin supporting Perfetto file recorder"
This is a reland of commit 7424295ce9

The differences between this reland and the original CL are: now the
Perfetto file recorder does not get built in PRODUCT builds, and it does
not get built unless the SUPPORT_PERFETTO macro is defined.

TEST=Recorded traces with the Perfetto file recorder and explored them
in Perfetto's trace viewer, CI

Original change's description:
> [VM] Begin supporting Perfetto file recorder
>
> This CL adds the `TimelineEventPerfettoFileRecorder` class, which is a
> timeline recorder that writes a trace to file in Perfetto's proto
> format. This CL supports the recording of all types of timeline events
> except flow events. Support for flow events will be added in a future
> CL.
>
> TEST=Recorded traces with the Perfetto file recorder and explored them
> in Perfetto's trace viewer, CI
>
> Change-Id: Iaa2051e536589a473c5e15f9de9bb9c251f13d38
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278942
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Derek Xu <derekx@google.com>

Change-Id: I8713f704b5fbeed5f1231012bce8a32aaf566ae4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286020
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
2023-04-17 14:42:50 +00:00
Ryan Macnak 8b0f4a8c61 [vm] Update to constexpr in runtime/platform and lib.
TEST=build
Change-Id: I61ebb04f414d36813a99b424fb1bcdb440f68ad5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/293801
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2023-04-11 17:43:58 +00:00
Ryan Macnak 1eae028526 [vm] Fix zero-as-null-pointer-constant warnings in the VM.
Unfortunately we cannot enable -Wzero-as-null-pointer-constant because of zlib and icu headers we include.

TEST=build
Change-Id: I9ffd7d7e26f2b3dd616ce54d164b92e60a2366dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/293882
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2023-04-10 21:38:57 +00:00
Ryan Macnak e0e14c370a [vm] Update NULL to nullptr in runtime/platform.
TEST=build
Change-Id: I1a01702451f1be91f14c3a860fa3f84b0c686409
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292062
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-03-30 20:49:48 +00:00
Ryan Macnak 25548a912f [vm, reload] NoSuchMethodError after a static target goes missing or changes signature.
- Mostly restores the pre-kernel behavior, except that actual arguments are not passed to the NoSuchMethodError.
 - Instead of a vague error message, the call site whose target is missing is now part of the stack trace and the name/kind of the missing target is part of the exception message.
 - If the missing target is on a branch not taken, things properly succeed.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/37517
Bug: https://github.com/flutter/flutter/issues/122626
Change-Id: Ic3d5a386caa8d1627d8452a5601bc728ad519987
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/291055
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-03-30 19:27:17 +00:00
Vyacheslav Egorov 56ad16f4de [vm] Make VM strict about min required Mac OS X version.
Refuse to initialize VM if current Mac OS X version is below
MAC_OS_X_VERSION_MIN_REQUIRED, which was configured in build
time through mac_sdk_min GN argument.

TEST=tested manually by changing return value of MacOSXVersion

Cq-Include-Trybots: luci.dart.try:dart-sdk-mac-arm64-try
Bug: https://github.com/flutter/flutter/issues/121739
Change-Id: I9d913516077dbb196e8d6ebd7fe9f3914de7c748
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286932
Reviewed-by: Michael Thomsen <mit@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2023-03-08 09:08:39 +00:00
Ryan Macnak 63e6041ca9 [vm] Update to variadic FATAL.
TEST=ci
Change-Id: Ic6bc784605e10760bb28ea6df34242336a33b4d0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286947
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-03-06 22:06:59 +00:00
Oleh Prypin 2e01df50e9 Revert "[VM] Begin supporting Perfetto file recorder"
This reverts commit 7424295ce9.

Reason for revert: Causes VM crash in internal tests - b/271081079

Original change's description:
> [VM] Begin supporting Perfetto file recorder
>
> This CL adds the `TimelineEventPerfettoFileRecorder` class, which is a
> timeline recorder that writes a trace to file in Perfetto's proto
> format. This CL supports the recording of all types of timeline events
> except flow events. Support for flow events will be added in a future
> CL.
>
> TEST=Recorded traces with the Perfetto file recorder and explored them
> in Perfetto's trace viewer, CI
>
> Change-Id: Iaa2051e536589a473c5e15f9de9bb9c251f13d38
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278942
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Derek Xu <derekx@google.com>

Change-Id: I1c4b0b78be3748b525b82ea66079dd95d412d82e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/285906
Auto-Submit: Oleh Prypin <oprypin@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2023-02-28 12:41:00 +00:00
Derek Xu 7424295ce9 [VM] Begin supporting Perfetto file recorder
This CL adds the `TimelineEventPerfettoFileRecorder` class, which is a
timeline recorder that writes a trace to file in Perfetto's proto
format. This CL supports the recording of all types of timeline events
except flow events. Support for flow events will be added in a future
CL.

TEST=Recorded traces with the Perfetto file recorder and explored them
in Perfetto's trace viewer, CI

Change-Id: Iaa2051e536589a473c5e15f9de9bb9c251f13d38
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278942
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
2023-02-27 18:37:48 +00:00
Ryan Macnak f56c45eb37 [vm] Speed up JSON encoding.
TEST=ci
Change-Id: I8bfe00472f3a5e4e6680de631072cea0dacc3f55
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/283980
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-02-21 18:58:42 +00:00
Josh Soref 01b28894e7 Spelling pkg dev compiler
Closes https://github.com/dart-lang/sdk/pull/50861

GitOrigin-RevId: 71005e6f5bf5a151cb5c1aefb6a2a300fc40f592
Change-Id: Iadfafb5787a62e9a379437f6a3763d31f99ba7c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/277743
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2023-01-26 09:12:41 +00:00
Josh Soref ef42a0b110 Spelling pkg analyzer lib
Closes https://github.com/dart-lang/sdk/pull/50860

GitOrigin-RevId: b27066c37f93c8c6d1123d6ebd6a4c0afcf59844
Change-Id: I15fa4aea1dad45daf168e34d1c4450320ec9b40a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/277742
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2023-01-25 14:08:27 +00:00
Josh Soref f7a2ea5e06 Spelling
Closes https://github.com/dart-lang/sdk/pull/50922

GitOrigin-RevId: 58fd7cfd5ef470a65a52ea28e0407244d853c917
Change-Id: I2e5a5ed991cb05270170a18b8f0169daa9eabdb7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278537
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2023-01-24 18:00:25 +00:00
Josh Soref 84e3c8b50f Spelling tests
Closes https://github.com/dart-lang/sdk/pull/50920

GitOrigin-RevId: fa87531bd0f52b69485c9d02ff9e44a4a29c6a91
Change-Id: I0ae8574a5b77087895e004079f221201bb550cf3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278535
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2023-01-19 16:24:29 +00:00
Ryan Macnak 8a37b70a53 [vm, compiler] More consistently support large load/store offsets.
TEST=ci
Change-Id: I7691808679c57c6f16f3859641cf52b9e606e304
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/275381
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-12-14 22:17:35 +00:00
Vyacheslav Egorov c704e11ee2 [vm/api] Introduce Dart_WriteHeapSnapshot API
Allows embedders to programmatically request heap snapshot from a
running isolate group instead of relying on a vm-service or hidden
internal Dart APIs

Additionally we allow to include snapshot writing functionality into
PRODUCT builds using DART_ENABLE_HEAP_SNAPSHOT_WRITER define.

TEST=vm/cc/DartAPI_WriteHeapSnapshot

Bug: b/259115846
Change-Id: Ic3ef76e5fb9adcf8f23a1959f5238742b64ecde2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/273181
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2022-12-01 14:38:44 +00:00
Tess Strickland 4f925105cf [vm] Use a hash map for larger type argument instantiation caches.
Previously, the VM used a linear array to cache previous instantiations
of a type arguments object. Now once the cache hits a certain number
of occupied entries, the VM changes to using a hash-based approach.

The InstantiateTypeArguments stubs have not yet been updated to
traverse the hash-based cache, so once the cache has grown too large,
all attempts at instantiations, even those that are in the cache, go to
the runtime. Thus, until the stubs are updated, this is only an
improvement if the cost of traversing the linear cache dominates the
cost of making a runtime call. Our benchmarks see a ~40% performance
regression for hash-based caches of size 100 but a ~400% performance
improvement for hash-based caches of size 1000. Thus, we currently
split the difference and set the maximum size of linear caches to 500.

TEST=vm/cc/TypeArguments_Cache_ManyInstantiations

Bug: https://github.com/dart-lang/sdk/issues/48344
Change-Id: I7f1376943523bb5bcd8b175cfb1936779ea73d60
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-debug-simriscv64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/265325
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2022-11-28 15:00:20 +00:00
Ryan Macnak 28c4d87b34 [vm, gc] Extend alignment helpers to work with alignment offsets.
TEST=ci
Change-Id: I02bad1feba3a16360aa52414c94cdef81b158ed6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/262000
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2022-10-06 00:50:45 +00:00
Tess Strickland b0c4ddf919 [pkg/dart2native] Avoid overwriting section contents in MachO files.
To create a Dart standalone executable on MacOS, we modify the
dartaotruntime executable to add the snapshot contents, and the
VM looks into the executable on disk to find the snapshot to load.

Previously, we did this by adding a new 64-bit segment load command
with a single section, where the section's file offset and size
describes the inserted snapshot. This meant the Mach-O header size increased by 152 bytes.

Originally, this wasn't an issue as there was plenty of padding, but
later clang updates removed most of this padding, and so writing the
new header actually overwrote the initial contents of the first section
in the file, which happens to be the __text section. In addition, since
the first section's offset was now declared to be within the header,
utilities that strictly validated the Mach-O format, like codesign,
would report errors.

This CL changes it so that we actually reserve space in the
dartaotruntime header using the -add_empty_section flag to the linker.
In addition, we change from using a segment load command to using a
(40 byte) note load command. This is because a segment load command
specifies that the contents should be loaded in memory, but we don't
use that loaded version. Instead, the VM reloads it from the executable
on disk so it can appropriately mmap the different parts of the
snapshot. A note section instead just declares a section of the
executable as arbitrary data that the owner can read from the file
and use as desired, which is semantically closer to our current usage.

This CL also adds a test to pkg/dartdev/test/commands/compile_test to
ensure that corrupting a random part of the snapshot in the executable
causes signature verification to fail.

This CL also reverts CL 256208, thus relanding the clang changes
starting from June that originally raised awareness of the issue by
greatly reduced the amount of padding after the load commands.

TEST=pkg/dartdev/test/commands/compile_test

Bug: https://github.com/dart-lang/sdk/issues/49783
Change-Id: Iee554d87b0eabaecd7a534ca4e4facfefbce6385
Cq-Include-Trybots: luci.dart.try:analyzer-mac-release-try,dart-sdk-mac-arm64-try,dart-sdk-mac-try,pkg-mac-release-arm64-try,pkg-mac-release-try,vm-kernel-precomp-mac-product-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/260108
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2022-09-29 08:32:47 +00:00
Ryan Macnak cba1a09e5c [vm, compiler] Inform MSan about parameters coming from FFI.
- Mark the outgoing arguments stack area as initialized.
 - Mark the trampoline's incoming parameter slots as initialized in case they are used as local handles.
 - Mark the outgoing register arguments as initialized.

TEST=msan
Bug: https://github.com/dart-lang/sdk/issues/42314
Bug: https://github.com/dart-lang/sdk/issues/49298
Bug: https://github.com/dart-lang/sdk/issues/49957
Change-Id: Ifa978e1b905a424ec9f64b89879cea6e82f70d12
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/259102
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-09-23 21:31:14 +00:00
Vyacheslav Egorov 83ab5d5ca3 [vm] Remove dynamic field unboxing in JIT
Instead apply the same approach as we do in AOT: unbox based on the
static type information. There are no TFA results available in JIT,
but we could still unbox fields when running in sound null-safety.

TEST=ci

Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try
Change-Id: Ide2e78c6659261ef8d245a4586cf699ea0fbb459
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256211
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2022-09-20 13:42:45 +00:00
Vyacheslav Egorov 4a4eedd860 [vm] Clean up ClassTable
* Merge ClassTable and SharedClassTable back together;
* Simplify handling of multiple arrays growing in sync;
* Refactor how reload deals with ClassTable.

The last change is the most important because it makes it
much easier to reason about the code. We move away from
copying bits and pieces of the class table and shared
class table into reload contexts.

Having two class table fields in the isolate group makes
it easier to reason about. One field contains program
class table (one modified by kernel loader and accessed
by various program structure cid lookups) and heap
walk class table (used by GC visitors). Normally these
two fields point to the same class table, but during
hot reload we temporary split them apart: original
class table is kept as a heap walk class table, while
program class table is replaced by a clone and updated
by reload.

If reload succeeds we drop original class table and
set program class table as heap walk one.

If reload fails we drop the program class table and
restore original one from heap walk table.

TEST=ci

Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-asan-linux-release-x64-try
Change-Id: I8b66259fcc474dea7dd2af063e4772df99be06c4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/258361
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-09-10 15:12:35 +00:00
Vyacheslav Egorov afb34fb9e3 [vm] Fix kNumberOfReservedCpuRegisters on ARM64
Instead of using an integer literal use constexpr popcount
function instead.

The misalignment causes an incorrect generation of a call
to a shared write barrier stub for the last allocatable register
on ARM64.

Reported by Lin Zuojian <zuojian.lzj@alibaba-inc.com>

TEST=ci

Change-Id: I69ce32da958573be0ec8967e462f8a378a778a28
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/254401
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-08-10 18:04:09 +00:00
Tess Strickland 24683da915 [vm] Add OS and architecture to non-symbolic stack traces.
Examples of the new line added to non-symbolic stack traces:

os: linux arch: x64 comp: yes sim: no
(Running on linux-x64c)

os: macos arch: arm64 comp: no sim: yes
(Running on mac-simarm64)

This CL also abstracts out the separate hardcoded strings across
the codebase for host and target OS and architecture into
definitions in platform/globals.h to ensure that they stay
in sync across different uses.

TEST=vm/dart{,_2}/use_dwarf_stack_traces_flag

Issue: https://github.com/flutter/flutter/pull/101586
Change-Id: Ifdfea5138dd1003f561da0174e89aebc165bf9b0
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-mac-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-ffi-android-release-arm-try,vm-ffi-android-release-arm64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/253283
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2022-08-05 11:53:37 +00:00
Kapandaria 7fa9a6776a [vm] Update bit algorithms in utils.cc
Closes https://github.com/dart-lang/sdk/pull/48942

GitOrigin-RevId: 484c201343c546abdfdc8669f87e844c26c0fd43
Change-Id: Ib918e0deca4cccbeef2cc6a17a84ec95a74e80f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/243361
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2022-05-16 07:30:40 +00:00
Ryan Macnak 3c01efcbb0 Reland "[build] Fix Android build for ARM64 Mac."
Account for simarm_x64 in TargetCpuForArch.

TEST=local
Bug: https://github.com/dart-lang/sdk/issues/48792
Change-Id: I820cb49ee8925c50196cb2472266f9e248c9089d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/243642
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-05-10 16:11:53 +00:00
Ryan Macnak 5f1bda06ee Revert "[build] Fix Android build for ARM64 Mac."
This reverts commit b50396740f.

Reason for revert: vm-kernel-precomp-linux-debug-simarm_x64

Original change's description:
> [build] Fix Android build for ARM64 Mac.
>
> TEST=local
> Bug: https://github.com/dart-lang/sdk/issues/48792
> Change-Id: Ia43dd46f90ef5f592e324bb889c20c221f69998a
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/241102
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Reviewed-by: William Hesse <whesse@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>

TBR=whesse@google.com,rmacnak@google.com,dacoharkes@google.com

Change-Id: I0cfdb5653dff7675afdb005df0c37f7560b26ee3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/48792
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/241860
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-04-20 23:13:29 +00:00
Ryan Macnak b50396740f [build] Fix Android build for ARM64 Mac.
TEST=local
Bug: https://github.com/dart-lang/sdk/issues/48792
Change-Id: Ia43dd46f90ef5f592e324bb889c20c221f69998a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/241102
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-04-20 17:15:48 +00:00
Ryan Macnak 32715d1d5b [vm] Mark ELF libraries produced directly by the VM as not requiring an executable stack.
Even if the main program disables executable stacks, dlopen'ing a library that doesn't itself disable executable stack will switch the stack to executable. (Presumably in the name of compatability with GNU nested functions.)

Dart does not need executable stacks, and executable stacks are an additional attack surface.

TEST=readelf
Bug: b/229648756
Change-Id: Ia8c234ebc6178a26528d37b1359a10dd42039a9b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/241540
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-04-19 18:51:47 +00:00