Commit graph

348 commits

Author SHA1 Message Date
Liam Appelbe e8d7425c4e [vm/ffi] Closure callbacks for sync callbacks
Bug: https://github.com/dart-lang/sdk/issues/52689
Change-Id: I54be397cfbf8519fe5b5a51b793fe46d602124d9
Fixes: https://github.com/dart-lang/sdk/issues/52689
Bug: https://github.com/dart-lang/sdk/issues/53096
TEST=isolate_local_function_callbacks_test.dart, plus generated tests and additions to existing tests
CoreLibraryReviewExempt: The isolate and FFI packages are VM-only
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/317060
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2023-08-25 03:35:44 +00:00
Ryan Macnak 7d23f32389 [vm, compiler] Kick CSP ahead of SP in FFI callback stub.
TEST=profiler
Bug: https://github.com/dart-lang/sdk/issues/53303
Change-Id: Id73046f91c52e39e675c89797e8896ba09d4baa6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322200
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2023-08-23 00:17:02 +00:00
Liam Appelbe 5c723e996e [vm/ffi] Weak RawReceivePorts and NativeCallables
Bug: https://github.com/dart-lang/sdk/issues/49083
Change-Id: I9643fe54e46042021146a19e6ec1c1d967cd6496
Fixes: https://github.com/dart-lang/sdk/issues/49083
CoreLibraryReviewExempt: The isolate and FFI packages are VM-only
TEST=raw_port_test.dart and async_void_function_callbacks_test.dart
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/314360
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-08-14 02:30:11 +00:00
Liam Appelbe edeac698c2 [vm/ffi] Closure callbacks for async callbacks
This change is almost trivial. The closure is stored on the callback's
RawReceivePort, not in the VM. So we can basically just remove the CFE
check and it pretty much works. The only problem is that we can't set
function.FfiCallbackTarget anymore, so most of the CL is dealing with
that.

A few places were deciding whether an FFI trampoline was a call or a
callback based on whether function.FfiCallbackTarget() was null. But
now the target will be null for async callbacks. So instead I've added
a new value to the FfiCallbackKind enum (and renamed it), and changed
those checks.

Sync callback closures will be a separate CL, because they're more
complicated.

Bug: https://github.com/dart-lang/sdk/issues/52689
Change-Id: I8e5dfb557362e679f66195b735c3c382e6792840
TEST=async_void_function_callbacks_test.dart
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316160
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2023-07-26 23:23:26 +00:00
Daco Harkes 3915c9cd44 [test/ffi] Fix status file
https://dart-review.googlesource.com/c/sdk/+/302322 didn't match the
behavior and didn't cover aot.

Bug: https://github.com/dart-lang/sdk/issues/39487
Change-Id: Id1bf70ab72d19cd4c1e531362099389eebb46835
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316383
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-07-26 16:50:46 +00:00
asiva 5d872a8737 [VM/Runtime] - Deprecate Dart_UpdateExternalSize and Dart_UpdateFinalizableExternalSize from the Dart C API
https://github.com/dart-lang/sdk/issues/52568

TEST=new test added.

Change-Id: I4f4b6b607fa2e0daa14fc25e3bdf99214f6ccb6b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/314900
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2023-07-20 17:06:44 +00:00
Liam Appelbe eb6baa33ad [vm] Async callback fast path when on target isolate
Skip creating the temporary isolate if the current thread is already
entered into the target isolate. We can marshall the args and send the
message directly from this isolate.

Bug: https://github.com/dart-lang/sdk/issues/52764
Change-Id: I94a1358ad7d2632b53c122cc38cd5675e6f5ecbc
Fixes: https://github.com/dart-lang/sdk/issues/52764
TEST=CI, specifically async_void_function_callbacks_test.dart
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/312351
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-07-11 21:26:48 +00:00
Ryan Macnak 99ef36064f [infra] Make the sanitizer a first-class status variable.
Makes the TSAN skips apply to
 `test.py -mrelease --sanitizer=tsan`
and not just
 `test.py -n vm-tsan-linux-release-x64`.

Also makes the timeouts agree.

Change-Id: I10315e754a4ebb3020f3c2f6cecfac6b77e77a9b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311828
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-07-10 17:46:31 +00:00
Liam Appelbe f4eb5392e5 Refactor vmspecific_static_checks_test
Bug: https://github.com/dart-lang/sdk/issues/52754
Change-Id: I13d729f0caf61fa0ef261ffd1c382177a4115d8f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/312342
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2023-07-10 00:07:28 +00:00
Liam Appelbe c3a532bc36 [vm] Deflake async callbacks test
Change-Id: Ia12ac5380f90760a23297b449d24e324f6156daa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311800
Auto-Submit: Liam Appelbe <liama@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-06-28 10:26:21 +00:00
Liam Appelbe 13ec07415b [vm] Async FFI callbacks
More details about the design:
https://docs.google.com/document/d/1QDjyY_6wOTOgURwpeYMKU9qEz0gKxx2MUrdruC6Kp6c/edit?usp=sharing

Change-Id: Ie3985d86dca7f5010044ca46c33ca177588c0f69
Bug: #37022
CoreLibraryReviewExempt: Reviewed by vm and api groups. web and wasm groups not affected because FFI isn't on those platforms.
TEST=async_void_function_callbacks_test.dart, ffi_callback_metadata_test.cc, other front end tests
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/305900
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-06-28 01:00:18 +00:00
Simon Binder 62488db4da [vm/ffi] Add DynamicLibrary.close()
This adds the `close()` method to `DynamicLibrary`, which uses
`dlclose()` on Unix and `FreeLibrary` on Windows to close a dynamic FFI
library.

TEST=tests/ffi/dylib_close_test.dart
Closes https://github.com/dart-lang/sdk/issues/40159

Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-riscv64-try,vm-ffi-qemu-linux-release-arm-try,dart2wasm-linux-d8-try,dart2wasm-linux-chrome-try,vm-reload-rollback-linux-debug-x64-try,vm-reload-linux-debug-x64-try,vm-win-debug-x64c-try,vm-win-debug-x64-try,vm-aot-android-release-arm_x64-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-asan-linux-release-x64-try,vm-aot-msan-linux-release-x64-try,vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try
CoreLibraryReviewExempt: `dart:ffi` VM only API.
Change-Id: I73af98677b481902fe548bdcee56964a0195faf0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/307680
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-06-15 17:19:37 +00:00
Sergey G. Grekhov ef2fd96213 [tests] Remove obsolete Dart 3.0 experiments from FFI tests
Change-Id: I6909ccbbdd3e2a7e9ef403155f3ce0a41d5c173f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309680
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2023-06-15 08:25:18 +00:00
Liam Appelbe 07f587504b Reland "[vm] Migrate FFI callbacks to the new metadata system."
This reverts https://dart-review.googlesource.com/c/sdk/+/306674

Patchset 1 is a pure rollback of the rollback.
Patchset 2 is https://dart-review.googlesource.com/c/sdk/+/306316
Patchset 4+ is the forward fix for the Fuchsia issues.

The Fuchsia bug that we're fixing (or working around), is that
VirtualMemory::DuplicateRX doesn't work on Fuchsia. A proper fix will
require special casing it, like on MacOS. In the mean time we can avoid
using this function by only allowing one page of trampolines on Fuchsia.
Unfortunately, when I removed the BSS stuff from the original CL, it
was necessary to duplicate even the first page, so I've had to add that
stuff back just for Fuchsia.

Change-Id: Id42de78ee5de126bcc83bfa4148f6efb4045f976
Bug: https://github.com/dart-lang/sdk/issues/52579
Bug: https://buganizer.corp.google.com/issues/284959841
Fixes: https://github.com/dart-lang/sdk/issues/52581
TEST=CI, especially vm-fuchsia-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306676
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2023-06-06 02:07:58 +00:00
Liam Appelbe 8ea3e9fd0d Revert "[vm] Migrate FFI callbacks to the new metadata system."
Roll back https://dart-review.googlesource.com/c/sdk/+/302903
and https://dart-review.googlesource.com/c/sdk/+/306124

Since the bug is in VirtualMemory::DuplicateRX, also roll back
https://dart-review.googlesource.com/c/sdk/+/303960
https://dart-review.googlesource.com/c/sdk/+/304400
https://dart-review.googlesource.com/c/sdk/+/304620

Bug: https://buganizer.corp.google.com/issues/284959841
Change-Id: Id212de209fb57b2c3395bb61064dc8f569832884
TEST=CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306674
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-06-01 01:00:58 +00:00
Ryan Macnak b0170463af [infra] Run JIT FFI tests on QEMU.
Change-Id: I97dd32cff783265de6a431276434e79bb030939b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299780
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-05-26 23:22:32 +00:00
Daco Harkes 157f6bb8a3 [test/ffi] Stop printing uninitialized union members
TEST=function_structs_by_value_generated_ret_*
TEST=function_callbacks_structs_by_value_generated_test

Closes: https://github.com/dart-lang/sdk/issues/52527
Change-Id: Ie910ca319e0b2bc35ee2bcededd4094b209e3282
Cq-Include-Trybots: luci.dart.try:vm-aot-msan-linux-release-x64-try,vm-msan-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/305982
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-05-26 16:22:25 +00:00
Liam Appelbe f0f732b1a6 [vm] Migrate FFI callbacks to the new metadata system.
This is the same metadata system that will support async callbacks. The
main thing this does is take the trampolines that used to be JIT only,
and use them in AOT too. This is partly a safety thing (there's some
extra checks that used to be skipped on AOT), but mostly just so that
the metadata system is unified between the sync and async callbacks.

More details about the design:
https://docs.google.com/document/d/1QDjyY_6wOTOgURwpeYMKU9qEz0gKxx2MUrdruC6Kp6c/edit?usp=sharing

I split this off the async CL. Some of the comments refer to async
stuff that doesn't exist yet, but it's coming immediately after this so
I didn't update them.

Change-Id: Icd5e86934ee9ae34c2c0e2ed2bbd1b928a7184ac
TEST=ffi_callback_metadata_test.cc and CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302903
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2023-05-24 22:22:04 +00:00
Martin Kustermann 2d230aa0b5 [vm/ffi] Move ffi-callback related state from Thread to ObjectStore, move jit trampolines from Isolate to IsolateGroup
The ffi-callback related information on the [Thread] object is metadata
corresponding to ffi-callback-trampoline [Function] objects. There is
nothing thread or isolate specific about it.

Moving it away from [Thread] is needed because an [Isolate] can
have different [Thread] objects across its lifetime (see [0]): When
the stack of an isolate is empty, we reserve now the right to
re-cycle the [Thread]. If the isolate later runs again, it may
get a new [Thread] object.

This CL moves this information from [Thread] to the [ObjectStore]. In
addition we make the compiler be responsible for populating this
metadata - instead of doing this per-call site of
`Pointer.fromFunction()`. It will be preserved across snapshot writing
& snapshot reading (for AppJIT as well as AppAOT).

Similarly the JIT trampolines that are on Isolate aren't isolate
specific and can go to [IsolateGroup]. This simplifies doing the above
as the compiler can allocate those as well.

The effect is that [Thread] object gets smaller, GC doesn't have to
visit the 2 slots per-thread. It comes at expense of 2 more loads
when invoking the callback.

[0] https://dart-review.googlesource.com/c/sdk/+/297920

TEST=Regression test is vm/ffi{,_2}/invoke_callback_after_suspension_test

Change-Id: Ifde46a9f6e79819b5c0e359c3d3998d1d93b9b1e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/303700
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2023-05-17 11:23:28 +00:00
Alexander Aprelev 3a668dcf03 [vm/win/aot] Provide unwinding information for Windows AOT snapshots.
Fixes https://github.com/dart-lang/sdk/issues/52045
TEST=ffi_induce_a_crash_test

Change-Id: Ic047df10732d6ff8cf695ce098d80ec3a098dbf9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302380
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2023-05-11 03:14:27 +00:00
Daco Harkes e5bb28bc4d [vm/ffi] Pointer.asTypedList finalizer
TEST=tests/ffi/external_typed_data_finalizer_test.dart

Closes: https://github.com/dart-lang/sdk/issues/50507
CoreLibraryReviewExempt: https://github.com/dart-lang/sdk/issues/52261
Change-Id: I1a82dcca15961b28c0de64637970fe38a39286e5
Cq-Include-Trybots: luci.dart.try:vm-asan-linux-release-x64-try,vm-aot-asan-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-qemu-linux-release-arm-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-aot-win-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-aot-mac-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/301001
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2023-05-10 11:38:57 +00:00
Daco Harkes 65630baac9 [gardening] Update status file
Mark in status file instead of with test workflow so that test.py is
fully green and all current triaged failures are marked in status file.

Bug: https://github.com/dart-lang/sdk/issues/39487
Change-Id: Ib0729a9ee68d7f0e0f8a9ad30238e537467f8426
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302322
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Auto-Submit: Daco Harkes <dacoharkes@google.com>
2023-05-09 18:22:56 +00:00
Daco Harkes 8beb778e64 [analyzer/ffi] Fix spurious FFI errors
Closes: https://github.com/dart-lang/sdk/issues/52298
Change-Id: Ibfccd34a877d5786f4720790b5ff88dcc643c9d6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/301980
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-05-08 15:16:05 +00:00
Ryan Macnak d2ef2ce26f [ffi] Add failing callback test with an Error that is not an UnhandledException.
Bug: https://github.com/dart-lang/sdk/issues/39487
Change-Id: I54d09341fa4b919e7efe9ea0714c089d2ee91826
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299880
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-05-01 16:13:51 +00:00
Daco Harkes 624ce64b01 [gardening] Skip FFI test on Windows AOT
Test in https://dart-review.googlesource.com/c/sdk/+/293541 should be
skipped on Windows AOT.

Bug: https://github.com/dart-lang/sdk/issues/40579
Bug: https://github.com/dart-lang/sdk/issues/51794
Closes: https://github.com/dart-lang/sdk/issues/51966
Change-Id: I06c4f492f97c903cd31f973163b264526a341681
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/293685
Reviewed-by: Siva Annamalai <asiva@google.com>
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-05-01 08:13:48 +00:00
Daco Harkes a045aea9bc [test/ffi] Format record types
Now that https://github.com/dart-lang/dart_style/issues/1128 has been
addressed, format the files testing varargs.

And skips `printf` executable lookup on Android.

Closes: https://github.com/dart-lang/sdk/issues/52112
Change-Id: I31158ad7a5ddd8520bbee5d67e427ee4c1f3da9e
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296800
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-04-21 08:23:57 +00:00
Martin Kustermann f9eb97fdff Reland "[vm] Expose Dart_{CurrentIsolate,ExitIsolate,EnterIsolate}"
For applications that want to have arbitrary number of isolates call
into native code that may be blocking, we expose the API functions that
allows those native threads to exit an isolate before running
long/blocking code.

Without the ability to exit/re-enter isolate, one may experience
deadlocks as we have a fixed limit on the number of concurrently
executing isolates atm.

In the longer term we may find a way to do this automatically
with low overhead, see [0]. But since those API functions are quite
stable and we already expose e.g. `Dart_{Enter,Exit}Scope`, I don't
see a reason not to expose `Dart_{Enter,Exit}Isolate`.

Difference to original CL:

  Do use STL synchronization primitives (as the ones in runtime/bin
  are not always available in shared libraries)


[0] Issue https://github.com/dart-lang/sdk/issues/51261

Issue https://github.com/dart-lang/sdk/issues/51254

TEST=ffi{,_2}/dl_api_exit_enter_isolate_test

Change-Id: Id817e8d4edb3db35f029248d62388cbd0682001d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294980
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2023-04-13 09:20:17 +00:00
Ryan Macnak 783d0ef90a Revert "[vm] Expose Dart_{CurrentIsolate,ExitIsolate,EnterIsolate}"
This reverts commit a2512819f8.

Reason for revert: FFI tests fail to link on Windows, fail to load on product-mode Android

Original change's description:
> [vm] Expose Dart_{CurrentIsolate,ExitIsolate,EnterIsolate}
>
> For applications that want to have arbitrary number of isolates call
> into native code that may be blocking, we expose the API functions that
> allows those native threads to exit an isolate before running
> long/blocking code.
>
> Without the ability to exit/re-enter isolate, one may experience
> deadlocks as we have a fixed limit on the number of concurrently
> executing isolates atm.
>
> In the longer term we may find a way to do this automatically
> with low overhead, see [0]. But since those API functions are quite
> stable and we already expose e.g. `Dart_{Enter,Exit}Scope`, I don't
> see a reason not to expose `Dart_{Enter,Exit}Isolate`.
>
> [0] Issue https://github.com/dart-lang/sdk/issues/51261
>
> Issue https://github.com/dart-lang/sdk/issues/51254
>
> TEST=ffi{,_2}/dl_api_exit_enter_isolate_test
>
> Change-Id: I91c772ca962fddb87919663fea07939a498fa205
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292722
> Commit-Queue: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

Change-Id: I05ad5b9ce24754a68693160e470f8eb71a812c75
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294860
Auto-Submit: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2023-04-12 16:39:19 +00:00
Martin Kustermann a2512819f8 [vm] Expose Dart_{CurrentIsolate,ExitIsolate,EnterIsolate}
For applications that want to have arbitrary number of isolates call
into native code that may be blocking, we expose the API functions that
allows those native threads to exit an isolate before running
long/blocking code.

Without the ability to exit/re-enter isolate, one may experience
deadlocks as we have a fixed limit on the number of concurrently
executing isolates atm.

In the longer term we may find a way to do this automatically
with low overhead, see [0]. But since those API functions are quite
stable and we already expose e.g. `Dart_{Enter,Exit}Scope`, I don't
see a reason not to expose `Dart_{Enter,Exit}Isolate`.

[0] Issue https://github.com/dart-lang/sdk/issues/51261

Issue https://github.com/dart-lang/sdk/issues/51254

TEST=ffi{,_2}/dl_api_exit_enter_isolate_test

Change-Id: I91c772ca962fddb87919663fea07939a498fa205
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292722
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-04-12 13:22:23 +00:00
Alexander Aprelev 386a87a890 [vm/win] Better exception handling on Windows x64.
This change adds x64 unwinding information for generated code allowing Windows to actually call registered unhandled exception filters(Crashpad or dart-builtin crash handler).

BUG=https://github.com/dart-lang/sdk/issues/51726
TEST=ffi\ffi_induce_a_crash_test.dart

Change-Id: I1d9dd208f2c030b51a1146afa1048d419847fef7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294129
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2023-04-07 21:22:35 +00:00
Daco Harkes 563425a46e [vm/ffi] Fix test on Android
Change-Id: Id7a4255c6729678a95f3f055fa9772b88ac0418a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/293686
Reviewed-by: Alexander Aprelev <aam@google.com>
2023-04-05 19:51:09 +00:00
Daco Harkes 2d61150905 [cfe/ffi] Fix missing type argument on @Native
TEST=tests/ffi/regress_51913_test.dart

Bug: https://github.com/dart-lang/sdk/issues/51913
Change-Id: I79746f3b3852f34d7e31b32260ce5abcb2d6801a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/293602
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-04-05 16:24:30 +00:00
Daco Harkes f0adc91cc6 [test/ffi] Test null in Handle
TEST=tests/ffi/vmspecific_regress_51794_test.dart

Bug: https://github.com/dart-lang/sdk/issues/51794
Change-Id: I600f48b79e513840ec4613c25e60364b40957253
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/293541
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Auto-Submit: Daco Harkes <dacoharkes@google.com>
2023-04-05 14:11:48 +00:00
Ryan Macnak 3d8424e9fc [test] Introduce status variable $simulator, and fix some cases missing simarm_x64.
TEST=ci
Change-Id: Ie2a8ac015b2d316527d648956c86d4c7cb319d26
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292962
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-04-04 03:10:26 +00:00
Daco Harkes 7cfab4e12c [infra/ffi] Skip FFI tests on simulator
Some recent infra shuffling [0] caused FFI tests not to be ingored on
some simulator architectures.

FFI is not supported on simulators, so ignore tests.

[0] https://dart-review.googlesource.com/c/sdk/+/286360

Bug: https://github.com/dart-lang/sdk/issues/48365
Change-Id: Id623733e89f4269feabbe1f1212bdb64a43d7540
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292821
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-04-03 16:14:02 +00:00
Daco Harkes 3de5d25429 [vm] Class modifiers for NativeFieldWrapper subtypes
Split off https://dart-review.googlesource.com/c/sdk/+/291761.
Landing separately, so that the breaking change itself is a smaller
CL.

TEST=ci build (see touched _test files)

bug: https://github.com/dart-lang/sdk/issues/51896
Change-Id: Ic8d218845ccb6a277689e911b2c34c44639e13cf
Cq-Include-Trybots: luci.dart.try:flutter-linux-try,vm-aot-linux-debug-x64c-try,vm-ffi-android-debug-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292000
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-03-31 13:20:19 +00:00
Alexander Aprelev 67683c3973 [vm/isolates] Introduce 'vm:isolate-unsendable' pragma.
Decorate Zone, Future, Completer, Timer and Stream with newly-introduced pragma to make sure that message verification stops earlier, produces shorter retaining path for the user.

BUG=https://github.com/dart-lang/sdk/issues/51722
TEST=send_unsupported_objects_test,isolate_exit_unsendable_test.dart
CoreLibraryReviewExempt: vm-specific pragmas
Change-Id: I499eea542d228ac9cf0797a682664f93f360dc80
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289027
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-03-27 20:54:11 +00:00
Daco Harkes 92e5746494 Reland "[vm/ffi] Add class modifiers"
This is a reland of commit 1755f89092

Can land after (or with) the Flutter PR:
https://github.com/flutter/engine/pull/40434

Original change's description:
> [vm/ffi] Add class modifiers
>
> Adds class modifiers to `dart:ffi`.
>
> Migrates all user-defined subclasses of `Struct`, `Union`, `Opaque`,
> and `AbiSpecificInteger` to be `final class`es.
>
> Does not remove the manual error checking, so some errors will show up
> twice now in language version 3.0. In language version <3.0, only the
> FFI-specific error will show up.
>
> In a follow-up CL, we will try to make the language-errors to show up
> also <3.0 so that we can remove the FFI-specific errors.
>
> Examples of duplicated errors:
> pkg/analyzer/test/src/diagnostics/subtype_of_ffi_class_test.dart
>
> TEST=pkg/analyzer/test/ (for the analyzer)
> TEST=pkg/front_end/testcases/ (for the CFE)
> TEST=test/ffi/ (for the VM)
>
> CoreLibraryReviewExempt: No need for dart2js to review.
> Bug: https://github.com/dart-lang/sdk/issues/51683
> Change-Id: I2964ceccb7db59fbdaf6be5319f5e4ec2dabe0f3
> Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-win-release-try,pkg-mac-release-try,vm-precomp-ffi-qemu-linux-release-riscv64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-reload-rollback-linux-debug-x64-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289223
> Reviewed-by: Johnni Winther <johnniwinther@google.com>
> Reviewed-by: Devon Carew <devoncarew@google.com>
> Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
> Reviewed-by: Jackson Gardner <jacksongardner@google.com>
> Reviewed-by: Lasse Nielsen <lrn@google.com>
> Commit-Queue: Daco Harkes <dacoharkes@google.com>

TEST=pkg/analyzer/test/ (for the analyzer)
TEST=pkg/front_end/testcases/ (for the CFE)
TEST=test/ffi/ (for the VM)
CoreLibraryReviewExempt: No need for dart2js to review.
Bug: https://github.com/dart-lang/sdk/issues/51683
Change-Id: I2ee3f0ac31d4162068a2346a06320029b2263ee2
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-win-release-try,pkg-mac-release-try,vm-precomp-ffi-qemu-linux-release-riscv64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-reload-rollback-linux-debug-x64-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289781
Reviewed-by: Devon Carew <devoncarew@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
2023-03-21 15:25:10 +00:00
Zach Anderson 7471994270 Revert "[vm/ffi] Add class modifiers"
This reverts commit 1755f89092.

Reason for revert: This is a breaking change and is blocking the Dart -> Flutter roll. See https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20Unopt/26896/overview.

Original change's description:
> [vm/ffi] Add class modifiers
>
> Adds class modifiers to `dart:ffi`.
>
> Migrates all user-defined subclasses of `Struct`, `Union`, `Opaque`,
> and `AbiSpecificInteger` to be `final class`es.
>
> Does not remove the manual error checking, so some errors will show up
> twice now in language version 3.0. In language version <3.0, only the
> FFI-specific error will show up.
>
> In a follow-up CL, we will try to make the language-errors to show up
> also <3.0 so that we can remove the FFI-specific errors.
>
> Examples of duplicated errors:
> pkg/analyzer/test/src/diagnostics/subtype_of_ffi_class_test.dart
>
> TEST=pkg/analyzer/test/ (for the analyzer)
> TEST=pkg/front_end/testcases/ (for the CFE)
> TEST=test/ffi/ (for the VM)
>
> CoreLibraryReviewExempt: No need for dart2js to review.
> Bug: https://github.com/dart-lang/sdk/issues/51683
> Change-Id: I2964ceccb7db59fbdaf6be5319f5e4ec2dabe0f3
> Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-win-release-try,pkg-mac-release-try,vm-precomp-ffi-qemu-linux-release-riscv64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-reload-rollback-linux-debug-x64-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289223
> Reviewed-by: Johnni Winther <johnniwinther@google.com>
> Reviewed-by: Devon Carew <devoncarew@google.com>
> Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
> Reviewed-by: Jackson Gardner <jacksongardner@google.com>
> Reviewed-by: Lasse Nielsen <lrn@google.com>
> Commit-Queue: Daco Harkes <dacoharkes@google.com>

Bug: https://github.com/dart-lang/sdk/issues/51683
Change-Id: Ie5b8a08aea6d64b1991ace4814322b21ffb670c7
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-win-release-try,pkg-mac-release-try,vm-precomp-ffi-qemu-linux-release-riscv64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-reload-rollback-linux-debug-x64-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289640
Commit-Queue: Slava Egorov <vegorov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2023-03-20 11:15:59 +00:00
Daco Harkes 1755f89092 [vm/ffi] Add class modifiers
Adds class modifiers to `dart:ffi`.

Migrates all user-defined subclasses of `Struct`, `Union`, `Opaque`,
and `AbiSpecificInteger` to be `final class`es.

Does not remove the manual error checking, so some errors will show up
twice now in language version 3.0. In language version <3.0, only the
FFI-specific error will show up.

In a follow-up CL, we will try to make the language-errors to show up
also <3.0 so that we can remove the FFI-specific errors.

Examples of duplicated errors:
pkg/analyzer/test/src/diagnostics/subtype_of_ffi_class_test.dart

TEST=pkg/analyzer/test/ (for the analyzer)
TEST=pkg/front_end/testcases/ (for the CFE)
TEST=test/ffi/ (for the VM)

CoreLibraryReviewExempt: No need for dart2js to review.
Bug: https://github.com/dart-lang/sdk/issues/51683
Change-Id: I2964ceccb7db59fbdaf6be5319f5e4ec2dabe0f3
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-win-release-try,pkg-mac-release-try,vm-precomp-ffi-qemu-linux-release-riscv64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-reload-rollback-linux-debug-x64-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289223
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Devon Carew <devoncarew@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Jackson Gardner <jacksongardner@google.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-03-17 19:29:41 +00:00
Daco Harkes 8218ee0840 [cfe/ffi] Fix Finalizable in for( in ) loops
The `Finalizable` visitor was visiting for-in loops in AST order:
(1) variable, (2) iterable, (3) body. This caused the `variable` to be
fenced in the `iterable` expression. The `variable` should only be
fenced in the `body`.

TEST=tests/ffi/regress_51538_test.dart
TEST=pkg/vm/test/transformations/ffi_test.dart
     with pkg/vm/testcases/transformations/ffi/regress_51538.dart

Closes: https://github.com/dart-lang/sdk/issues/51538
Change-Id: Idacf87b6de3ee0d2d5c6c5046060c55135593fed
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286182
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2023-03-02 09:05:19 +00:00
Daco Harkes 08b57e0457 [test/ffi] Fix test use RTLD_GLOBAL on Android
Android arm has a different const for `RTLD_GLOBAL` which we take care
of in the helper function `dlopenGlobalPlatformSpecific`. So use that
instead.

Change-Id: Icb5d0b84f7d76ec1f6e24d43a644cffe8f46fd42
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286140
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2023-03-01 10:24:24 +00:00
Daco Harkes cb1b6905ac [vm/ffi] Fix @Natives with VarArgs
This CL fixes the processing of `@Native` annotations to take into
account the native types nested in `VarArgs`.

TEST=tests/ffi/regress_51504_test.dart
TEST=tests/ffi/function_varargs_generated_native_leaf_test.dart
TEST=tests/ffi/function_varargs_generated_native_test.dart

Closes: https://github.com/dart-lang/sdk/issues/51504
Change-Id: Ifb0a08e1e52c9b51cb5143e7ac487a91da4a0e0c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/285623
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-03-01 09:08:31 +00:00
Daco Harkes 579d964406 [test/ffi] Beef up @Native testing
We only tested a handful of native signatures for `@Native`s and
`@FfiNative`s. This CL duplicates the generated FFI call tests which
cover a lot of signatures to also run for `@Native`s.

Bug: https://github.com/dart-lang/sdk/issues/47625
Change-Id: I475f028c79e13470c2ea4e24040209a915631779
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-win-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-precomp-ffi-qemu-linux-release-riscv64-try,vm-precomp-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284901
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-02-24 21:16:50 +00:00
Daco Harkes 14e4a52657 [vm] Fix Dart_CObject_Type breaking change
https://dart-review.googlesource.com/c/sdk/+/257925 added a new entry
in the middle of the `Dart_CObject_Type` enum, which changed the
value of the entries below. However, this enum is part of
`dart_api_dl.h` and versioned by `dart_version.h`.

New entries to `Dart_CObject_Type` should be added at the end of the
enum to avoid making breaking changes to the type.

TEST=tests/ffi/vmspecific_handle_dynamically_linked_test.dart

Bug: https://github.com/dart-lang/sdk/issues/51459

Change-Id: I367b54f62e59ddf925e255bb56c0f8660be7c227
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284161
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-02-20 11:59:19 +00:00
Daco Harkes dc45e67c5e Reland "[vm/ffi] Disallow nullable struct fields"
This is a reland of commit 4e7d71b593

Flutter's dependencies have been rolled.
g3 has also been rolled.

TEST=tests/ffi/regress_51041_test.dart

Original change's description:
> [vm/ffi] Disallow nullable struct fields
>
> TEST=tests/ffi/regress_51041_test.dart
>
> Closes: https://github.com/dart-lang/sdk/issues/51041
> Change-Id: I44d82c5f378c5d001bfb7526be87c179b8160be1
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/279511
> Commit-Queue: Daco Harkes <dacoharkes@google.com>
> Reviewed-by: Johnni Winther <johnniwinther@google.com>

Change-Id: I66580ad7c2f032c17cc708ce875e8dba942ae44b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/283600
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-02-16 11:26:38 +00:00
Daco Harkes 7a18d3c6c5 [test/ffi] @Native assets test AOT with --strip
This more closely resembles --release compilation in Flutter on Linux
and Android.

TEST=tests/ffi/native_assets/asset_absolute_test.dart

Bug: https://github.com/dart-lang/sdk/issues/49803
Change-Id: I51ea100966393cd25b11a44dc2c40ce43552ae87
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/283184
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2023-02-15 13:21:36 +00:00
Daco Harkes 6a90c21992 [vm/ffi] Fix MemoryCopyInstr on ia32
The MemoryCopyInstr destroys the value in src-TypedDataBase
location. It overwrites it by loading the data_ field.

This requires using a WritableRegister, rather than a Register.

This issue seems to have been already present before the recent
optimizations (the leaq instruction overwrites array_reg), but the
recent optimizations made it exercise this instruction in a loop where
it was assumed the src-position was not destroyed.

TEST=tests/ffi/regress_51315_test.dart

Closes: https://github.com/dart-lang/sdk/issues/51315
Change-Id: If37c6c39de8fd0cba9b8a25a1f38f025a39123d9
Cq-Include-Trybots: luci.dart.try:vm-kernel-win-debug-ia32-try,vm-kernel-nnbd-win-release-ia32-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-eager-optimization-linux-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/283021
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-02-14 13:20:43 +00:00
Daco Harkes dc2c739a6c [vm/ffi] Fix Struct use in @Natives
The FfiNative transform was trying to access the generated struct
constructor which is added to the AST in the FFI definitions transform.

The FFI definitions transform has to run _before_ the FFI native
transform.

TEST=tests/ffi/regress_51321_test.dart

Closes: https://github.com/dart-lang/sdk/issues/51321
Change-Id: I6b5caf6e5851455cece6aec13ef73d85cc201ebf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/282103
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-02-13 09:40:24 +00:00
Daco Harkes 732e4a43d5 Revert "[vm/ffi] Disallow nullable struct fields"
Revert submission 279511

Reason for revert: Flutter dependencies are using nullable struct
fields and need to be migrated first
https://github.com/flutter/flutter/issues/120260

Reverted changes: /q/submissionid:279510 /q/submissionid:279511

Change-Id: I7ab66ef6c6b5f21303804b2930af4ad6aa000413
Tested: Revert
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/281600
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2023-02-08 08:51:28 +00:00