Commit graph

275 commits

Author SHA1 Message Date
blagoev
50e18b8160 [vm/ffi] Add Dart_IsNull in dart_api_dl.h
TEST=tests/ffi/vmspecific_handle_dynamically_linked_test.dart

Closes: https://github.com/dart-lang/sdk/pull/50466
Closes: https://github.com/dart-lang/sdk/issues/48331

GitOrigin-RevId: ce49a64788310a755cd9bb5c7ff6d281830bcc33
Change-Id: Ia8e23c57c76767d9e3db799b8fe6b172071582a2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/269742
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2022-11-15 10:32:16 +00:00
Ryan Macnak
ae13d34299 [test] Avoid tests fighting over the default service port.
With no argument, --enable-vm-service and --observe default to port 8181. Passing 0 means to take any available port. If two tests try to use the default port at the same time, one will fail to bind that port.

TEST=ci
Change-Id: Ib7615911db9fb9ab037de1efdd4918b54f59d9e5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/268840
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-11-10 17:28:48 +00:00
Daco Harkes
b411015395 [test/ffi] Skip tests on opcounter that already have optcounter flag
https://dart-review.googlesource.com/c/sdk/+/265021 increased the
optcounter flag in some tests and the loop count. The optcounter bot
forces the optcounter flag at 5, causing these tests to now take
longer.

Since we're already running these tests with optcounter flags on all
bots, skip them on the optcounter bot.

Change-Id: Ib7f7de0fe6da5216ea8e6e606144c5b943d4b4f2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/266082
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2022-10-27 12:42:42 +00:00
Daco Harkes
f524ec74ce [vm/ffi] FfiNative process lookup
This CL makes `FfiNative`s use `DynamicLibrary.process()` lookup if
resolving with the resolver set by `Dart_SetFfiNativeResolver` fails.

Also moves the implementation over from ffi.cc to
ffi_dynamic_library.cc so the implementation can be shared with
`DynamicLibrary.process()`.

Moves the implementation behind non-simulator and non-precompiler.
However, the implementation is tested in vm/cc tests which are in
precompiler mode. So enables the implementation if TESTED is defined.

This CL massages the build files so that TESTED is properly
defined when compiling the runtime for the vm/cc tests, and links
the ole32 symbols on windows for vm/cc tests.

(And some unrelated small cleanup changes here and there.)

TEST=tests/ffi/native_assets/process_test.dart

Change-Id: I25395d381db1d9b4b7a5759171a798a1140a6140
Cq-Include-Trybots: luci.dart.try:vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,dart-sdk-win-try,vm-kernel-win-release-x64-try,vm-kernel-win-release-ia32-try,vm-kernel-precomp-win-product-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/264982
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-10-25 10:57:07 +00:00
Ryan Macnak
9319d07e11 [vm] Fill in ArgumentValue.invalidObject when objects are rejected in isolate messages.
TEST=ci
Change-Id: I5a2816ce7fd40463aaffd971d867955a7b2bd7e6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/265043
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-10-24 17:49:29 +00:00
Alexander Markov
882e3711ff [ffi/tests] Split large auto-generated tests into multiple tests
This change tries to avoid timeouts on debug/ia32 bots by replacing
each auto-generated

  tests/ffi{,_2}/function_structs_by_value_generated{,_leaf}_test.dart

with 3 separate tests.
Also, --optimization-counter-threshold is increased.

Fixes https://github.com/dart-lang/sdk/issues/48627
Issue: https://github.com/dart-lang/sdk/issues/45007

Change-Id: I8a9c72927d05a3da3da35a21e9ffba0f3bda2112
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/265021
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-10-20 22:08:53 +00:00
Alexander Markov
24b8399086 [tests] Avoid small --optimization-counter-threshold in tests
Small --optimization-counter-threshold makes tests very slow,
especially on architectures where kernel service runs from
kernel and not from app-jit snapshot.

TEST=change in tests, *-ia32 bots
Fixes https://github.com/dart-lang/sdk/issues/48627

Change-Id: I63e7e201ef9a0e4f645016c39a5be1819b61822d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/263421
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-10-10 21:04:18 +00:00
Daco Harkes
b113fffb65 Reland "[vm/ffi] Throw on returning Error in Handle"
Fix: Check handle contents for Smi.
Closes: https://github.com/flutter/flutter/issues/112726

Orignal CL description:

Makes `Dart_Handle` FFI returns behave as the following snippet:

```
Dart_Handle ExampleSnippet() {
  Dart_Handle result = ...;
  if (Dart_IsError(result)) {
    Dart_PropagateError(result);
  }
  return result;
}
```

Also makes FFI consistent with Dart_NativeFunctions, which will
automatically throw upon return if Dart_SetReturnValue set the result
to an error.

`UnhandledExceptions` cannot flow out into Dart generated code. So,
the implementation needs to be in `FfiCallInstr::EmitNativeCode`.

Using `Dart_IsError` is slow compared to a machine code class id
check. So, we should do the handle unwrapping and class id check in
machine code.

Unwrapping Handles in machine code is only safe when the GC is
guaranteed to not run: Either (1) in `kThreadInGenerated`, or (2) in
`kThreadInNative`, but only when transitioned into safepoint. So, the
handle cannot be unwrapped immediately after the FFI call in machine code. We first need to transition back to generated.

This means we need to transition again to native to do the actual
`Dart_PropagateError` call. We can do so without the stub in JIT
because we never return with normal control flow.

Performance impact of this change is within benchmark noise in both
JIT and AOT.
Size impact is 42 bytes on x64, which is 10% in AOT and 12% in JIT.

For more numbers see: go/dart-ffi-handle-error

TEST=runtime/bin/ffi_test/ffi_test_functions_vmspecific.cc
TEST=tests/ffi/vmspecific_handle_test.dart

Closes: https://github.com/dart-lang/sdk/issues/49936
Change-Id: Id8edfd841a7d6246438386007d83747868a0a151
Cq-Include-Trybots: luci.dart.try:vm-canary-linux-debug-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-kernel-gcc-linux-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-msvc-windows-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-precomp-ffi-qemu-linux-release-riscv64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-mac-release-arm64-try,vm-kernel-precomp-win-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/262342
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-10-03 12:25:08 +00:00
Zach Anderson
2742d90033 Revert "[vm/ffi] Throw on returning Error in Handle"
This reverts commit d9c442bce8.

Reason for revert: https://github.com/flutter/flutter/issues/112726

Original change's description:
> [vm/ffi] Throw on returning `Error` in `Handle`
>
> Makes `Dart_Handle` FFI returns behave as the following snippet:
>
> ```
> Dart_Handle ExampleSnippet() {
>   Dart_Handle result;
>   if (Dart_IsError(result)) {
>     Dart_PropagateError(result);
>   }
>   return result;
> }
> ```
>
> Also makes FFI consistent with Dart_NativeFunctions, which will
> automatically throw upon return if Dart_SetReturnValue set the result
> to an error.
>
> `UnhandledExceptions` cannot flow out into Dart generated code. So,
> the implementation needs to be in `FfiCallInstr::EmitNativeCode`.
>
> Using `Dart_IsError` is slow compared to a machine code class id
> check. So, we should do the handle unwrapping and class id check in
> machine code.
>
> Unwrapping Handles in machine code is only safe when the GC is
> guaranteed to not run: Either (1) in `kThreadInGenerated`, or (2) in
> `kThreadInNative`, but only when transitioned into safepoint. So, the
> handle cannot be unwrapped immediately after the FFI call in machine code. We first need to transition back to generated.
>
> This means we need to transition again to native to do the actual
> `Dart_PropagateError` call. We can do so without the stub in JIT
> because we never return with normal control flow.
>
> Performance impact of this change is within benchmark noise in both
> JIT and AOT.
> Size impact is 42 bytes on x64, which is 10% in AOT and 12% in JIT.
>
> For more numbers see: go/dart-ffi-handle-error
>
> TEST=runtime/bin/ffi_test/ffi_test_functions_vmspecific.cc
> TEST=tests/ffi/vmspecific_handle_test.dart
>
> Closes: https://github.com/dart-lang/sdk/issues/49936
> Change-Id: Ie8fabeb6d53bc80689541bc4470cb37ee2200581
> Cq-Include-Trybots: luci.dart.try:vm-canary-linux-debug-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-kernel-gcc-linux-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-msvc-windows-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-precomp-ffi-qemu-linux-release-riscv64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-mac-release-arm64-try,vm-kernel-precomp-win-debug-x64c-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/261603
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Commit-Queue: Daco Harkes <dacoharkes@google.com>

TBR=kustermann@google.com,rmacnak@google.com,dacoharkes@google.com,dart-scoped@luci-project-accounts.iam.gserviceaccount.com

Change-Id: I94cc63de16b54db2b0a4f92759c39a1e569b8e63
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-canary-linux-debug-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-kernel-gcc-linux-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-msvc-windows-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-precomp-ffi-qemu-linux-release-riscv64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-mac-release-arm64-try,vm-kernel-precomp-win-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/262270
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2022-09-30 21:51:59 +00:00
Daco Harkes
d9c442bce8 [vm/ffi] Throw on returning Error in Handle
Makes `Dart_Handle` FFI returns behave as the following snippet:

```
Dart_Handle ExampleSnippet() {
  Dart_Handle result;
  if (Dart_IsError(result)) {
    Dart_PropagateError(result);
  }
  return result;
}
```

Also makes FFI consistent with Dart_NativeFunctions, which will
automatically throw upon return if Dart_SetReturnValue set the result
to an error.

`UnhandledExceptions` cannot flow out into Dart generated code. So,
the implementation needs to be in `FfiCallInstr::EmitNativeCode`.

Using `Dart_IsError` is slow compared to a machine code class id
check. So, we should do the handle unwrapping and class id check in
machine code.

Unwrapping Handles in machine code is only safe when the GC is
guaranteed to not run: Either (1) in `kThreadInGenerated`, or (2) in
`kThreadInNative`, but only when transitioned into safepoint. So, the
handle cannot be unwrapped immediately after the FFI call in machine code. We first need to transition back to generated.

This means we need to transition again to native to do the actual
`Dart_PropagateError` call. We can do so without the stub in JIT
because we never return with normal control flow.

Performance impact of this change is within benchmark noise in both
JIT and AOT.
Size impact is 42 bytes on x64, which is 10% in AOT and 12% in JIT.

For more numbers see: go/dart-ffi-handle-error

TEST=runtime/bin/ffi_test/ffi_test_functions_vmspecific.cc
TEST=tests/ffi/vmspecific_handle_test.dart

Closes: https://github.com/dart-lang/sdk/issues/49936
Change-Id: Ie8fabeb6d53bc80689541bc4470cb37ee2200581
Cq-Include-Trybots: luci.dart.try:vm-canary-linux-debug-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-kernel-gcc-linux-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-msvc-windows-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-precomp-ffi-qemu-linux-release-riscv64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-mac-release-arm64-try,vm-kernel-precomp-win-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/261603
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-09-30 13:29:51 +00:00
Daco Harkes
8a78aaf463 [analyzer/ffi] Fix FfiNative Pointer params
TEST=pkg/analyzer/test/src/diagnostics/ffi_native_test.dart
TEST=tests/ffi/regress_49684_test.dart

Closes: https://github.com/dart-lang/sdk/issues/49684
Change-Id: I756635c0a34aa18f3a3a2cbdcc0657b08cb5050e
Cq-Include-Trybots: luci.dart.try:analyzer-linux-release-try,analyzer-mac-release-try,analyzer-win-release-try,analyzer-analysis-server-linux-try,vm-ffi-android-debug-arm64c-try,vm-precomp-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/261400
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Brian Wilkerson <brianwilkerson@google.com>
2022-09-27 15:58:16 +00:00
Daco Harkes
c04673f44b [vm/ffi] Support DynamicLibrary.process() on Windows
Support looking up a symbol in the process on Windows by using the
Windows Process Status API to iterate over all loaded Modules.

TEST=tests/ffi/has_symbol_test.dart
TEST=tests/ffi/vmspecific_dynamic_library_test.dart

Change-Id: I1029f1c7dae9a193b662d942388affb681842c90
Cq-Include-Trybots: luci.dart.try:vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,dart-sdk-win-try,vm-kernel-win-release-x64-try,vm-kernel-win-release-ia32-try,vm-kernel-precomp-win-product-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/260760
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2022-09-27 08:42:03 +00:00
Ryan Macnak
cba1a09e5c [vm, compiler] Inform MSan about parameters coming from FFI.
- Mark the outgoing arguments stack area as initialized.
 - Mark the trampoline's incoming parameter slots as initialized in case they are used as local handles.
 - Mark the outgoing register arguments as initialized.

TEST=msan
Bug: https://github.com/dart-lang/sdk/issues/42314
Bug: https://github.com/dart-lang/sdk/issues/49298
Bug: https://github.com/dart-lang/sdk/issues/49957
Change-Id: Ifa978e1b905a424ec9f64b89879cea6e82f70d12
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/259102
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-09-23 21:31:14 +00:00
Ahmed Ashour
5fe480b788 Fix typos
Fix #49864

TEST=ci

Change-Id: I9a7e06d604cd0b4f56f2ac229ab3fc9f01cb9d76
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256824
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: Mayank Patke <fishythefish@google.com>
2022-09-16 19:35:00 +00:00
Alexander Markov
a7f7c0c5ac [vm] Support serialization of FFI callbacks in IL serialization
TEST=tests/ffi/function_callbacks_test.dart
TEST=Manual run of vm-kernel-precomp-linux-debug-x64-try with --test_il_serialization enabled.

Issue: https://github.com/dart-lang/sdk/issues/43299
Change-Id: Ia57021d9091e8a80de3645cb4723ebdbb5a3d33d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256371
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2022-08-26 17:37:21 +00:00
Daco Harkes
f592a11e51 [vm/ffi] NativeFieldWrapper FfiNative check receivers for nullptr
Changes `FfiNative` instance methods on `NativeFieldWrapperClass`es
that have `Pointer<Void>` conversion to check for `nullptr`.

If the native field has `0`/`nullptr` a `StateError` is thrown:
"Bad state: Native field is nullptr."

This only makes sense if the first native field is used to point to
the C++ object corresponding to the Dart object. As far as we know
all current use cases do so.

TEST=pkg/vm/testcases/transformations/ffi/ffinative.dart.expect
TEST=tests/ffi/vmspecific_ffi_native_test.dart

Closes: https://github.com/dart-lang/sdk/issues/49620
Change-Id: I92f760c33d391476010722358f9713fa4491ab61
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/254200
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-08-11 17:39:57 +00:00
Daco Harkes
e364fda064 Reland "[cfe/ffi] Improve FFI call mismatched types compile errors"
This is a reland of commit 206fdf148c

Original change's description:
> [cfe/ffi] Improve FFI call mismatched types compile errors
>
> This CL fixes two issues.
>
> 1. `FfiNative`s now check the Dart and native type for compatiblity.
> 2. Both `FfiNative`, `asFunction`, and `lookupFunction` check the type
>    correspondence between native and Dart type with a subtype check of
>    the expected Dart type and the provided Dart type. For functions,
>    any return type is a subtype of a void type. This is fine for Dart,
>    but not for native calls. This CL manually checks the return type
>    for void.
>
> This CL does not fix the inconsistency between `asFunction` and
> `FfiNative` with regard to allowing more strict return types than
> `Object` for `Handle`s
> Issue: https://github.com/dart-lang/sdk/issues/49518
>
> Analyzer fixes in follow up CL.
>
> TEST=tests/ffi/vmspecific_static_checks_ffinative_test.dart
>
> Closes: https://github.com/dart-lang/sdk/issues/49471
> Change-Id: Ibc7bd6a1a0db59cc5fa5d755d76999fd7e9a06a4
> Cq-Include-Trybots: luci.dart.try:analyzer-linux-release-try,analyzer-mac-release-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/252601
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Commit-Queue: Daco Harkes <dacoharkes@google.com>

TEST=tests/ffi/vmspecific_static_checks_ffinative_test.dart

Change-Id: Ic1efba45ae8ff2585fc67fdf63c653ce090d0337
Cq-Include-Trybots: luci.dart.try:analyzer-linux-release-try,analyzer-mac-release-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/252663
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2022-08-01 16:09:03 +00:00
Daco Harkes
48693c6b3d [analyzer/ffi] Error on FfiNative mismatched types
This CL applies `_validateCompatibleFunctionTypes` to `FfiNative`s.
The validation is enhanced to include `Pointer<Void>` to native field
wrapper conversion.

Also, this CL explicitly checks for `void` return types to match the
previous CL CFE behavior.

Closes: https://github.com/dart-lang/sdk/issues/49412
Change-Id: I96243ac1038321febf32f5a03507c6431fdb03d6
Cq-Include-Trybots: luci.dart.try:analyzer-linux-release-try,analyzer-mac-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/252602
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-08-01 15:44:53 +00:00
Daco Harkes
f9722142ee Revert "[cfe/ffi] Improve FFI call mismatched types compile errors"
This reverts commit 206fdf148c.

Reason for revert: Flutter issues are caught by this CL, preventing
Flutter from building.
https://github.com/flutter/flutter/issues/108309

Original change's description:
> [cfe/ffi] Improve FFI call mismatched types compile errors
>
> This CL fixes two issues.
>
> 1. `FfiNative`s now check the Dart and native type for compatiblity.
> 2. Both `FfiNative`, `asFunction`, and `lookupFunction` check the type
>    correspondence between native and Dart type with a subtype check of
>    the expected Dart type and the provided Dart type. For functions,
>    any return type is a subtype of a void type. This is fine for Dart,
>    but not for native calls. This CL manually checks the return type
>    for void.
>
> This CL does not fix the inconsistency between `asFunction` and
> `FfiNative` with regard to allowing more strict return types than
> `Object` for `Handle`s
> Issue: https://github.com/dart-lang/sdk/issues/49518
>
> Analyzer fixes in follow up CL.
>
> TEST=tests/ffi/vmspecific_static_checks_ffinative_test.dart
>
> Closes: https://github.com/dart-lang/sdk/issues/49471
> Change-Id: Ibc7bd6a1a0db59cc5fa5d755d76999fd7e9a06a4
> Cq-Include-Trybots: luci.dart.try:analyzer-linux-release-try,analyzer-mac-release-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/252601
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Commit-Queue: Daco Harkes <dacoharkes@google.com>

TBR=kustermann@google.com,dacoharkes@google.com

Change-Id: Id82b129d491adcc94cdd685a0a0f6a43248c71f2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Issue: https://github.com/dart-lang/sdk/issues/49518
Cq-Include-Trybots: luci.dart.try:analyzer-linux-release-try,analyzer-mac-release-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/252662
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2022-07-25 18:18:29 +00:00
Daco Harkes
206fdf148c [cfe/ffi] Improve FFI call mismatched types compile errors
This CL fixes two issues.

1. `FfiNative`s now check the Dart and native type for compatiblity.
2. Both `FfiNative`, `asFunction`, and `lookupFunction` check the type
   correspondence between native and Dart type with a subtype check of
   the expected Dart type and the provided Dart type. For functions,
   any return type is a subtype of a void type. This is fine for Dart,
   but not for native calls. This CL manually checks the return type
   for void.

This CL does not fix the inconsistency between `asFunction` and
`FfiNative` with regard to allowing more strict return types than
`Object` for `Handle`s
Issue: https://github.com/dart-lang/sdk/issues/49518

Analyzer fixes in follow up CL.

TEST=tests/ffi/vmspecific_static_checks_ffinative_test.dart

Closes: https://github.com/dart-lang/sdk/issues/49471
Change-Id: Ibc7bd6a1a0db59cc5fa5d755d76999fd7e9a06a4
Cq-Include-Trybots: luci.dart.try:analyzer-linux-release-try,analyzer-mac-release-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/252601
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-07-25 14:34:39 +00:00
Anis Alibegić
40e18905f2 Fixed various typos in a lot of files
Closes https://github.com/dart-lang/sdk/pull/49478

TEST=Manual

GitOrigin-RevId: f4c9c6869dfe73639295e86574a021523b3d374d
Change-Id: I134a97caed4eec59d70e9cbca16b7e9a472cf2c1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/251902
Reviewed-by: Michael Thomsen <mit@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Kevin Chisholm <kevinjchisholm@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2022-07-25 12:21:59 +00:00
Daco Harkes
aa6abbf464 [cfe/ffi] Transform FfiNatives in a single pass
Before this CL, `FfiNative`s were first transformed to `asFunction`
calls, which were then immediately transformed to `_asFunctionInternal`
calls.

This caused the the static checks to be done in two steps, the second
step happening after the first transform. It is cleaner to first do all
checks.

This refactoring enables implementing `_asFunctionInternal` variants
for `FfiNative`s that don't use a `Pointer` for the address.

Besides the transform change, this CL
- moves shared logic over to pkg/vm/lib/transformations/ffi/common.dart,
- splits up the ffi-native tests in to positive and negative tests, and
- adds negative tests for mismatches between Dart and native types.

These new tests do _not yet_ pass on the analyzer. This is tracked in:
https://github.com/dart-lang/sdk/issues/49412

TEST=tests/ffi/ffi_native_test.dart
TEST=tests/ffi/vmspecific_static_checks_ffinative_test.dart

Closes: https://github.com/dart-lang/sdk/issues/49413
Change-Id: I5baded43eab7ff1dc1ffb16550b2a638e4b7a34e
Cq-Include-Trybots: luci.dart.try:analyzer-linux-release-try,analyzer-mac-release-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/250843
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-07-08 12:36:09 +00:00
Daco Harkes
d3ea8bff5c [vm/ffi] Fix constant Finalizables
The kernel builder relied on only seeing expressions emitted by the
finalizable transform (variable get and this) as arguments to the fence.

However, other transforms can run later. In this case the expression
was turned into a constant.

This CL changes the implementation to accept any expression rather than
accepting a subset of expressions.

TEST=tests/ffi/regress_49402_test.dart

Closes: https://github.com/dart-lang/sdk/issues/49402
Change-Id: I1a962a5b7a38099eb5c3bbf5a5a8145b16727d97
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/250744
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2022-07-07 08:39:21 +00:00
Daco Harkes
a37502c1b8 [misc] Cleanup stale todos
TEST=tests/ffi

Change-Id: Iae97f62b1adb8dbe99ad075dfe750604db0de62c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249301
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2022-06-28 16:02:53 +00:00
Daco Harkes
1fdd242e4b [deps] Roll package:ffi to 2.0.1
Bug: https://github.com/dart-lang/ffi/issues/148
Change-Id: I406c6a0fbdb53c07507519f1237e09362e3475d0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249160
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
Reviewed-by: Devon Carew <devoncarew@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-06-19 18:29:11 +00:00
Daco Harkes
a855030120 [vm/ffi] Allow struct nesting with looser packing
C compilers don't enforce nesting rules. `dart:ffi` was enforcing these
rules causing issues for binding to C libraries having looser packing
for nested structs than outer structs.

This CL completely removes the error from the analyzer and CFE.
(As an alternative we could have kept a hint/lint, but this might create
more noise than value.)

TEST=pkg/analyzer/test/src/diagnostics/packed_nesting_non_packed_test.dart

Closes: https://github.com/dart-lang/sdk/issues/46644
Change-Id: Iae2d5c885546f7799bc6dea2f8cd7dd508216a0c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/247382
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
2022-06-09 19:08:43 +00:00
Daco Harkes
90f118484f [vm/ffi] Add Char C type
The diff from
https://dart-review.googlesource.com/c/sdk/+/229156/1..3

This was blocked by https://github.com/timsneath/win32/issues/349
earlier. Now that that has been resolved, we should be able to land
this.

We should land this to make package:ffigen be able to bind to `char` in
Dart 2.17.
https://github.com/dart-lang/ffigen/pull/287#discussion_r835734993

TEST=tests/ffi/c_types_test.dart

Change-Id: Ia3bc3785a3d5c0c9f7475106eb77f944f2d0a838
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-ia32-try,vm-kernel-nnbd-mac-debug-arm64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-precomp-ffi-qemu-linux-release-riscv64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/239020
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2022-03-28 09:46:56 +00:00
Daco Harkes
532c116cd2 [vm] Implement NativeFinalizer
This CL implements `NativeFinalizer` in the GC.

`FinalizerEntry`s are extended to track `external_size` and in which
`Heap::Space` the finalizable value is.

On attaching a native finalizer, the external size is added to the
relevant heap. When the finalizable value is promoted from new to old
space, the external size is promoted as well. And when a native
finalizer is run or is detached, the external size is removed from the
relevant heap again.

In contrast to Dart `Finalizer`s, `NativeFinalizer`s are run on isolate
shutdown.

When the `NativeFinalizer`s themselves are collected, the finalizers are
not run. Users should stick the native finalizer in a global variable to
ensure finalization. We will revisit this design when we add send and
exit support, because there is a design space to explore what to do in
that case. This current solution promises the least to users.

In this implementation native finalizers have a Dart entry to clean up
the entries from the `all_entries` field of the finalizer. We should
consider using another data structure that avoids the need for this Dart
entry. See the TODO left in the code.

Bug: https://github.com/dart-lang/sdk/issues/47777

TEST=runtime/tests/vm/dart(_2)/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
TEST=tests/ffi(_2)/vmspecific_native_finalizer_*

Change-Id: I8f594c80c3c344ad83e1f2de10de028eb8456121
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/236320
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-03-26 09:41:21 +00:00
Clement Skau
2c0ba56ea1 [vm/ffi] Add error for FFI annotation with opt. args.
Optional arguments (positional and named) in @FfiNative
annotations are not meaningful, and should result in a
compile-time error.

TEST=tests/ffi/ffi_native_test.dart
Bug: https://github.com/dart-lang/sdk/issues/47169
Change-Id: I8896e6a43f9399b537e6ee7c7a0e2857a370203f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/232622
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
2022-03-18 20:58:51 +00:00
Ryan Macnak
b6ab7350be [test] Inform status files about the new simulated architectures.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/37299
Bug: https://github.com/dart-lang/sdk/issues/38587
Change-Id: I6f1a749690b894b14f465d608f37acf62568cdef
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231521
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-02-03 20:32:16 +00:00
Alexander Markov
3d977bc191 [vm/test] Fix memory leak in ffi/regress_flutter97301_test
TEST=ci
Fixes https://github.com/dart-lang/sdk/issues/48283

Change-Id: I22d9ebac3240e9ca84fec833340d71b58d01119d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231335
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2022-02-03 00:28:55 +00:00
Alexander Markov
e4942db83e [vm/compiler] Avoid speculative conversion in ffi Pointer.asTypedList
On 32-bit ARM in AOT mode Pointer.asTypedList is generated so that
there is a LoadField from Pointer.data_field which has kUnboxedFfiIntPtr
representation (uint32) and then the value is passed to a
StoreInstanceField for TypedDataBase.data_field which has kUnboxedIntPtr
representation (int32). As a result, a speculative uint32->int32
IntConverter instruction is inserted by SelectRepresentations pass.
AOT doesn't support deoptimization so code generation crashes after
retrying without speculative inlining.

This change fixes the type incompatibility by loading value with
LoadUntagged and then converting it with ConvertUntaggedToUnboxed(kUnboxedIntPtr).

TEST=ffi/regress_flutter97301_test
Fixes https://github.com/flutter/flutter/issues/97301

Change-Id: I4a00d4ac7978b4775add0ddae510841a2b4cbae0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/230956
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-02-01 16:49:56 +00:00
Daco Harkes
045b9c1715 Reland "[vm/ffi] Add common C types"
We're adding these types to `dart:ffi` rather than `package:ffi` so that
they can be used with `FfiNative`s.

Adds `NativeType`s for the following C types:

* unsigned char
* signed char
* short
* unsigned short
* int
* unsigned int
* long
* unsigned long
* long long
* unsigned long long
* uintptr_t
* size_t
* wchar_t

Because the C standard only defines minimum sizes for many of these
types, future platforms might diverge from the typical size even if all
platforms currently agree on a size. To avoid having to reification
later, we define all types as AbiSpecificIntegers rather than typedefs,
even if all current target platforms agree on the size.

Closes: https://github.com/dart-lang/sdk/issues/36140

TEST=tests/ffi/c_types_test.dart

Original patch in patchset 1.

* Removes `Char` for now until package:win32 has rolled to 2.3.8 in
  Flutter. https://pub.dev/packages/win32/versions/2.3.8/changelog
  https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8824468064587445729/+/u/Android_Views_Integration_Tests/stdout
* Adds `c_type.dart` in `ffi_source.gni` which should fix `IntPtr`
  missing when analyzing `path_provider_linux`. (However, I was unable
  to reproduce the issue locally.)
  https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8824468064571399025/+/u/run_test.dart_for_flutter_plugins_shard_and_subshard_analyze/test_stdout
  `/tmp/flutter_plugins.KZMNMC/packages/path_provider/path_provider_linux$ ~/flt/engine/src/out/host_debug/dart-sdk/bin/dart  analyze --fatal-infos`

Change-Id: I89130cccba285fc9c483bb53f5710a302f2b104f
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-canary-linux-debug-try,vm-fuchsia-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-mac-debug-x64-try,vm-kernel-mac-release-arm64-try,vm-kernel-nnbd-win-release-ia32-try,vm-kernel-nnbd-win-release-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-win-release-x64-try,flutter-analyze-try,flutter-engine-linux-try,flutter-frontend-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229156
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-01-27 13:09:01 +00:00
Daco Harkes
5183ba3ca4 Revert "[vm/ffi] Add common C types"
This reverts commit 85a87ca7ae.

Reason for revert: 

* Adding `Char` breaks `package:win32` which is used in Flutter so it
breaks the Flutter build:
https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8824468064587445729/+/u/Android_Views_Integration_Tests/stdout

For reference: full list of Flutter failues:
https://github.com/flutter/flutter/runs/4890844911

Original change's description:
> [vm/ffi] Add common C types
>
> We're adding these types to `dart:ffi` rather than `package:ffi` so that
> they can be used with `FfiNative`s.
>
> Adds `NativeType`s for the following C types:
>
> * char
> * unsigned char
> * signed char
> * short
> * unsigned short
> * int
> * unsigned int
> * long
> * unsigned long
> * long long
> * unsigned long long
> * uintptr_t
> * size_t
> * wchar_t
>
> Because the C standard only defines minimum sizes for many of these
> types, future platforms might diverge from the typical size even if all
> platforms currently agree on a size. To avoid having to reification
> later, we define all types as AbiSpecificIntegers rather than typedefs,
> even if all current target platforms agree on the size.
>
> Closes: https://github.com/dart-lang/sdk/issues/36140
>
> TEST=tests/ffi/c_types_test.dart
>
> Change-Id: Ie97d253856d787386529231e8060f879069be886
> Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-canary-linux-debug-try,vm-fuchsia-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-mac-debug-x64-try,vm-kernel-mac-release-arm64-try,vm-kernel-nnbd-win-release-ia32-try,vm-kernel-nnbd-win-release-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-win-release-x64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/228541
> Reviewed-by: Martin Kustermann <kustermann@google.com>

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: Ic56df88c653b1395ed5e5a71af5e571b1adc3671
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229152
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-01-21 17:05:42 +00:00
Daco Harkes
85a87ca7ae [vm/ffi] Add common C types
We're adding these types to `dart:ffi` rather than `package:ffi` so that
they can be used with `FfiNative`s.

Adds `NativeType`s for the following C types:

* char
* unsigned char
* signed char
* short
* unsigned short
* int
* unsigned int
* long
* unsigned long
* long long
* unsigned long long
* uintptr_t
* size_t
* wchar_t

Because the C standard only defines minimum sizes for many of these
types, future platforms might diverge from the typical size even if all
platforms currently agree on a size. To avoid having to reification
later, we define all types as AbiSpecificIntegers rather than typedefs,
even if all current target platforms agree on the size.

Closes: https://github.com/dart-lang/sdk/issues/36140

TEST=tests/ffi/c_types_test.dart

Change-Id: Ie97d253856d787386529231e8060f879069be886
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-canary-linux-debug-try,vm-fuchsia-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-mac-debug-x64-try,vm-kernel-mac-release-arm64-try,vm-kernel-nnbd-win-release-ia32-try,vm-kernel-nnbd-win-release-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/228541
Reviewed-by: Martin Kustermann <kustermann@google.com>
2022-01-20 14:41:46 +00:00
Ryan Macnak
04ba20aa98 [vm] Support RISC-V.
Implements a backend targeting RV32GC and RV64GC, based on Linux standardizing around GC. The assembler is written to make it easy to disable usage of C, but because the sizes of some instruction sequences are compile-time constants, an additional build configuration would need to be defined to make use of it.

The assembler and disassembler cover every RV32/64GC instruction. The simulator covers all instructions except accessing CSRs and the floating point state accessible through such, include accrued exceptions and dynamic rounding mode.

Quirks:
  - RISC-V is a compare-and-branch architecture, but some existing "architecture-independent" parts of the Dart compiler assume a condition code architecture. To avoid rewriting these parts, we use a peephole in the assembler to map to compare-and-branch. See Assembler::BranchIf. Luckily nothing depended on taking multiple branches on the same condition code set.
  - There are no hardware overflow checks, so we must use Hacker's Delight style software checks. Often these are very cheap: if the sign of one operand is known, a single branch is needed.
  - The ranges of RISC-V branches and jumps are such that we use 3 levels of generation for forward branches, instead of the 2 levels of near and far branches used on ARM[64]. Nearly all code is handled by the first two levels with 20-bits of range, with enormous regex matchers triggering the third level that uses aupic+jalr to get 32-bits of range.
  - For PC-relative calls in AOT, we always generate auipc+jalr pairs with 32-bits of range, so we never generate trampolines.
  - Only a subset of registers are available in some compressed instructions, so we assign the most popular uses to these registers. In particular, THR, TMP[2], CODE and PP. This has the effect of assigning CODE and PP to volatile registers in the C calling convention, whereas they are assigned preserved registers on the other architectures. As on ARM64, PP is untagged; this is so short indices can be accessed with a compressed instruction.
  - There are no push or pop instructions, so combining pushes and pops is preferred so we can update SP once.
  - The C calling convention has a strongly aligned stack, but unlike on ARM64 we don't need to use an alternate stack pointer. The author ensured language was added to the RISC-V psABI making the OS responsible for realigning the stack pointer for signal handlers, allowing Dart to leave the stack pointer misaligned from the C calling convention's point of view until a foreign call.
  - We don't bother with the link register tracking done on ARM[64]. Instead we make use of an alternate link register to avoid inline spilling in the write barrier.

Unimplemented:
 - non-trivial FFI cases
 - Compressed pointers - No intention to implement.
 - Unboxed SIMD - We might make use of the V extension registers when the V extension is ratified.
 - BigInt intrinsics

TEST=existing tests for IL level, new tests for assembler/disassembler/simulator
Bug: https://github.com/dart-lang/sdk/issues/38587
Bug: https://github.com/dart-lang/sdk/issues/48164
Change-Id: I991d1df4be5bf55efec5371b767b332d37dfa3e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217289
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-01-20 00:57:57 +00:00
Simon Binder
0a3d896274 [ffi] Add .ref= setter for pointers of structs or unions
Adds the `ref` setter to the `StructPointer` and `UnionPointer`
extensions, copying a compound structure into native memory.
The FFI use-site transformation transforms invocations of this setter
into an appropriate memcopy call.

Closes https://github.com/dart-lang/sdk/issues/44768

TEST=tests/ffi(_2)/extension_methods_test.dart

Change-Id: I3ef06ad08b8e71e39b05d662e8082fc5d0ad876d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/227542
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-01-12 20:48:28 +00:00
Daco Harkes
acdf82de17 [vm/ffi] ABI-specific integers
This CL adds support for users defining integers which are mapped to
differing sizes and signedness based on the application binary interface
the Dart VM is running on.

Notable implementation design decisions:
- ABIs are open world, so that adding an ABI to the Dart VM does not
  break existing definitions. Thus, we only figure out in the VM that
  we're missing a mapping. We throw compile-time errors.
  - In AOT, these show up in the precompilation step.
  - In JIT, these show up as `_CompileTimeError` at runtime. Note that
    these can be caught. So in subsequent compilation steps we need to
    ensure that we also throw the same compile-time error.
- We match on the call-sites (streaming_flowgraph_builder) rather than
  method bodies (kernel_to_il) of AbiSpecific loads and stores so that
  we can compile for the int-size of the call site.

API design decisions:
https://github.com/dart-lang/sdk/issues/42563#issuecomment-981774001

Closes: https://github.com/dart-lang/sdk/issues/42563

TEST=tests/ffi_2/abi_*_test.dart
TEST=tests/ffi/function_*_generated_test.dart
TEST=tests/ffi/vmspecific_static_checks_test.dart
Change-Id: I8c8df36fab939b6fb614c5f1ee8e1bf46b6e9521
Cq-Include-Trybots: luci.dart.try:analyzer-linux-release-try,analyzer-nnbd-linux-release-try,app-kernel-linux-debug-x64-try,benchmark-linux-try,dart-sdk-linux-try,front-end-linux-release-x64-try,front-end-nnbd-linux-release-x64-try,pkg-linux-debug-try,vm-canary-linux-debug-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-fuchsia-release-x64-try,vm-kernel-checked-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-linux-debug-x64c-try,vm-kernel-mac-debug-x64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-nnbd-win-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-android-release-arm64c-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221501
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-12-16 22:07:00 +00:00
Daco Harkes
663357209c [vm/ffi] Introduce Abi
This introduces the application binary interface (ABI) concept to
`dart:ffi` as a class with opaque instances.

We will use this for providing mappings for ABI-specific types.
Bug: https://github.com/dart-lang/sdk/issues/42563
Bug: https://github.com/dart-lang/sdk/issues/42816
Closes: https://github.com/dart-lang/sdk/issues/45254

Later, we might open up the contents of `Abi` if need be.

Some API design discussion notes:
https://github.com/dart-lang/sdk/issues/42563#issuecomment-981774001

TEST=tests/ffi/abi_test.dart

Change-Id: Iebf290fc12f37d6f4236432ddf27ab3e12bde06d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221627
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2021-12-01 19:19:37 +00:00
Clement Skau
7344e5e619 [test] Skip regress_47594_test in Product
Bug: https://github.com/dart-lang/sdk/issues/47811
Change-Id: I16af738c8d01c1b82052d6f1aaf55d1650143f11
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-product-arm-try,app-kernel-linux-product-x64-try,vm-ffi-android-product-arm64c-try,vm-kernel-linux-product-x64-try,vm-kernel-mac-product-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-win-product-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221620
Auto-Submit: Clement Skau <cskau@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-11-30 11:11:04 +00:00
Clement Skau
034824f80e [test] Skip failing bots on regress_47594_test
- DartDev is not available on Android.
- DynamicLibrary.process() is not available on Windows.

Change-Id: I6606159168dd01e0fa0b9bc97b5316641123d42b
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-android-release-arm64c-try,vm-ffi-android-release-arm64c-try,vm-ffi-android-release-arm-try,vm-ffi-android-product-arm64c-try,vm-ffi-android-product-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-kernel-win-release-x64-try,vm-kernel-win-release-ia32-try,vm-kernel-win-product-x64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-precomp-nnbd-win-release-x64-try,vm-kernel-nnbd-win-release-x64-try,vm-kernel-nnbd-win-release-ia32-try,vm-kernel-nnbd-win-debug-x64-try,pkg-win-release-try,dart-sdk-win-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221500
Auto-Submit: Clement Skau <cskau@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2021-11-29 22:36:04 +00:00
Clement Skau
bd4a27ff81 [vm/ffi] Adds transition information to FFI leaf calls.
This change sets `top_exit_frame_info` and `vmtag` on FFI leaf calls.
We have to set this info on the thread so that the stack walker can
interpret the frame correctly.
Without it, anyone trying to walk the stack during an FFI leaf call
- like the profiler - will misinterpret the top frame and cause
a segfault.

TEST=Added regression test.

Bug: https://github.com/dart-lang/sdk/issues/47594
Change-Id: If83aeab194aa0213aee82558bb9541cd7294a935
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220360
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
2021-11-29 10:12:36 +00:00
Vyacheslav Egorov
71c9c2cd83 [vm] Remove support for non-bare AOT snapshots
Every AOT user out there has been using bare instructions mode
and continuing to maintaining non-bare instructions mode simply
adds costs (both in terms of time spent making changes to work
in a mode that is not used and CI resources spent on testing it).

This change removes FLAG_use_bare_instructions and changes the code
to assume that FLAG_precompiled_mode implies bare instructions.

TEST=ci

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,app-kernel-linux-release-x64-try,app-kernel-linux-debug-x64-try
Change-Id: I5032b13bfcb613f79865f2cfa139cca8d1b42556
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220964
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-11-24 13:03:52 +00:00
Daco Harkes
00ea62f713 [cfe/ffi] Handle invalid types in Array
Closes: https://github.com/dart-lang/sdk/issues/47673

TEST=tests/ffi(_2)/regress_47673(_2)_test.dart

Change-Id: I8fdafc5378797ed071250b0caed1ab944873a218
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220760
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
2021-11-19 13:38:47 +00:00
Martin Kustermann
96d40d1caa [vm/concurrency] Remove --(no-)enable-isolate-groups flag usage in tests
This is a preparation CL to remove the --enable-isolate-groups flag in
the VM.

The following tests were only running in --no-enable-isolate-groups and
are therefore obsolete now:

  - runtime/tests/vm/dart/regress_47468_test.dart
  - runtime/tests/vm/dart_2/regress_47468_test.dart
  - tests/lib/isolate/illegal_msg_function_test.dart
  - tests/lib_2/isolate/illegal_msg_function_test.dart

TEST=Changes tests only.

Change-Id: I6257cb667eebca66a649614d3010139dd2cdd3ab
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/219100
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-11-04 18:57:21 +00:00
Daco Harkes
757c2b82ce [vm/ffi] Add Bool
This CL adds the `Bool` `NativeType` with support for loading/storing
from `Pointer<Bool>`, `Array<Bool>`, and `Struct` and `Union` fields,
and support for passing booleans through FFI calls and callbacks.

The assumption is that `bool` is always treated as `uint8_t` in the
native ABIs. Including: (1) whether there can be garbage in the upper
bytes in CPU registers, (2) stack alignment, and (3) alignment in
compounds.

The conversion from `bool` to `uint8_t` is implemented as follows:
- bool to int: `value ? 1 : 0`
- int to bool: `value != 0`
The conversion is implemented in Dart in patch files for memory loads
and stores (pointer, array, and struct fields) and kernel_to_il for
FFI call and callback arguments and return value.

TEST=runtime/vm/compiler/ffi/native_type_vm_test.cc
TEST=tests/ffi/bool_test.dart
TEST=tests/ffi/function_callbacks_structs_by_value_generated_test.dart
TEST=tests/ffi/function_structs_by_value_generated_test.dart

Closes: https://github.com/dart-lang/sdk/issues/36855

Change-Id: I75d100340ba41771abfb41c598ca92066a89370b
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-ffi-android-debug-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-ffi-android-debug-arm64c-try,vm-kernel-mac-release-arm64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-android-release-arm64c-try,vm-kernel-precomp-android-release-arm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/216900
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
2021-10-18 10:44:11 +00:00
Daco Harkes
be195ad329 [analyzer/ffi] Report non-const fromFunction argument
* Adds missing error to analyzer.
* Adds analyzer test.
* Adds analyzer/cfe negative test.
* Formats the analyzer/cfe negative tests file.

TEST=pkg/analyzer/lib/src/generated/ffi_verifier.dart
TEST=tests/ffi/vmspecific_static_checks_test.dart

Bug: https://github.com/dart-lang/sdk/issues/36780

Change-Id: Ibc2a3d1c3b4b4d8b2285605f084237a55253fb20
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217007
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
2021-10-15 11:21:23 +00:00
Daco Harkes
8516cdc587 [test/ffi] Split generated files on isLeaf
Splits up tests/ffi/function_structs_by_value_generated_test.dart in
- compounds
- non-leaf calls
- leaf calls

We could also consider splitting on chunks from the `functions` from
tests/ffi/generator/structs_by_value_tests_configuration.dart as a
follow up.

TEST=This only splits up the tests, please reapprove failures.
Expected failures:
https://dart-ci.firebaseapp.com/current_results/#/filter=ffi
- windows precompiled https://github.com/dart-lang/sdk/issues/40564
- mac arm64 https://github.com/dart-lang/sdk/issues/46349

Bug: https://github.com/dart-lang/sdk/issues/45007

Change-Id: Id3d9987cbc1e09f579b8cc68ce72fe5d36348b80
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-win-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-precomp-win-debug-x64c-try,pkg-mac-release-arm64-try,vm-kernel-mac-release-arm64-try,vm-kernel-nnbd-mac-debug-arm64-try,vm-kernel-nnbd-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/216666
Reviewed-by: Clement Skau <cskau@google.com>
2021-10-14 13:05:33 +00:00
Daco Harkes
e512402b05 [test/ffi] Make the test generator async
The `dart format` step takes 8 seconds currently on the large files.
This cuts down the total time from 21 seconds to 9 on my machine.

TEST=This only changes the test generator, nothing should change.

Bug: https://github.com/dart-lang/sdk/issues/45007

Change-Id: Ie4940fdd3a43a923ca28cb39b0673cc19f81a10a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/216664
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
2021-10-14 13:05:33 +00:00
Clement Skau
998d5f5a9b [vm/ffi] Adds FFI transform FfiNative instance methods.
- Allows FfiNative annotation to be attached to non-static
  methods of classes.
- Transforms non-static instance methods to static, adding
  wrappers with the receiver as an extra, implicit first paramters.
- Transform all parameters and arguments to Pointer if the object
  being passed supports it (i.e. extends NativeFieldWrapperClass1).
- Adds compile time errors for cases where the FfiNative annotation
  doesn't align with the annotated function. Taking into account
  implicit receivers and converted Pointers.
- Adds complimentary Analyzer checks for the above errors as well.
- Adds tests for the transforms, compile time errors and analyzer
  changes.

TEST=Adds new tests for instance methods, analyzer changes.

Change-Id: Idf54430acf2728a650008333b149b254941290ad
Cq-Do-Not-Cancel-Tryjobs: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/213773
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-10-12 10:58:44 +00:00