dart-sdk/runtime/vm
Daco Harkes e16bb210d2 [vm/ffi] Optimize @Native calls
This CL removes static fields for storing the `@Native`'s function
addresses. Instead, the function addresses are stored in the object
pool for all archs except for ia32. ia32 has no AOT and no AppJit
snapshots, so the addresses are directly embedded in the code.

This CL removes the closure wrapping for `@Native`s. Instead of
`pointer.asFunctionInternal()()` where `asFunction` returns a closure
which contains the trampoline, the function is compiled to a body
which contains the trampoline `Native()`. This is possible for
`@Native`s because the dylib and symbol names are known statically.

Doing the compilation in kernel_to_il instead of a CFE transform
enables supporting static linking later. (The alternative would have
been to transform in the cfe to a `@pragma('vm:cachable-idempotent')`
instead of constructing the IL in kernel_to_il.

To enable running resolution in ia32 in kernel_to_il.cc, the
resolution function has been made available via
`runtime/lib/ffi_dynamic_library.h`.

Because the new calls are simply static calls, the TFA can figure
out const arguments flowing to these calls. This leads to constant
locations in the parameters to FfiCalls. So, this CL also introduces
logic to move constants into `NativeLocation`s.

TEST=runtime/vm/compiler/backend/il_test.cc
TEST=tests/ffi/function_*_native_(leaf_)test.dart
TEST=pkg/vm/testcases/transformations/ffi/ffinative_compound_return.dart

Closes: https://github.com/dart-lang/sdk/issues/47625
Closes: https://github.com/dart-lang/sdk/issues/51618
Change-Id: Ic5d017005dedcedea40c455c4d24dbe774f91603
CoreLibraryReviewExempt: Internal FFI implementation changes
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284300
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-11-02 16:00:53 +00:00
..
compiler [vm/ffi] Optimize @Native calls 2023-11-02 16:00:53 +00:00
ffi
heap Revert "[vm, gc] Mark through new-space." 2023-10-18 07:47:39 +00:00
libfuzzer
protos
service Remove deprecated APIs from dart_io_extensions.dart and add new is*Available RPCs. 2023-10-11 22:22:12 +00:00
allocation.cc
allocation.h
allocation_test.cc
ama_test.cc
analyze_snapshot_api_impl.cc
app_snapshot.cc [vm] Introduce cachable idempotent calls 2023-10-27 07:55:47 +00:00
app_snapshot.h
assert_test.cc
atomic_test.cc
base64.cc
base64.h
base64_test.cc
base_isolate.h
benchmark_test.cc
benchmark_test.h
bit_set.h
bit_set_test.cc
bit_vector.cc
bit_vector.h
bit_vector_test.cc
bitfield.h
bitfield_test.cc
bitmap.cc
bitmap.h
bitmap_test.cc
boolfield.h
boolfield_test.cc
bootstrap.cc
bootstrap.h
bootstrap_natives.cc
bootstrap_natives.h Reland "[vm/compiler] Change MemoryCopy to also take untagged addresses." 2023-10-17 07:41:49 +00:00
bss_relocs.cc
bss_relocs.h
BUILD.gn
canonical_tables.cc
canonical_tables.h
catch_entry_moves_test.cc
class_finalizer.cc
class_finalizer.h
class_finalizer_test.cc
class_id.h
class_table.cc
class_table.h
closure_functions_cache.cc
closure_functions_cache.h
code_comments.cc
code_comments.h
code_descriptors.cc [vm/aot/tfa] Whole-program propagation of closure values 2023-10-12 16:16:21 +00:00
code_descriptors.h
code_descriptors_test.cc
code_entry_kind.h
code_observers.cc
code_observers.h
code_patcher.cc
code_patcher.h
code_patcher_arm.cc
code_patcher_arm64.cc
code_patcher_arm64_test.cc
code_patcher_arm_test.cc
code_patcher_ia32.cc
code_patcher_ia32_test.cc
code_patcher_riscv.cc
code_patcher_riscv_test.cc
code_patcher_x64.cc
code_patcher_x64_test.cc
compiler_test.cc [vm] Pass offset and script uri for expression compilation 2023-10-12 10:22:38 +00:00
constants.h
constants_arm.cc
constants_arm.h
constants_arm64.cc
constants_arm64.h
constants_base.h
constants_ia32.cc
constants_ia32.h
constants_riscv.cc
constants_riscv.h
constants_x64.cc
constants_x64.h
constants_x86.h
cpu.h
cpu_arm.cc
cpu_arm.h
cpu_arm64.cc
cpu_arm64.h
cpu_ia32.cc
cpu_ia32.h
cpu_riscv.cc
cpu_riscv.h
cpu_test.cc
cpu_x64.cc
cpu_x64.h
cpuid.cc
cpuid.h
cpuinfo.h
cpuinfo_android.cc
cpuinfo_fuchsia.cc
cpuinfo_linux.cc
cpuinfo_macos.cc
cpuinfo_test.cc
cpuinfo_win.cc
custom_isolate_test.cc
dart.cc
dart.h
dart_api_impl.cc [vm] Enable waitFor for 3.2 release 2023-10-05 13:24:34 +00:00
dart_api_impl.h
dart_api_impl_test.cc Revert "[vm, gc] Mark through new-space." 2023-10-18 07:47:39 +00:00
dart_api_message.h
dart_api_state.cc
dart_api_state.h
dart_entry.cc
dart_entry.h
datastream.cc
datastream.h
datastream_test.cc
debugger.cc
debugger.h
debugger_api_impl_test.cc [vm] Pass offset and script uri for expression compilation 2023-10-12 10:22:38 +00:00
debugger_api_impl_test.h
debugger_arm.cc
debugger_arm64.cc
debugger_ia32.cc
debugger_riscv.cc
debugger_x64.cc
deferred_objects.cc [vm, compiler] Implement unboxed SIMD for RISC-V via lowering. 2023-09-27 16:23:38 +00:00
deferred_objects.h
deopt_instructions.cc Reland "[vm/compiler] Change MemoryCopy to also take untagged addresses." 2023-10-17 07:41:49 +00:00
deopt_instructions.h [vm, compiler] Implement unboxed SIMD for RISC-V via lowering. 2023-09-27 16:23:38 +00:00
dispatch_table.cc
dispatch_table.h
double_conversion.cc
double_conversion.h
double_internals.h
dwarf.cc [vm] Fix synthetic line number program 2023-10-23 13:54:04 +00:00
dwarf.h
elf.cc
elf.h
exceptions.cc [vm, compiler] Implement unboxed SIMD for RISC-V via lowering. 2023-09-27 16:23:38 +00:00
exceptions.h [vm, compiler] Implement unboxed SIMD for RISC-V via lowering. 2023-09-27 16:23:38 +00:00
exceptions_test.cc
experimental_features.cc Reland "Bump version to 3.3." 2023-10-06 19:00:58 +00:00
experimental_features.h Reland "Bump version to 3.3." 2023-10-06 19:00:58 +00:00
ffi_callback_metadata.cc [vm] Make sure to use MAP_JIT for callback pages 2023-11-01 22:38:27 +00:00
ffi_callback_metadata.h
ffi_callback_metadata_test.cc
field_table.cc
field_table.h
finalizable_data.h
fixed_cache.h
fixed_cache_test.cc
flag_list.h
flags.cc
flags.h
flags_test.cc
frame_layout.h [vm] Fix Windows ARM64 MSVC build. 2023-10-20 19:45:49 +00:00
gdb_helpers.cc
globals.h [vm] Fix Windows ARM64 MSVC build. 2023-10-20 19:45:49 +00:00
growable_array.h
growable_array_test.cc
guard_field_test.cc
handle_visitor.h
handles.cc
handles.h
handles_impl.h
handles_test.cc
hash.h
hash_map.h
hash_map_test.cc
hash_table.h
hash_table_test.cc
image_snapshot.cc [vm] More fixes for ARM64 MSVC. 2023-10-24 15:04:36 +00:00
image_snapshot.h
instructions.cc
instructions.h
instructions_arm.cc
instructions_arm.h
instructions_arm64.cc
instructions_arm64.h
instructions_arm64_test.cc
instructions_arm_test.cc
instructions_ia32.cc
instructions_ia32.h
instructions_ia32_test.cc
instructions_riscv.cc
instructions_riscv.h
instructions_riscv_test.cc
instructions_x64.cc
instructions_x64.h
instructions_x64_test.cc
intrusive_dlist.h
intrusive_dlist_test.cc
isolate.cc Revert "[vm, gc] Mark through new-space." 2023-10-18 07:47:39 +00:00
isolate.h Revert "[vm, gc] Mark through new-space." 2023-10-18 07:47:39 +00:00
isolate_reload.cc [vm, reload] Preserve libraries' ffi_native_resolver. 2023-10-19 17:38:10 +00:00
isolate_reload.h
isolate_reload_test.cc
isolate_test.cc
json_stream.cc
json_stream.h
json_test.cc
json_writer.cc
json_writer.h
kernel.cc
kernel.h
kernel_binary.cc
kernel_binary.h [cfe] Add offset serialization to remaining Expression nodes without them. 2023-10-30 16:05:27 +00:00
kernel_isolate.cc [vm] Pass offset and script uri for expression compilation 2023-10-12 10:22:38 +00:00
kernel_isolate.h [vm] Pass offset and script uri for expression compilation 2023-10-12 10:22:38 +00:00
kernel_loader.cc [vm/ffi] Optimize @Native calls 2023-11-02 16:00:53 +00:00
kernel_loader.h [vm/ffi] Optimize @Native calls 2023-11-02 16:00:53 +00:00
kernel_test.cc
lockers.cc
lockers.h
log.cc
log.h
log_test.cc
longjump.cc
longjump.h
longjump_test.cc
megamorphic_cache_table.cc
megamorphic_cache_table.h
memory_region.cc
memory_region.h
memory_region_test.cc
message.cc
message.h
message_handler.cc
message_handler.h
message_handler_test.cc
message_snapshot.cc
message_snapshot.h
message_test.cc
metrics.cc
metrics.h
metrics_test.cc
mixin_test.cc
native_api_impl.cc
native_arguments.h [vm/ffi] Optimize @Native calls 2023-11-02 16:00:53 +00:00
native_entry.cc
native_entry.h
native_entry_test.cc
native_entry_test.h
native_function.h
native_message_handler.cc
native_message_handler.h
native_symbol.h
native_symbol_posix.cc
native_symbol_win.cc
object.cc [vm/ffi] Optimize @Native calls 2023-11-02 16:00:53 +00:00
object.h [vm/ffi] Optimize @Native calls 2023-11-02 16:00:53 +00:00
object_arm64_test.cc
object_arm_test.cc
object_graph.cc
object_graph.h
object_graph_copy.cc Revert "[vm, gc] Mark through new-space." 2023-10-18 07:47:39 +00:00
object_graph_copy.h
object_graph_test.cc
object_ia32_test.cc
object_id_ring.cc
object_id_ring.h
object_id_ring_test.cc
object_reload.cc
object_riscv_test.cc
object_service.cc
object_set.h
object_store.cc [vm/ffi] Optimize @Native calls 2023-11-02 16:00:53 +00:00
object_store.h [vm/ffi] Optimize @Native calls 2023-11-02 16:00:53 +00:00
object_store_test.cc
object_test.cc
object_x64_test.cc
os.h
os_android.cc
os_fuchsia.cc
os_linux.cc
os_macos.cc
os_test.cc
os_thread.cc [VM/Timeline] Fix race between Timeline::Cleanup() and OSThread's destructor 2023-10-12 16:58:27 +00:00
os_thread.h [vm] Explicitly exclude interrupt related state in PRODUCT 2023-09-28 19:53:33 +00:00
os_thread_absl.cc
os_thread_absl.h
os_thread_android.cc
os_thread_android.h
os_thread_fuchsia.cc
os_thread_fuchsia.h
os_thread_linux.cc
os_thread_linux.h
os_thread_macos.cc
os_thread_macos.h
os_thread_win.cc
os_thread_win.h
os_win.cc
parser.cc
parser.h
pending_deopts.cc
pending_deopts.h
perfetto_utils.h
pointer_tagging.h
port.cc
port.h
port_set.h
port_test.cc
proccpuinfo.cc
proccpuinfo.h
profiler.cc [vm, profiler] Don't assume the isolate has a mutator during a profile sample. 2023-10-02 17:57:33 +00:00
profiler.h
profiler_service.cc
profiler_service.h
profiler_test.cc [vm, profiler] Don't assume the isolate has a mutator during a profile sample. 2023-10-02 17:57:33 +00:00
program_visitor.cc
program_visitor.h
random.cc
random.h
raw_object.cc Revert "[vm, gc] Mark through new-space." 2023-10-18 07:47:39 +00:00
raw_object.h Revert "[vm, gc] Mark through new-space." 2023-10-18 07:47:39 +00:00
raw_object_fields.cc
raw_object_fields.h
regexp.cc
regexp.h
regexp_assembler.cc
regexp_assembler.h
regexp_assembler_bytecode.cc
regexp_assembler_bytecode.h
regexp_assembler_bytecode_inl.h
regexp_assembler_ir.cc [vm/compiler] Convert more LoadUntagged -> LoadField. 2023-10-17 16:14:29 +00:00
regexp_assembler_ir.h
regexp_ast.cc
regexp_ast.h
regexp_bytecodes.h
regexp_interpreter.cc
regexp_interpreter.h
regexp_parser.cc
regexp_parser.h
regexp_test.cc
report.cc
report.h
resolver.cc
resolver.h
reusable_handles.h
reverse_pc_lookup_cache.cc
reverse_pc_lookup_cache.h
ring_buffer.h
ring_buffer_test.cc
runtime_entry.cc Revert "[vm, gc] Mark through new-space." 2023-10-18 07:47:39 +00:00
runtime_entry.h
runtime_entry_arm.cc
runtime_entry_arm64.cc
runtime_entry_ia32.cc
runtime_entry_list.h
runtime_entry_riscv.cc
runtime_entry_x64.cc
scope_timer.h
scopes.cc
scopes.h Mark operator== const to avoid ambiguity in C++20. 2023-10-04 13:18:55 +00:00
scopes_test.cc
service.cc [vm] Pass offset and script uri for expression compilation 2023-10-12 10:22:38 +00:00
service.h
service_event.cc
service_event.h
service_isolate.cc
service_isolate.h
service_test.cc
signal_handler.h
signal_handler_android.cc
signal_handler_fuchsia.cc
signal_handler_linux.cc
signal_handler_macos.cc
signal_handler_win.cc
simulator.h
simulator_arm.cc
simulator_arm.h
simulator_arm64.cc
simulator_arm64.h
simulator_riscv.cc
simulator_riscv.h
simulator_x64.cc
simulator_x64.h
snapshot.cc
snapshot.h
snapshot_test.cc
source_report.cc
source_report.h
source_report_test.cc
splay-tree.h
stack_frame.cc [vm] Fix Windows ARM64 MSVC build. 2023-10-20 19:45:49 +00:00
stack_frame.h
stack_frame_arm.h
stack_frame_arm64.h
stack_frame_ia32.h
stack_frame_riscv.h
stack_frame_test.cc
stack_frame_x64.h
stack_trace.cc
stack_trace.h
static_type_exactness_state.h
stub_code.cc
stub_code.h
stub_code_arm64_test.cc
stub_code_arm_test.cc
stub_code_ia32_test.cc
stub_code_list.h
stub_code_riscv_test.cc
stub_code_test.cc
stub_code_x64_test.cc
symbols.cc
symbols.h [vm/ffi] Optimize @Native calls 2023-11-02 16:00:53 +00:00
tagged_pointer.h Revert "[vm, gc] Mark through new-space." 2023-10-18 07:47:39 +00:00
tags.cc
tags.h
thread.cc Revert "[vm, gc] Mark through new-space." 2023-10-18 07:47:39 +00:00
thread.h Revert "[vm, gc] Mark through new-space." 2023-10-18 07:47:39 +00:00
thread_barrier.h
thread_barrier_test.cc
thread_interrupter.cc
thread_interrupter.h
thread_interrupter_android.cc
thread_interrupter_android_arm.S
thread_interrupter_fuchsia.cc
thread_interrupter_linux.cc
thread_interrupter_macos.cc
thread_interrupter_win.cc [vm] Populate LR in Windows ARM[64] thread interrupter. 2023-10-09 17:23:23 +00:00
thread_pool.cc
thread_pool.h
thread_pool_test.cc
thread_registry.cc Revert "[vm, gc] Mark through new-space." 2023-10-18 07:47:39 +00:00
thread_registry.h Revert "[vm, gc] Mark through new-space." 2023-10-18 07:47:39 +00:00
thread_stack_resource.cc
thread_stack_resource.h
thread_state.cc
thread_state.h
thread_test.cc [gardening] Fix non-protected use of DisableThreadInterrupts. 2023-09-29 10:08:02 +00:00
timeline.cc [VM/Timeline] Print deprecation warning when --systrace-timeline flag is used 2023-10-16 14:02:31 +00:00
timeline.h [VM/Timeline] Fix race between Timeline::Cleanup() and OSThread's destructor 2023-10-12 16:58:27 +00:00
timeline_android.cc
timeline_fuchsia.cc
timeline_linux.cc
timeline_macos.cc
timeline_test.cc
timer.cc
timer.h
token.cc [vm, compiler] Implement unboxed SIMD for RISC-V via lowering. 2023-09-27 16:23:38 +00:00
token.h [vm, compiler] Implement unboxed SIMD for RISC-V via lowering. 2023-09-27 16:23:38 +00:00
token_position.cc
token_position.h
type_testing_stubs.cc
type_testing_stubs.h
type_testing_stubs_test.cc
unibrow-inl.h
unibrow.cc
unibrow.h
unicode.cc
unicode_data.cc
unicode_test.cc
unit_test.cc [vm] Pass offset and script uri for expression compilation 2023-10-12 10:22:38 +00:00
unit_test.h [vm/test] Support loading from pool in 32 bit assembler tests 2023-09-27 18:18:47 +00:00
unwinding_records.cc
unwinding_records.h
unwinding_records_win.cc
uri.cc
uri.h
uri_test.cc
utils_test.cc [platform] Fix Utils::IsInt on unsigned values. 2023-10-31 19:39:51 +00:00
v8_snapshot_writer.cc
v8_snapshot_writer.h
version.h
version_in.cc
virtual_memory.cc [vm] Make sure to use MAP_JIT for callback pages 2023-11-01 22:38:27 +00:00
virtual_memory.h
virtual_memory_compressed.cc
virtual_memory_compressed.h
virtual_memory_fuchsia.cc [vm] Fail describing error if VMO operations fail for a reason other than no memory. 2023-10-17 20:32:52 +00:00
virtual_memory_posix.cc
virtual_memory_test.cc
virtual_memory_win.cc
visitor.cc
visitor.h
vm_sources.gni
zone.cc
zone.h
zone_test.cc
zone_text_buffer.cc
zone_text_buffer.h