This moderately reduces code size without large regressions on Golem. Going further to -Os still produces noticeable regressions.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/38647
Change-Id: I8409908ca37d4a2b18cb989d9748e35a15072df6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217504
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Some artifacts are .exe instead of .bat.
Library uris with file schemes use the Uri.path (using forward slashes
on Windows), so the tests emitting native asset mappings should use
Uri.path as well. (Not Uri.toFilePath() which uses backwards slashes.)
The paths of the dynamic libraries in the assets should use the
toFilePath(), as these are passed to the system API which expects
correct paths.
The Platform.script Uri has a path with file:///C: and back slashes.
The uri RFC does not support back slashes, and our ResolveUri does
neither. So for Windows we replace backslashes with forward slashes.
TEST=tests/ffi/native_assets/
Bug: https://github.com/dart-lang/sdk/issues/51066
Bug: https://github.com/dart-lang/sdk/issues/51067
Change-Id: I2e168e0549fe80d9a5366d636c6f1ef971942130
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-win-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-nnbd-win-release-x64-try,vm-kernel-nnbd-win-release-ia32-try,vm-kernel-msvc-windows-try,dart-sdk-win-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/279356
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Adds support for `asset: 'asset-id'` in `FfiNative`s.
Symbols in assets are resolved by
1. the provided [asset] (the bulk of this CL),
2. the native resolver set with `Dart_SetFfiNativeResolver` in
`dart_api.h` (old behavior), and
3. the current process.
Symbols in assets are resolved by the Dart VM through the mapping
provided in the `vm:ffi:native_assets` library:
```
@pragma('vm:ffi:native-assets', {
"linux_x64": {
"<asset1>": [
"absolute",
"<absolute-path>"
],
"<asset2>": [
"relative",
"<relative-path>"
],
"<asset3>": [
"system",
"<system-path>"
],
"<asset4>": [
"process",
],
"<asset5>": [
"executable",
],
}
})
library;
```
There are 5 asset path types. Symbols resolved in these asset types are
resolved as follows.
A. Absolute paths are dlopened, after which the symbol is looked up.
B. Relative paths are resolved relative to `Platform.script.uri`, and
dlopened, after which the symbol is looked up.
C. System paths are treated as absolute paths. (But we might explicitly
try to use `PATH` in the future.)
D. Symbols looked up in Process assets are looked up in the current
process.
E. Symbols looked up in Executable assets are looked up in the current
executable.
The native assets mapping can be provided in three ways.
X. In the invocation to `gen_kernel` with the `--native-assets`
argument. This uses the gen_kernel entry point for compiling
kernel.
Y. By placing a `native_assets.yaml` next to the
`package_config.json` used by the isolate group. This works for
`dart <source file>` and `Isolate.spawnUri(<dart source file>)`.
This uses the kernel_service entry point for compiling kernel.
Z. By sending a String containing the native assets mapping when
invoking the kernel_service directly from within the VM. Currently,
only used for unit tests.
TEST=tests/ffi/native_assets/asset_*_test.dart (X)
TEST=tests/ffi/native_assets/infer_native_assets_yaml_*.dart (Y)
TEST=runtime/vm/ffi/native_assets_test.cc (Z)
The library is synthesized from yaml in pkg:vm as AST.
Design doc: go/dart-native-assets
Bug: https://github.com/dart-lang/sdk/issues/49803
Change-Id: I8bf7000bfcc03b362948efd3baf168838948e6b9
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-mac-debug-arm64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/264842
Reviewed-by: Martin Kustermann <kustermann@google.com>
The representation of record shape in record instances and record
types is changed from a pair
int num_fields
Array field_names
to a single integer - packed bitfield
int num_fields(16)
int field_names_index(kSmiBits-16)
where field names index is an index in the array available from
ObjectStore.
With the new representation of record shapes:
1) Size of record instances is reduced.
2) Number of comparisons for a shape test reduced from 2 to 1
(shape test is used during type checks).
3) A few operations removed from Record.hashCode.
4) Type testing stubs (TTS) are now supported for record types with
named fields. Previously it was not possible to check shape of records
with named fields in TTS as TTS cannot access object pool and cannot
load field names array).
TEST=existing
Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I7cdcbb53938aba5d561cd24dc99530395dbbea7e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276201
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This lets the GC visit FfiTrampolineData::c_signature again.
https://dart-review.googlesource.com/c/sdk/+/272201 stopped adding
FfiTrampolineData::c_signature to snapshots. However, instead of
skipping it manually in app_shapshot.cc, we skipped it in
raw_object.h, which also caused the GC to skip it.
This CL adds it back in as we need it in JIT snapshots. This way we
keep it consistent between AOT/JIT snapshots.
TEST=tests/ffi/regress_b_261224444_test.dart
The c signatures of FFI trampolines were not properly traced in the
precompiler, causing us to hit an assert when the classes mentioned in
those types where only referenced from a signature and not retained
for any other reason.
TEST=tests/ffi/native_assets/process_test.dart (dartkp)
Closes: https://github.com/dart-lang/sdk/issues/50678
Bug: b/261224444
Change-Id: I84fc880744c2045ea3e2ef4f37df454b80b2faeb
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,app-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/274387
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Right each `Pointer.fromFunction()` invocation will lead to creation of
a new ffi trampoline function & it's following JITed code. In AOT we
have exactly one ffi trampoline per target/native-signature/exceptional-return
combination.
=> This CL ensures we have only one such function.
Furthermore each `Pointer.fromFunction()` will currently perform 2
runtime calls in JIT: One to create a `Function` object, the other to
JIT that function & register callback metadata.
=> This CL ensures we won't do a runtime call to get a function, instead
do it at compile-time (as in AOT)
Furthermore we eagerly assign a callback-id to the unique/deduped ffi
trampoline callbacks. Only when the application requests a pointer, do
we populate metadata on the `Thread` object.
This CL doesn't (yet) change the fact that in JIT mode we have
isolate-specific jit trampolines (that will call now shared ffi trampoline
functions).
We also avoid baking in C++ runtime function pointers in generated
code. As a result we can now preserve ffi trampolines across AppJIT
serialization.
As a nice side-effect, we remove 100 lines of code.
TEST=ffi{,_2}/ffi_callback_unique_test
Issue https://github.com/dart-lang/sdk/issues/50611
Change-Id: I458831a47b041a088086f28f825de2a3849f6adc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/273420
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Allows embedders to programmatically request heap snapshot from a
running isolate group instead of relying on a vm-service or hidden
internal Dart APIs
Additionally we allow to include snapshot writing functionality into
PRODUCT builds using DART_ENABLE_HEAP_SNAPSHOT_WRITER define.
TEST=vm/cc/DartAPI_WriteHeapSnapshot
Bug: b/259115846
Change-Id: Ic3ef76e5fb9adcf8f23a1959f5238742b64ecde2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/273181
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
All `Pointer`s will now have the `Never` type argument at runtime.
TEST=tests/ffi(_2)/*
Bug: https://github.com/dart-lang/sdk/issues/49935
Change-Id: I83c8bba9461c2cab22992ba2e3cf42b7b5f43c36
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-mac-debug-x64-try,dart-sdk-mac-arm64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/272201
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
This makes things nicer to read in places that display implementation names, such as stack traces, debuggers, profilers and inspectors.
TEST=ci
Change-Id: I959f70d9e51be59801c4455f8c5ccac3c214c21a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/270502
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This CL makes `FfiNative`s use `DynamicLibrary.process()` lookup if
resolving with the resolver set by `Dart_SetFfiNativeResolver` fails.
Also moves the implementation over from ffi.cc to
ffi_dynamic_library.cc so the implementation can be shared with
`DynamicLibrary.process()`.
Moves the implementation behind non-simulator and non-precompiler.
However, the implementation is tested in vm/cc tests which are in
precompiler mode. So enables the implementation if TESTED is defined.
This CL massages the build files so that TESTED is properly
defined when compiling the runtime for the vm/cc tests, and links
the ole32 symbols on windows for vm/cc tests.
(And some unrelated small cleanup changes here and there.)
TEST=tests/ffi/native_assets/process_test.dart
Change-Id: I25395d381db1d9b4b7a5759171a798a1140a6140
Cq-Include-Trybots: luci.dart.try:vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,dart-sdk-win-try,vm-kernel-win-release-x64-try,vm-kernel-win-release-ia32-try,vm-kernel-precomp-win-product-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/264982
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Support looking up a symbol in the process on Windows by using the
Windows Process Status API to iterate over all loaded Modules.
TEST=tests/ffi/has_symbol_test.dart
TEST=tests/ffi/vmspecific_dynamic_library_test.dart
Change-Id: I1029f1c7dae9a193b662d942388affb681842c90
Cq-Include-Trybots: luci.dart.try:vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,dart-sdk-win-try,vm-kernel-win-release-x64-try,vm-kernel-win-release-ia32-try,vm-kernel-precomp-win-product-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/260760
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This is a reland of commit b8d4e24338
How the failures were fixed:
1. My ExternalSizeLimit test crashed on msvc because I was using a
0-sized array. I have now changed that array to have size 1.
2. My ExternalSizeLimit test crashed on x64c because
ExternalTypedData::MaxElements(kExternalTypedDataUint8ArrayCid) is much
smaller than kMaxAddrSpaceMB/4 on x64c. I now call
ExternalTypedData::New() with a length argument of 1, and just pretend
that the external allocations are larger when calling
FinalizablePersistentHandle::New().
Original change's description:
> [VM - Runtime] Return nullptr when allocating a
> FinalizablePersistentHandle fails
>
> This CL adds checks to ensure that the tracked total size of
> externally allocated objects never exceeds the amount of memory on the
> system. When the limit is exceeded, then
> FinalizablePersistentHandle::New() will return nullptr.
>
> Resolves https://github.com/dart-lang/sdk/issues/49332
>
> TEST=ci
>
> Change-Id: Ib6cc92325b1d5efcb2965098fa45cfecc90995e3
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256201
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Derek Xu <derekx@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>
TEST=I ran the tryjobs for the configurations that broke CI.
Change-Id: I813aa74667c59a4dbec7f53440ca8d0bf21256ce
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256973
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
This reverts commit b8d4e24338.
Reason for revert: breaks msvc, msan, x64c builds
Original change's description:
> [VM - Runtime] Return nullptr when allocating a FinalizablePersistentHandle fails
>
> This CL adds checks to ensure that the tracked total size of externally
> allocated objects never exceeds the amount of memory on the system. When
> the limit is exceeded, then FinalizablePersistentHandle::New() will
> return nullptr.
>
> Resolves https://github.com/dart-lang/sdk/issues/49332
>
> TEST=ci
>
> Change-Id: Ib6cc92325b1d5efcb2965098fa45cfecc90995e3
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256201
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Derek Xu <derekx@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>
TBR=bkonyi@google.com,asiva@google.com,dnfield@google.com,derekx@google.com
Change-Id: I934bfbf5dc2e8e2ead5c74fe6b1d84e7b311788c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256972
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This CL adds checks to ensure that the tracked total size of externally
allocated objects never exceeds the amount of memory on the system. When
the limit is exceeded, then FinalizablePersistentHandle::New() will
return nullptr.
Resolves https://github.com/dart-lang/sdk/issues/49332
TEST=ci
Change-Id: Ib6cc92325b1d5efcb2965098fa45cfecc90995e3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256201
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
These types now work with Dart_TypedDataAcquireData.
The presence of these types no longer degrades the performance of typed data indexed loads.
The presence of these types degrades the performance of typed data indexed stores much less. The performance of indexed stores is somewhat regressed if these types were not used.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/32028
Bug: https://github.com/dart-lang/sdk/issues/40924
Bug: https://github.com/dart-lang/sdk/issues/42785
Change-Id: I05ac5c9543f6f61ac37533b9efe511254778caed
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/253700
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Heap snapshots currently produced don't visit isolate stacks. As such
analyzing such snapshots may lead one to conclude there is a lot of
garbage while objects are actually reachable.
=> This CL makes us visit isolate stacks when building heap snapshots.
Furthermore we add a new `VMInternals.writeHeapSnapshotToFile` helper
that can be used to programmatically write snapshots and can be handy
for internal use at times. (We also use this helper in a test)
TEST=vm/dart{,_2}/heap_snapshot_test
Change-Id: I976544b7f6d20863764af9a40bf1ffb3c319bbce
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/253785
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This is a reland of commit 08c13f173c
Fixes test failures on non-x64 architectures, both in the test
harness and due to DWARF5 line number program headers having a
non-backwards compatible format. (We generate DWARF2 in the
ELF snapshot writer, but the assembler used for assembly snapshots
may generate DWARF5.)
TEST=vm/dart{,_2}/use_dwarf_stack_traces_flag
Original change's description:
> [pkg/native_stack_traces] Support Mach-O dSYM debugging information.
>
> TEST=vm/dart{,_2}/use_dwarf_stack_traces_flag
>
> Bug: https://github.com/dart-lang/sdk/issues/43612
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-product-x64-try,pkg-mac-release-arm64-try,vm-kernel-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
> Change-Id: Icda21bb14dcc0cf4784cea118e6ba7dd4edd35aa
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/250381
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Slava Egorov <vegorov@google.com>
Bug: https://github.com/dart-lang/sdk/issues/43612
Change-Id: I8a9cb70e78bc8594bcae004809c5a1be778d691d
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-product-x64-try,pkg-mac-release-arm64-try,vm-kernel-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-nnbd-mac-release-simarm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/251464
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
