Previously constants were referenced via there relative binary offset,
i.e. a constant reference saying 42 meant at byte offset 42 relative to
the start of the constant table in the binary. This was done to be able
to (on the VM side) read the needed constants lazily. It meant, though,
that constants had to be stored in a map, mapping from the byte position
to the constant.
This change adds a level of indirection when needing the lazy reading,
but lets the constant references reference the constant number instead
so that a constant reference saying 42 means constant number 42,
i.e. constants can be stored in a list instead of in a map.
This is done on the dart side, but the VM still stores it in a map.
The level of indirection is a tabel next to the constant table where
each entry has constant size (4 bytes) from which one can read the
relative byte offset into the constant table from the constant number,
thus still being able to read needed constants lazily.
This CL also cleans up a leftover where for instance double constants
had their textual representation saved as a string in the string indexer
(and thus the output dill) even though they were never referenced.
File size changes:
* Platform: increses 7,816 bytes.
* Compile of dart2js (including platform): decreases 71,424 bytes.
Speed changes:
* Adding `UserTag`s to the code and looking at observatories cpu profile
on a `pkg/kernel/test/binary_bench.dart --golem AstFromBinaryLazy`
run of a compile of dart2js reading the constant table has gone
from ~10% to ~5%.
* Doing statistics on
`pkg/kernel/test/binary_bench.dart --raw AstFromBinaryLazy` run of a
compile of dart2js says -6.28169% +/- 4.97269%.
* Golem runs has nothing above the noise level.
It does say "AstFromBinaryLazy (Intel Xeon) -4.006% (0.4 noise)" and
"AstFromBinaryEagerP50 (Intel Core i5) -8.929% (0.6 noise)" though.
TEST=All tests running the VM tests this.
Change-Id: I07ead457527a4477de803ee55ba742f5557413d3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196925
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
When handling hot reload via library tag handler's Kernel tag
VM must guarantee that external typed data it receives from the
embedder stays alive for as long as VM has any structures
referencing this kernel binary. This is achieved by attaching
original typed data to KernelProgramInfo objects containing
the views into it.
Unfortunately kernel::Reader APIs were somewhat unsafe: allowing
to construct reader object from Program's raw buffer and forget to
set the link betwen the reader and original typed data.
This created a situation where reloading using a multicomponent
Kernel binary would result in KernelProgramInfo objects without
link to the original typed data, which in turn leads to premature
finalization of external typed data and subsequent crashes
when trying to use delete kernel binary.
This CL reworks kernel::Reader API (by introducing kernel::ProgramBinary
wrapper) to make it safer and make sure that connection is preserved.
TEST=vm/cc/IsolateReload_RegressB179030011
Bug: b/179030011
Change-Id: I05f8c31c3cb8e67de6e94a20d9501a5f476b7e27
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182280
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
When creating new instructions that inherit a token position that
represents a source location from another instruction, the inheriting
instruction must also have the same inlining ID in order for the source
position represented by the token position to be looked up in the
correct script.
Force this by wrapping both in a single InstructionSource struct which
is taken by instructions which take token positions instead of just a
token position. That way, it's more work to manually transfer the token
position separately from the inlining ID of the instruction than doing
the right thing of transfering both at once.
To ensure this information is kept consistent, we pass InstructionSource
structs through the FlowGraphCompiler all the way down to the
CodeSourceMapBuilder.
This CL also makes the following changes:
* Cache the upper bound of source positions in scripts and use it to add
a check for if a given real token position is valid for the script
without iterating over the line starts data for each token position.
* Start inlining intervals appropriately when adding descriptor and
null check information to the code source map.
Code size changes are minimal on Flutter gallery in release mode
(<0.05% decrease).
TEST=Existing tests on trybots, with manual checking with
--check-token-positions that previous errors are now removed.
Bug: https://github.com/dart-lang/sdk/issues/44436
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-ia32-try
Change-Id: I23ced262cb4e9fe9d81356f409e7e8d220d63ee0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173967
Reviewed-by: Régis Crelier <regis@google.com>
First, change LocationForPosition and TokenRangeAtLine to return
whether they successfully located the offset/line. Also change the
wrappers in Script (GetTokenLocation and TokenRangeAtLine) to return
whether or not any information was successfully located.
These methods now only change the out parameters in the case that
appropriate information was found, so any unconditional uses of the out
parameters need the out parameters to be appropriately initialized.
We now allow negative lines to be requested in TokenRangeAtLine (with
the failure behavior described above) so the line returned by
LocationAtPosition can be fed into TokenRangeAtLine without intermediate
checking.
The calculation of the token length, which uses the script source and
not the line starts array, has been moved to a new method,
Script::GetTokenLength. The new method returns the length (or a
negative value if the token length could not be determined) instead of
using an out parameter.
TEST=Existing tests on trybots, to ensure tests using current
line/number info aren't affected.
Bug: https://github.com/dart-lang/sdk/issues/44436
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try
Change-Id: Ibc048a226d11ff9a340a8d249654d07720fdf115
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175365
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
This CL makes use of the now included constant constructor coverage
in the dill file.
It works like this:
* When the CFE evaluates constants, every constant constructor
invocation evaluated saves the reference to the constructor in the
`Source` (from the Components uri to source table) for the callers
Library.
* This data is loaded into the VM in a "raw" format.
* When a request for coverage comes in, the VM - on top of the normal
coverage processing - goes through all scripts to find constant
constructor coverage for the requested script and offset. Note that
all scripts must be checked because library A can have evaluated a
constructor from library B - so even if only coverage for library B
was requested, library A has to be checked.
For all constructors found the start and end position is reported as
covered. Note that this does not mark any initializes and there are
(at least currently) no good way of marking which initializes were
evaluated (because it has to be stable across edits even when the
`advanced invalidation feature` is enabled).
* Note that the reason for the coverage to work on references - as
hinted above - is because we want it to be stable across hot reloads
even if/when advanced invalidation is enabled. This means, that
library A cannot record "positional coverage" for library B because
library B might get (for instance) new comments that will make any old
offsets invalid. By using references we always lookup in the current
world and use the correct offsets.
https://github.com/dart-lang/sdk/issues/38934
TEST=Existing test suite, new tests for the new coverage added.
Change-Id: I29531247a4b91a99d9a459cfdefbb9798e9c948f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175246
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Remove the use of Fields, which have since the metadata mechanism was introduced come to require global registration.
Retain some sloppiness matching of declarations by name instead of identity for the sake of hot reload.
TEST=ci
Change-Id: Ifa4cc7724096c606f994b057b7838e124ceb3626
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175141
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit bfd382820f.
Reason for revert: Breaks several lib_2/mirrors/ tests
Original change's description:
> [vm] Track metadata by identity instead of name.
>
> Remove the use of Fields, which have since the metadata mechanism was introduced come to require global registration.
>
> TEST=ci
> Change-Id: Ic2e0c4408557514507fd732e0cb19b022b2539c7
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161631
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
TBR=kustermann@google.com,aam@google.com,rmacnak@google.com
Change-Id: Icf75e91872d9c85ce9be29a9a178d72ca024c98b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174920
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
Remove the use of Fields, which have since the metadata mechanism was introduced come to require global registration.
TEST=ci
Change-Id: Ic2e0c4408557514507fd732e0cb19b022b2539c7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161631
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit 5370c56c80.
Reason for revert: Failures on
dartk-android-product-arm
dartk-android-product-arm
dartkp-linux-release-arm-qemu
like this:
sizeof(ScriptLayout) got 64, Script_InstanceSize expected 56
../../runtime/vm/dart.cc: 160: error: CheckOffsets failed. Try updating offsets by running ./tools/run_offsets_extractor.sh
sizeof(ScriptLayout) got 64, AOT_Script_InstanceSize expected 56
../../runtime/vm/dart.cc: 160: error: CheckOffsets failed. Try updating offsets by running ./tools/run_offsets_extractor.sh
Original change's description:
> [VM] Read and report constant constructor coverage from dill
>
> This CL makes use of the now included constant constructor coverage
> in the dill file.
>
> It works like this:
> * When the CFE evaluates constants, every constant constructor
> invocation evaluated saves the reference to the constructor in the
> `Source` (from the Components uri to source table) for the callers
> Library.
> * This data is loaded into the VM in a "raw" format.
> * When a request for coverage comes in, the VM - on top of the normal
> coverage processing - goes through all scripts to find constant
> constructor coverage for the requested script and offset. Note that
> all scripts must be checked because library A can have evaluated a
> constructor from library B - so even if only coverage for library B
> was requested, library A has to be checked.
> For all constructors found the start and end position is reported as
> covered. Note that this does not mark any initializes and there are
> (at least currently) no good way of marking which initializes were
> evaluated (because it has to be stable across edits even when the
> `advanced invalidation feature` is enabled).
> * Note that the reason for the coverage to work on references - as
> hinted above - is because we want it to be stable across hot reloads
> even if/when advanced invalidation is enabled. This means, that
> library A cannot record "positional coverage" for library B because
> library B might get (for instance) new comments that will make any old
> offsets invalid. By using references we always lookup in the current
> world and use the correct offsets.
>
> https://github.com/dart-lang/sdk/issues/38934
>
> TEST=Existing test suite, new tests for the new coverage added.
>
> Change-Id: I925963d1a9b9907efe621c72deb7348fa3be5ae8
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/171949
> Commit-Queue: Jens Johansen <jensj@google.com>
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
TBR=vegorov@google.com,bkonyi@google.com,jensj@google.com
Change-Id: I5187e6749d59ded250ec0933a94db0536485b70a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174470
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
This CL makes use of the now included constant constructor coverage
in the dill file.
It works like this:
* When the CFE evaluates constants, every constant constructor
invocation evaluated saves the reference to the constructor in the
`Source` (from the Components uri to source table) for the callers
Library.
* This data is loaded into the VM in a "raw" format.
* When a request for coverage comes in, the VM - on top of the normal
coverage processing - goes through all scripts to find constant
constructor coverage for the requested script and offset. Note that
all scripts must be checked because library A can have evaluated a
constructor from library B - so even if only coverage for library B
was requested, library A has to be checked.
For all constructors found the start and end position is reported as
covered. Note that this does not mark any initializes and there are
(at least currently) no good way of marking which initializes were
evaluated (because it has to be stable across edits even when the
`advanced invalidation feature` is enabled).
* Note that the reason for the coverage to work on references - as
hinted above - is because we want it to be stable across hot reloads
even if/when advanced invalidation is enabled. This means, that
library A cannot record "positional coverage" for library B because
library B might get (for instance) new comments that will make any old
offsets invalid. By using references we always lookup in the current
world and use the correct offsets.
https://github.com/dart-lang/sdk/issues/38934
TEST=Existing test suite, new tests for the new coverage added.
Change-Id: I925963d1a9b9907efe621c72deb7348fa3be5ae8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/171949
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
There is no need to call from C++ into Dart code just so the Dart code
can return a constant. Instead we can get the constant directly in C++
code.
This avoids calling into Dart code during class finalization to evaluate
enum constants.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: Ia0b09482188be2d0ce83ce9ffc20e5c15e52d070
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/170540
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Changes:
Doing this always in precompiled mode meant increased data segment sizes
when CodeSourceMaps are stored, since encoded line/column information is
larger in the LEB-like encoding used. Now we only store column
information when we produce non-symbolic stacks, since the increased
space needed to store the columns is instead in DWARF sections and can
be stripped or elided.
Original description:
Previously, we passed line number information to the stack trace printer
and to DWARF by changing the non-special positions in the CodeSourceMap
to line numbers in precompiled mode. However, doing this lost column
information.
We get the column information back in the majority of cases by encoding
the line number and column information when neither is too large to pack
together into 30 bits. (Here, 20 bits for line and 10 bits for column.)
Otherwise, we just store the line information as before, though due to
using a bit to encode whether column info exists, it's reduced to 30
bits. If the line info is too big for that, we just return kNoSourcePos.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: Ia8baee71468da6100a170fa305d03059ffd17f78
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151822
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit 7c34a80c11.
Reason for revert: This change seems to cause >2% code size regression
in flutter-release mode.
Original change's description:
> [vm/aot] Keep column information when possible for precompiled mode.
>
> Previously, we passed line number information to the stack trace printer
> and to DWARF by changing the non-special positions in the CodeSourceMap
> to line numbers in precompiled mode. However, doing this lost column
> information.
>
> We get the column information back in the majority of cases by encoding
> the line number and column information when neither is too large to pack
> together into 30 bits. (Here, 20 bits for line and 10 bits for column.)
> Otherwise, we just store the line information as before, though due to
> using a bit to encode whether column info exists, it's reduced to 30
> bits. If the line info is too big for that, we just return kNoSourcePos.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
> Change-Id: Id1c826f10871e2f304fa40a59d8b704404d3a2c9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151507
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
TBR=rmacnak@google.com,alexmarkov@google.com,sstrickl@google.com
Change-Id: I5cf97543d1ac2731bb27bdb58ae97af6f22f2cfb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-linux-product-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151820
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Previously, we passed line number information to the stack trace printer
and to DWARF by changing the non-special positions in the CodeSourceMap
to line numbers in precompiled mode. However, doing this lost column
information.
We get the column information back in the majority of cases by encoding
the line number and column information when neither is too large to pack
together into 30 bits. (Here, 20 bits for line and 10 bits for column.)
Otherwise, we just store the line information as before, though due to
using a bit to encode whether column info exists, it's reduced to 30
bits. If the line info is too big for that, we just return kNoSourcePos.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: Id1c826f10871e2f304fa40a59d8b704404d3a2c9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151507
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
- Detect null safety when not specified before isolate
initialization
- for source files by having CFE parse the source file
for @dart annotations
- for kernel files by sniffing the kernel file for
compilation mode
- for appJIT files by sniffing the feature string
- for AOT snapshots by sniffing the feature string
- Remove workaround of returning null safety to false during
bootstrapping
- Add a new Dart C API call for detecting null safety
Bug: 41766
Change-Id: Ia8cf264323a2d0d58c2855ce6491456aa6f1da07
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150089
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
- change the null_safety flag to be a tri state flag (unspecified, no
null safety and null safety)
- added logic to set the null safety mode of an isolate based on the
value specified in the kernel file (.dill file) if the null safety
flag is not specified on the command line
- added logic to auto detect the null safety mode based on the language
version API provided by CFE
- added logic to pass the correct null safety option to CFE when
invoking it for compilation based on the null-safety flag setting
- Delete non-nullable-flag() function and adjust code that was using it.
- Remove 'nnbd-experiment' from the snapshot string.
https://github.com/dart-lang/sdk/issues/41206https://github.com/dart-lang/sdk/issues/41207
Change-Id: I006bf3c9229980fc7986faac6a5850d3722aec92
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143160
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
This works around bugs in UndefinedBehaviorSanitizer and Clang.
Bug: b/28638298
Change-Id: I6be595f9664516019d28017d24559583a1ae3a21
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144354
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
For each selector, count the number of calls to the selector that are
both reachable and polymorphic according to the TFA. Only include
selectors in the table with non-zero counts.
This reduces the dispatch table size by 30% in dart2js and by 49% in
Flutter Gallery (312k memory use reduction on ARM64).
Call counts are transferred in a dedicated metadata block with
per-selector information. This mechanism also prepares for transferring
other per-selector information in the future.
Change-Id: Iba15aa4d6c50e67e53c3fd8e542123d3fc98bd07
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/132603
Reviewed-by: Alexander Markov <alexmarkov@google.com>
When creating a KernelProgramInfo, we create several logical views into the kernel buffer. These are fresh ExternalTypedDatas, rather than proper TypedDataViews, so they do not automically keep the original ExternalTypedData alive. Create an explicit reference to the orginal ExternalTypedData in the KernelProgramInfo. When creating snapshots, this reference is ignored/null'd and the views are turned into copies, effectively dropping the parts of the original buffer that do not have views.
Fixes a leak with reload and a use-after-free with eval.
Bug: https://github.com/dart-lang/sdk/issues/33973
Bug: https://github.com/dart-lang/sdk/issues/39610
Change-Id: I09d3830133314ccbaa0341d904127c2b6925c4ec
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/126825
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit ab6aeaa106.
Revert "[vm/compiler] Speed up the compiler part which deals with kernel reading up in DEBUG mode"
This reverts commit b316210d94.
Reason for revert: regression of snapshot sizes (DNO-599).
Original change's description:
> [vm/kernel] Use GC-tracked ExternalTypedData/TypedDataView for kernel buffers
>
> Until now we often leaked kernel buffers (e.g. hot reload buffers) because various
> objects were referencing ExternalTypedData objects pointing into the middle of
> c-allocated memory. This made it impossible for the GC to determine when the last
> reference is gone.
>
> This CL ensures that the actual buffers are *always* made available via
> ExternalTypedData and any inner pointers into it are created via TypedDataViews.
>
> The embedder guarantees to the free kernel buffers it has provided to:
> - Dart_CreateIsolateFromKernel
> - Dart_LoadScriptFromKernel
> - Dart_LoadLibraryFromKernel
> - Dart_SetDartLibrarySourcesKernel
> on isolate shutdown.
>
> All other kernel buffers will get a finalizer attached, which ensures the
> kernel buffers get freed by the GC once they are no longer referenced:
> - Kernel blobs for expression evaluation
> - Kernel blobs for Hot-Reload
> - Kernel blobs for cc tests
>
> Fixes https://github.com/dart-lang/sdk/issues/33973
> Fixes https://github.com/dart-lang/sdk/issues/36857
> Issue https://github.com/dart-lang/sdk/issues/37030
>
> Change-Id: I1cc410c94c0f4b229413e793728a261afcb10aaf
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103130
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Martin Kustermann <kustermann@google.com>
TBR=kustermann@google.com,rmacnak@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: I49715d2400f4a5c8806b7d6a2912b7258f671a0a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/104343
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Until now we often leaked kernel buffers (e.g. hot reload buffers) because various
objects were referencing ExternalTypedData objects pointing into the middle of
c-allocated memory. This made it impossible for the GC to determine when the last
reference is gone.
This CL ensures that the actual buffers are *always* made available via
ExternalTypedData and any inner pointers into it are created via TypedDataViews.
The embedder guarantees to the free kernel buffers it has provided to:
- Dart_CreateIsolateFromKernel
- Dart_LoadScriptFromKernel
- Dart_LoadLibraryFromKernel
- Dart_SetDartLibrarySourcesKernel
on isolate shutdown.
All other kernel buffers will get a finalizer attached, which ensures the
kernel buffers get freed by the GC once they are no longer referenced:
- Kernel blobs for expression evaluation
- Kernel blobs for Hot-Reload
- Kernel blobs for cc tests
Fixes https://github.com/dart-lang/sdk/issues/33973
Fixes https://github.com/dart-lang/sdk/issues/36857
Issue https://github.com/dart-lang/sdk/issues/37030
Change-Id: I1cc410c94c0f4b229413e793728a261afcb10aaf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103130
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This is preparation for handling implicit static getters as a bytecode.
Change-Id: Ic9fd186161d900c3b4c0bb773726b0cd0f35451e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99105
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This CL also includes a service test for setting
a breakpoint in a part file from a package.
Fixes#35859.
Change-Id: I0199006a87746dc1c27721ba0d51e502e76cb107
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97104
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
This change extracts code which reads kernel in order to get
covariance attributes of parameters into a separate function.
This removes code duplication and makes it easier to hook up
reading of covariance attributes from bytecode in future.
Change-Id: Ia19caf3edaf45df9e0bb1bdc715fac21d26788dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96560
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
It is used by hot-reload to check if a file is a kernel binary - so it makes
no sense to output failures to the stderr.
Change-Id: Ie2ebd5ff47990eec950550e333a41aaf5f8d7973
Reviewed-on: https://dart-review.googlesource.com/c/80224
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
TFA is improved to detect if a method was called using dynamic selector,
was ever called not via this, and whether the tear-off was taken.
This information is used to remove parameter type checks in AOT.
Less accurate selector-based analysis, which was used previously to get
this information, is now used only if TFA is disabled.
Also, precompiler is improved to omit method extractors and implicit
closure functions if analysis proved that tear-off is not taken.
Flutter gallery snapshot size: -0.8%
Change-Id: Iec01257dfdc78104752104df14e2ce078d326a96
Reviewed-on: https://dart-review.googlesource.com/77005
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
This way requests to compile won't end prematurely after reading
bytecode if interpreter is enabled.
Change-Id: Ic06cae98ea65bc4656ef491fbd306d34b0e1ef4c
Reviewed-on: https://dart-review.googlesource.com/75628
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This will retain the kernel blobs for the lifetime of the isolate, since
our normal heap objects don't properly retain the c-heap allocated
kernel blob buffers.
This work-around to suppress lsan is required for a customer.
Issue https://github.com/dart-lang/sdk/issues/33973
Change-Id: I8af092fcc5000970b2018ca694d06a9cc8b3208c
Reviewed-on: https://dart-review.googlesource.com/66565
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This reverts commit 78462ecbdd.
Reason for revert: There is code which doesn't hang on to the ExternalTypedData and therefore causes use-after-free.
Original change's description:
> [VM] Move kernel buffer created by IKG into external typed data with finalizer
>
> This fixes a memory leak which shows up by lsan in certain situations.
>
> Change-Id: Ib0d18520a2e57562bc72e79c8e4f070c16509268
> Reviewed-on: https://dart-review.googlesource.com/66383
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
TBR=kustermann@google.com,rmacnak@google.com,zra@google.com
Change-Id: I71981b17264b50592b60d844fb3ca795371faaa6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/66580
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This fixes a memory leak which shows up by lsan in certain situations.
Change-Id: Ib0d18520a2e57562bc72e79c8e4f070c16509268
Reviewed-on: https://dart-review.googlesource.com/66383
Reviewed-by: Ryan Macnak <rmacnak@google.com>
In certain cases StreamingFlowGraphBuilder was used not for building
flow graph, but as an advanced kernel reader. This CL extracts all such
functionality from StreamingFlowGraphBuilder and revises all such uses.
StreamingFlowGraphBuilder constructors without FlowGraphBuilder are
removed. After this CL StreamingFlowGraphBuilder is only used to build
flow graph, as intended.
Change-Id: I69b08e24d37f8f2f336bee85334af11be1639b24
Reviewed-on: https://dart-review.googlesource.com/64821
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This CL continues refactoring aimed at isolating StreamingFlowGraphBuilder.
StreamingFlowGraphBuilder dependency is removed from classes
* StreamingConstantEvaluator (it still depends on FlowGraphBuilder)
* ConstantHelper
* SimpleExpressionConverter
and several methods in object.cc.
StreamingConstantEvaluator and ConstantHelper are moved from
kernel_binary_flowgraph{.h, .cc} to a new source file
constant_evaluator{.h, .cc}. StreamingConstantEvaluator is renamed
to ConstantEvaluator.
KernelFingerprintHelper and KernelSourceFingerprintHelper are moved to
a new source file kernel_fingerprints{.h, .cc}.
Instances of kernel::FlowGraphBuilder no longer contain back reference
to a StreamingFlowGraphBuilder. In order to drop this circular dependency
TranslateFinallyFinalizers() is moved from FlowGraphBuilder to
StreamingFlowGraphBuilder.
Change-Id: Id550d22b3567dea9512328a900935bd6145a8107
Reviewed-on: https://dart-review.googlesource.com/64463
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Instead of bailing out with FATAL et al (which shuts down the VM),
give an ApiError when the dill file is invalid.
Bug: #33577
Change-Id: I8354b4e68ee95e36584284fd15b3abab3c3bf980
Reviewed-on: https://dart-review.googlesource.com/61937
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Metadata is no longer written ahead of all nodes. Instead, metadata for
each node is written in the same context as the node itself (into a separate
buffer). This allows metadata to contain (serialize) arbitrary nodes
(for example, arbitrary DartTypes) and use serialization context of parent
nodes (such as declared type parameters).
However, with this change metadata looses the ability to reference
arbitrary AST nodes. This ability was overly restricted and had no
practical uses. (It was not possible to reference nodes which are not
reachable from root Component. As a consequence, it was not possible to
write references to arbitrary DartTypes.)
This change aligns the serialization capabilities of metadata with
how kernel AST nodes are serialized.
Change-Id: I027299a33b599b62572eccd4aa7083ad1dd2b3b3
Reviewed-on: https://dart-review.googlesource.com/54481
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Update status for simulators, which the test harness invokes with DIL files instead of source files.
Change-Id: I8444ad7e17a0a71a1ce3c0021487397baa1c3e65
Reviewed-on: https://dart-review.googlesource.com/54622
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
- Fix const-ness of Dart_FileReadCallback and callers.
- Fix leak on read error in DartUtils::FileRead.
This is progress towards sharing kernel memory on Fuchsia.
Bug: https://github.com/dart-lang/sdk/issues/32618
Change-Id: I47f8b224905d6a105a5ca0ab2ee4ab6a42b5e342
Reviewed-on: https://dart-review.googlesource.com/47102
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Dart_LoadKernel is only used in Flutter engine and dart_runner, where it is always used as Dart_LoadKernel(Dart_ReadKernelBinary). These uses should be replaced with Dart_LoadScriptFromKernel.
Dart_LoadLibraryFromKernel is needed for loading split kernel files and implementing IsolateMirror.loadUri.
Change-Id: Ib505350eff53ec889406747f8f99393ebbdb4c7d
Reviewed-on: https://dart-review.googlesource.com/46220
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
