Commit graph

23 commits

Author SHA1 Message Date
Ryan Macnak 0c07cd6bc7 [vm] Reduce peak memory usage of --write_v8_snapshot_profile_to.
Remove unnecessary members, reduce field sizes and reduce initial map capacities.

/usr/bin/time -v gen_snapshot --snapshot-kind=app-aot-elf --write-v8-snapshot-profile-to=... dart2js.dill
Maximum resident set size (kbytes): 1347220 -> 902280 (-33.0%)

TEST=ci
Bug: b/196510517
Change-Id: I2d4dd55ac6db8ccbe9f8a834bab9fe906ba1228d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212269
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-09-02 21:46:35 +00:00
Tess Strickland c49560cd37 [platform] Make NBitMask, Bit, and TestBit constexpr.
Also make NBitMask generic, replace uses of SignedNBitMask
with generic NBitMask, and remove SignedNBitMask.

TEST=No changes to functionality for existing callers, so existing tests.

Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try
Change-Id: Ic33785e50be98123c75782c6e62ef32ec45f4191
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/208181
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-07-27 11:22:47 +00:00
Ryan Macnak d2e460675b [vm] Fix gcc build.
TEST=local build
Change-Id: I572aba5798a07ce5cb0cbda83d6eb3e44811ba5b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203287
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-06-17 16:05:02 +00:00
Ivan Inozemtsev a5ff656da1 Revert "[vm] Switch datastream Write/WriteUnsigned to (S)LEB128."
This reverts commit 80102c981b.

Reason for revert: b/186359854

Original change's description:
> [vm] Switch datastream Write/WriteUnsigned to (S)LEB128.
>
> This reduces the number of variable-length integer encodings in our
> database from 2 to 1, and chooses the more standard one.
>
> TEST=Existing tests, in particular any that involve snapshots.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
> Change-Id: Ia700158ac873ad32ac28c1027a669895961bc715
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196321
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: Idf32bdd879cf8bb7407f6dae764312140ad6eeb2
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196920
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
2021-04-26 07:52:47 +00:00
Tess Strickland 80102c981b [vm] Switch datastream Write/WriteUnsigned to (S)LEB128.
This reduces the number of variable-length integer encodings in our
database from 2 to 1, and chooses the more standard one.

TEST=Existing tests, in particular any that involve snapshots.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: Ia700158ac873ad32ac28c1027a669895961bc715
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196321
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-04-23 07:35:41 +00:00
Tess Strickland 927c59104f [vm] Re-tighten requirements that the node type be set before writing.
Fixes the case where the Image name and type was not written
to the v8 snapshot profile in non-precompiled mode.

Adds core-jit snapshot tests to the v8 snapshot profile writer test.

In core-jit and app-aot-elf modes, verify that all bytes in the
snapshot sections are accounted for in the profile.

Bug: https://github.com/dart-lang/sdk/issues/45787

TEST=vm/dart{,_2}/v8_snapshot_profile_writer_test

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try
Change-Id: I88be309cdea6293d768e2fa458492387d70fde22
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196323
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-04-22 15:27:38 +00:00
Tess Strickland 8f30423bee [vm] Allow unknown type nodes in v8 snapshot profile in non-DEBUG.
Fixes https://github.com/dart-lang/sdk/issues/45787

TEST=Flutter HHH/web tests.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: Id223f667b4571d0733e578207d160e86f24687bb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196322
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-04-21 14:03:10 +00:00
Tess Strickland a2b8da3fed [vm] Cast to unsigned before left shift in V8 snapshot profile writer.
TEST=vm/dart_2/v8_snapshot_profile_writer_test on UBSAN.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-ubsan-linux-release-x64-try
Change-Id: Ide25bf08c27b6962637639429aa8ff9deadbe283
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196121
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-04-20 16:40:18 +00:00
Tess Strickland 6c8f4aa190 Reland "[vm] Fix V8 snapshot profile handling of the dispatch table."
This is a reland of 5909fd111d

Does a large refactoring on the V8 snapshot profile writer
to clean things up, add more debugging support, and to fix
the problems that surfaced during the original landing.

Other changes:

Changes Serializer::CreateArtificialNodeIfNeeded() to create
artificial nodes for Code objects and immutable arrays.

Fixes CodeSerializationCluster::Trace() to only push needed parts of
discarded code objects, instead of tracing them like full code objects.

Adds test cases to v8_snapshot_profile_writer_test that exercise
the following situations (both separately and together):

* Non-symbolic stack traces are enabled and code and function objects
  are dropped when not needed at runtime.

* Creation of the dispatch table is disabled.

TEST=vm/dart{,_2}/v8_snapshot_profile_writer_test

Original change's description:
> [vm] Fix V8 snapshot profile handling of the dispatch table.
>
> Fixes https://github.com/dart-lang/sdk/issues/45702.
>
> TEST=Tests listed in the issue above.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
> Change-Id: Ibf5e3ccf3828c01f9dda47de360314dabe8cb8a9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195272
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>

Change-Id: I8e7030267fe190079a8f68d00fe20bf7170e5719
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-obfuscate-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195513
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-04-20 11:17:36 +00:00
Tess Strickland 2f56446250 [vm] Change missed Hashcode -> Hash() from CL submitted in parallel.
TEST=Build succeeds.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try
Change-Id: Idc107eb4e84e7e80cdb87454c2c6e27787682525
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195262
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-04-14 10:34:43 +00:00
Tess Strickland 0f8861e3ba [vm] Standardize hashing instance methods in C++.
In the codebase, we have several different interfaces for instance
methods that return a hash:

- uword Hash() const;
- intptr_t Hash() const;
- uint32_t Hash() const;
- intptr_t Hashcode() const;

This CL standardizes on `uword Hash() const` and adjusts any related
functions to match.

TEST=Existing test suites, as this is an internal refactoring.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Change-Id: If2cbce57f3fae0f0d24031b6e324f0323c965f41
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195067
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-04-14 10:04:48 +00:00
Tess Strickland dc88076767 [vm] Handle WSRs more generally in v8 snapshot profile writing.
Instead of special-casing the current fields that may have
WeakSerializationReferences, handle WSRs appearing as elements
or properties of objects more generally. This removes existing
special casing and avoids the need for it in case of new
future uses of WSRs.

For artificial nodes being added for dropped
WeakSerializationReference targets, add them as kArtificial nodes
(not kSnapshot) that has the original offset (element) or
name (property). The replacement is added as a kSnapshot node
that has a negative offset with the same magnitude as the
artificial node (element) or ":real_<property name>" (property).
This simplifies the work done in pkg/vm_snapshot_analysis to
use the artificial nodes instead of replacement ones for
reassembling hierarchies and the like.

This CL also cleans up the old SerializerWritingObjectScope
class, both moving it to Serializer::WritingObjectScope and
allowing nesting of WritingObjectScopes with the correct
semantics.

Thanks to this, not only can we recur when under a WritingObjectScope
instead of lifting recursion outside of those scopes, but we can
also create artificial nodes for non-empty per-code object pools and
static call target tables instead of attributing their contents
as supposed elements of the Code object being written.

TEST=pkg/vm_snapshot_analysis/test/instruction_sizes_test

Cq-Include-Trybots: luci.dart.try:pkg-linux-release-try,pkg-mac-release-try,pkg-win-release-try
Change-Id: Ib945c5afcd89b1458b8be3559b6eae24048aba2f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194243
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-04-14 08:44:58 +00:00
Vyacheslav Egorov ff6757f380 [vm/snapshot] Improve precision of --write-v8-snapshot-profile-to.
Attribute bytes written outside of any particular object to the
artificial root node.

https://github.com/dart-lang/sdk/issues/41249

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try
Change-Id: I6d4be9b9a6a42cd0996330bb14552d12d6430ff9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148526
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2020-05-26 10:46:05 +00:00
Vyacheslav Egorov a634ef8159 [vm/snapshot] Add root names into write_v8_snapshot_profile_to output
R=sjindel@google.com

Change-Id: Ic178da22d2e97a86067808d4eb14df723efd03df
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136582
Reviewed-by: Samir Jindel <sjindel@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2020-02-20 13:07:18 +00:00
Samir Jindel 8d39e3456f [vm/aot] Minor improvements to make snapshot profiles more useful.
Change-Id: I614d0af3fb3b2bcb642f13d767c385d255b43d5c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105580
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2019-06-17 13:09:50 +00:00
Samir Jindel c4541e63f8 [vm/aot] Update serializer for V8's "snapshot profile" format
Fixes https://github.com/dart-lang/sdk/issues/37183
Addresses https://github.com/dart-lang/sdk/issues/37126

Change-Id: I1c1e516ac2bcc45e46059f4439f16aaee6d824fe
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105302
Reviewed-by: Martin Kustermann <kustermann@google.com>
2019-06-06 19:54:12 +00:00
Samir Jindel 96dda6a516 Re-land "[vm/aot] Fill in names of properties in snapshot profile."
This reverts commit dc346e6fdd.

Cq-Include-Trybots: luci.dart.try:vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-mac-release-simarm64-try, vm-kernel-precomp-win-release-x64-try
Change-Id: I1b9181683757cf47f917274d21c8fc1c463218c7
Reviewed-on: https://dart-review.googlesource.com/c/85709
Commit-Queue: Samir Jindel <sjindel@google.com>
Auto-Submit: Samir Jindel <sjindel@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-12-03 19:11:42 +00:00
Alexander Aprelev dc346e6fdd Revert "[vm/aot] Fill in names of properties in snapshot profile."
This reverts commit 465f8595ec as it
breaks windows and mac bots.

Change-Id: I94a939d711f69a28f405ee32680315ad6677b09e
Reviewed-on: https://dart-review.googlesource.com/c/85722
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2018-11-30 19:52:23 +00:00
Samir Jindel 465f8595ec [vm/aot] Fill in names of properties in snapshot profile.
Change-Id: Ic709a3dc40390463ffb5fd76fd8eb63ebd87acc0
Cq-Include-Trybots: luci.dart.try:vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-mac-release-simarm64-try
Reviewed-on: https://dart-review.googlesource.com/c/85292
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-11-30 18:38:05 +00:00
Samir Jindel 8aa0af0654 [vm] Fix C++ incompatibility in V8SnapshotProfileWriter
Change-Id: Ib676762594783b2bc755b8362d7a2dd3522efcb1
Reviewed-on: https://dart-review.googlesource.com/c/85201
Commit-Queue: Samir Jindel <sjindel@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Auto-Submit: Samir Jindel <sjindel@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2018-11-22 17:18:05 +00:00
Samir Jindel cde4270e47 Re-land "[vm/aot] Snapshot size analysis."
Minor fixes have been applied, see diff against patchset 1.

This reverts commit 9b937f1226.

Change-Id: I8e4bbc0b88e33d3b554e91c17d1f849e24a1ccb3
Cq-Include-Trybots: luci.dart.try:vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-win-product-x64-try, vm-kernel-precomp-win-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/84845
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Auto-Submit: Samir Jindel <sjindel@google.com>
2018-11-22 15:14:46 +00:00
Zach Anderson 9b937f1226 Revert "[vm/aot] Snapshot size analysis."
This reverts commit 06c89287b9.

Reason for revert:
- Failure of v8_snapshot_profile_writer_test on vm-kernel-precomp-android-release
- many failures on windows precomp (?)
- link failures in windows product builds.

Original change's description:
> [vm/aot] Snapshot size analysis.
> 
> Cq-Include-Trybots: luci.dart.try:vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try
> Change-Id: Ie5d5d9f0f064f2c9ef8b7c9a0537b1e633bf8175
> Reviewed-on: https://dart-review.googlesource.com/c/84425
> Commit-Queue: Samir Jindel <sjindel@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TBR=vegorov@google.com,rmacnak@google.com,sjindel@google.com

Change-Id: I461f06236f27bb5c6548bd08f4b73507d1b913fa
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/84860
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-11-19 22:29:09 +00:00
Samir Jindel 06c89287b9 [vm/aot] Snapshot size analysis.
Cq-Include-Trybots: luci.dart.try:vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try
Change-Id: Ie5d5d9f0f064f2c9ef8b7c9a0537b1e633bf8175
Reviewed-on: https://dart-review.googlesource.com/c/84425
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-11-19 20:29:05 +00:00