* Move parsing functions to static methods of the classes that
represent the data being parsed.
* Move calculations for adjusting offsets in load commands into the
corresponding classes.
* Make function to insert a new payload segment an instance method
of MachOFile.
* Remove unnecessary abstractions and definitions for MachO load
commands that we don't need to parse.
These refactorings are being done to make later changes easier to
review, by first lifting the initial refactorings out into a
separate CL.
Issue: https://github.com/dart-lang/sdk/issues/49783
Change-Id: I133eb368cbb9ee0d8e4f3998ba1a0bbe8555b8aa
Cq-Include-Trybots: luci.dart.try:analyzer-mac-release-try,dart-sdk-mac-arm64-try,dart-sdk-mac-try,pkg-mac-release-arm64-try,pkg-mac-release-try,vm-kernel-precomp-mac-product-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256821
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This reverts commit 161f8fd942.
Reason for revert: breaks google3 (b/243899439)
Original change's description:
> [pkg/dart2native] Add negative padding check to MachO writer.
>
> This catches an inadvertent partial overwrite of the first section
> in the `dartaotruntime` executable that happens when there's not
> enough padding after the headers. The first section is a text section,
> so unless there are no calls to code in this overwritten portion, there
> are no runtime failures when this happens.
>
> However, having a MachO section which has a file offset within the
> headers is checked by MachO verification code within the `codesign`
> utility when run with the `-f` flag, and thus caused failures in
> certain tests and builds. Note that we only use that flag when
> `codesign` does not have the `linker-signed` option, which is for
> MacOS versions prior to 12.0.
>
> Example failure with this change when this case occurs (e.g., prior to
> the recent clang revert):
> ```
> $ xcodebuild/ReleaseARM64/dart-sdk/bin/dart compile exe -o test test.dart
> Info: Compiling with sound null safety
> Error: AOT compilation failed
> FormatException: The MachO header overlaps with the first 120 bytes of the section contents
> ```
>
> Change-Id: Ib27db910777f61b90f162f7a0bcfa4ba6592a5a0
> Bug: https://github.com/dart-lang/sdk/issues/49783
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-mac-product-x64-try,dart-sdk-mac-arm64-try,dart-sdk-mac-try,pkg-mac-release-arm64-try,pkg-mac-release-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256264
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
TBR=kustermann@google.com,dacoharkes@google.com,sstrickl@google.com
Change-Id: Iccf27ffab6c6b6272e366b77c5092b346145e263
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/49783
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-mac-product-x64-try,dart-sdk-mac-arm64-try,dart-sdk-mac-try,pkg-mac-release-arm64-try,pkg-mac-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256460
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Emmanuel Pellereau <emmanuelp@google.com>
Reviewed-by: Emmanuel Pellereau <emmanuelp@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
This catches an inadvertent partial overwrite of the first section
in the `dartaotruntime` executable that happens when there's not
enough padding after the headers. The first section is a text section,
so unless there are no calls to code in this overwritten portion, there
are no runtime failures when this happens.
However, having a MachO section which has a file offset within the
headers is checked by MachO verification code within the `codesign`
utility when run with the `-f` flag, and thus caused failures in
certain tests and builds. Note that we only use that flag when
`codesign` does not have the `linker-signed` option, which is for
MacOS versions prior to 12.0.
Example failure with this change when this case occurs (e.g., prior to
the recent clang revert):
```
$ xcodebuild/ReleaseARM64/dart-sdk/bin/dart compile exe -o test test.dart
Info: Compiling with sound null safety
Error: AOT compilation failed
FormatException: The MachO header overlaps with the first 120 bytes of the section contents
```
Change-Id: Ib27db910777f61b90f162f7a0bcfa4ba6592a5a0
Bug: https://github.com/dart-lang/sdk/issues/49783
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-mac-product-x64-try,dart-sdk-mac-arm64-try,dart-sdk-mac-try,pkg-mac-release-arm64-try,pkg-mac-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256264
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Instead of assuming that linker-signed is a valid option flag for
codesign, only attempt to use it on macOS 11.0 or later, where
signatures on linker-signed binaries need not be force overwritten.
Otherwise, fall back to a regular signature that must be force
overwritten.
Bug: https://github.com/dart-lang/sdk/issues/49010
Cq-Include-Trybots: luci.dart.try:pkg-mac-release-arm64-try,pkg-mac-release-try
Change-Id: Ia2bff2f501dbe37b3269a2a2f16ce9a29026664e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/245000
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Instead of simply concatenating the dartaotruntime executable and the
ELF snapshot, we instead insert the ELF snapshot as a COFF section,
so the resulting PE file can be signed.
TEST=pkg/dartdev/test/commands/compile_test
Bug: https://github.com/dart-lang/sdk/issues/39106
Change-Id: Iaafb5833fcb9c69c0dd374b64769f22b8b9f53a2
Cq-Include-Trybots: luci.dart.try:pkg-win-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/236100
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Since the official Dart SDK releases are fully signed, attempting
to re-sign them will fail unless forced.
TEST=Manually.
Bug: https://github.com/dart-lang/sdk/issues/39106
Change-Id: I0cf74fa7fc74a3cc52967abc4bb0dc2d656765d9
Cq-Include-Trybots: luci.dart.try:pkg-mac-release-arm64-try,pkg-mac-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238343
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This CL implements a slightly altered compilation process for `dart
compile` on MacOS. Instead of the traditional approach of
concatenating a dart snapshot with the dart runtime, this new
implementation uses a new MachO segment to store the snapshot. Doing
so ensure users can properly sign the resulting binaries.
The dart runtime has also been updated to look for the snapshots in
this new segment.
There are new unit tests ensuring the dart runtime can correctly
identify MachO-formatted executables and an end-to-end test ensuring
`dart compile` produces code that has the new segment and produces
code that can be signed.
TEST=pkg/dartdev/test/commands/compile_test
Change-Id: Ibdce80455e02bc2b6fd345d80be8d75f989a9c28
Cq-Include-Trybots: luci.dart.try:analyzer-mac-release-try,dart-sdk-mac-arm64-try,dart-sdk-mac-try,pkg-mac-release-arm64-try,pkg-mac-release-try,vm-kernel-mac-debug-x64-try,vm-kernel-mac-product-x64-try,vm-kernel-mac-release-arm64-try,vm-kernel-mac-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/228080
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
* Add team "groups" in tools/OWNERS_<group name>.
* Add top-level OWNERS as a fallback.
* Add OWNERS for all top-level directories.
* Add OWNERS to all packages.
For additional background information see go/dart-sdk-owners.
TEST=No op until code-owners is enabled.
Bug: b/200915407
Change-Id: I7fe6116cc599c749cd50ca16151d6d6a801d99d7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229147
Reviewed-by: Jonas Termansen <sortie@google.com>
This adds a new messages kind 'info' to the CFE for showing general
information during compilation. A 'configuration' options is added
to `CompilerOptions` for telling the CFE how it is run.
The configuration 'compile' is added for when the CFE is invoked to
produces an "executable" as when running `dart compile`. When
configuration is set, the CFE emits an info message about the
null safety compilation mode.
Support for `dart compile exe` and `dart compile js` is added in this
CL. Support for `dart compile kernel|app-jit|aot` is not included.
In response to https://github.com/dart-lang/sdk/issues/44234
TEST=pkg/dartdev/test/commands/compile_test.dart
Change-Id: I08f51e2a3f5ad4841c4d703bcd266b7afb63c7c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178982
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
The path normalization done by dart2native for "sourceFile" can cause
lower-casing of the path. We should do the same for the "packages" path.
TEST=Makes pkg/vm_snapshot_analysis/test/precompiler_trace_test pass on
Windows in unittest-asserts-release-win config.
Closes https://github.com/dart-lang/sdk/issues/43955
Change-Id: I3de0f2cc33d7a30280d4e815b0b7e3588f24e4f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/170093
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Supports compiling to snapshots, JS, native executables, and kernel:
$ dart help compile
Compile Dart to various formats
Usage: dart compile <subcommand> [arguments]
-h, --help Print this usage information.
Available subcommands:
aot Compile Dart to an AOT snapshot
exe Compile Dart to a self-contained executable
jit Compile Dart to a JIT snapshot
js Compile Dart to JavaScript
kernel Compile Dart to Kernel intermediate format
Change-Id: I1143f8dd0eec6f87a8271144f92dedc1fcf82923
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146806
Commit-Queue: Michael Thomsen <mit@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Jaime Wren <jwren@google.com>
The flutter engine DEPS got new icu/boringssl so this CL can safely
land and roll into engine. A small change is made in Patchset 1..3.
This removes special logic for creating the `dart-sdk` we distribute
which used to build release and product mode and copied some binaries
from the latter into the former, before the SDK was actuallly ready to
test and distribute.
This changes the GN build rules to build the necessary
gen_snapshot/dart_precompiled_runtime product binaries during the normal
release build.
Normally during --mode=product builds the global build config in
//build/config/BUILDCONFIG.gn will set `-fvisibility=false`.
=> Doing so results in much smaller binaries - because only explicitly
exported symbols are visible, the rest can be tree shaken by the linker.
Since we are building --mode=release, the `-fvisibility=false` will not
be set. In order to set the flag for the 2 special product-mode binaries
we need to add -fvisibility=hidden manually, in:
* dart_product_config: Which is used for compiling VM sources.
* 3rd party double-conversion library
* 3rd party boringssl library
* 3rd party icu library
The upstream CLs are:
* BoringSSL: https://dart-review.googlesource.com/c/boringssl_gen/+/150482
* ICU: https://chromium-review.googlesource.com/c/chromium/deps/icu/+/2236407
Issue https://github.com/dart-lang/sdk/issues/42230
Change-Id: I8d9b37acbb5eb94e44d97652f838948a946fd372
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151505
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
This reverts commit a88a328b7f.
Reason for revert: Auto roller hasn't rolled previous changes from
Dart yet. Those changes updated DEPS for icu/boringssl. We have to
wait with this CL until the DEPS changes have been done in the engine.
Original change's description:
> [vm] Build dart2native dependencies with the normal "create_sdk" target
>
> This removes special logic for creating the `dart-sdk` we distribute
> which used to build release and product mode and copied some binaries
> from the latter into the former, before the SDK was actuallly ready to
> test and distribute.
>
> This changes the GN build rules to build the necessary
> gen_snapshot/dart_precompiled_runtime product binaries during the normal
> release build.
>
> Normally during --mode=product builds the global build config in
> //build/config/BUILDCONFIG.gn will set `-fvisibility=false`.
>
> => Doing so results in much smaller binaries - because only explicitly
> exported symbols are visible, the rest can be tree shaken by the linker.
>
> Since we are building --mode=release, the `-fvisibility=false` will not
> be set. In order to set the flag for the 2 special product-mode binaries
> we need to add -fvisibility=hidden manually, in:
>
> * dart_product_config: Which is used for compiling VM sources.
> * 3rd party double-conversion library
> * 3rd party boringssl library
> * 3rd party icu library
>
> The upstream CLs are:
>
> * BoringSSL: https://dart-review.googlesource.com/c/boringssl_gen/+/150482
> * ICU: https://chromium-review.googlesource.com/c/chromium/deps/icu/+/2236407
>
> Issue https://github.com/dart-lang/sdk/issues/42230
>
> Change-Id: I3e47664d9fadb9ed1ad033bb17d46e769442f741
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150524
> Commit-Queue: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Alexander Thomas <athom@google.com>
> Reviewed-by: Zach Anderson <zra@google.com>
TBR=kustermann@google.com,zra@google.com,athom@google.com,asiva@google.com
Change-Id: Ied0f2f5b58dceb2b59bc2ac6e11e5fd717c8885e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151382
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This removes special logic for creating the `dart-sdk` we distribute
which used to build release and product mode and copied some binaries
from the latter into the former, before the SDK was actuallly ready to
test and distribute.
This changes the GN build rules to build the necessary
gen_snapshot/dart_precompiled_runtime product binaries during the normal
release build.
Normally during --mode=product builds the global build config in
//build/config/BUILDCONFIG.gn will set `-fvisibility=false`.
=> Doing so results in much smaller binaries - because only explicitly
exported symbols are visible, the rest can be tree shaken by the linker.
Since we are building --mode=release, the `-fvisibility=false` will not
be set. In order to set the flag for the 2 special product-mode binaries
we need to add -fvisibility=hidden manually, in:
* dart_product_config: Which is used for compiling VM sources.
* 3rd party double-conversion library
* 3rd party boringssl library
* 3rd party icu library
The upstream CLs are:
* BoringSSL: https://dart-review.googlesource.com/c/boringssl_gen/+/150482
* ICU: https://chromium-review.googlesource.com/c/chromium/deps/icu/+/2236407
Issue https://github.com/dart-lang/sdk/issues/42230
Change-Id: I3e47664d9fadb9ed1ad033bb17d46e769442f741
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150524
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
This change actually migrates vm/dart/* tests to NNBD by fixing
compile-time errors and adjusting tests where needed.
vm/dart/null_float32x4_simd_ops_test and vm/dart/null_float64x2_simd_ops_test
are deleted as they are superseded by static type checks.
vm/dart/regress_40462_test.dart is a huge source code auto-generated
by fuzzer and migrating and maintaining this source doesn't have much
value, so it is deleted.
There are still failures in strong mode due to dependencies on
the packages which are not migrated yet (https://github.com/dart-lang/sdk/issues/42146).
Migrated vm/dart/null_checks_with_dwarf_stack_traces_test fails both in
weak and strong mode due to https://github.com/dart-lang/sdk/issues/42149.
Issue: https://github.com/dart-lang/sdk/issues/41314
Change-Id: I5561f1c8705ec16def0c4e0efa495d15f4ea7259
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149493
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Previously the output used scrubbed qualified names which hides some
of the information and causes names of various synthetic functions
collide. We change it to use internal qualified names instead and add
additional information for disambiguation:
- For functions this means adding a prefix for various synthetic
methods;
- For anonymous closures we add their token position;
- For types we prefix type parameters with their owner's name.
We also add a unit test covering parsing of
--print-instructions-sizes-to output.
As part of adding a test dart2native was changed to support passing
extra arguments to gen_snapshot binary.
Fixes https://github.com/dart-lang/sdk/issues/41137
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-win-release-try,pkg-mac-release-try
Change-Id: I299ab0fb5993c8af9b51246483608d1e65970d67
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146581
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
This should enable strong mode tests to be fully NNBD opted in. At the
same time, legacy tests when run with the experiment off should
hopefully be able to still consume these libraries because they don't
actually use any NNBD features.
To do this, I changed the script that generates the SDK repo's package
config to not put in a language version for any package whose pubspec
has no SDK version. According to language versioning, that means the
package should be considered to be at the "current" version of the SDK
running the code.
In NNBD, that's opted in. With the experiment off, that is (presumably)
"opted out" in the sense that NNBD doesn't exist.
In order to *not* opt in some of the other packages that currently
lack SDK constraints, I put those in and pinned them to "^2.7.0" which
was what the package config script used to implicitly fill in for them.
I see a bunch of other changes in the generated package config too. The
update script probably hasn't been run in a while.
Change-Id: I55193d42eac0696a6b0105546551efa45a1f3252
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144305
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Leaf Petersen <leafp@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Change-Id: I132a1bd47324ba9f546af81c21c84c3f108a7797
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/139220
Auto-Submit: Kevin Moore <kevmoo@google.com>
Reviewed-by: Michael Thomsen <mit@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Michael Thomsen <mit@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Currently we bundle only one platform file, which dart2native uses as
well as other tools. Since the platform file has annotations already
evaluated, our AOT compiler ends up including service related
functionality in the AOT snapshot.
This CL bundles the product-mode vm_platform.dill as well, which will
reduce dart2native-created apps slightly.
Change-Id: I223811b30704cde1739739d97a8ea2d34a681292
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125665
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Fixes symbol resolution on Darwin.
The original commit is in patchset 1.
Change-Id: If71e9a72b8a2f45db2591bfa45ec4018745ee2c4
Cq-Include-Trybots:luci.dart.try:dart-sdk-mac-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119646
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Usage: dart2native <main-dart-file> [<options>]
Generates an executable or an AOT snapshot from <main-dart-file>.
-D, --define=<key=value> Set values of environment variables. To specify
multiple variables, use multiple options or use
commas to separate key-value pairs.
E.g.: dart2native -Da=1,b=2 main.dart
--enable-asserts Enable assert statements.
-h, --help Display this help message.
-o, --output=<path> Set the output filename. <path> can be relative
or absolute.
E.g.: dart2native main.dart -o ../bin/my_app.exe
-k, --output-kind=<aot|exe>
[aot] Generate an AOT snapshot.
[exe] (default) Generate a standalone executable.
-p, --packages=<path> Get package locations from the specified file
instead of .packages. <path> can be relative or
absolute.
E.g.: dart2native --packages=/tmp/pkgs main.dart
-v, --verbose Show verbose output.
Change-Id: Id4d5542a946326fb8c009643f16fa1be99c9a97e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118994
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Kathy Walrath <kathyw@google.com>
Based on copy discussion with kathyw@ and mit@.
Change-Id: Iac41e66a431b4fd67550bd35633baa4cc79e197e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118571
Reviewed-by: Michael Thomsen <mit@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
This reverts commit a6141ff5c9.
Reason for revert: Blocking Dart SDK -> Flutter engine roll. See logs here: https://github.com/flutter/engine/pull/11836/checks?check_run_id=210988794
Original change's description:
> Add dart2native tool for building either an aot file or a stand-alone executable.
>
> *dart2aot has been rewritten in Dart accompanied by a trampoline script.
> *dart2exec is still missing implementation.
>
> Change-Id: I4b662ce86c7365fa4d043b48a691881c8ef08a8c
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108601
> Commit-Queue: Sarah Zakarias <zarah@google.com>
> Reviewed-by: Clement Skau <cskau@google.com>
TBR=cskau@google.com,zarah@google.com
Change-Id: I4c5946ce0f0a66484e243b98cdcb7e2b24e2d705
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/115340
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
