Converts `package:` import uris into `/packages/` modules.
Also renames the output modules to append `.lib.js` instead of just `.js`. This allows us to distinguish between modules and applications based on extension.
Updates DDC source map code to be able to convert absolute file uris in sources so that they are relative to the source map.
Bug: https://github.com/dart-lang/webdev/issues/865
Change-Id: I55d70aa3761f10cc8bd7e92f5b567478040660de
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/132300
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Jonah Williams <jonahwilliams@google.com>
Commit-Queue: Jake Macdonald <jakemac@google.com>
- fixes check_nnbd_sdk.dart to run on dart2js patch files
- splits golden file used by that script by target
- adds it to the old-rti bot (until we get the new nnbd bot)
- adds a suite that runs legacy tests in nnbd weak mode
Change-Id: I0a89534dcd1cf7338849d59ab3e5d989d2ecf3d0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131885
Reviewed-by: Vijay Menon <vsm@google.com>
This reverts commit 24a0601d5b.
Reason for revert: This breaks the flutter build because flutter has
a build target producing the same "gen/frontend_server.dart.snapshot" file and is therefore conflicting.
See e.g. head-head-head bot failure:
https://ci.chromium.org/p/dart/builders/ci.sandbox/flutter-engine-linux/9747
Original change's description:
> Create the frontend_server snapshot as a part of the sdk
>
> This will allow us to use it from webdev, as it now supports DDC output.
>
> Change-Id: I4b82d1c45d2c058abb062dddd2a2c41cc6318c45
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/130134
> Reviewed-by: Keerti Parthasarathy <keertip@google.com>
> Reviewed-by: Jonah Williams <jonahwilliams@google.com>
> Reviewed-by: Vijay Menon <vsm@google.com>
> Commit-Queue: Jake Macdonald <jakemac@google.com>
TBR=keertip@google.com,jakemac@google.com,vsm@google.com,jonahwilliams@google.com
Change-Id: Ib2d782560cc005416ece429f9506dca055bffd9b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/130377
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This will allow us to use it from webdev, as it now supports DDC output.
Change-Id: I4b82d1c45d2c058abb062dddd2a2c41cc6318c45
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/130134
Reviewed-by: Keerti Parthasarathy <keertip@google.com>
Reviewed-by: Jonah Williams <jonahwilliams@google.com>
Reviewed-by: Vijay Menon <vsm@google.com>
Commit-Queue: Jake Macdonald <jakemac@google.com>
We are temporarily building a smaller SDK platform to make progress building
applications against the forked sdk_nnbd sources. This is because some of the
libraries have not yet been ported and they don't compose well.
I needed to thread the status of the non-nullable experiment through the front
end entry point and into the target options, and native types.
Also includes some standardization of the flag name in DDC.
https://github.com/dart-lang/sdk/issues/39698
Change-Id: I4bdd503be694b28d7f95828d4af28d1d5fb3691f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/127486
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
This is the first step in deleting legacy DDC!
* Default all invocations of DDC to run the kernel version.
* Disable all tests that run analyzer DDC.
* Detect the most likely case when invoked with an analyzer
summary and print instruction to update build_web_compilers
dependency.
I will follow up with deletions, and a rename of the test configurations/
builders to reclaim the names (ddc vs ddk, dartdevc vs dartdevk).
Issue #38777
Change-Id: I1477f0274b6b7f345a5e19d2b3f93797c454a09e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/127067
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Mark Zhou <markzipan@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
This is a reland of 3c9e924073
Original change's description:
> Revamp patch_sdk.dart mainly to use libraries.json.
>
> This will make it possible to use the script to create a patched sdk
> for dart2js and the vm (some followup needed), which will be useful
> to migrate their libraries to nnbd.
>
> In the future, we should be able to leverage this new logic to
> write tests that ensure our sdk is warning-free (there are > 80 real
> warnings today even on the non-nnbd sdk)
>
> In detail, this CL includes:
>
> * loading the libraries data from libraries.json, this removes the need for a
> hand-crafted libraries.dart that replicates the data in libraries.json.
>
> * auto-generates a libraries.dart in the patched sdk that can be used to run
> dartanalyzer or to compile an sdk with dartdevc-legacy.
>
> * script simplifications in patch_sdk.dart
> * removed tracking of modification stamps. This didn't appeared to be
> used by the ninja rules. I believe it was used to reduce local
> iteration cycles when we were first developing the sdk itself.
>
> * use Uri directly instead of package:path - this simplifies things,
> especially since the libraries_specification is Uri-based as well.
>
> * switch to use package:args for parsing options
>
> * added an option to merge part files in a single file. I've noticed that the
> analyzer provides some false positive errors in part files, but once merged
> they are no longer shown. In particular, we are clearly hitting limitations of
> dartanalyzer on the patched sdk. At this time the old patch-script generated
> an sdk with 1500 warnings, this new script reduces it to 500 (not sure why),
> with merging of part files it goes down to 80.
>
>
> Change-Id: I6bbaf92ef4554f00c8bf6b38d19bf79d44fb3e94
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125780
> Reviewed-by: Nicholas Shahan <nshahan@google.com>
> Commit-Queue: Sigmund Cherem <sigmund@google.com>
Change-Id: Ifcb9dbef36452e41e8e96e086761d76b5a3d988e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125998
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Auto-Submit: Sigmund Cherem <sigmund@google.com>
The compile_platform template assumes that the output will be in the root-out-dir, and because of that the generated .deps file contains relative paths that only work if the output is in that folder.
It would be nice to fix compile_platform to be more flexible about this, but for now moving the output up fixes the issue that modifications in the sdk were not triggering a rebuild.
Change-Id: Ifc1e7adb23d6911a9f71c341b7968e0a9b1d4973
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/126122
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
This reverts commit 3c9e924073.
This CL was breaking the internal build.
Change-Id: Ic4e6df070f2a87ac88ef97f0e6b07d53c13ceadc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125925
Auto-Submit: Vijay Menon <vsm@google.com>
Commit-Queue: Vijay Menon <vsm@google.com>
Commit-Queue: David Morgan <davidmorgan@google.com>
Reviewed-by: David Morgan <davidmorgan@google.com>
This will make it possible to use the script to create a patched sdk
for dart2js and the vm (some followup needed), which will be useful
to migrate their libraries to nnbd.
In the future, we should be able to leverage this new logic to
write tests that ensure our sdk is warning-free (there are > 80 real
warnings today even on the non-nnbd sdk)
In detail, this CL includes:
* loading the libraries data from libraries.json, this removes the need for a
hand-crafted libraries.dart that replicates the data in libraries.json.
* auto-generates a libraries.dart in the patched sdk that can be used to run
dartanalyzer or to compile an sdk with dartdevc-legacy.
* script simplifications in patch_sdk.dart
* removed tracking of modification stamps. This didn't appeared to be
used by the ninja rules. I believe it was used to reduce local
iteration cycles when we were first developing the sdk itself.
* use Uri directly instead of package:path - this simplifies things,
especially since the libraries_specification is Uri-based as well.
* switch to use package:args for parsing options
* added an option to merge part files in a single file. I've noticed that the
analyzer provides some false positive errors in part files, but once merged
they are no longer shown. In particular, we are clearly hitting limitations of
dartanalyzer on the patched sdk. At this time the old patch-script generated
an sdk with 1500 warnings, this new script reduces it to 500 (not sure why),
with merging of part files it goes down to 80.
Change-Id: I6bbaf92ef4554f00c8bf6b38d19bf79d44fb3e94
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125780
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
This is to align how we pipeline each step in the build with other tools. It
will also help with caching and incremental rebuilds when doing local development
in ddc.
Change-Id: I8ff9988927d5632301926fca173c01dd281f923f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125447
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Vijay Menon <vsm@google.com>
This moves ddc to use the same logic to compile the sdk.dill as all other
backends. This will also enable us in a subsequent CL to generate the dart_sdk.js
files from a precompiled .dill file, and hence improve overall caching on local
rebuilds.
Change-Id: I02e178baa5497a5bee2de42e3423176ca97ceaef
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125446
Reviewed-by: Nicholas Shahan <nshahan@google.com>
This allows us to compile each package in parallel (for those with no deps), and
will make it possible to enable nnbd on a subset of them (expect, async_helper,
js, meta)
Change-Id: I032bafe3e43b14340ee35509d1f228d18571f524
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125484
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
We still have a single `dartdevc_test` rule that invokes both, but now they can
run in parallel and this unblocks a subsequent change to build each pacakge
separately under kernel.
Depending on the timing of deleting dartdevc-legacy, we may want to consider
adding a `dartdevc_legacy_test` vs `dartdevc_kernel_test` to keep the two more
separate/siloed.
Change-Id: Icc3f8fd21248aa09b553c41df708452d21b39b2f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125500
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
This will make the nnbd-ddc bot green again. To properly support compiling these
packages in the future, we need to either migrate them or add to the CFE the
machanism to select whether a package is opt-out.
Change-Id: Ia8c5fa1a8000e233af20c87e877e2c666cb4354e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125420
Auto-Submit: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
The goal of this change is to make it possible to start iterating on nnbd tests
for DDC. This keeps the current philosophy of keeping NNBD and non-NNBD build
configurations separate, except that NNBD builds are allowed to consume library
sources from the non-nnbd sdk.
This change addresses the first set of errors that are encountered when trying
to build a fully NNBD configuration:
* CFE has issues with type promotion, for that reason we remove
`--enable-experiment=non-nullable` in vm build rules, and we use a smaller
training data when creating the dartdevc app-jit snapshot.
* We'd hit compile-time errors with nnbd libraries with inconsistent
patchfiles. To avoid that, the libarries.json points to the non-nnbd SDK
sources for most dart:* libraries.
Change-Id: Ie6226b3bd8a92b4a1632dd84a5db2f04238fd4f8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125080
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Problem:
We have been having a lot of reports of failures when initializing the incremental compiler
(see https://github.com/dart-lang/sdk/issues/38102).
These appear to be the result of empty kernel files but we have added other logic to check
for that when kernel files are written, which has had zero reports of throwing from users
(https://github.com/dart-lang/build/pull/2387).
We also use the same mechanism for temp directory management as we did with analyzer, but only
see this problem with the switch to kernel, and specifically the incremental compiler.
Solution:
Add retry logic and see if that fixes the problem. Any time there is a retry log a warning
so that we don't just silently do retries all the time.
This is a general defensive mechanism to cover a broad spectrum of potential failures with the
incremental compiler in the wild. I don't intend to remove it as it isn't harmful, and the
warning logs should be enough to encourage issues to be filed if it is happening often.
I have no direct reason to believe this will actually solve the particular linked problem other
than that we only see this issue when the incremental compiler is enabled.
Bug: https://github.com/dart-lang/sdk/issues/39122, https://github.com/dart-lang/sdk/issues/38102
Change-Id: Iaabb4497d6da69684692c1d7b9c030c59ebc6072
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123001
Commit-Queue: Jake Macdonald <jakemac@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Auto-Submit: Jake Macdonald <jakemac@google.com>
I confirmed that this prevents the local paths from leaking into the ddc_sdk.dill
file created by this build target.
Change-Id: I777482f5aea6d70106072f070ab48233ec302682
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/122178
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
With these changes, I can run:
./tools/build.py -m release --nnbd dartdevc_test
In a checkout that also includes a migrated core library that uses
NNBD features.
I haven't verified if the resulting JS is *correct*, but the build
doesn't error out.
Change-Id: I7d89efe5da8c46e2a9805743e4e61858da8097dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/122280
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
When use_nnbd is set, make the build steps for generating the DDK SDK
summary and JS use the forked SDK directory and enable the non-nullable
experiment flag.
Change-Id: Ia483a621dccd40245cdf5e2c1f7fd86b9bc1e266
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/122162
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
To support JavaScript compilation, the frontend_server will require a dependency on the dev_compiler. To avoid conflating this with the vm specific functionality, the frontend server will be split from its current location.
This change will require a small corresponding update in flutter/engine, documented in the patches directory
Change-Id: I47923765546f7f6fa43e36ef38f8f466d3a7b2fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/120321
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Our //build/config/BUILDCONFIG.gn isn't used for Flutter, so we must
put the default version where the Flutter build can see it.
Change-Id: I99afc99209b3721c48aa56ef413910d34df1bb8c
Cq-Include-Trybots: luci.dart.try:flutter-engine-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/120580
Reviewed-by: Teagan Strickland <sstrickl@google.com>
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Built the revert CL by hand because it conflicts with other changes.
Change-Id: I50fed06e9b5311d38b0f242f8207d5c51c795783
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/120360
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Consensus seems to be that there should be different mechanism to support host-targeting-host vs host-targeting-target configuration: comparing toolchains names won't work for that.
Also, dart_host_toolchain was set up to be used by Fuchsia, but it is no longer being used.
Change-Id: Ic2e63d8cef00b18bf6866122199027459eaf32c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118910
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
Not doing it can lead to the same input producing "different" outputs.
Note that this is a follow-up to cdcec63569
where I forgot about the incremental summary-only case.
Change-Id: Idf76c3839f46c468a62350968b353be7235d91b9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118287
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
It is included in the 'dart' binary thus making it change depending on
where it is compiled.
Change-Id: I6f3ad8cf8f8da386ada7809402b4711c4fcab3ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117728
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Not doing it can lead to the same input producing "different" outputs.
Change-Id: I31fa1d728c2d26cbf1ad008472ea8fce4310794f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117725
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
This reverts commit d9489622be.
Reason for revert: flutter/engine@04f567bdde
Original change's description:
> Revert "[build] Speed up debug and simulator builds by running steps on the prebuilt VM."
>
> This reverts commit 74cff6c7df as it
> introduces a breakage of flutter build process.
>
> Revert "[build] Fix application_snapshot.gni for uses outside of utils/xyz."
>
> This reverts commit 351acd155d as a
> collateral damage.
>
> Change-Id: Ic175c464c78e76a0adf176b41294344901bfe798
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117063
> Commit-Queue: Alexander Aprelev <aam@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
TBR=aam@google.com,rmacnak@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: Ie27eb455eddd3e60b96eb65bb3dad888b369baf6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117286
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This is a reland of https://dart-review.googlesource.com/c/sdk/+/113123
after stack overflow in the interpreter was fixed in https://dart-review.googlesource.com/c/sdk/+/117061
Original change's description:
Temporary setting of FLAG_enable_interpreter is avoided in the VM unit tests as
this flag is global and used by kernel service isolate while running tests.
Flipping this flag causes assertion failures in kernel isolate's background compiler
or infinite loop between LazyCompile stub and CompileFunction as AttachBytecode
doesn't set entry point to InterpretCall.
The less intrusive way to ensure compilation of functions in unit tests is to set
FLAG_compilation_counter_threshold to 0.
Change-Id: Ibbcf4b5bb0df558851c80cfa40a9a54f949dd187
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117122
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Before this CL the kernel service was during the build trained with
an attempt at a full file uri for dart2js.
There was a suddle bug though: Under non-Windows the uri would become
something like "file:////full/path/to/dart2js.dart" (i.e. have 4 slashes)
whereas under Windows it would look something like
"file:///C:/full/path/to/dart2js.dart" (i.e. correctly with 3 slashes).
Mostly, having 4 slashes instead of 3 doesn't matter much:
Everything will just look weird and have 4 slashes instead of 3,
but when at the same time giving the compiler a relative .packages file
(so it will be translated according to the current directory and become
a uri with 3 slashes) with relative uris in it (so those uris also get
3 slashes in them) and then doing a sub-string match to - in some
cases - interpreting the input as a package uri things go awry.
"file:///path/to/whatever" is not a substring of
"file:////path/to/whatever" and thus the interpreting the input as a
package uri doesn't happen, which can cause weird compilation errors
when mixing relative and package uri imports, e.g.:
file:////b/s/w/ir/cache/builder/sdk/pkg/compiler/lib/src/common/codegen.dart:439:38: Error: The argument type 'MemberEntity/*1*/' can't be assigned to the parameter type 'MemberEntity/*2*/'.
- 'MemberEntity/*1*/' is from 'file:////b/s/w/ir/cache/builder/sdk/pkg/compiler/lib/src/elements/entities.dart'.
- 'MemberEntity/*2*/' is from 'package:compiler/src/elements/entities.dart' ('file:///b/s/w/ir/cache/builder/sdk/pkg/compiler/lib/src/elements/entities.dart').
Try changing the type of the parameter, or casting the argument to 'MemberEntity/*2*/'.
return _functionCompiler.compile(member);
^
Change-Id: I4c6b5ea2e7e2e4a75331847138a79708ac365769
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103126
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
This reverts commit 74cff6c7df as it
introduces a breakage of flutter build process.
Revert "[build] Fix application_snapshot.gni for uses outside of utils/xyz."
This reverts commit 351acd155d as a
collateral damage.
Change-Id: Ic175c464c78e76a0adf176b41294344901bfe798
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117063
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Only generating a snapshot really requires running the VM produced during the build.
Change-Id: Ic07bf8b61609c39c9e2dc910737ed56e64decd43
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116749
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
