Commit graph

98 commits

Author SHA1 Message Date
Martin Kustermann 23109940e0 [vm] Separate for-snapshot vs embed-sources bools when compiling to kernel
Fixes https://github.com/dart-lang/sdk/issues/53010

TEST=Fixes vm/dart/regress_48523_test

Change-Id: I2066ffd250228c01bb7aa2de5d1d0362ba6f7071
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316581
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2023-07-28 10:01:53 +00:00
Ryan Macnak b68351fbc3 [vm] Update NULL to nullptr in runtime/bin.
TEST=build
Change-Id: Ie3be570c274b0275a995a0f54b5e6ccdfc77ccd3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292287
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Brian Quinlan <bquinlan@google.com>
2023-04-12 01:11:05 +00:00
Ryan Macnak c81b00938d [standalone] Fix memory leak in the tag handler.
TEST=asan
Bug: https://github.com/dart-lang/sdk/issues/37030
Bug: https://github.com/dart-lang/sdk/issues/51210
Change-Id: Ia62a4d7d1805c6ac4a311ef9952fff248e7b4693
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280284
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-02-02 21:35:47 +00:00
Alexander Markov 211da364be [vm] Experimental ability to spawn isolate from kernel blob in memory
TEST=runtime/tests/vm/dart/spawn_uri_from_kernel_blob_test.dart

Change-Id: Ieb327f0350d5d8ea1d344c64aa3dd217125da5fe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/232682
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-02-23 16:55:52 +00:00
Vyacheslav Egorov bda70c8489 [vm] Remove support for dart-ext: imports
Per breaking change #45451 we are removing support for dart-ext:
style native extensions from the Dart VM.

This CL removes the associated VM code, tests and samples. It also ports
a single test which used dart-ext: import to use FFI instead.

TEST=ci

Bug: https://github.com/dart-lang/sdk/issues/45451
Change-Id: Iae984bce32baf29a950b5de1323939006a217b94
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212050
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-09-02 14:14:35 +00:00
Alexander Aprelev e70690e415 [native_extensions] Convert uri to path so that on Windows native extension loading works.
Lookup of native extension library on Windows works when the library is next
to Platform.executable, this CL fixes lookup when it's next to the script.

Fixes https://github.com/dart-lang/sdk/issues/45370

TEST=updated sample_extension test

Change-Id: I91226be19ceec05e45cf90fd85428a62d95769c2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193741
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-04-01 03:22:35 +00:00
Johnni Winther 55b34a467c Reland "[vm] Pass snapshot flag to kernel_service"
Enable reporting of null safety compilation mode when running
`dart compile aot-snapshot`, `dart compile jit-snapshot`,
and `dart compile kernel`.

Closes #44234

TEST=pkg/dartdev/test/commands/compile_test.dart

Change-Id: Id9e72751068d5f208055ab953d2c8392e84c4798
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/180187
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2021-01-21 21:18:32 +00:00
Aske Simon Christensen 903e92a196 Revert "[vm] Pass snapshot flag to kernel_service"
This reverts commit f8b0d26cc3.

Reason for revert: Multiple test failures in app-jit and simarm(64)

Original change's description:
> [vm] Pass snapshot flag to kernel_service
>
> Enable reporting of null safety compilation mode when running
> `dart compile aot-snapshot`, `dart compile jit-snapshot`,
> and `dart compile kernel`.
>
> Closes #44234
>
> TEST=pkg/dartdev/test/commands/compile_test.dart
>
> Change-Id: I0d4b35c6ccb4167c0c7539a4eb24a5139e29cf53
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178990
> Commit-Queue: Johnni Winther <johnniwinther@google.com>
> Reviewed-by: Ben Konyi <bkonyi@google.com>

TBR=bkonyi@google.com,johnniwinther@google.com

Change-Id: I83aeaa8620c640d02d5ccfd1fe8112d8209a9ad4
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/179773
Auto-Submit: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2021-01-18 21:00:03 +00:00
Johnni Winther f8b0d26cc3 [vm] Pass snapshot flag to kernel_service
Enable reporting of null safety compilation mode when running
`dart compile aot-snapshot`, `dart compile jit-snapshot`,
and `dart compile kernel`.

Closes #44234

TEST=pkg/dartdev/test/commands/compile_test.dart

Change-Id: I0d4b35c6ccb4167c0c7539a4eb24a5139e29cf53
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178990
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2021-01-18 09:27:59 +00:00
Ryan Macnak 9cb5d8e5be [standalone] Fix Isolate.spawnUri to work with AOT snapshots.
Fixes missing initialization of dart:_builtin during AOT.

Change-Id: If9f24f3658b91f490fc8215f2e9343bd437b2744
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/156050
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-08-04 19:00:18 +00:00
Daco Harkes 202e32afeb [vm] Native API: Introduce Dart_FinalizableHandle
Introduces Dart_NewFinalizableHandle which does auto delete itself,
but does not allow accessing the weak referenced object.

Issue: https://github.com/dart-lang/sdk/issues/42312

Change-Id: I24ea732925122c453213c4fa3f629761c352f838
Cq-Include-Trybots:dart/try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154695
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-07-23 11:30:39 +00:00
Ryan Macnak c5a94db091 [vm] Initial implementation of deferred loading.
(Assignment of libraries to loading units is already done in the kernel generation step.)

After compiling and before serializing, we walk the program and for each Code we assign its Instructions, CodeSourceMap and CompressedStackMap to the loading unit of that Code's defining library. Deduplication may cause Instructions, CodeSourceMaps and CompressedStackMaps to belong to more than one loading unit; in this case the objects are assigned to the root loading unit. Later they can be more precisely assigned to the dominating loading unit.

All objects except some Instructions, CodeSourceMaps and CompressedStackMaps belong to the root loading unit's snapshot. This snapshot is written like an unsplit snapshot, except that when serializing Code, we will write a reference to a stub or null when the Code's Instructions, CodeSourceMap or CompressedStackMap belongs to a non-root loading unit. The snapshots of non-root loading units contain these deferred objects and references to the corresponding Code objects to patch. The types of objects we defer (Instructions, CodeSourceMaps and CompressedStackMaps) usually represent 70+% of the snapshot size.

Bare instructions mode must be disabled when splitting because we cannot have PC-relative calls between loading units. Later we can re-enable this for calls within loading units.

Broken: Compactor probably crashes we can now have an unbounded number of image pages and the compactor assumes a fixed number.

Embedder's guide:

At compile-time, gen_snapshot should be passed --loading_unit_manifest with a path, which will enable splitting and output a mapping from loading unit ids to snapshot output paths.

At runtime, sometime during isolate startup, an embedder should call Dart_SetDeferredLoadHandler, probably near an existing call to Dart_SetLibraryTagHandler. The callback is given a loading unit id, and should eventually call Dart_DeferredLoadComplete[Error].

Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Ib597eb87c8cd634416d5ee1f00629c5550aebb00
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152427
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-07-17 19:29:47 +00:00
Alexander Aprelev d33e51df3c [build] Support --clang when building dart sdk on Windows.
It relies on flutter copy of clang distribution, same one that is used to build flutter/engine.

It addressed several deprecated warnings from clang compiler for functions like strdup, unlink, etc.

It allows few warnings still since they are triggered in third_party sources.

Change-Id: Ieb13792c011438d46dbbc0fa030e1b5e4ea14315
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142704
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-06-25 03:36:41 +00:00
Siva Annamalai d46a696957 Revert "Reland : [VM/Runtime] Cleanup package config initialization code"
This reverts commit 03777dba31.

Reason for revert: Failures in couple of bots

Original change's description:
> Reland : [VM/Runtime] Cleanup package config initialization code
> 
> - Use _Init function for initialization of package config
> - Remove DartUtils::SetWorkingDirectory, SingleArgDart_Invoke,
> 
>   DartUtils::SetupPackageConfig
> - Rename Loader::InitForSnapshot to Loader::Init
> - Remove _setPackagesConfig, _setWorkingDirectory from builtin library
> - Rename _setPackagesMap to _setPackagesConfig
> Change-Id: I1e4a25e79706d7764247c9fbfcc8e8ce3231343c
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151633
> Reviewed-by: Alexander Aprelev <aam@google.com>
> Commit-Queue: Siva Annamalai <asiva@google.com>

TBR=aam@google.com,asiva@google.com

Change-Id: Ibea8c35fe310ac7708f00bb9ffd58715b6924cb5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151869
Reviewed-by: Siva Annamalai <asiva@google.com>
Auto-Submit: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-06-20 08:02:53 +00:00
asiva 03777dba31 Reland : [VM/Runtime] Cleanup package config initialization code
- Use _Init function for initialization of package config
- Remove DartUtils::SetWorkingDirectory, SingleArgDart_Invoke,

  DartUtils::SetupPackageConfig
- Rename Loader::InitForSnapshot to Loader::Init
- Remove _setPackagesConfig, _setWorkingDirectory from builtin library
- Rename _setPackagesMap to _setPackagesConfig
Change-Id: I1e4a25e79706d7764247c9fbfcc8e8ce3231343c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151633
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-06-20 05:50:49 +00:00
asiva 4b9aa2bd7e Revert "[VM/Runtime] Cleanup package config initialization code"
This reverts commit 29e93bcdbd.

Reason for revert: Windows bots are failing with package resolution
errors. Looks like we are missing a windows path sanitization step.

Change-Id: Ib56f7e926b4f385fa3fde74c9c71947f64b8be1c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151469
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-06-17 00:03:19 +00:00
asiva 29e93bcdbd [VM/Runtime] Cleanup package config initialization code
- Use _Init function for initialization of package config
- Remove DartUtils::SetWorkingDirectory, SingleArgDart_Invoke,
  DartUtils::SetupPackageConfig
- Rename Loader::InitForSnapshot to Loader::Init
- Remove _setPackagesConfig, _setWorkingDirectory from builtin library
- Rename _setPackagesMap to _setPackagesConfig
Change-Id: I0b83574593fa95e1b948134fcd4a7b4858ea19c2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151045
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-06-16 23:24:26 +00:00
Ryan Macnak 77ddd77c0a [standalone] Fix memory leak when failing to load a native extension.
Change-Id: Ib7996abc122077f8c7191e741f06fc6ff53a0038
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/137650
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-02-28 00:57:16 +00:00
asiva 01c9c18d24 [VM/Runtime] Propagate error from Loader::InitForSnapshot instead of asserting
Change signature of Loader::InitForSnapshot to return an error and check
for this error when initializing an isolate. This should fix the flaky
crashes reported in https://github.com/dart-lang/sdk/issues/39950

Bug:39950
Change-Id: I564b2f8f924378c156f5bb22238a760be187f8b0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/130041
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-01-02 23:00:36 +00:00
asiva b1c6d89fb0 [VM/runtime] Remove support for loading URLs through service isolate
- Removes the load port functionality from the service isolate, this was
  used to load sources in Dart 1 and is pretty much dead code in Dart 2
- Moves resolution of URIs and Package URI (based on package map) to the
  individual isolates instead of sending a request to the service isolate
- Setups the package map lazily as it is needed only if resolveURI is
  called in Dart code, it is not needed for loading/compiling sources
  (this is now done in the front end).
- Removed Dart_ServiceWaitForLoadPort() methods from the Dart C API as
  it is not used anywhere.

Change-Id: I6c3704bc431bdcd49bf074eb58bee1ed492ccccb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/129742
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2019-12-27 17:30:05 +00:00
asiva d0139e8e38 [VM] More cleanup in the loader code
- Removed ResolveDependenciesAsFilePaths functionality as the loader
  does not track any dependencies (this is now provided by the kernel
  isolate)

- Removed ResolveAsFilePath functionality from the loader class

- Removed some left over code related to deferred loading

Change-Id: Ic1b1a79b07020bb7082e9efb7544b73931290faf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/122220
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2019-10-21 17:44:18 +00:00
asiva cd95fcc810 [VM] Start the kernel isolate only when compilations are needed
- Start the kernel isolate only when a compilation is needed
- Remove some dead code and state in the loader path

Change-Id: I39e14cd79cda18b155af6334a11635c44e2fc11f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121923
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2019-10-18 00:27:16 +00:00
Martin Kustermann a64b06ec83 [gardening] Migrate most files to be implicit-bool-conversion free
Issue https://dart-review.googlesource.com/c/sdk/+/115701

Change-Id: Ib579f0bbc8d694aec74afd837217316a10baf910
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/115707
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2019-09-05 21:41:42 +00:00
Alexander Aprelev 0425997b31 Second attempt to reland "[vm/concurrency] Introduce concept of Isolate Groups"
This reverts commit 3d14b75f97.

An Isolate Group (IG) is a collection of isolates which were spawned from the
same source. This allows the VM to:

  * have a guarantee that all isolates within one IG can safely exchange
    structured objects (currently we rely on embedder for this
    guarantee)

  * hot-reload all isolates together (currently we only reload one
    isolate, leaving same-source isolates in inconsistent state)

  * make a shared heap for all isolates from the same IG, which paves
    the way for faster communication and sharing of immutable objects.

All isolates within one IG will share the same IsolateGroupSource.

**Embedder changes**

This change makes breaking embedder API changes to support this new
concept of Isolate Groups: The existing isolate lifecycle callbacks
given to Dart_Initialize will become Isolate Group lifecycle callbacks.
A new callback `initialize_isolate` callback will be added which can
initialize a new isolate within an existing IG.

Existing embedders can be updated by performing the following renames

  Dart_CreateIsolate -> Dart_CreateIsolateGroup
  Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
  Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
  Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
  Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
  Dart_IsolateData -> Dart_IsolateGroupData
  Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
  Dart_InitializeParams.create -> Dart_InitializeParams.create_group
  Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
  Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate

By default `Isolate.spawn` will cause the creation of a new IG.

Though an embedder can opt-into supporting multiple isolates within one IG by
providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
The responsibility of this new callback is to initialize an existing
isolate (which was setup by re-using source code from the spawning
isolate - i.e. the one which used `Isolate.spawn`) by setting native
resolvers, initializing global state, etc.

Issue https://github.com/dart-lang/sdk/issues/36648
Issue https://github.com/dart-lang/sdk/issues/36097

Original review: https://dart-review.googlesource.com/c/sdk/+/105241

Difference to original review:

  * Give each isolate it's own [Loader] (for now)
  * Sort classes during initialization for spawned isolates if app-jit is used (to match main isolate)
  * Fix IsolateData memory leak if isolate startup fails

Difference to first reland(Patchset 2):

  * Fix typo where memory was freed twice.

Change-Id: Ib1c83fe83b629cd50ae6af90ee99fdd44da882d4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108367
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2019-07-08 23:49:05 +00:00
Vyacheslav Egorov 3d14b75f97 Revert "Reland "[vm/concurrency] Introduce concept of Isolate Groups""
This reverts commit 67ab3be10d.

Reason for revert: Causes non-deterministic failures on front-end bots on Windows with "Isolate creation failed" error. See https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket.appspot.com/8909292129328681248/+/steps/unit_tests/0/stdout

Original change's description:
> Reland "[vm/concurrency] Introduce concept of Isolate Groups"
> 
> An Isolate Group (IG) is a collection of isolates which were spawned from the
> same source. This allows the VM to:
> 
>   * have a guarantee that all isolates within one IG can safely exchange
>     structured objects (currently we rely on embedder for this
>     guarantee)
> 
>   * hot-reload all isolates together (currently we only reload one
>     isolate, leaving same-source isolates in inconsistent state)
> 
>   * make a shared heap for all isolates from the same IG, which paves
>     the way for faster communication and sharing of immutable objects.
> 
> All isolates within one IG will share the same IsolateGroupSource.
> 
> **Embedder changes**
> 
> This change makes breaking embedder API changes to support this new
> concept of Isolate Groups: The existing isolate lifecycle callbacks
> given to Dart_Initialize will become Isolate Group lifecycle callbacks.
> A new callback `initialize_isolate` callback will be added which can
> initialize a new isolate within an existing IG.
> 
> Existing embedders can be updated by performing the following renames
> 
>   Dart_CreateIsolate -> Dart_CreateIsolateGroup
>   Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
>   Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
>   Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
>   Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
>   Dart_IsolateData -> Dart_IsolateGroupData
>   Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
>   Dart_InitializeParams.create -> Dart_InitializeParams.create_group
>   Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
>   Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate
> 
> By default `Isolate.spawn` will cause the creation of a new IG.
> 
> Though an embedder can opt-into supporting multiple isolates within one IG by
> providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
> The responsibility of this new callback is to initialize an existing
> isolate (which was setup by re-using source code from the spawning
> isolate - i.e. the one which used `Isolate.spawn`) by setting native
> resolvers, initializing global state, etc.
> 
> Issue https://github.com/dart-lang/sdk/issues/36648
> Issue https://github.com/dart-lang/sdk/issues/36097
> 
> Original review: https://dart-review.googlesource.com/c/sdk/+/105241
> 
> Difference to original review:
> 
>   * Give each isolate it's own [Loader] (for now)
>   * Sort classes during initialization for spawned isolates if app-jit is used (to match main isolate)
>   * Fix IsolateData memory leak if isolate startup fails
> 
> Change-Id: I98277d3d10fe275aa9b8a16b6bdd446bbea0b100
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107506
> Commit-Queue: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TBR=kustermann@google.com,aam@google.com,rmacnak@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: Ia4e0f4f9fc317499d3570a371c5bdf9aed799e77
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108101
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2019-07-03 12:19:15 +00:00
Martin Kustermann 67ab3be10d Reland "[vm/concurrency] Introduce concept of Isolate Groups"
An Isolate Group (IG) is a collection of isolates which were spawned from the
same source. This allows the VM to:

  * have a guarantee that all isolates within one IG can safely exchange
    structured objects (currently we rely on embedder for this
    guarantee)

  * hot-reload all isolates together (currently we only reload one
    isolate, leaving same-source isolates in inconsistent state)

  * make a shared heap for all isolates from the same IG, which paves
    the way for faster communication and sharing of immutable objects.

All isolates within one IG will share the same IsolateGroupSource.

**Embedder changes**

This change makes breaking embedder API changes to support this new
concept of Isolate Groups: The existing isolate lifecycle callbacks
given to Dart_Initialize will become Isolate Group lifecycle callbacks.
A new callback `initialize_isolate` callback will be added which can
initialize a new isolate within an existing IG.

Existing embedders can be updated by performing the following renames

  Dart_CreateIsolate -> Dart_CreateIsolateGroup
  Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
  Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
  Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
  Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
  Dart_IsolateData -> Dart_IsolateGroupData
  Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
  Dart_InitializeParams.create -> Dart_InitializeParams.create_group
  Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
  Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate

By default `Isolate.spawn` will cause the creation of a new IG.

Though an embedder can opt-into supporting multiple isolates within one IG by
providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
The responsibility of this new callback is to initialize an existing
isolate (which was setup by re-using source code from the spawning
isolate - i.e. the one which used `Isolate.spawn`) by setting native
resolvers, initializing global state, etc.

Issue https://github.com/dart-lang/sdk/issues/36648
Issue https://github.com/dart-lang/sdk/issues/36097

Original review: https://dart-review.googlesource.com/c/sdk/+/105241

Difference to original review:

  * Give each isolate it's own [Loader] (for now)
  * Sort classes during initialization for spawned isolates if app-jit is used (to match main isolate)
  * Fix IsolateData memory leak if isolate startup fails

Change-Id: I98277d3d10fe275aa9b8a16b6bdd446bbea0b100
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107506
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2019-06-29 18:45:11 +00:00
Martin Kustermann b2171f16f1 Revert "[vm/concurrency] Introduce concept of Isolate Groups"
This reverts commit b75057b687.

Reason for revert: Caused some failures on app-jit builders

Original change's description:
> [vm/concurrency] Introduce concept of Isolate Groups
> 
> An Isolate Group (IG) is a collection of isolates which were spawned from the
> same source. This allows the VM to:
> 
>   * have a guarantee that all isolates within one IG can safely exchange
>     structured objects (currently we rely on embedder for this
>     guarantee)
> 
>   * hot-reload all isolates together (currently we only reload one
>     isolate, leaving same-source isolates in inconsistent state)
> 
>   * make a shared heap for all isolates from the same IG, which paves
>     the way for faster communication and sharing of immutable objects.
> 
> All isolates within one IG will share the same IsolateGroupSource.
> 
> **Embedder changes**
> 
> This change makes breaking embedder API changes to support this new
> concept of Isolate Groups: The existing isolate lifecycle callbacks
> given to Dart_Initialize will become Isolate Group lifecycle callbacks.
> A new callback `initialize_isolate` callback will be added which can
> initialize a new isolate within an existing IG.
> 
> Existing embedders can be updated by performing the following renames
> 
>   Dart_CreateIsolate -> Dart_CreateIsolateGroup
>   Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
>   Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
>   Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
>   Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
>   Dart_IsolateData -> Dart_IsolateGroupData
>   Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
>   Dart_InitializeParams.create -> Dart_InitializeParams.create_group
>   Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
>   Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate
> 
> By default `Isolate.spawn` will cause the creation of a new IG.
> 
> Though an embedder can opt-into supporting multiple isolates within one IG by
> providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
> The responsibility of this new callback is to initialize an existing
> isolate (which was setup by re-using source code from the spawning
> isolate - i.e. the one which used `Isolate.spawn`) by setting native
> resolvers, initializing global state, etc.
> 
> Issue https://github.com/dart-lang/sdk/issues/36648
> Issue https://github.com/dart-lang/sdk/issues/36097
> 
> Change-Id: I82437ac017ca33018d45e02f353b0672db155f6a
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105241
> Commit-Queue: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Alexander Aprelev <aam@google.com>

TBR=kustermann@google.com,aam@google.com,rmacnak@google.com

Change-Id: Ibd90992a01d61188f27b445c21532e0c46f996ea
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107405
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2019-06-26 16:04:27 +00:00
Martin Kustermann b75057b687 [vm/concurrency] Introduce concept of Isolate Groups
An Isolate Group (IG) is a collection of isolates which were spawned from the
same source. This allows the VM to:

  * have a guarantee that all isolates within one IG can safely exchange
    structured objects (currently we rely on embedder for this
    guarantee)

  * hot-reload all isolates together (currently we only reload one
    isolate, leaving same-source isolates in inconsistent state)

  * make a shared heap for all isolates from the same IG, which paves
    the way for faster communication and sharing of immutable objects.

All isolates within one IG will share the same IsolateGroupSource.

**Embedder changes**

This change makes breaking embedder API changes to support this new
concept of Isolate Groups: The existing isolate lifecycle callbacks
given to Dart_Initialize will become Isolate Group lifecycle callbacks.
A new callback `initialize_isolate` callback will be added which can
initialize a new isolate within an existing IG.

Existing embedders can be updated by performing the following renames

  Dart_CreateIsolate -> Dart_CreateIsolateGroup
  Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
  Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
  Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
  Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
  Dart_IsolateData -> Dart_IsolateGroupData
  Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
  Dart_InitializeParams.create -> Dart_InitializeParams.create_group
  Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
  Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate

By default `Isolate.spawn` will cause the creation of a new IG.

Though an embedder can opt-into supporting multiple isolates within one IG by
providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
The responsibility of this new callback is to initialize an existing
isolate (which was setup by re-using source code from the spawning
isolate - i.e. the one which used `Isolate.spawn`) by setting native
resolvers, initializing global state, etc.

Issue https://github.com/dart-lang/sdk/issues/36648
Issue https://github.com/dart-lang/sdk/issues/36097

Change-Id: I82437ac017ca33018d45e02f353b0672db155f6a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105241
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2019-06-26 11:34:48 +00:00
Matthew Dempsky 6d5f763952 [vm] Eliminate Mutex/Monitor indirection where possible
In many cases, the Mutexes and Monitors have to be marked "mutable"
because they're used to synchronize const accessor methods.

Small text segment improvement for Product builds:

$ size dart.{arm,x64}.{before,after}
   text	   data	    bss	    dec	    hex	filename
19726069	 409960	 392332	20528361	1393ce9	dart.arm.before
19725525	 409960	 392332	20527817	1393ac9	dart.arm.after
22576021	 600376	1782824	24959221	17cd8f5	dart.x64.before
22574821	 600376	1782824	24958021	17cd445	dart.x64.after

Change-Id: I68f5cd5ad452044df8bfebd160910496036a3e6b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/101745
Commit-Queue: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2019-05-13 21:17:51 +00:00
Ryan Macnak 971f4845d7 [build] Remove last platform -> vm and bin -> vm dependencies. Fix some ODR violations.
Change some static_libraries to source_sets to make ODR violations link-time errors.

This is needed to enable (stop suppressing) -fvisibility=hidden in Fuchsia product builds.

Change-Id: I699cec8d4b516beab9cebf9db0a522a7ff99e004
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99822
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2019-04-22 20:15:43 +00:00
Ryan Macnak 235a918233 [vm] Remove Dart_LoadScript, Dart_LoadLibrary, Dart_LoadSource, etc.
Change-Id: I9c19625702f1df48c03de2eb3a0f7bb581d2e0b3
Reviewed-on: https://dart-review.googlesource.com/c/79140
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-10-11 00:38:37 +00:00
asiva 30aa464f77 [VM] Remove 'corelib' and 'dart-io' sources generation into C++ arrays
- Remove all code that generates the corelib and dart:io library sources into C++ arrays and link them into dart_bootstrap
- Remove the executable dart_bootstrap and all uses of it
- Remove bootstrap_nolib.cc and builtin_nocore.cc
- Remove the Dart 1 code in core lib bootstrap path

Change-Id: Ifd33496204285a08b42fe09e39428e7a92b416b6
Reviewed-on: https://dart-review.googlesource.com/c/77241
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2018-10-02 23:27:05 +00:00
Ryan Macnak 2f25dcda16 [vm] Remove script snapshots.
Change-Id: I5bc692b754c0909de4aa5f05af81b074578b16f4
Reviewed-on: https://dart-review.googlesource.com/76706
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2018-09-26 22:49:27 +00:00
asiva 0a29264117 [VM] Remove vm_platform from the dart binary
- Remove vm_platform from the dart binary
- kernel isolate always compiles in strong mode

Change-Id: I4e3973fdaee0bda1c43244ea04673f253eaae249
Reviewed-on: https://dart-review.googlesource.com/65788
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-08-24 15:45:04 +00:00
Samir Jindel dd1b7005a6 [vm] Fix typo in 7e358186a
Change-Id: I682263fe4bfa5b2485d42dd28dc93d1450387070
Reviewed-on: https://dart-review.googlesource.com/71240
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
2018-08-22 19:19:58 +00:00
Samir Jindel 7e358186ab [vm] Fix resolution of native extensions imported within packages.
We need to use the actual file URL of the importing library to find the extension library,
such as "file:///library/lib.dart" rather than the package URL (like "package:library/lib.dart").


Change-Id: I32a36f7a67321f356c9af34d1a458aeb122d0c24
Reviewed-on: https://dart-review.googlesource.com/71167
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-08-22 17:45:34 +00:00
Samir Jindel 5c7d257978 [kernel/vm] Fix native extensions.
Tested with imports through current directory, VM binary directory,
and LD_LIBRARY_PATH. This also restores the Dart 1 behavior of not supporting
relative extension paths.

Change-Id: I090bf8592fef74d4ccde40e6f550baa84c98e3bc
Reviewed-on: https://dart-review.googlesource.com/69162
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
2018-08-10 11:06:56 +00:00
Alexander Markov 4552beb514 Revert "[vm/kernel-service] Report an error if isolate compilation options change."
This reverts commit e0845f9359.

Reason for revert: large number of test failures on vm-kernel-reload-* bots.

Original change's description:
> [vm/kernel-service] Report an error if isolate compilation options change.
> 
> This is follow-up to https://dart.googlesource.com/sdk/+/f67cba10676e6464ed137656d04373392e6d5c03.
> 
> Change-Id: I81832ad7fb948cf31c39530f8bb193ef7f45f58d
> Reviewed-on: https://dart-review.googlesource.com/62280
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Alexander Aprelev <aam@google.com>

TBR=aam@google.com,rmacnak@google.com,asiva@google.com

Change-Id: Id7f1d26d11261cda46d5f552a468c2e3c3f50ec5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/62860
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2018-06-28 15:56:45 +00:00
Alexander Aprelev e0845f9359 [vm/kernel-service] Report an error if isolate compilation options change.
This is follow-up to https://dart.googlesource.com/sdk/+/f67cba10676e6464ed137656d04373392e6d5c03.

Change-Id: I81832ad7fb948cf31c39530f8bb193ef7f45f58d
Reviewed-on: https://dart-review.googlesource.com/62280
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2018-06-27 23:31:07 +00:00
Ryan Macnak c30af41b96 Reapply "[mirrors] Add IsolateMirror.loadUri."
- Forward only kImportTag requests to DFE. This restores the current handling of a kScriptTag request when loading the kernel isolate in the simulators.
 - Mark dynamic_load_test as failing in the reload stress tests (reload fails to preserve library identity).

Change-Id: Ibe6f0a3505b99736a38d566abf3b2151505d7a7e
Reviewed-on: https://dart-review.googlesource.com/56706
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-05-29 22:01:01 +00:00
Ryan Macnak de545b4079 Revert "[mirrors] Add IsolateMirror.loadUri."
This reverts commit ad4cfa0260.

Reason for revert: Assertion failures on DBC

Original change's description:
> [mirrors] Add IsolateMirror.loadUri.
> 
> This allows a programmer to dynamically load code into an isolate. The closest existing API is Isolate.spawnUri, but communication with the dynamically loaded code in that case is limited to asynchronous message passing of JSON-like objects.
> 
> Change-Id: Icb23e9dacfb0035622c119f11d4e0f892ba2ccd1
> Reviewed-on: https://dart-review.googlesource.com/45363
> Reviewed-by: Zach Anderson <zra@google.com>

TBR=rmacnak@google.com,zra@google.com,asiva@google.com,kmillikin@google.com

Change-Id: I80669188b9f40b3b527e8e268ade0d0d514a8753
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/56640
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2018-05-24 20:03:40 +00:00
Ryan Macnak ad4cfa0260 [mirrors] Add IsolateMirror.loadUri.
This allows a programmer to dynamically load code into an isolate. The closest existing API is Isolate.spawnUri, but communication with the dynamically loaded code in that case is limited to asynchronous message passing of JSON-like objects.

Change-Id: Icb23e9dacfb0035622c119f11d4e0f892ba2ccd1
Reviewed-on: https://dart-review.googlesource.com/45363
Reviewed-by: Zach Anderson <zra@google.com>
2018-05-24 17:07:15 +00:00
Ryan Macnak a5c11d7d03 [vm] Remove Dart_ReadKernelBinary, take 2.
Update status for simulators, which the test harness invokes with DIL files instead of source files.

Change-Id: I8444ad7e17a0a71a1ce3c0021487397baa1c3e65
Reviewed-on: https://dart-review.googlesource.com/54622
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2018-05-14 20:58:08 +00:00
Ryan Macnak 407872b9e5 Revert "[vm] Remove Dart_ReadKernelBinary."
This reverts commit 2998c32951.

Reason for revert: Failures with kernel + simulators

Original change's description:
> [vm] Remove Dart_ReadKernelBinary.
> 
>  - Fix const-ness of Dart_FileReadCallback and callers.
>  - Fix leak on read error in DartUtils::FileRead.
> 
> This is progress towards sharing kernel memory on Fuchsia.
> 
> Bug: https://github.com/dart-lang/sdk/issues/32618
> Change-Id: I47f8b224905d6a105a5ca0ab2ee4ab6a42b5e342
> Reviewed-on: https://dart-review.googlesource.com/47102
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>

TBR=aam@google.com,rmacnak@google.com,asiva@google.com

Change-Id: Ibb636b817de1629a062fe0a6ef9462a2fef669ef
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/32618
Reviewed-on: https://dart-review.googlesource.com/54560
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2018-05-10 01:36:39 +00:00
Ryan Macnak 2998c32951 [vm] Remove Dart_ReadKernelBinary.
- Fix const-ness of Dart_FileReadCallback and callers.
 - Fix leak on read error in DartUtils::FileRead.

This is progress towards sharing kernel memory on Fuchsia.

Bug: https://github.com/dart-lang/sdk/issues/32618
Change-Id: I47f8b224905d6a105a5ca0ab2ee4ab6a42b5e342
Reviewed-on: https://dart-review.googlesource.com/47102
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-05-10 00:35:37 +00:00
Ryan Macnak 3989d09c95 [vm] Move writing snapshot magic value from embedder to VM.
Add Dart_IsSnapshot and Dart_IsKernel to the embedding API.

Change-Id: I7d4e1d145557e8f11dbc3c143767ad38b3615a4a
Reviewed-on: https://dart-review.googlesource.com/46360
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-03-19 17:47:55 +00:00
Ryan Macnak 558a931409 [vm] Move some string utils from runtime/vm to runtime/platform.
Change-Id: I552086445a7e07792f9da85afa5edf23b6c450d5
Reviewed-on: https://dart-review.googlesource.com/44402
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2018-03-01 02:07:46 +00:00
Ben Konyi 6338914d31 - Removed dead code (pulled from review 36121)
- Enabled incremental compiler in the dart frontend for the command-line VM
- Added method to update in-memory file system with in-memory sources provided by tests

Change-Id: Ica26b7235383185dbb6b4b41064193d432dfcba0
Reviewed-on: https://dart-review.googlesource.com/36126
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-02-07 19:35:39 +00:00
Siva Chandra 71dba8a0e5 Link the platform kernel with the command line dart executable.
Change-Id: Ib376a259e2e9ff9b487820c041d6795f8f7192d1
Reviewed-on: https://dart-review.googlesource.com/37420
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Chandra <sivachandra@google.com>
2018-02-01 15:11:29 +00:00
Samir Jindel 7d00b79477 [kernel] Support native extensions in the VM through Kernel.
Change-Id: I860e66b3e66a882ff771e477c318362cefbd4eaa
Reviewed-on: https://dart-review.googlesource.com/35925
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-01-29 14:04:35 +00:00