[vm] Rework awaiter stack unwinding.

The main contribution of this CL is unification of disparate
handling of various functions like `Future.timeout`,
`Future.wait`, `_SuspendState.createAsyncCallbacks` and
`_SuspendState._createAsyncStarCallback` into a single
`@pragma('vm:awaiter-link')` which allows Dart developers
to specify where awaiter unwinder should look for the next
awaiter.

For example this allows unwinding to succeed for the code like this:

    Future<int> outer(Future<int> inner) {
      @pragma('vm:awaiter-link')
      final completer = Completer<int>();

      inner.then((v) => completer.complete(v));

      return completer.future;
   }

This refactoring also ensures that we preserve information
(including Function & Code objects) required for awaiter
unwinding across all modes (JIT, AOT and AOT with DWARF stack
traces). This guarantees users will get the same information
no matter which mode they are running in. Previously
we have been disabling awaiter_stacks tests in some AOT
modes - which led to regressions in the quality of produced
stacks.

This CL also cleans up relationship between debugger and awaiter
stack returned by StackTrace.current - which makes stack trace
displayed by debugger (used for stepping out and determinining
whether exception is caught or not) and `StackTrace.current`
consistent.

Finally we make one user visible change to the stack trace:
awaiter stack will no always include intermediate listeners
created through `Future.then`. Previously we would sometimes
include these listeners at the tail of the stack trace,
which was inconsistent.

Ultimately this means that code like this:

    Future<int> inner() async {
      await null;  // asynchronous gap
      print(StackTrace.current); // (*)
      return 0;
    }

    Future<int> outer() async {
      int process(int v) {
        return v + 1;
      }

      return await inner().then(process);
    }

    void main() async {
      await outer();
    }

Produces stack trace like this:

    inner
    <asynchronous suspension>
    outer.process
    <asynchronous suspension>
    outer
    <asynchronous suspension>
    main
    <asynchronous suspension>

And when stepping out of `inner` execution will stop at `outer.process`
first and the next step out will bring execution to `outer` next.

Fixes https://github.com/dart-lang/sdk/issues/52797
Fixes https://github.com/dart-lang/sdk/issues/52203
Issue https://github.com/dart-lang/sdk/issues/47985

TEST=ci

Bug: b/279929839
CoreLibraryReviewExempt: CL just adds @pragma to facilitate unwinding
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-product-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-dwarf-linux-product-x64-try
Change-Id: If377d5329d6a11c86effb9369dc603a7ae616fe7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311680
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>

pkg/native_stack_traces/lib/src/dwarf.dart

@@ -816,7 +816,11 @@ class DebugInformationEntry {
       DartCallInfo(
           function: unit.nameOfOrigin(abstractOrigin ?? -1),
           inlined: inlined,
-          internal: isArtificial,
+          // Don't hide artificial (invisible) methods which appear as
+          // Future listeners. This is because tear-offs of static methods
+          // are marked as invisible (even if the method itself is visible)
+          // and we want these to appear in stack traces.
+          internal: isArtificial && address != lowPC,
           filename: filename,
           line: line,
           column: column)

pkg/vm_service/test/get_stack_test.dart

@@ -89,12 +89,11 @@ final tests = <IsolateTest>[
   (VmService service, IsolateRef isolateRef) async {
     final result = await service.getStack(isolateRef.id!);
 
-    expect(result.frames, hasLength(6));
+    expect(result.frames, hasLength(5));
     expect(result.asyncCausalFrames, hasLength(26));
 
     expectFrames(result.frames!, [
       [equals('Regular'), endsWith(' func10')],
-      [equals('Regular'), endsWith(' _RootZone.runUnary')],
       [equals('Regular'), anything], // Internal mech. ..
       [equals('Regular'), anything],
       [equals('Regular'), anything],

runtime/docs/awaiter_stack_traces.md

@@ -0,0 +1,127 @@
+# Awaiter Stack Traces
+
+One of the common challenges associated with debugging asynchronous code is that stack traces do not reference the code which led to the exception. The context is lost when execution cross asynchronous gap.
+
+Consider the following code:
+
+```dart
+Future<int> inner() async {
+  await null;  // asynchronous gap
+  print(StackTrace.current); // (*)
+  return 0;
+}
+
+Future<int> outer() async {
+  int process(int v) {
+    return v + 1;
+  }
+
+  return await inner().then(process);
+}
+
+void main() async {
+  await outer();
+}
+```
+
+Producing synchronous stack trace at the line marked `(*)` will yield the following:
+
+```
+#0 inner
+#1 _SuspendState._createAsyncCallbacks.thenCallback
+#2 _RootZone.runUnary
+#3 _SuspendState._awaitNotFuture.run
+#4 _microtaskLoop
+#5 _startMicrotaskLoop
+#6 _startMicrotaskLoop
+#7 _runPendingImmediateCallback
+#8 _RawReceivePort._handleMessage
+```
+
+Only a single frame corresponds to user code (`#0  inner`) and the rest are `dart:async` internals. Nothing in this stack trace mentions `outer` or `main`, which called `inner`. This makes diagnosing issues based on a stack trace much harder.
+
+To address this problem runtime system augments synchronous portion of the stack trace with an _awaiter stack trace_. Each awaiter frame represents a closure or a suspended `async` function which will be invoked when the currently running asynchronous computation completes.
+
+This support allows runtime system to produce the following output for the example given above:
+
+```
+#0  inner
+<asynchronous suspension>
+#1  outer.process
+<asynchronous suspension>
+#2  outer
+<asynchronous suspension>
+#3  main
+<asynchronous suspension>
+```
+
+## Algorithm
+
+To recover awaiter stack trace runtime follows through a chain of `_Future`, `_StreamController` and `SuspendState` objects. The following diagram illustrates the path it takes to produce asynchronous stack trace in our initial example:
+
+![Heap structure used for unwinding](awaiter_stack_unwinding.png)
+
+Each awaiter frame is a pair of `(closure, nextFrame)`:
+
+* `closure` is a listener which will be invoked when the future this frame is waiting on completes.
+  * This might be one of the callbacks associated with [suspendable functions](async.md) internals, e.g. `_SuspendState.thenCallback` which resumes execution after the `await`.
+* `next` is an object representing the next awaiter frame, which is waiting for the completion of this awaiter frame.
+
+Unwinding rules can be summarised as follows:
+
+* If at any point `closure` has a captured variable marked with `@pragma('vm:awaiter-link')` variable then the value of that variable will be used as `nextFrame`.
+* If `nextFrame` is a `_SuspendState` then `_SuspendState.function_data` gives us `_FutureImpl` or `_AsyncStarStreamController` to look at.
+* If `nextFrame` is `_FutureImpl` then we can take the first `_FutureListener` in `listeners` and then the next frame is `(listener.callback, listener.result)`.
+* If `nextFrame` is `_AsyncStarStreamController` then we get  `asyncStarStreamController.controller.subscription._onData`, which should give us an instance of `_StreamIterator`, which inside contains a `_FutureImpl` on which `await for` is waiting.
+
+Awaiter unwinding is implemented in by [`dart::StackTraceUtils::CollectFrames`] in [`runtime/vm/stack_trace.cc`].
+
+### `@pragma('vm:awaiter-link')`
+
+Dart code which does not use `async`/`async*` functions and instead uses callbacks and lower-level primitives can integrate with awaiter frame unwinding by annotating variables which link to the next awaiter frame with `@pragma('vm:awaiter-link')`.
+
+Consider the following variation of the example:
+
+```dart
+Future<int> outer() {
+  final completer = Completer<int>();
+
+  int process(int v) {
+    completer.complete(v);
+  }
+
+  inner().then(v);
+
+  return completer.future;
+}
+```
+
+Running this would produce the following stack trace:
+
+```
+#0 inner
+<asynchronous suspension>
+#1 outer.process
+<asynchronous suspension>
+```
+
+Runtime is unable to unwind the awaiter stack past `process`. However if `completer` is annotated with `@pragma('vm:awaiter-link')` then unwinder will know where to continue:
+
+```dart
+Future<int> outer() {
+  @pragma('vm:awaiter-link')
+  final completer = Completer<int>();
+  // ...
+}
+```
+
+```
+#0      inner
+<asynchronous suspension>
+#1      outer.process
+<asynchronous suspension>
+#2      main
+<asynchronous suspension>
+```
+
+`vm:awaiter-link` can be used in `dart:async` internals to avoid hardcoding recognition of specific methods into the runtime system, see for example `_SuspendState.thenCallback` or `Future.timeout` implementations.

runtime/docs/pragmas.md

@@ -17,6 +17,7 @@ These pragmas are part of the VM's API and are safe for use in external code.
 | `vm:platform-const` | Marks a static getter or a static field with an initializer where the getter body or field initializer evaluates to a constant value if the target operating system is known. |
 | `weak-tearoff-reference` | [Declaring a static weak reference intrinsic method.](compiler/pragmas_recognized_by_compiler.md#declaring-a-static-weak-reference-intrinsic-method) |
 | `vm:isolate-unsendable` | Marks a class, instances of which won't be allowed to be passed through ports or sent between isolates. |
+| `vm:awaiter-link` | [Specifying variable to follow for awaiter stack unwinding](awaiter_stack_traces.md) |
 
 ## Unsafe pragmas for general use
 

runtime/lib/stacktrace.cc

@@ -41,11 +41,14 @@ static StackTracePtr CurrentStackTrace(Thread* thread,
 
   const auto& code_array = GrowableObjectArray::ZoneHandle(
       zone, GrowableObjectArray::New(kDefaultStackAllocation));
-  GrowableArray<uword> pc_offset_array;
+  GrowableArray<uword> pc_offset_array(kDefaultStackAllocation);
 
   // Collect the frames.
-  StackTraceUtils::CollectFrames(thread, code_array, &pc_offset_array,
+  StackTraceUtils::CollectFrames(thread, skip_frames,
-                                 skip_frames);
+                                 [&](const StackTraceUtils::Frame& frame) {
+                                   code_array.Add(frame.code);
+                                   pc_offset_array.Add(frame.pc_offset);
+                                 });
 
   return CreateStackTraceObject(zone, code_array, pc_offset_array);
 }

runtime/observatory/tests/service/async_step_out_test.dart

@@ -14,13 +14,15 @@ import 'test_helper.dart';
 //
 // dart runtime/observatory/tests/service/update_line_numbers.dart <test.dart>
 //
-const int LINE_A = 27;
+const int LINE_A = 29;
-const int LINE_B = 28;
+const int LINE_B = 30;
-const int LINE_C = 29;
+const int LINE_C = 31;
-const int LINE_0 = 38;
+const int LINE_G = 36;
-const int LINE_D = 39;
+const int LINE_0 = 40;
-const int LINE_E = 40;
+const int LINE_D = 41;
-const int LINE_F = 41;
+const int LINE_E = 42;
+const int LINE_F = 43;
+const int LINE_H = 44;
 // AUTOGENERATED END
 
 Future<int> helper() async {
@@ -31,14 +33,15 @@ Future<int> helper() async {
 }
 
 Future<void> testMain() async {
-  int process(int value) {
+  int process(int value) /* LINE_G */ {
     return value + 1;
   }
 
   debugger(); // LINE_0.
   print('line d'); // LINE_D.
   await helper().then(process); // LINE_E.
-  print('line f'); // LINE_F.
+  final v = process(10); // LINE_F.
+  print('line h'); // LINE_H.
 }
 
 var tests = <IsolateTest>[
@@ -66,8 +69,16 @@ var tests = <IsolateTest>[
   stoppedAtLine(LINE_C),
   stepOut, // out of helper to awaiter testMain.
 
+  hasStoppedAtBreakpoint,
+  stoppedAtLine(LINE_G),
+  stepOut, // out of helper to awaiter testMain.
+
   hasStoppedAtBreakpoint,
   stoppedAtLine(LINE_F),
+  stepOver,
+
+  hasStoppedAtBreakpoint,
+  stoppedAtLine(LINE_H),
 ];
 
 main(args) => runIsolateTests(args, tests,

runtime/observatory/tests/service/get_stack_limit_rpc_test.dart

@@ -53,10 +53,18 @@ var tests = <IsolateTest>[
     expect(asyncFrames.length, greaterThan(frames.length));
     expect(stack['truncated'], false);
     verifyStack(frames, [
-      'bar', 'foo', 'bar', 'foo',
+      'bar',
-      'bar', 'foo', 'bar', 'foo',
+      'foo',
-      'bar', 'foo', 'bar', 'foo',
+      'bar',
-      '_RootZone.runUnary', // Internal async. mech. ..
+      'foo',
+      'bar',
+      'foo',
+      'bar',
+      'foo',
+      'bar',
+      'foo',
+      'bar',
+      'foo'
     ]);
 
     final fullStackLength = frames.length;
@@ -71,10 +79,18 @@ var tests = <IsolateTest>[
     expect(asyncFrames.length, fullStackLength + 1);
     expect(stack['truncated'], true);
     verifyStack(frames, [
-      'bar', 'foo', 'bar', 'foo',
+      'bar',
-      'bar', 'foo', 'bar', 'foo',
+      'foo',
-      'bar', 'foo', 'bar', 'foo',
+      'bar',
-      '_RootZone.runUnary', // Internal async. mech. ..
+      'foo',
+      'bar',
+      'foo',
+      'bar',
+      'foo',
+      'bar',
+      'foo',
+      'bar',
+      'foo'
     ]);
 
     // Try a limit < actual stack depth and expect to get a stack of depth

runtime/observatory_2/tests/service_2/async_step_out_test.dart

@@ -4,6 +4,8 @@
 //
 // VMOptions=--async-debugger --verbose-debug
 
+// @dart=2.9
+
 import 'dart:developer';
 import 'service_test_common.dart';
 import 'test_helper.dart';
@@ -14,30 +16,34 @@ import 'test_helper.dart';
 //
 // dart runtime/observatory/tests/service/update_line_numbers.dart <test.dart>
 //
-const int LINE_A = 27;
+const int LINE_A = 31;
-const int LINE_B = 28;
+const int LINE_B = 32;
-const int LINE_C = 29;
+const int LINE_C = 33;
-const int LINE_0 = 37;
+const int LINE_G = 38;
-const int LINE_D = 38;
+const int LINE_0 = 42;
-const int LINE_E = 39;
+const int LINE_D = 43;
-const int LINE_F = 40;
+const int LINE_E = 44;
+const int LINE_F = 45;
+const int LINE_H = 46;
 // AUTOGENERATED END
 
-helper() async {
+Future<int> helper() async {
   await null; // LINE_A.
   print('line b'); // LINE_B.
   print('line c'); // LINE_C.
+  return 0;
 }
 
-testMain() async {
+Future<void> testMain() async {
-  int process(int value) {
+  int process(int value) /* LINE_G */ {
     return value + 1;
   }
 
   debugger(); // LINE_0.
   print('line d'); // LINE_D.
   await helper().then(process); // LINE_E.
-  print('line f'); // LINE_F.
+  final v = process(10); // LINE_F.
+  print('line h'); // LINE_H.
 }
 
 var tests = <IsolateTest>[
@@ -65,8 +71,16 @@ var tests = <IsolateTest>[
   stoppedAtLine(LINE_C),
   stepOut, // out of helper to awaiter testMain.
 
+  hasStoppedAtBreakpoint,
+  stoppedAtLine(LINE_G),
+  stepOut, // out of helper to awaiter testMain.
+
   hasStoppedAtBreakpoint,
   stoppedAtLine(LINE_F),
+  stepOver,
+
+  hasStoppedAtBreakpoint,
+  stoppedAtLine(LINE_H),
 ];
 
 main(args) => runIsolateTests(args, tests,

runtime/observatory_2/tests/service_2/get_stack_limit_rpc_test.dart

@@ -53,10 +53,18 @@ var tests = <IsolateTest>[
     expect(asyncFrames.length, greaterThan(frames.length));
     expect(stack['truncated'], false);
     verifyStack(frames, [
-      'bar', 'foo', 'bar', 'foo',
+      'bar',
-      'bar', 'foo', 'bar', 'foo',
+      'foo',
-      'bar', 'foo', 'bar', 'foo',
+      'bar',
-      '_RootZone.runUnary', // Internal async. mech. ..
+      'foo',
+      'bar',
+      'foo',
+      'bar',
+      'foo',
+      'bar',
+      'foo',
+      'bar',
+      'foo'
     ]);
 
     final fullStackLength = frames.length;
@@ -71,10 +79,18 @@ var tests = <IsolateTest>[
     expect(asyncFrames.length, fullStackLength + 1);
     expect(stack['truncated'], true);
     verifyStack(frames, [
-      'bar', 'foo', 'bar', 'foo',
+      'bar',
-      'bar', 'foo', 'bar', 'foo',
+      'foo',
-      'bar', 'foo', 'bar', 'foo',
+      'bar',
-      '_RootZone.runUnary', // Internal async. mech. ..
+      'foo',
+      'bar',
+      'foo',
+      'bar',
+      'foo',
+      'bar',
+      'foo',
+      'bar',
+      'foo'
     ]);
 
     // Try a limit < actual stack depth and expect to get a stack of depth

runtime/tests/vm/dart/awaiter_stacks/async_stacks_test.dart

@@ -280,11 +280,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #3    allYield (%test%)
 <asynchronous suspension>
-#4    doTestAwaitThen (%test%)
+#4    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#5    runTest (harness.dart)
+#5    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#6    main (%test%)
+#6    runTest (harness.dart)
+<asynchronous suspension>
+#7    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwSync (%test%)
@@ -350,11 +352,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #2    mixedYields (%test%)
 <asynchronous suspension>
-#3    doTestAwaitThen (%test%)
+#3    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#4    runTest (harness.dart)
+#4    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#5    main (%test%)
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwAsync (%test%)
@@ -389,11 +393,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #2    syncSuffix (%test%)
 <asynchronous suspension>
-#3    doTestAwaitThen (%test%)
+#3    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#4    runTest (harness.dart)
+#4    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#5    main (%test%)
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwAsync (%test%)
@@ -428,11 +434,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #2    nonAsyncNoStack (%test%)
 <asynchronous suspension>
-#3    doTestAwaitThen (%test%)
+#3    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#4    runTest (harness.dart)
+#4    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#5    main (%test%)
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwAsync (%test%)
@@ -465,11 +473,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #2    awaitEveryAsyncStarThrowSync (%test%)
 <asynchronous suspension>
-#3    doTestAwaitThen (%test%)
+#3    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#4    runTest (harness.dart)
+#4    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#5    main (%test%)
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwSync (%test%)
@@ -503,11 +513,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #2    awaitEveryAsyncStarThrowAsync (%test%)
 <asynchronous suspension>
-#3    doTestAwaitThen (%test%)
+#3    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#4    runTest (harness.dart)
+#4    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#5    main (%test%)
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwAsync (%test%)
@@ -593,11 +605,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #2    awaitTimeout (%test%)
 <asynchronous suspension>
-#3    doTestAwaitThen (%test%)
+#3    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#4    runTest (harness.dart)
+#4    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#5    main (%test%)
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwAsync (%test%)
@@ -632,11 +646,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #2    awaitWait (%test%)
 <asynchronous suspension>
-#3    doTestAwaitThen (%test%)
+#3    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#4    runTest (harness.dart)
+#4    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#5    main (%test%)
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwAsync (%test%)
@@ -654,33 +670,7 @@ final currentExpectations = [
   """
 #0    throwAsync (%test%)
 <asynchronous suspension>
-#1    doTestAwait (%test%)
+#1    futureSyncWhenComplete.<anonymous closure> (%test%)
-<asynchronous suspension>
-#2    runTest (harness.dart)
-<asynchronous suspension>
-#3    main (%test%)
-<asynchronous suspension>""",
-  """
-#0    throwAsync (%test%)
-<asynchronous suspension>
-#1    doTestAwaitThen (%test%)
-<asynchronous suspension>
-#2    runTest (harness.dart)
-<asynchronous suspension>
-#3    main (%test%)
-<asynchronous suspension>""",
-  """
-#0    throwAsync (%test%)
-<asynchronous suspension>
-#1    doTestAwaitCatchError (%test%)
-<asynchronous suspension>
-#2    runTest (harness.dart)
-<asynchronous suspension>
-#3    main (%test%)
-<asynchronous suspension>""",
-  """
-#0    throwSync (%test%)
-#1    futureThen.<anonymous closure> (%test%)
 <asynchronous suspension>
 #2    doTestAwait (%test%)
 <asynchronous suspension>
@@ -689,10 +679,24 @@ final currentExpectations = [
 #4    main (%test%)
 <asynchronous suspension>""",
   """
-#0    throwSync (%test%)
+#0    throwAsync (%test%)
-#1    futureThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#2    doTestAwaitThen (%test%)
+#1    futureSyncWhenComplete.<anonymous closure> (%test%)
+<asynchronous suspension>
+#2    doTestAwaitThen.<anonymous closure> (%test%)
+<asynchronous suspension>
+#3    doTestAwaitThen (%test%)
+<asynchronous suspension>
+#4    runTest (harness.dart)
+<asynchronous suspension>
+#5    main (%test%)
+<asynchronous suspension>""",
+  """
+#0    throwAsync (%test%)
+<asynchronous suspension>
+#1    futureSyncWhenComplete.<anonymous closure> (%test%)
+<asynchronous suspension>
+#2    doTestAwaitCatchError (%test%)
 <asynchronous suspension>
 #3    runTest (harness.dart)
 <asynchronous suspension>
@@ -702,11 +706,39 @@ final currentExpectations = [
 #0    throwSync (%test%)
 #1    futureThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#2    doTestAwaitCatchError (%test%)
+#2    _doSomething (%test%)
 <asynchronous suspension>
-#3    runTest (harness.dart)
+#3    doTestAwait (%test%)
 <asynchronous suspension>
-#4    main (%test%)
+#4    runTest (harness.dart)
+<asynchronous suspension>
+#5    main (%test%)
+<asynchronous suspension>""",
+  """
+#0    throwSync (%test%)
+#1    futureThen.<anonymous closure> (%test%)
+<asynchronous suspension>
+#2    _doSomething (%test%)
+<asynchronous suspension>
+#3    doTestAwaitThen.<anonymous closure> (%test%)
+<asynchronous suspension>
+#4    doTestAwaitThen (%test%)
+<asynchronous suspension>
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
+<asynchronous suspension>""",
+  """
+#0    throwSync (%test%)
+#1    futureThen.<anonymous closure> (%test%)
+<asynchronous suspension>
+#2    _doSomething (%test%)
+<asynchronous suspension>
+#3    doTestAwaitCatchError (%test%)
+<asynchronous suspension>
+#4    runTest (harness.dart)
+<asynchronous suspension>
+#5    main (%test%)
 <asynchronous suspension>"""
 ];
 // CURRENT EXPECTATIONS END

runtime/tests/vm/dart/awaiter_stacks/harness.dart

@@ -120,17 +120,15 @@ late final Dwarf? _dwarf;
 
 void configure(List<String> currentExpectations,
     {String debugInfoFilename = 'debug.so'}) {
-  if (debugInfoFilename != null) {
+  try {
-    try {
+    final testCompilationDir = Platform.environment['TEST_COMPILATION_DIR'];
-      final testCompilationDir = Platform.environment['TEST_COMPILATION_DIR'];
+    if (testCompilationDir != null) {
-      if (testCompilationDir != null) {
+      debugInfoFilename = path.join(testCompilationDir, debugInfoFilename);
-        debugInfoFilename = path.join(testCompilationDir, debugInfoFilename);
-      }
-      _dwarf = Dwarf.fromFile(debugInfoFilename)!;
-    } on FileSystemException {
-      // We're not running in precompiled mode, so the file doesn't exist and
-      // we can continue normally.
     }
+    _dwarf = Dwarf.fromFile(debugInfoFilename)!;
+  } on FileSystemException {
+    // We're not running in precompiled mode, so the file doesn't exist and
+    // we can continue normally.
   }
   _currentExpectations = currentExpectations;
 }
@@ -232,5 +230,10 @@ final currentExpectations = [${updatedExpectationsString}];
 // then we don't have a way to deobfuscate the stack trace.
 bool shouldSkip() {
   final stack = StackTrace.current.toString();
-  return !stack.contains('shouldSkip') && !stack.contains('*** ***');
+  final isObfuscateMode = !stack.contains('shouldSkip');
+  final isDwarfStackTracesMode = stack.contains('*** ***');
+
+  // We should skip the test if we are running without DWARF stack
+  // traces enabled but with obfuscation.
+  return !isDwarfStackTracesMode && isObfuscateMode;
 }

runtime/tests/vm/dart/awaiter_stacks/zone_callback_stack_traces_test.dart

@@ -8,6 +8,15 @@
 // VMOptions=--save-debugging-info=$TEST_COMPILATION_DIR/debug.so
 // VMOptions=--dwarf-stack-traces --save-debugging-info=$TEST_COMPILATION_DIR/debug.so
 
+// This test check that awaiter stack unwinding can produce useful and readable
+// stack traces when unwinding through custom Zone which use
+// [Zone.registerUnaryCallback] and [Zone.registerBinaryCallback] hooks when
+// corresponding hooks are properly annotated with `@pragma('vm:awaiter-link')`.
+//
+// `package:stack_trace` which is heavily used in the Dart ecosystem is heavily
+// reliant on these hooks and we want to make sure that native awaiter stack
+// unwinding works correctly even within `package:stack_trace` zones.
+
 import 'dart:async';
 
 import 'package:expect/expect.dart';
@@ -16,50 +25,43 @@ import 'harness.dart' as harness;
 
 bool barRunning = false;
 
-Future<void> foo() async {}
+Future<void> foo() async {
+  await null;
+  stacktraces.add(StackTrace.current);
+}
 
 Future<void> bar() async {
-  try {
+  await foo();
-    barRunning = true;
+  stacktraces.add(StackTrace.current);
-    await foo();
-  } finally {
-    barRunning = false;
-  }
 }
 
 Future<void> runTest() {
   final Zone testZone = Zone.current.fork(
       specification: ZoneSpecification(
-          registerUnaryCallback: _registerUnaryCallback,
+    registerUnaryCallback: _registerUnaryCallback,
-          registerBinaryCallback: _registerBinaryCallback));
+    registerBinaryCallback: _registerBinaryCallback,
+  ));
   return testZone.run(bar);
 }
 
-StackTrace? registerUnaryCallbackStackTrace;
+final stacktraces = <StackTrace>[];
-StackTrace? registerBinaryCallbackStackTrace;
 
 ZoneUnaryCallback<R, T> _registerUnaryCallback<R, T>(
-    Zone self, ZoneDelegate parent, Zone zone, R Function(T) f) {
+    Zone self,
-  final stackTrace = StackTrace.current;
+    ZoneDelegate parent,
-  print('registerUnaryCallback got stack trace:');
+    Zone zone,
-  print(stackTrace);
+    @pragma('vm:awaiter-link') R Function(T) f) {
-  if (barRunning) {
+  stacktraces.add(StackTrace.current);
-    Expect.isNull(registerUnaryCallbackStackTrace);
+  return parent.registerUnaryCallback(zone, (v) => f(v));
-    registerUnaryCallbackStackTrace = stackTrace;
-  }
-  return parent.registerUnaryCallback(zone, f);
 }
 
 ZoneBinaryCallback<R, T1, T2> _registerBinaryCallback<R, T1, T2>(
-    Zone self, ZoneDelegate parent, Zone zone, R Function(T1, T2) f) {
+    Zone self,
-  final stackTrace = StackTrace.current;
+    ZoneDelegate parent,
-  print('registerBinaryCallback got stack trace:');
+    Zone zone,
-  print(stackTrace);
+    @pragma('vm:awaiter-link') R Function(T1, T2) f) {
-  if (barRunning) {
+  stacktraces.add(StackTrace.current);
-    Expect.isNull(registerBinaryCallbackStackTrace);
+  return parent.registerBinaryCallback(zone, (a, b) => f(a, b));
-    registerBinaryCallbackStackTrace = stackTrace;
-  }
-  return parent.registerBinaryCallback(zone, f);
 }
 
 Future<void> main() async {
@@ -70,8 +72,10 @@ Future<void> main() async {
   harness.configure(currentExpectations);
 
   await runTest();
-  await harness.checkExpectedStack(registerUnaryCallbackStackTrace!);
+  for (var st in stacktraces) {
-  await harness.checkExpectedStack(registerBinaryCallbackStackTrace!);
+    await harness.checkExpectedStack(st);
+  }
+  Expect.equals(6, stacktraces.length);
 
   harness.updateExpectations();
 }
@@ -81,6 +85,28 @@ final currentExpectations = [
   """
 #0    _registerUnaryCallback (%test%)
 #1    _CustomZone.registerUnaryCallback (zone.dart)
+#2    foo (%test%)
+#3    bar (%test%)
+#4    _rootRun (zone.dart)
+#5    _CustomZone.run (zone.dart)
+#6    runTest (%test%)
+#7    main (%test%)
+#8    _delayEntrypointInvocation.<anonymous closure> (isolate_patch.dart)
+#9    _RawReceivePort._handleMessage (isolate_patch.dart)""",
+  """
+#0    _registerBinaryCallback (%test%)
+#1    _CustomZone.registerBinaryCallback (zone.dart)
+#2    foo (%test%)
+#3    bar (%test%)
+#4    _rootRun (zone.dart)
+#5    _CustomZone.run (zone.dart)
+#6    runTest (%test%)
+#7    main (%test%)
+#8    _delayEntrypointInvocation.<anonymous closure> (isolate_patch.dart)
+#9    _RawReceivePort._handleMessage (isolate_patch.dart)""",
+  """
+#0    _registerUnaryCallback (%test%)
+#1    _CustomZone.registerUnaryCallback (zone.dart)
 #2    bar (%test%)
 #3    _rootRun (zone.dart)
 #4    _CustomZone.run (zone.dart)
@@ -97,6 +123,18 @@ final currentExpectations = [
 #5    runTest (%test%)
 #6    main (%test%)
 #7    _delayEntrypointInvocation.<anonymous closure> (isolate_patch.dart)
-#8    _RawReceivePort._handleMessage (isolate_patch.dart)"""
+#8    _RawReceivePort._handleMessage (isolate_patch.dart)""",
+  """
+#0    foo (%test%)
+<asynchronous suspension>
+#1    bar (%test%)
+<asynchronous suspension>
+#2    main (%test%)
+<asynchronous suspension>""",
+  """
+#0    bar (%test%)
+<asynchronous suspension>
+#1    main (%test%)
+<asynchronous suspension>"""
 ];
 // CURRENT EXPECTATIONS END

runtime/tests/vm/dart/checked_parameter_assert_assignable_stacktrace_test.dart

@@ -25,6 +25,10 @@ class B extends A {
 StackTrace? trace = null;
 
 void main() async {
+  if (harness.shouldSkip()) {
+    // Skip the test in this configuration.
+    return;
+  }
   harness.configure(currentExpectations);
 
   A a = new A();

runtime/tests/vm/dart/invisible_function_pragma_test.dart

@@ -1,10 +1,21 @@
 // Copyright (c) 2022, the Dart project authors.  Please see the AUTHORS file
 // for details. All rights reserved. Use of this source code is governed by a
 // BSD-style license that can be found in the LICENSE file.
+//
+// Note: we pass --save-debugging-info=* without --dwarf-stack-traces to
+// make this test pass on vm-aot-dwarf-* builders.
+//
+// VMOptions=--save-debugging-info=$TEST_COMPILATION_DIR/debug.so
+// VMOptions=--dwarf-stack-traces --save-debugging-info=$TEST_COMPILATION_DIR/debug.so
 
 import 'awaiter_stacks/harness.dart' as harness;
 
 main() async {
+  if (harness.shouldSkip()) {
+    // Skip the test in this configuration.
+    return;
+  }
+
   harness.configure(currentExpectations);
 
   StackTrace trace = StackTrace.empty;
@@ -32,22 +43,31 @@ main() async {
 
 class A {
   A.visible(void Function() fun) {
+    print('A.visible');
     fun();
   }
 
   @pragma('vm:invisible')
   A.invisible(void Function() fun) {
+    print('A.invisible');
     fun();
   }
 }
 
-void visible(void Function() fun) => fun();
+void visible(void Function() fun) {
+  print('visible()');
+  fun();
+}
 
 @pragma('vm:invisible')
-void invisible(void Function() fun) => fun();
+void invisible(void Function() fun) {
+  print('invisible()');
+  fun();
+}
 
 void visibleClosure(void Function() fun) {
   visibleInner() {
+    print('visibleInner');
     fun();
   }
 
@@ -57,6 +77,7 @@ void visibleClosure(void Function() fun) {
 void invisibleClosure(void Function() fun) {
   @pragma('vm:invisible')
   invisibleInner() {
+    print('invisibleInner');
     fun();
   }
 

runtime/tests/vm/dart_2/awaiter_stacks/async_stacks_test.dart

@@ -282,11 +282,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #3    allYield (%test%)
 <asynchronous suspension>
-#4    doTestAwaitThen (%test%)
+#4    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#5    runTest (harness.dart)
+#5    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#6    main (%test%)
+#6    runTest (harness.dart)
+<asynchronous suspension>
+#7    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwSync (%test%)
@@ -352,11 +354,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #2    mixedYields (%test%)
 <asynchronous suspension>
-#3    doTestAwaitThen (%test%)
+#3    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#4    runTest (harness.dart)
+#4    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#5    main (%test%)
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwAsync (%test%)
@@ -391,11 +395,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #2    syncSuffix (%test%)
 <asynchronous suspension>
-#3    doTestAwaitThen (%test%)
+#3    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#4    runTest (harness.dart)
+#4    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#5    main (%test%)
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwAsync (%test%)
@@ -430,11 +436,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #2    nonAsyncNoStack (%test%)
 <asynchronous suspension>
-#3    doTestAwaitThen (%test%)
+#3    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#4    runTest (harness.dart)
+#4    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#5    main (%test%)
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwAsync (%test%)
@@ -467,11 +475,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #2    awaitEveryAsyncStarThrowSync (%test%)
 <asynchronous suspension>
-#3    doTestAwaitThen (%test%)
+#3    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#4    runTest (harness.dart)
+#4    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#5    main (%test%)
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwSync (%test%)
@@ -505,11 +515,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #2    awaitEveryAsyncStarThrowAsync (%test%)
 <asynchronous suspension>
-#3    doTestAwaitThen (%test%)
+#3    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#4    runTest (harness.dart)
+#4    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#5    main (%test%)
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwAsync (%test%)
@@ -595,11 +607,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #2    awaitTimeout (%test%)
 <asynchronous suspension>
-#3    doTestAwaitThen (%test%)
+#3    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#4    runTest (harness.dart)
+#4    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#5    main (%test%)
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwAsync (%test%)
@@ -634,11 +648,13 @@ final currentExpectations = [
 <asynchronous suspension>
 #2    awaitWait (%test%)
 <asynchronous suspension>
-#3    doTestAwaitThen (%test%)
+#3    doTestAwaitThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#4    runTest (harness.dart)
+#4    doTestAwaitThen (%test%)
 <asynchronous suspension>
-#5    main (%test%)
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
 <asynchronous suspension>""",
   """
 #0    throwAsync (%test%)
@@ -656,33 +672,7 @@ final currentExpectations = [
   """
 #0    throwAsync (%test%)
 <asynchronous suspension>
-#1    doTestAwait (%test%)
+#1    futureSyncWhenComplete.<anonymous closure> (%test%)
-<asynchronous suspension>
-#2    runTest (harness.dart)
-<asynchronous suspension>
-#3    main (%test%)
-<asynchronous suspension>""",
-  """
-#0    throwAsync (%test%)
-<asynchronous suspension>
-#1    doTestAwaitThen (%test%)
-<asynchronous suspension>
-#2    runTest (harness.dart)
-<asynchronous suspension>
-#3    main (%test%)
-<asynchronous suspension>""",
-  """
-#0    throwAsync (%test%)
-<asynchronous suspension>
-#1    doTestAwaitCatchError (%test%)
-<asynchronous suspension>
-#2    runTest (harness.dart)
-<asynchronous suspension>
-#3    main (%test%)
-<asynchronous suspension>""",
-  """
-#0    throwSync (%test%)
-#1    futureThen.<anonymous closure> (%test%)
 <asynchronous suspension>
 #2    doTestAwait (%test%)
 <asynchronous suspension>
@@ -691,10 +681,24 @@ final currentExpectations = [
 #4    main (%test%)
 <asynchronous suspension>""",
   """
-#0    throwSync (%test%)
+#0    throwAsync (%test%)
-#1    futureThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#2    doTestAwaitThen (%test%)
+#1    futureSyncWhenComplete.<anonymous closure> (%test%)
+<asynchronous suspension>
+#2    doTestAwaitThen.<anonymous closure> (%test%)
+<asynchronous suspension>
+#3    doTestAwaitThen (%test%)
+<asynchronous suspension>
+#4    runTest (harness.dart)
+<asynchronous suspension>
+#5    main (%test%)
+<asynchronous suspension>""",
+  """
+#0    throwAsync (%test%)
+<asynchronous suspension>
+#1    futureSyncWhenComplete.<anonymous closure> (%test%)
+<asynchronous suspension>
+#2    doTestAwaitCatchError (%test%)
 <asynchronous suspension>
 #3    runTest (harness.dart)
 <asynchronous suspension>
@@ -704,11 +708,39 @@ final currentExpectations = [
 #0    throwSync (%test%)
 #1    futureThen.<anonymous closure> (%test%)
 <asynchronous suspension>
-#2    doTestAwaitCatchError (%test%)
+#2    _doSomething (%test%)
 <asynchronous suspension>
-#3    runTest (harness.dart)
+#3    doTestAwait (%test%)
 <asynchronous suspension>
-#4    main (%test%)
+#4    runTest (harness.dart)
+<asynchronous suspension>
+#5    main (%test%)
+<asynchronous suspension>""",
+  """
+#0    throwSync (%test%)
+#1    futureThen.<anonymous closure> (%test%)
+<asynchronous suspension>
+#2    _doSomething (%test%)
+<asynchronous suspension>
+#3    doTestAwaitThen.<anonymous closure> (%test%)
+<asynchronous suspension>
+#4    doTestAwaitThen (%test%)
+<asynchronous suspension>
+#5    runTest (harness.dart)
+<asynchronous suspension>
+#6    main (%test%)
+<asynchronous suspension>""",
+  """
+#0    throwSync (%test%)
+#1    futureThen.<anonymous closure> (%test%)
+<asynchronous suspension>
+#2    _doSomething (%test%)
+<asynchronous suspension>
+#3    doTestAwaitCatchError (%test%)
+<asynchronous suspension>
+#4    runTest (harness.dart)
+<asynchronous suspension>
+#5    main (%test%)
 <asynchronous suspension>"""
 ];
 // CURRENT EXPECTATIONS END

runtime/tests/vm/dart_2/awaiter_stacks/harness.dart

@@ -233,5 +233,10 @@ final currentExpectations = [${updatedExpectationsString}];
 // then we don't have a way to deobfuscate the stack trace.
 bool shouldSkip() {
   final stack = StackTrace.current.toString();
-  return !stack.contains('shouldSkip') && !stack.contains('*** ***');
+  final isObfuscateMode = !stack.contains('shouldSkip');
+  final isDwarfStackTracesMode = stack.contains('*** ***');
+
+  // We should skip the test if we are running without DWARF stack
+  // traces enabled but with obfuscation.
+  return !isDwarfStackTracesMode && isObfuscateMode;
 }

runtime/tests/vm/dart_2/awaiter_stacks/zone_callback_stack_traces_test.dart

@@ -8,6 +8,15 @@
 // VMOptions=--save-debugging-info=$TEST_COMPILATION_DIR/debug.so
 // VMOptions=--dwarf-stack-traces --save-debugging-info=$TEST_COMPILATION_DIR/debug.so
 
+// This test check that awaiter stack unwinding can produce useful and readable
+// stack traces when unwinding through custom Zone which use
+// [Zone.registerUnaryCallback] and [Zone.registerBinaryCallback] hooks when
+// corresponding hooks are properly annotated with `@pragma('vm:awaiter-link')`.
+//
+// `package:stack_trace` which is heavily used in the Dart ecosystem is heavily
+// reliant on these hooks and we want to make sure that native awaiter stack
+// unwinding works correctly even within `package:stack_trace` zones.
+
 // @dart=2.9
 
 import 'dart:async';
@@ -18,50 +27,43 @@ import 'harness.dart' as harness;
 
 bool barRunning = false;
 
-Future<void> foo() async {}
+Future<void> foo() async {
+  await null;
+  stacktraces.add(StackTrace.current);
+}
 
 Future<void> bar() async {
-  try {
+  await foo();
-    barRunning = true;
+  stacktraces.add(StackTrace.current);
-    await foo();
-  } finally {
-    barRunning = false;
-  }
 }
 
 Future<void> runTest() {
   final Zone testZone = Zone.current.fork(
       specification: ZoneSpecification(
-          registerUnaryCallback: _registerUnaryCallback,
+    registerUnaryCallback: _registerUnaryCallback,
-          registerBinaryCallback: _registerBinaryCallback));
+    registerBinaryCallback: _registerBinaryCallback,
+  ));
   return testZone.run(bar);
 }
 
-StackTrace registerUnaryCallbackStackTrace;
+final stacktraces = <StackTrace>[];
-StackTrace registerBinaryCallbackStackTrace;
 
 ZoneUnaryCallback<R, T> _registerUnaryCallback<R, T>(
-    Zone self, ZoneDelegate parent, Zone zone, R Function(T) f) {
+    Zone self,
-  final stackTrace = StackTrace.current;
+    ZoneDelegate parent,
-  print('registerUnaryCallback got stack trace:');
+    Zone zone,
-  print(stackTrace);
+    @pragma('vm:awaiter-link') R Function(T) f) {
-  if (barRunning) {
+  stacktraces.add(StackTrace.current);
-    Expect.isNull(registerUnaryCallbackStackTrace);
+  return parent.registerUnaryCallback(zone, (v) => f(v));
-    registerUnaryCallbackStackTrace = stackTrace;
-  }
-  return parent.registerUnaryCallback(zone, f);
 }
 
 ZoneBinaryCallback<R, T1, T2> _registerBinaryCallback<R, T1, T2>(
-    Zone self, ZoneDelegate parent, Zone zone, R Function(T1, T2) f) {
+    Zone self,
-  final stackTrace = StackTrace.current;
+    ZoneDelegate parent,
-  print('registerBinaryCallback got stack trace:');
+    Zone zone,
-  print(stackTrace);
+    @pragma('vm:awaiter-link') R Function(T1, T2) f) {
-  if (barRunning) {
+  stacktraces.add(StackTrace.current);
-    Expect.isNull(registerBinaryCallbackStackTrace);
+  return parent.registerBinaryCallback(zone, (a, b) => f(a, b));
-    registerBinaryCallbackStackTrace = stackTrace;
-  }
-  return parent.registerBinaryCallback(zone, f);
 }
 
 Future<void> main() async {
@@ -72,8 +74,10 @@ Future<void> main() async {
   harness.configure(currentExpectations);
 
   await runTest();
-  await harness.checkExpectedStack(registerUnaryCallbackStackTrace);
+  for (var st in stacktraces) {
-  await harness.checkExpectedStack(registerBinaryCallbackStackTrace);
+    await harness.checkExpectedStack(st);
+  }
+  Expect.equals(6, stacktraces.length);
 
   harness.updateExpectations();
 }
@@ -83,6 +87,28 @@ final currentExpectations = [
   """
 #0    _registerUnaryCallback (%test%)
 #1    _CustomZone.registerUnaryCallback (zone.dart)
+#2    foo (%test%)
+#3    bar (%test%)
+#4    _rootRun (zone.dart)
+#5    _CustomZone.run (zone.dart)
+#6    runTest (%test%)
+#7    main (%test%)
+#8    _delayEntrypointInvocation.<anonymous closure> (isolate_patch.dart)
+#9    _RawReceivePort._handleMessage (isolate_patch.dart)""",
+  """
+#0    _registerBinaryCallback (%test%)
+#1    _CustomZone.registerBinaryCallback (zone.dart)
+#2    foo (%test%)
+#3    bar (%test%)
+#4    _rootRun (zone.dart)
+#5    _CustomZone.run (zone.dart)
+#6    runTest (%test%)
+#7    main (%test%)
+#8    _delayEntrypointInvocation.<anonymous closure> (isolate_patch.dart)
+#9    _RawReceivePort._handleMessage (isolate_patch.dart)""",
+  """
+#0    _registerUnaryCallback (%test%)
+#1    _CustomZone.registerUnaryCallback (zone.dart)
 #2    bar (%test%)
 #3    _rootRun (zone.dart)
 #4    _CustomZone.run (zone.dart)
@@ -99,6 +125,18 @@ final currentExpectations = [
 #5    runTest (%test%)
 #6    main (%test%)
 #7    _delayEntrypointInvocation.<anonymous closure> (isolate_patch.dart)
-#8    _RawReceivePort._handleMessage (isolate_patch.dart)"""
+#8    _RawReceivePort._handleMessage (isolate_patch.dart)""",
+  """
+#0    foo (%test%)
+<asynchronous suspension>
+#1    bar (%test%)
+<asynchronous suspension>
+#2    main (%test%)
+<asynchronous suspension>""",
+  """
+#0    bar (%test%)
+<asynchronous suspension>
+#1    main (%test%)
+<asynchronous suspension>"""
 ];
 // CURRENT EXPECTATIONS END

runtime/tests/vm/dart_2/checked_parameter_assert_assignable_stacktrace_test.dart

@@ -27,6 +27,10 @@ class B extends A {
 StackTrace trace = null;
 
 void main() async {
+  if (harness.shouldSkip()) {
+    // Skip the test in this configuration.
+    return;
+  }
   harness.configure(currentExpectations);
 
   A a = new A();

runtime/tests/vm/dart_2/invisible_function_pragma_test.dart

@@ -1,15 +1,27 @@
 // Copyright (c) 2022, the Dart project authors.  Please see the AUTHORS file
 // for details. All rights reserved. Use of this source code is governed by a
 // BSD-style license that can be found in the LICENSE file.
+//
+// Note: we pass --save-debugging-info=* without --dwarf-stack-traces to
+// make this test pass on vm-aot-dwarf-* builders.
+//
+// VMOptions=--save-debugging-info=$TEST_COMPILATION_DIR/debug.so
+// VMOptions=--dwarf-stack-traces --save-debugging-info=$TEST_COMPILATION_DIR/debug.so
 
 // @dart = 2.9
 
 import 'awaiter_stacks/harness.dart' as harness;
 
 main() async {
+  if (harness.shouldSkip()) {
+    // Skip the test in this configuration.
+    return;
+  }
+
   harness.configure(currentExpectations);
 
   StackTrace trace = StackTrace.empty;
+
   A.visible(() => trace = StackTrace.current);
   await harness.checkExpectedStack(trace);
 
@@ -33,22 +45,31 @@ main() async {
 
 class A {
   A.visible(void Function() fun) {
+    print('A.visible');
     fun();
   }
 
   @pragma('vm:invisible')
   A.invisible(void Function() fun) {
+    print('A.invisible');
     fun();
   }
 }
 
-void visible(void Function() fun) => fun();
+void visible(void Function() fun) {
+  print('visible()');
+  fun();
+}
 
 @pragma('vm:invisible')
-void invisible(void Function() fun) => fun();
+void invisible(void Function() fun) {
+  print('invisible()');
+  fun();
+}
 
 void visibleClosure(void Function() fun) {
   visibleInner() {
+    print('visibleInner');
     fun();
   }
 
@@ -58,6 +79,7 @@ void visibleClosure(void Function() fun) {
 void invisibleClosure(void Function() fun) {
   @pragma('vm:invisible')
   invisibleInner() {
+    print('invisibleInner');
     fun();
   }
 

runtime/tests/vm/vm.status

@@ -447,12 +447,6 @@ dart/run_appended_aot_snapshot_test: SkipByDesign # Tests the precompiled runtim
 dart_2/run_appended_aot_snapshot_test: SkipByDesign # Tests the precompiled runtime.
 
 [ $builder_tag == dwarf || $builder_tag == obfuscated ]
-dart/awaiter_stacks/async_throws_stack_lazy_test: SkipByDesign # Asserts exact stacktrace output.
-dart/awaiter_stacks/async_throws_stack_no_causal_test: SkipByDesign # Asserts exact stacktrace output.
-dart/awaiter_stacks/flutter_regress_100441_test: SkipByDesign # Asserts exact stacktrace output.
-dart/awaiter_stacks/sync_async_start_pkg_test_test: SkipByDesign # Asserts exact stacktrace output.
-dart/awaiter_stacks/zone_callback_stack_traces_test: SkipByDesign # Asserts exact stacktrace output.
-dart/checked_parameter_assert_assignable_stacktrace_test: SkipByDesign # Asserts exact stacktrace output.
 dart/error_messages_in_null_checks_test: SkipByDesign # Relies symbol names in stack traces
 dart/extension_names_test: SkipByDesign # Relies symbol names in stack traces
 dart/extension_unnamed_names_test: SkipByDesign # Relies symbol names in stack traces
@@ -461,12 +455,6 @@ dart/invisible_function_pragma_test: SkipByDesign # Relies symbol names in stack
 dart/optimized_stacktrace_line_and_column_test: SkipByDesign # Relies symbol names in stack traces
 dart/optimized_stacktrace_line_test: SkipByDesign # Relies symbol names in stack traces
 dart/stacktrace_mixin_application_test: SkipByDesign # Relies symbol names in stack traces
-dart_2/awaiter_stacks/async_throws_stack_lazy_test: SkipByDesign # Asserts exact stacktrace output.
-dart_2/awaiter_stacks/async_throws_stack_no_causal_test: SkipByDesign # Asserts exact stacktrace output.
-dart_2/awaiter_stacks/flutter_regress_100441_test: SkipByDesign # Asserts exact stacktrace output.
-dart_2/awaiter_stacks/sync_async_start_pkg_test_test: SkipByDesign # Asserts exact stacktrace output.
-dart_2/awaiter_stacks/zone_callback_stack_traces_test: SkipByDesign # Asserts exact stacktrace output.
-dart_2/checked_parameter_assert_assignable_stacktrace_test: SkipByDesign # Asserts exact stacktrace output.
 dart_2/error_messages_in_null_checks_test: SkipByDesign # Relies on uris / symbol names
 dart_2/extension_names_test: SkipByDesign # Relies on uris / symbol names
 dart_2/extension_unnamed_names_test: SkipByDesign # Relies on uris / symbol names

runtime/vm/app_snapshot.cc

@@ -1316,8 +1316,7 @@ class ClosureDataSerializationCluster : public SerializationCluster {
       }
       WriteCompressedField(data, parent_function);
       WriteCompressedField(data, closure);
-      s->WriteUnsigned(
+      s->WriteUnsigned(static_cast<uint32_t>(data->untag()->packed_fields_));
-          static_cast<intptr_t>(data->untag()->default_type_arguments_kind_));
     }
   }
 
@@ -1351,8 +1350,7 @@ class ClosureDataDeserializationCluster : public DeserializationCluster {
       }
       data->untag()->parent_function_ = static_cast<FunctionPtr>(d.ReadRef());
       data->untag()->closure_ = static_cast<ClosurePtr>(d.ReadRef());
-      data->untag()->default_type_arguments_kind_ =
+      data->untag()->packed_fields_ = d.ReadUnsigned<uint32_t>();
-          static_cast<ClosureData::DefaultTypeArgumentsKind>(d.ReadUnsigned());
     }
   }
 };

runtime/vm/compiler/aot/precompiler.cc

@@ -51,6 +51,7 @@
 #include "vm/regexp_parser.h"
 #include "vm/resolver.h"
 #include "vm/runtime_entry.h"
+#include "vm/stack_trace.h"
 #include "vm/symbols.h"
 #include "vm/tags.h"
 #include "vm/timeline.h"
@@ -122,8 +123,9 @@ struct RetainReasons : public AllStatic {
   static constexpr const char* kImplicitClosure = "implicit closure";
   // The object is a local closure.
   static constexpr const char* kLocalClosure = "local closure";
-  // The object is a sync or async function or in the parent chain of one.
+  // The object is needed for async stack unwinding.
-  static constexpr const char* kIsSyncAsyncFunction = "sync or async function";
+  static constexpr const char* kAsyncStackUnwinding =
+      "needed for async stack unwinding";
   // The object is the initializer for a static field.
   static constexpr const char* kStaticFieldInitializer =
       "static field initializer";
@@ -1135,15 +1137,6 @@ void Precompiler::AddTypesOf(const Function& function) {
     return;
   }
 
-  // Special case to allow walking of lazy async stacks to work.
-  // Should match parent checks in CallerClosureFinder::FindCaller.
-  if (parent_function.recognized_kind() == MethodRecognizer::kFutureTimeout ||
-      parent_function.recognized_kind() == MethodRecognizer::kFutureWait) {
-    AddRetainReason(parent_function, RetainReasons::kIsSyncAsyncFunction);
-    AddTypesOf(parent_function);
-    return;
-  }
-
   // We're not retaining the parent due to this function, so wrap it with
   // a weak serialization reference.
   const auto& data = ClosureData::CheckedHandle(Z, function.data());
@@ -1407,6 +1400,10 @@ const char* Precompiler::MustRetainFunction(const Function& function) {
     return "dynamic invocation forwarder";
   }
 
+  if (StackTraceUtils::IsNeededForAsyncAwareUnwinding(function)) {
+    return RetainReasons::kAsyncStackUnwinding;
+  }
+
   return nullptr;
 }
 
@@ -2207,6 +2204,9 @@ void Precompiler::DropFunctions() {
       // Dynamic resolution of entry points also checks for valid arguments.
       return AddRetainReason(sig, RetainReasons::kEntryPointPragmaSignature);
     }
+    if (StackTraceUtils::IsNeededForAsyncAwareUnwinding(function)) {
+      return AddRetainReason(sig, RetainReasons::kAsyncStackUnwinding);
+    }
     if (FLAG_trace_precompiler) {
       THR_Print("Clearing signature for function %s\n",
                 function.ToLibNamePrefixedQualifiedCString());
@@ -2920,6 +2920,7 @@ void Precompiler::DiscardCodeObjects() {
                        const FunctionSet& functions_called_dynamically)
         : zone_(zone),
           function_(Function::Handle(zone)),
+          parent_function_(Function::Handle(zone)),
           class_(Class::Handle(zone)),
           library_(Library::Handle(zone)),
           loading_unit_(LoadingUnit::Handle(zone)),
@@ -2975,9 +2976,8 @@ void Precompiler::DiscardCodeObjects() {
 
       function_ = code.function();
       if (functions_to_retain_.ContainsKey(function_)) {
-        // Retain Code objects corresponding to:
+        // Retain Code objects corresponding to native functions
-        // * async/async* closures (to construct async stacks).
+        // (to find native implementation).
-        // * native functions (to find native implementation).
         if (function_.is_native()) {
           ++codes_with_native_function_;
           return;
@@ -2989,6 +2989,11 @@ void Precompiler::DiscardCodeObjects() {
           ++codes_with_dynamically_called_function_;
           return;
         }
+
+        if (StackTraceUtils::IsNeededForAsyncAwareUnwinding(function_)) {
+          ++codes_with_function_needed_for_async_unwinding_;
+          return;
+        }
       } else {
         ASSERT(!functions_called_dynamically_.ContainsKey(function_));
       }
@@ -3011,6 +3016,10 @@ void Precompiler::DiscardCodeObjects() {
       }
 
       code.set_is_discarded(true);
+      if (FLAG_trace_precompiler) {
+        THR_Print("Discarding code object corresponding to %s\n",
+                  function_.ToFullyQualifiedCString());
+      }
       ++discarded_codes_;
     }
 
@@ -3037,6 +3046,8 @@ void Precompiler::DiscardCodeObjects() {
                 codes_with_native_function_);
       THR_Print("    %8" Pd " Codes with dynamically called functions\n",
                 codes_with_dynamically_called_function_);
+      THR_Print("    %8" Pd " Codes with async unwinding related functions\n",
+                codes_with_function_needed_for_async_unwinding_);
       THR_Print("    %8" Pd " Codes with deferred functions\n",
                 codes_with_deferred_function_);
       THR_Print("    %8" Pd " Codes with ffi trampoline functions\n",
@@ -3050,6 +3061,7 @@ void Precompiler::DiscardCodeObjects() {
    private:
     Zone* zone_;
     Function& function_;
+    Function& parent_function_;
     Class& class_;
     Library& library_;
     LoadingUnit& loading_unit_;
@@ -3067,6 +3079,7 @@ void Precompiler::DiscardCodeObjects() {
     intptr_t codes_with_pc_descriptors_ = 0;
     intptr_t codes_with_native_function_ = 0;
     intptr_t codes_with_dynamically_called_function_ = 0;
+    intptr_t codes_with_function_needed_for_async_unwinding_ = 0;
     intptr_t codes_with_deferred_function_ = 0;
     intptr_t codes_with_ffi_trampoline_function_ = 0;
     intptr_t codes_used_as_call_targets_ = 0;

runtime/vm/compiler/backend/slot.h

@@ -175,8 +175,7 @@ NONNULLABLE_BOXED_NATIVE_SLOTS_LIST(FOR_EACH_NATIVE_SLOT)
 #define UNBOXED_NATIVE_SLOTS_LIST(V)                                           \
   AOT_ONLY_UNBOXED_NATIVE_SLOTS_LIST(V)                                        \
   V(AbstractType, UntaggedAbstractType, flags, Uint32, FINAL)                  \
-  V(ClosureData, UntaggedClosureData, default_type_arguments_kind, Uint8,      \
+  V(ClosureData, UntaggedClosureData, packed_fields, Uint32, FINAL)            \
-    FINAL)                                                                     \
   V(FinalizerBase, UntaggedFinalizerBase, isolate, IntPtr, VAR)                \
   V(FinalizerEntry, UntaggedFinalizerEntry, external_size, IntPtr, VAR)        \
   V(Function, UntaggedFunction, entry_point, Uword, FINAL)                     \

runtime/vm/compiler/frontend/kernel_binary_flowgraph.cc

@@ -6060,7 +6060,8 @@ Fragment StreamingFlowGraphBuilder::BuildFunctionNode(
 
         LocalScope* scope = scopes()->function_scopes[i].scope;
         const ContextScope& context_scope = ContextScope::Handle(
-            Z, scope->PreserveOuterScope(flow_graph_builder_->context_depth_));
+            Z, scope->PreserveOuterScope(function,
+                                         flow_graph_builder_->context_depth_));
         function.set_context_scope(context_scope);
         function.set_kernel_offset(offset);
         type_translator_.SetupFunctionParameters(Class::Handle(Z), function,

runtime/vm/compiler/frontend/kernel_to_il.cc

@@ -2545,9 +2545,9 @@ Fragment FlowGraphBuilder::BuildClosureCallDefaultTypeHandling(
   LocalVariable* closure_data = MakeTemporary("closure_data");
 
   store_default += LoadLocal(closure_data);
-  const auto& slot = Slot::ClosureData_default_type_arguments_kind();
+  store_default += BuildExtractUnboxedSlotBitFieldIntoSmi<
-  store_default += LoadNativeField(slot);
+      ClosureData::PackedDefaultTypeArgumentsKind>(
-  store_default += Box(slot.representation());
+      Slot::ClosureData_packed_fields());
   LocalVariable* default_tav_kind = MakeTemporary("default_tav_kind");
 
   // Two locals to drop after join, closure_data and default_tav_kind.

runtime/vm/compiler/frontend/scope_builder.cc

@@ -550,8 +550,6 @@ void ScopeBuilder::VisitFunctionNode() {
   FunctionNodeHelper function_node_helper(&helper_);
   function_node_helper.ReadUntilExcluding(FunctionNodeHelper::kTypeParameters);
 
-  const auto& function = parsed_function_->function();
-
   intptr_t list_length =
       helper_.ReadListLength();  // read type_parameters list length.
   for (intptr_t i = 0; i < list_length; ++i) {
@@ -573,19 +571,6 @@ void ScopeBuilder::VisitFunctionNode() {
     VisitStatement();  // Read body
     first_body_token_position_ = helper_.reader_.min_position();
   }
-
-  // Mark known chained futures such as _Future::timeout()'s _future.
-  if (function.recognized_kind() == MethodRecognizer::kFutureTimeout &&
-      depth_.function_ == 1) {
-    LocalVariable* future = scope_->LookupVariableByName(Symbols::_future());
-    ASSERT(future != nullptr);
-    future->set_is_chained_future();
-  } else if (function.recognized_kind() == MethodRecognizer::kFutureWait &&
-             depth_.function_ == 1) {
-    LocalVariable* future = scope_->LookupVariableByName(Symbols::_future());
-    ASSERT(future != nullptr);
-    future->set_is_chained_future();
-  }
 }
 
 void ScopeBuilder::VisitInitializer() {
@@ -1332,6 +1317,8 @@ void ScopeBuilder::VisitVariableDeclaration() {
   const intptr_t kernel_offset =
       helper_.data_program_offset_ + helper_.ReaderOffset();
   VariableDeclarationHelper helper(&helper_);
+  helper.ReadUntilExcluding(VariableDeclarationHelper::kAnnotations);
+  const intptr_t annotations_offset = helper_.ReaderOffset();
   helper.ReadUntilExcluding(VariableDeclarationHelper::kType);
   AbstractType& type = BuildAndVisitVariableType();
 
@@ -1356,6 +1343,9 @@ void ScopeBuilder::VisitVariableDeclaration() {
   }
   LocalVariable* variable =
       MakeVariable(helper.position_, end_position, name, type, kernel_offset);
+  if (helper.annotation_count_ > 0) {
+    variable->set_annotations_offset(annotations_offset);
+  }
   if (helper.IsFinal()) {
     variable->set_is_final();
   }
@@ -1646,6 +1636,8 @@ void ScopeBuilder::AddVariableDeclarationParameter(
   const InferredTypeMetadata parameter_type =
       inferred_type_metadata_helper_.GetInferredType(helper_.ReaderOffset());
   VariableDeclarationHelper helper(&helper_);
+  helper.ReadUntilExcluding(VariableDeclarationHelper::kAnnotations);
+  const intptr_t annotations_offset = helper_.ReaderOffset();
   helper.ReadUntilExcluding(VariableDeclarationHelper::kType);
   String& name = H.DartSymbolObfuscate(helper.name_index_);
   ASSERT(name.Length() > 0);
@@ -1656,6 +1648,9 @@ void ScopeBuilder::AddVariableDeclarationParameter(
   LocalVariable* variable =
       MakeVariable(helper.position_, helper.position_, name, type,
                    kernel_offset, &parameter_type);
+  if (helper.annotation_count_ > 0) {
+    variable->set_annotations_offset(annotations_offset);
+  }
   if (helper.IsFinal()) {
     variable->set_is_final();
   }
@@ -1731,7 +1726,7 @@ LocalVariable* ScopeBuilder::MakeVariable(
     TokenPosition token_pos,
     const String& name,
     const AbstractType& type,
-    intptr_t kernel_offset,
+    intptr_t kernel_offset /* = LocalVariable::kNoKernelOffset */,
     const InferredTypeMetadata* param_type_md /* = nullptr */) {
   CompileType* param_type = nullptr;
   const Object* param_value = nullptr;

runtime/vm/compiler/recognized_methods_list.h

@@ -131,10 +131,6 @@ namespace dart {
   V(_SuspendState, set:_errorCallback, SuspendState_setErrorCallback,          \
     0xc3fa77cc)                                                                \
   V(_SuspendState, _clone, SuspendState_clone, 0xae0bb4c0)                     \
-  V(_SuspendState, _createAsyncCallbacks, SuspendState_createAsyncCallbacks,   \
-    0x8625a677)                                                                \
-  V(_SuspendState, _createAsyncStarCallback,                                   \
-    SuspendState_createAsyncStarCallback, 0x98ecfd9c)                          \
   V(_SuspendState, _resume, SuspendState_resume, 0x5d6bf8a9)                   \
   V(_IntegerImplementation, toDouble, IntegerToDouble, 0x9763ff66)             \
   V(_Double, _add, DoubleAdd, 0xea57d747)                                      \
@@ -321,9 +317,6 @@ namespace dart {
   V(::, _getNativeField, GetNativeField, 0xa0050fa5)                           \
   V(::, reachabilityFence, ReachabilityFence, 0x73009f9f)                      \
   V(_Utf8Decoder, _scan, Utf8DecoderScan, 0xb98ea301)                          \
-  V(_Future, timeout, FutureTimeout, 0xa0e8b7f4)                               \
-  V(Future, wait, FutureWait, 0x29515d77)                                      \
-  V(_RootZone, runUnary, RootZoneRunUnary, 0x642af2eb)                         \
   V(_FutureListener, handleValue, FutureListenerHandleValue, 0xec1745d2)       \
   V(::, has63BitSmis, Has63BitSmis, 0xf60ccb11)                                \
   V(::, get:extensionStreamHasListener, ExtensionStreamHasListener, 0xfaa5d763)\

runtime/vm/compiler/runtime_api.h

@@ -1482,7 +1482,7 @@ class Closure : public AllStatic {
 
 class ClosureData : public AllStatic {
  public:
-  static word default_type_arguments_kind_offset();
+  static word packed_fields_offset();
   static word InstanceSize();
   FINAL_CLASS();
 };

runtime/vm/compiler/runtime_offsets_extracted.h

@@ -159,8 +159,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x18;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x4;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x10;
+    0x10;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x18;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x14;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x1c;
@@ -262,7 +262,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x84;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x280;
+    ObjectStore_ffi_callback_code_offset = 0x278;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x224;
 static constexpr dart::compiler::target::word
@@ -869,8 +869,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x30;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x8;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x20;
+    0x20;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x30;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x28;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x38;
@@ -973,7 +973,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x500;
+    ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -1581,8 +1581,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x18;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x4;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x10;
+    0x10;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x18;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x14;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x1c;
@@ -1684,7 +1684,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x84;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x280;
+    ObjectStore_ffi_callback_code_offset = 0x278;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x224;
 static constexpr dart::compiler::target::word
@@ -2290,8 +2290,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x30;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x8;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x20;
+    0x20;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x30;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x28;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x38;
@@ -2394,7 +2394,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x500;
+    ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -3005,8 +3005,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x1c;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x8;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x14;
+    0x14;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x30;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x28;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x38;
@@ -3109,7 +3109,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x500;
+    ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -3717,8 +3717,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x1c;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x8;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x14;
+    0x14;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x30;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x28;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x38;
@@ -3821,7 +3821,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x500;
+    ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -4430,8 +4430,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x18;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x4;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x10;
+    0x10;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x18;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x14;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x1c;
@@ -4533,7 +4533,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x84;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x280;
+    ObjectStore_ffi_callback_code_offset = 0x278;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x224;
 static constexpr dart::compiler::target::word
@@ -5141,8 +5141,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x30;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x8;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x20;
+    0x20;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x30;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x28;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x38;
@@ -5245,7 +5245,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x500;
+    ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -5851,8 +5851,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x18;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x4;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x10;
+    0x10;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x18;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x14;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x1c;
@@ -5951,7 +5951,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x84;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x280;
+    ObjectStore_ffi_callback_code_offset = 0x278;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x224;
 static constexpr dart::compiler::target::word
@@ -6553,8 +6553,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x30;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x8;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x20;
+    0x20;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x30;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x28;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x38;
@@ -6654,7 +6654,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x500;
+    ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -7257,8 +7257,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x18;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x4;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x10;
+    0x10;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x18;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x14;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x1c;
@@ -7357,7 +7357,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x84;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x280;
+    ObjectStore_ffi_callback_code_offset = 0x278;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x224;
 static constexpr dart::compiler::target::word
@@ -7958,8 +7958,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x30;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x8;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x20;
+    0x20;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x30;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x28;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x38;
@@ -8059,7 +8059,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x500;
+    ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -8665,8 +8665,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x1c;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x8;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x14;
+    0x14;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x30;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x28;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x38;
@@ -8766,7 +8766,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x500;
+    ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -9369,8 +9369,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x1c;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x8;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x14;
+    0x14;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x30;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x28;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x38;
@@ -9470,7 +9470,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x500;
+    ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -10074,8 +10074,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x18;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x4;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x10;
+    0x10;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x18;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x14;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x1c;
@@ -10174,7 +10174,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x84;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x280;
+    ObjectStore_ffi_callback_code_offset = 0x278;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x224;
 static constexpr dart::compiler::target::word
@@ -10777,8 +10777,8 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word Closure_hash_offset = 0x30;
 static constexpr dart::compiler::target::word
     Closure_instantiator_type_arguments_offset = 0x8;
-static constexpr dart::compiler::target::word
+static constexpr dart::compiler::target::word ClosureData_packed_fields_offset =
-    ClosureData_default_type_arguments_kind_offset = 0x20;
+    0x20;
 static constexpr dart::compiler::target::word Code_instructions_offset = 0x30;
 static constexpr dart::compiler::target::word Code_object_pool_offset = 0x28;
 static constexpr dart::compiler::target::word Code_owner_offset = 0x38;
@@ -10878,7 +10878,7 @@ static constexpr dart::compiler::target::word ObjectStore_string_type_offset =
 static constexpr dart::compiler::target::word ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    ObjectStore_ffi_callback_code_offset = 0x500;
+    ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -11501,7 +11501,7 @@ static constexpr dart::compiler::target::word AOT_Closure_hash_offset = 0x18;
 static constexpr dart::compiler::target::word
     AOT_Closure_instantiator_type_arguments_offset = 0x4;
 static constexpr dart::compiler::target::word
-    AOT_ClosureData_default_type_arguments_kind_offset = 0x10;
+    AOT_ClosureData_packed_fields_offset = 0x10;
 static constexpr dart::compiler::target::word AOT_Code_instructions_offset =
     0x18;
 static constexpr dart::compiler::target::word AOT_Code_object_pool_offset =
@@ -11619,7 +11619,7 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word AOT_ObjectStore_type_type_offset =
     0x84;
 static constexpr dart::compiler::target::word
-    AOT_ObjectStore_ffi_callback_code_offset = 0x280;
+    AOT_ObjectStore_ffi_callback_code_offset = 0x278;
 static constexpr dart::compiler::target::word
     AOT_ObjectStore_suspend_state_await_offset = 0x224;
 static constexpr dart::compiler::target::word
@@ -12288,7 +12288,7 @@ static constexpr dart::compiler::target::word AOT_Closure_hash_offset = 0x30;
 static constexpr dart::compiler::target::word
     AOT_Closure_instantiator_type_arguments_offset = 0x8;
 static constexpr dart::compiler::target::word
-    AOT_ClosureData_default_type_arguments_kind_offset = 0x20;
+    AOT_ClosureData_packed_fields_offset = 0x20;
 static constexpr dart::compiler::target::word AOT_Code_instructions_offset =
     0x30;
 static constexpr dart::compiler::target::word AOT_Code_object_pool_offset =
@@ -12406,7 +12406,7 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word AOT_ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    AOT_ObjectStore_ffi_callback_code_offset = 0x500;
+    AOT_ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     AOT_ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -13078,7 +13078,7 @@ static constexpr dart::compiler::target::word AOT_Closure_hash_offset = 0x30;
 static constexpr dart::compiler::target::word
     AOT_Closure_instantiator_type_arguments_offset = 0x8;
 static constexpr dart::compiler::target::word
-    AOT_ClosureData_default_type_arguments_kind_offset = 0x20;
+    AOT_ClosureData_packed_fields_offset = 0x20;
 static constexpr dart::compiler::target::word AOT_Code_instructions_offset =
     0x30;
 static constexpr dart::compiler::target::word AOT_Code_object_pool_offset =
@@ -13196,7 +13196,7 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word AOT_ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    AOT_ObjectStore_ffi_callback_code_offset = 0x500;
+    AOT_ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     AOT_ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -13867,7 +13867,7 @@ static constexpr dart::compiler::target::word AOT_Closure_hash_offset = 0x1c;
 static constexpr dart::compiler::target::word
     AOT_Closure_instantiator_type_arguments_offset = 0x8;
 static constexpr dart::compiler::target::word
-    AOT_ClosureData_default_type_arguments_kind_offset = 0x14;
+    AOT_ClosureData_packed_fields_offset = 0x14;
 static constexpr dart::compiler::target::word AOT_Code_instructions_offset =
     0x30;
 static constexpr dart::compiler::target::word AOT_Code_object_pool_offset =
@@ -13985,7 +13985,7 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word AOT_ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    AOT_ObjectStore_ffi_callback_code_offset = 0x500;
+    AOT_ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     AOT_ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -14656,7 +14656,7 @@ static constexpr dart::compiler::target::word AOT_Closure_hash_offset = 0x1c;
 static constexpr dart::compiler::target::word
     AOT_Closure_instantiator_type_arguments_offset = 0x8;
 static constexpr dart::compiler::target::word
-    AOT_ClosureData_default_type_arguments_kind_offset = 0x14;
+    AOT_ClosureData_packed_fields_offset = 0x14;
 static constexpr dart::compiler::target::word AOT_Code_instructions_offset =
     0x30;
 static constexpr dart::compiler::target::word AOT_Code_object_pool_offset =
@@ -14774,7 +14774,7 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word AOT_ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    AOT_ObjectStore_ffi_callback_code_offset = 0x500;
+    AOT_ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     AOT_ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -15447,7 +15447,7 @@ static constexpr dart::compiler::target::word AOT_Closure_hash_offset = 0x18;
 static constexpr dart::compiler::target::word
     AOT_Closure_instantiator_type_arguments_offset = 0x4;
 static constexpr dart::compiler::target::word
-    AOT_ClosureData_default_type_arguments_kind_offset = 0x10;
+    AOT_ClosureData_packed_fields_offset = 0x10;
 static constexpr dart::compiler::target::word AOT_Code_instructions_offset =
     0x18;
 static constexpr dart::compiler::target::word AOT_Code_object_pool_offset =
@@ -15565,7 +15565,7 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word AOT_ObjectStore_type_type_offset =
     0x84;
 static constexpr dart::compiler::target::word
-    AOT_ObjectStore_ffi_callback_code_offset = 0x280;
+    AOT_ObjectStore_ffi_callback_code_offset = 0x278;
 static constexpr dart::compiler::target::word
     AOT_ObjectStore_suspend_state_await_offset = 0x224;
 static constexpr dart::compiler::target::word
@@ -16235,7 +16235,7 @@ static constexpr dart::compiler::target::word AOT_Closure_hash_offset = 0x30;
 static constexpr dart::compiler::target::word
     AOT_Closure_instantiator_type_arguments_offset = 0x8;
 static constexpr dart::compiler::target::word
-    AOT_ClosureData_default_type_arguments_kind_offset = 0x20;
+    AOT_ClosureData_packed_fields_offset = 0x20;
 static constexpr dart::compiler::target::word AOT_Code_instructions_offset =
     0x30;
 static constexpr dart::compiler::target::word AOT_Code_object_pool_offset =
@@ -16353,7 +16353,7 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word AOT_ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    AOT_ObjectStore_ffi_callback_code_offset = 0x500;
+    AOT_ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     AOT_ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -17020,7 +17020,7 @@ static constexpr dart::compiler::target::word AOT_Closure_hash_offset = 0x18;
 static constexpr dart::compiler::target::word
     AOT_Closure_instantiator_type_arguments_offset = 0x4;
 static constexpr dart::compiler::target::word
-    AOT_ClosureData_default_type_arguments_kind_offset = 0x10;
+    AOT_ClosureData_packed_fields_offset = 0x10;
 static constexpr dart::compiler::target::word AOT_Code_instructions_offset =
     0x18;
 static constexpr dart::compiler::target::word AOT_Code_object_pool_offset =
@@ -17134,7 +17134,7 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word AOT_ObjectStore_type_type_offset =
     0x84;
 static constexpr dart::compiler::target::word
-    AOT_ObjectStore_ffi_callback_code_offset = 0x280;
+    AOT_ObjectStore_ffi_callback_code_offset = 0x278;
 static constexpr dart::compiler::target::word
     AOT_ObjectStore_suspend_state_await_offset = 0x224;
 static constexpr dart::compiler::target::word
@@ -17798,7 +17798,7 @@ static constexpr dart::compiler::target::word AOT_Closure_hash_offset = 0x30;
 static constexpr dart::compiler::target::word
     AOT_Closure_instantiator_type_arguments_offset = 0x8;
 static constexpr dart::compiler::target::word
-    AOT_ClosureData_default_type_arguments_kind_offset = 0x20;
+    AOT_ClosureData_packed_fields_offset = 0x20;
 static constexpr dart::compiler::target::word AOT_Code_instructions_offset =
     0x30;
 static constexpr dart::compiler::target::word AOT_Code_object_pool_offset =
@@ -17912,7 +17912,7 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word AOT_ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    AOT_ObjectStore_ffi_callback_code_offset = 0x500;
+    AOT_ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     AOT_ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -18579,7 +18579,7 @@ static constexpr dart::compiler::target::word AOT_Closure_hash_offset = 0x30;
 static constexpr dart::compiler::target::word
     AOT_Closure_instantiator_type_arguments_offset = 0x8;
 static constexpr dart::compiler::target::word
-    AOT_ClosureData_default_type_arguments_kind_offset = 0x20;
+    AOT_ClosureData_packed_fields_offset = 0x20;
 static constexpr dart::compiler::target::word AOT_Code_instructions_offset =
     0x30;
 static constexpr dart::compiler::target::word AOT_Code_object_pool_offset =
@@ -18693,7 +18693,7 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word AOT_ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    AOT_ObjectStore_ffi_callback_code_offset = 0x500;
+    AOT_ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     AOT_ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -19359,7 +19359,7 @@ static constexpr dart::compiler::target::word AOT_Closure_hash_offset = 0x1c;
 static constexpr dart::compiler::target::word
     AOT_Closure_instantiator_type_arguments_offset = 0x8;
 static constexpr dart::compiler::target::word
-    AOT_ClosureData_default_type_arguments_kind_offset = 0x14;
+    AOT_ClosureData_packed_fields_offset = 0x14;
 static constexpr dart::compiler::target::word AOT_Code_instructions_offset =
     0x30;
 static constexpr dart::compiler::target::word AOT_Code_object_pool_offset =
@@ -19473,7 +19473,7 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word AOT_ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    AOT_ObjectStore_ffi_callback_code_offset = 0x500;
+    AOT_ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     AOT_ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -20139,7 +20139,7 @@ static constexpr dart::compiler::target::word AOT_Closure_hash_offset = 0x1c;
 static constexpr dart::compiler::target::word
     AOT_Closure_instantiator_type_arguments_offset = 0x8;
 static constexpr dart::compiler::target::word
-    AOT_ClosureData_default_type_arguments_kind_offset = 0x14;
+    AOT_ClosureData_packed_fields_offset = 0x14;
 static constexpr dart::compiler::target::word AOT_Code_instructions_offset =
     0x30;
 static constexpr dart::compiler::target::word AOT_Code_object_pool_offset =
@@ -20253,7 +20253,7 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word AOT_ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    AOT_ObjectStore_ffi_callback_code_offset = 0x500;
+    AOT_ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     AOT_ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word
@@ -20921,7 +20921,7 @@ static constexpr dart::compiler::target::word AOT_Closure_hash_offset = 0x18;
 static constexpr dart::compiler::target::word
     AOT_Closure_instantiator_type_arguments_offset = 0x4;
 static constexpr dart::compiler::target::word
-    AOT_ClosureData_default_type_arguments_kind_offset = 0x10;
+    AOT_ClosureData_packed_fields_offset = 0x10;
 static constexpr dart::compiler::target::word AOT_Code_instructions_offset =
     0x18;
 static constexpr dart::compiler::target::word AOT_Code_object_pool_offset =
@@ -21035,7 +21035,7 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word AOT_ObjectStore_type_type_offset =
     0x84;
 static constexpr dart::compiler::target::word
-    AOT_ObjectStore_ffi_callback_code_offset = 0x280;
+    AOT_ObjectStore_ffi_callback_code_offset = 0x278;
 static constexpr dart::compiler::target::word
     AOT_ObjectStore_suspend_state_await_offset = 0x224;
 static constexpr dart::compiler::target::word
@@ -21700,7 +21700,7 @@ static constexpr dart::compiler::target::word AOT_Closure_hash_offset = 0x30;
 static constexpr dart::compiler::target::word
     AOT_Closure_instantiator_type_arguments_offset = 0x8;
 static constexpr dart::compiler::target::word
-    AOT_ClosureData_default_type_arguments_kind_offset = 0x20;
+    AOT_ClosureData_packed_fields_offset = 0x20;
 static constexpr dart::compiler::target::word AOT_Code_instructions_offset =
     0x30;
 static constexpr dart::compiler::target::word AOT_Code_object_pool_offset =
@@ -21814,7 +21814,7 @@ static constexpr dart::compiler::target::word
 static constexpr dart::compiler::target::word AOT_ObjectStore_type_type_offset =
     0x108;
 static constexpr dart::compiler::target::word
-    AOT_ObjectStore_ffi_callback_code_offset = 0x500;
+    AOT_ObjectStore_ffi_callback_code_offset = 0x4f0;
 static constexpr dart::compiler::target::word
     AOT_ObjectStore_suspend_state_await_offset = 0x448;
 static constexpr dart::compiler::target::word

runtime/vm/compiler/runtime_offsets_list.h

@@ -131,7 +131,7 @@
   FIELD(Closure, function_type_arguments_offset)                               \
   FIELD(Closure, hash_offset)                                                  \
   FIELD(Closure, instantiator_type_arguments_offset)                           \
-  FIELD(ClosureData, default_type_arguments_kind_offset)                       \
+  FIELD(ClosureData, packed_fields_offset)                                     \
   FIELD(Code, instructions_offset)                                             \
   FIELD(Code, object_pool_offset)                                              \
   FIELD(Code, owner_offset)                                                    \

runtime/vm/debugger.cc

@@ -235,6 +235,7 @@ ActivationFrame::ActivationFrame(uword pc,
       sp_(sp),
       code_(Code::ZoneHandle(code.ptr())),
       function_(Function::ZoneHandle(code.function())),
+      closure_(Closure::null_closure()),
       deopt_frame_(Array::ZoneHandle(deopt_frame.ptr())),
       deopt_frame_offset_(deopt_frame_offset),
       kind_(kRegular),
@@ -243,10 +244,13 @@ ActivationFrame::ActivationFrame(uword pc,
   ASSERT(!function_.IsNull());
 }
 
-ActivationFrame::ActivationFrame(uword pc, const Code& code)
+ActivationFrame::ActivationFrame(uword pc,
+                                 const Code& code,
+                                 const Closure& closure)
     : pc_(pc),
       code_(Code::ZoneHandle(code.ptr())),
       function_(Function::ZoneHandle(code.function())),
+      closure_(Closure::ZoneHandle(closure.ptr())),
       deopt_frame_(Array::empty_array()),
       deopt_frame_offset_(0),
       kind_(kAsyncAwaiter) {}
@@ -254,6 +258,7 @@ ActivationFrame::ActivationFrame(uword pc, const Code& code)
 ActivationFrame::ActivationFrame(Kind kind)
     : code_(Code::ZoneHandle()),
       function_(Function::null_function()),
+      closure_(Closure::null_closure()),
       deopt_frame_(Array::empty_array()),
       deopt_frame_offset_(0),
       kind_(kind) {
@@ -636,25 +641,9 @@ intptr_t ActivationFrame::ContextLevel() {
   return context_level_;
 }
 
-ObjectPtr ActivationFrame::GetAsyncAwaiter(
-    CallerClosureFinder* caller_closure_finder) {
-  if (fp() != 0 && !function_.IsNull()) {
-    if (function_.IsAsyncFunction() || function_.IsAsyncGenerator()) {
-      const auto& suspend_state = Object::Handle(GetSuspendStateVar());
-      if (caller_closure_finder->WasPreviouslySuspended(function_,
-                                                        suspend_state)) {
-        return caller_closure_finder->FindCallerFromSuspendState(
-            SuspendState::Cast(suspend_state));
-      }
-    }
-  }
-
-  return Object::null();
-}
-
 bool ActivationFrame::HandlesException(const Instance& exc_obj) {
   if (kind_ == kAsyncSuspensionMarker) {
-    return false;
+    return has_catch_error();
   }
   intptr_t try_index = TryIndex();
   const auto& handlers = ExceptionHandlers::Handle(code().exception_handlers());
@@ -690,24 +679,6 @@ bool ActivationFrame::HandlesException(const Instance& exc_obj) {
     }
     try_index = handlers.OuterTryIndex(try_index);
   }
-  // Async functions might have indirect exception handlers in the form of
-  // `Future.catchError`. Check _FutureListeners.
-  if (kind_ == kRegular && function().IsAsyncFunction()) {
-    CallerClosureFinder caller_closure_finder(Thread::Current()->zone());
-    auto& suspend_state = Object::Handle(GetSuspendStateVar());
-    if (!caller_closure_finder.WasPreviouslySuspended(function(),
-                                                      suspend_state)) {
-      return false;
-    }
-    Object& futureOrListener =
-        Object::Handle(SuspendState::Cast(suspend_state).function_data());
-    futureOrListener =
-        caller_closure_finder.GetFutureFutureListener(futureOrListener);
-    if (futureOrListener.IsNull()) {
-      return false;
-    }
-    return caller_closure_finder.HasCatchError(futureOrListener);
-  }
 
   return false;
 }
@@ -1301,7 +1272,7 @@ void DebuggerStackTrace::AddActivation(ActivationFrame* frame) {
   }
 }
 
-void DebuggerStackTrace::AddAsyncSuspension() {
+void DebuggerStackTrace::AddAsyncSuspension(bool has_catch_error) {
   // We might start asynchronous unwinding in one of the internal
   // dart:async functions which would make synchronous part of the
   // stack empty. This would not happen normally but might happen
@@ -1310,10 +1281,15 @@ void DebuggerStackTrace::AddAsyncSuspension() {
       trace_.Last()->kind() != ActivationFrame::kAsyncSuspensionMarker) {
     trace_.Add(new ActivationFrame(ActivationFrame::kAsyncSuspensionMarker));
   }
+  if (has_catch_error) {
+    trace_.Last()->set_has_catch_error(true);
+  }
 }
 
-void DebuggerStackTrace::AddAsyncAwaiterFrame(uword pc, const Code& code) {
+void DebuggerStackTrace::AddAsyncAwaiterFrame(uword pc,
-  trace_.Add(new ActivationFrame(pc, code));
+                                              const Code& code,
+                                              const Closure& closure) {
+  trace_.Add(new ActivationFrame(pc, code, closure));
 }
 
 const uint8_t kSafepointKind = UntaggedPcDescriptors::kIcCall |
@@ -1716,67 +1692,41 @@ DebuggerStackTrace* DebuggerStackTrace::CollectAsyncAwaiters() {
   Thread* thread = Thread::Current();
   Zone* zone = thread->zone();
 
-  Code& code = Code::Handle(zone);
   Function& function = Function::Handle(zone);
 
   constexpr intptr_t kDefaultStackAllocation = 8;
   auto stack_trace = new DebuggerStackTrace(kDefaultStackAllocation);
 
-  const auto& code_array = GrowableObjectArray::ZoneHandle(
-      zone, GrowableObjectArray::New(kDefaultStackAllocation));
-  GrowableArray<uword> pc_offset_array(kDefaultStackAllocation);
   bool has_async = false;
+  StackTraceUtils::CollectFrames(
+      thread, /*skip_frames=*/0, [&](const StackTraceUtils::Frame& frame) {
+        if (frame.frame != nullptr) {  // Synchronous portion of the stack.
+          stack_trace->AppendCodeFrames(frame.frame, frame.code);
+        } else {
+          has_async = true;
 
-  std::function<void(StackFrame*)> on_sync_frame = [&](StackFrame* frame) {
+          if (frame.code.ptr() == StubCode::AsynchronousGapMarker().ptr()) {
-    code = frame->LookupDartCode();
+            stack_trace->AddAsyncSuspension(frame.has_async_catch_error);
-    stack_trace->AppendCodeFrames(frame, code);
+            return;
-  };
+          }
 
-  StackTraceUtils::CollectFrames(thread, code_array, &pc_offset_array,
+          // Skip invisible function frames.
-                                 /*skip_frames=*/0, &on_sync_frame, &has_async);
+          function ^= frame.code.function();
+          if (!function.is_visible()) {
+            return;
+          }
+
+          const uword absolute_pc = frame.code.PayloadStart() + frame.pc_offset;
+          stack_trace->AddAsyncAwaiterFrame(absolute_pc, frame.code,
+                                            frame.closure);
+        }
+      });
 
   // If the entire stack is sync, return no (async) trace.
   if (!has_async) {
     return nullptr;
   }
 
-  const intptr_t length = code_array.Length();
-  bool async_frames = false;
-  bool skip_next_gap_marker = false;
-  for (intptr_t i = 0; i < length; ++i) {
-    code ^= code_array.At(i);
-    if (code.ptr() == StubCode::AsynchronousGapMarker().ptr()) {
-      if (!skip_next_gap_marker) {
-        stack_trace->AddAsyncSuspension();
-      }
-      skip_next_gap_marker = false;
-
-      // Once we reach a gap, the rest is async.
-      async_frames = true;
-      continue;
-    }
-
-    // Skip the sync frames since they've been added (and un-inlined) above.
-    if (!async_frames) {
-      continue;
-    }
-
-    if (!code.IsFunctionCode()) {
-      continue;
-    }
-
-    // Skip invisible function frames.
-    function ^= code.function();
-    if (!function.is_visible()) {
-      skip_next_gap_marker = true;
-      continue;
-    }
-
-    const uword pc_offset = pc_offset_array[i];
-    const uword absolute_pc = code.PayloadStart() + pc_offset;
-    stack_trace->AddAsyncAwaiterFrame(absolute_pc, code);
-  }
-
   return stack_trace;
 }
 
@@ -1901,7 +1851,8 @@ bool Debugger::ShouldPauseOnException(DebuggerStackTrace* stack_trace,
   // If handler_frame's function is annotated with
   // @pragma('vm:notify-debugger-on-exception'), we specifically want to notify
   // the debugger of this otherwise ignored exception.
-  if (Library::FindPragma(Thread::Current(), /*only_core=*/false,
+  if (!handler_function.IsNull() &&
+      Library::FindPragma(Thread::Current(), /*only_core=*/false,
                           handler_function,
                           Symbols::vm_notify_debugger_on_exception())) {
     return true;
@@ -3067,23 +3018,19 @@ void Debugger::HandleSteppingRequest(bool skip_next_step /* = false */) {
                    stepping_fp_);
     }
   } else if (resume_action_ == kStepOut) {
-    if (stack_trace_->FrameAt(0)->function().IsAsyncFunction() ||
+    // Check if we have an asynchronous awaiter for the current frame.
-        stack_trace_->FrameAt(0)->function().IsAsyncGenerator()) {
+    if (async_awaiter_stack_trace_ != nullptr &&
-      CallerClosureFinder caller_closure_finder(Thread::Current()->zone());
+        async_awaiter_stack_trace_->Length() > 2 &&
-      // Request to step out of an async/async* closure.
+        async_awaiter_stack_trace_->FrameAt(1)->kind() ==
-      const Object& async_op = Object::Handle(
+            ActivationFrame::kAsyncSuspensionMarker) {
-          stack_trace_->FrameAt(0)->GetAsyncAwaiter(&caller_closure_finder));
+      auto awaiter_frame = async_awaiter_stack_trace_->FrameAt(2);
-      if (!async_op.IsNull()) {
+      AsyncStepInto(awaiter_frame->closure());
-        // Step out to the awaiter.
+      if (FLAG_verbose_debug) {
-        ASSERT(async_op.IsClosure());
+        OS::PrintErr("HandleSteppingRequest - continue to async awaiter %s\n",
-        AsyncStepInto(Closure::Cast(async_op));
+                     Function::Handle(awaiter_frame->closure().function())
-        if (FLAG_verbose_debug) {
+                         .ToFullyQualifiedCString());
-          OS::PrintErr("HandleSteppingRequest- kContinue to async_op %s\n",
-                       Function::Handle(Closure::Cast(async_op).function())
-                           .ToFullyQualifiedCString());
-        }
-        return;
       }
+      return;
     }
 
     // Fall through to synchronous stepping.
@@ -4094,11 +4041,9 @@ Breakpoint* Debugger::GetBreakpointByIdInTheList(intptr_t id,
 
 void Debugger::AsyncStepInto(const Closure& awaiter) {
   Zone* zone = Thread::Current()->zone();
-  CallerClosureFinder caller_closure_finder(zone);
+
-  if (caller_closure_finder.IsAsyncCallback(
+  auto& suspend_state = SuspendState::Handle(zone);
-          Function::Handle(zone, awaiter.function()))) {
+  if (StackTraceUtils::GetSuspendState(awaiter, &suspend_state)) {
-    const auto& suspend_state = SuspendState::Handle(
-        zone, caller_closure_finder.GetSuspendStateFromAsyncCallback(awaiter));
     const auto& function_data =
         Object::Handle(zone, suspend_state.function_data());
     SetBreakpointAtResumption(function_data);

runtime/vm/debugger.h

@@ -292,7 +292,12 @@ class ActivationFrame : public ZoneAllocated {
                   const Array& deopt_frame,
                   intptr_t deopt_frame_offset);
 
-  ActivationFrame(uword pc, const Code& code);
+  // Create a |kAsyncAwaiter| frame representing asynchronous awaiter
+  // waiting for the completion of a |Future|.
+  //
+  // |closure| is the listener which will be invoked when awaited
+  // computation completes.
+  ActivationFrame(uword pc, const Code& code, const Closure& closure);
 
   explicit ActivationFrame(Kind kind);
 
@@ -304,6 +309,10 @@ class ActivationFrame : public ZoneAllocated {
 
   uword GetCallerSp() const { return fp() + (kCallerSpSlotFromFp * kWordSize); }
 
+  // For |kAsyncAwaiter| frames this is the listener which will be invoked
+  // when the frame below (callee) completes.
+  const Closure& closure() const { return closure_; }
+
   const Function& function() const {
     return function_;
   }
@@ -374,11 +383,11 @@ class ActivationFrame : public ZoneAllocated {
 
   void PrintToJSONObject(JSONObject* jsobj);
 
-  // Get Closure that await'ed this async frame.
-  ObjectPtr GetAsyncAwaiter(CallerClosureFinder* caller_closure_finder);
-
   bool HandlesException(const Instance& exc_obj);
 
+  bool has_catch_error() const { return has_catch_error_; }
+  void set_has_catch_error(bool value) { has_catch_error_ = value; }
+
  private:
   void PrintToJSONObjectRegular(JSONObject* jsobj);
   void PrintToJSONObjectAsyncAwaiter(JSONObject* jsobj);
@@ -423,6 +432,7 @@ class ActivationFrame : public ZoneAllocated {
   Context& ctx_ = Context::ZoneHandle();
   const Code& code_;
   const Function& function_;
+  const Closure& closure_;
 
   bool token_pos_initialized_ = false;
   TokenPosition token_pos_ = TokenPosition::kNoSource;
@@ -444,6 +454,8 @@ class ActivationFrame : public ZoneAllocated {
   ZoneGrowableArray<intptr_t> desc_indices_;
   PcDescriptors& pc_desc_ = PcDescriptors::ZoneHandle();
 
+  bool has_catch_error_ = false;
+
   friend class Debugger;
   friend class DebuggerStackTrace;
   DISALLOW_COPY_AND_ASSIGN(ActivationFrame);
@@ -471,8 +483,8 @@ class DebuggerStackTrace : public ZoneAllocated {
 
  private:
   void AddActivation(ActivationFrame* frame);
-  void AddAsyncSuspension();
+  void AddAsyncSuspension(bool has_catch_error);
-  void AddAsyncAwaiterFrame(uword pc, const Code& code);
+  void AddAsyncAwaiterFrame(uword pc, const Code& code, const Closure& closure);
 
   void AppendCodeFrames(StackFrame* frame, const Code& code);
 

runtime/vm/dwarf.cc

@@ -248,6 +248,8 @@ void Dwarf::WriteAbbreviations(DwarfWriteStream* stream) {
   stream->uleb128(DW_FORM_udata);
   stream->uleb128(DW_AT_call_column);
   stream->uleb128(DW_FORM_udata);
+  stream->uleb128(DW_AT_artificial);
+  stream->uleb128(DW_FORM_flag);
   stream->uleb128(0);
   stream->uleb128(0);  // End of attributes.
 
@@ -415,6 +417,14 @@ InliningNode* Dwarf::ExpandInliningTree(const Code& code) {
       }
       case CodeSourceMapOps::kAdvancePC: {
         current_pc_offset += arg1;
+        if (arg1 == 0) {
+          // This happens at the start of the function where we emit a special
+          // kAdvancePC 0 instruction to record information about the function
+          // itself. We need to advance current_pc_offset a bit to prevent
+          // starting inlining interval directy at the start of the function
+          // itself.
+          current_pc_offset += 1;
+        }
         break;
       }
       case CodeSourceMapOps::kPushFunction: {
@@ -476,8 +486,10 @@ void Dwarf::WriteInliningNode(DwarfWriteStream* stream,
 
   // DW_AT_call_line
   stream->uleb128(node->position.line());
-  // DW_at_call_column
+  // DW_AT_call_column
   stream->uleb128(node->position.column());
+  // DW_AT_artificial
+  stream->u1(node->function.is_visible() ? 0 : 1);
 
   for (InliningNode* child = node->children_head; child != nullptr;
        child = child->children_next) {

runtime/vm/object.cc

@@ -749,6 +749,7 @@ void Object::Init(IsolateGroup* isolate_group) {
   *null_function_type_ = FunctionType::null();
   *null_record_type_ = RecordType::null();
   *null_type_arguments_ = TypeArguments::null();
+  *null_closure_ = Closure::null();
   *empty_type_arguments_ = TypeArguments::null();
   *null_abstract_type_ = AbstractType::null();
   *null_compressed_stackmaps_ = CompressedStackMaps::null();
@@ -4106,42 +4107,13 @@ FunctionPtr Class::GetRecordFieldGetter(const String& getter_name) const {
   return result.ptr();
 }
 
-bool Library::FindPragma(Thread* T,
+bool FindPragmaInMetadata(Thread* T,
-                         bool only_core,
+                          const Object& metadata_obj,
-                         const Object& obj,
+                          const String& pragma_name,
-                         const String& pragma_name,
+                          bool multiple,
-                         bool multiple,
+                          Object* options) {
-                         Object* options) {
   auto IG = T->isolate_group();
   auto Z = T->zone();
-  auto& lib = Library::Handle(Z);
-
-  if (obj.IsLibrary()) {
-    lib = Library::Cast(obj).ptr();
-  } else if (obj.IsClass()) {
-    auto& klass = Class::Cast(obj);
-    if (!klass.has_pragma()) return false;
-    lib = klass.library();
-  } else if (obj.IsFunction()) {
-    auto& function = Function::Cast(obj);
-    if (!function.has_pragma()) return false;
-    lib = Class::Handle(Z, function.Owner()).library();
-  } else if (obj.IsField()) {
-    auto& field = Field::Cast(obj);
-    if (!field.has_pragma()) return false;
-    lib = Class::Handle(Z, field.Owner()).library();
-  } else {
-    UNREACHABLE();
-  }
-
-  if (only_core && !lib.IsAnyCoreLibrary()) {
-    return false;
-  }
-
-  Object& metadata_obj = Object::Handle(Z, lib.GetMetadata(obj));
-  if (metadata_obj.IsUnwindError()) {
-    Report::LongJump(UnwindError::Cast(metadata_obj));
-  }
 
   // If there is a compile-time error while evaluating the metadata, we will
   // simply claim there was no @pragma annotation.
@@ -4191,7 +4163,46 @@ bool Library::FindPragma(Thread* T,
   if (found && options != nullptr) {
     *options = results.ptr();
   }
-  return found;
+  return false;
+}
+
+bool Library::FindPragma(Thread* T,
+                         bool only_core,
+                         const Object& obj,
+                         const String& pragma_name,
+                         bool multiple,
+                         Object* options) {
+  auto Z = T->zone();
+  auto& lib = Library::Handle(Z);
+
+  if (obj.IsLibrary()) {
+    lib = Library::Cast(obj).ptr();
+  } else if (obj.IsClass()) {
+    auto& klass = Class::Cast(obj);
+    if (!klass.has_pragma()) return false;
+    lib = klass.library();
+  } else if (obj.IsFunction()) {
+    auto& function = Function::Cast(obj);
+    if (!function.has_pragma()) return false;
+    lib = Class::Handle(Z, function.Owner()).library();
+  } else if (obj.IsField()) {
+    auto& field = Field::Cast(obj);
+    if (!field.has_pragma()) return false;
+    lib = Class::Handle(Z, field.Owner()).library();
+  } else {
+    UNREACHABLE();
+  }
+
+  if (only_core && !lib.IsAnyCoreLibrary()) {
+    return false;
+  }
+
+  Object& metadata_obj = Object::Handle(Z, lib.GetMetadata(obj));
+  if (metadata_obj.IsUnwindError()) {
+    Report::LongJump(UnwindError::Cast(metadata_obj));
+  }
+
+  return FindPragmaInMetadata(T, metadata_obj, pragma_name, multiple, options);
 }
 
 bool Function::IsDynamicInvocationForwarderName(const String& name) {
@@ -5430,7 +5441,7 @@ const char* Class::GenerateUserVisibleName() const {
   }
   String& name = String::Handle(Name());
   name = Symbols::New(Thread::Current(), String::ScrubName(name));
-  if (name.ptr() == Symbols::FutureImpl().ptr() &&
+  if (name.ptr() == Symbols::_Future().ptr() &&
       library() == Library::AsyncLibrary()) {
     return Symbols::Future().ToCString();
   }
@@ -8002,6 +8013,26 @@ void Function::set_context_scope(const ContextScope& value) const {
   UNREACHABLE();
 }
 
+Function::AwaiterLink Function::awaiter_link() const {
+  if (IsClosureFunction()) {
+    const Object& obj = Object::Handle(untag()->data());
+    ASSERT(!obj.IsNull());
+    return ClosureData::Cast(obj).awaiter_link();
+  }
+  UNREACHABLE();
+  return {};
+}
+
+void Function::set_awaiter_link(Function::AwaiterLink link) const {
+  if (IsClosureFunction()) {
+    const Object& obj = Object::Handle(untag()->data());
+    ASSERT(!obj.IsNull());
+    ClosureData::Cast(obj).set_awaiter_link(link);
+    return;
+  }
+  UNREACHABLE();
+}
+
 ClosurePtr Function::implicit_static_closure() const {
   if (IsImplicitStaticClosureFunction()) {
     const Object& obj = Object::Handle(untag()->data());
@@ -10017,6 +10048,7 @@ FunctionPtr Function::New(const FunctionType& signature,
       kind == UntaggedFunction::kImplicitClosureFunction) {
     ASSERT(space == Heap::kOld);
     const ClosureData& data = ClosureData::Handle(ClosureData::New());
+    data.set_awaiter_link({});
     result.set_data(data);
   } else if (kind == UntaggedFunction::kFfiTrampoline) {
     const FfiTrampolineData& data =
@@ -11263,20 +11295,43 @@ void FunctionType::set_num_implicit_parameters(intptr_t value) const {
 
 ClosureData::DefaultTypeArgumentsKind ClosureData::default_type_arguments_kind()
     const {
-  return LoadNonPointer(&untag()->default_type_arguments_kind_);
+  return untag()
+      ->packed_fields_
+      .Read<UntaggedClosureData::PackedDefaultTypeArgumentsKind>();
 }
 
 void ClosureData::set_default_type_arguments_kind(
     DefaultTypeArgumentsKind value) const {
-  StoreNonPointer(&untag()->default_type_arguments_kind_, value);
+  untag()
+      ->packed_fields_
+      .Update<UntaggedClosureData::PackedDefaultTypeArgumentsKind>(value);
+}
+
+Function::AwaiterLink ClosureData::awaiter_link() const {
+  const uint8_t depth =
+      untag()
+          ->packed_fields_.Read<UntaggedClosureData::PackedAwaiterLinkDepth>();
+  const uint8_t index =
+      untag()
+          ->packed_fields_.Read<UntaggedClosureData::PackedAwaiterLinkIndex>();
+  return {depth, index};
+}
+
+void ClosureData::set_awaiter_link(Function::AwaiterLink link) const {
+  untag()->packed_fields_.Update<UntaggedClosureData::PackedAwaiterLinkDepth>(
+      link.depth);
+  untag()->packed_fields_.Update<UntaggedClosureData::PackedAwaiterLinkIndex>(
+      link.index);
 }
 
 ClosureDataPtr ClosureData::New() {
   ASSERT(Object::closure_data_class() != Class::null());
-  ObjectPtr raw =
+  ClosureData& data = ClosureData::Handle();
+  data ^=
       Object::Allocate(ClosureData::kClassId, ClosureData::InstanceSize(),
                        Heap::kOld, ClosureData::ContainsCompressedPointers());
-  return static_cast<ClosureDataPtr>(raw);
+  data.set_packed_fields(0);
+  return data.ptr();
 }
 
 const char* ClosureData::ToCString() const {
@@ -18808,54 +18863,33 @@ void ContextScope::ClearFlagsAt(intptr_t scope_index) const {
   untag()->set_flags_at(scope_index, Smi::New(0));
 }
 
-bool ContextScope::GetFlagAt(intptr_t scope_index, intptr_t mask) const {
+bool ContextScope::GetFlagAt(intptr_t scope_index, intptr_t bit_index) const {
+  const intptr_t mask = 1 << bit_index;
   return (Smi::Value(untag()->flags_at(scope_index)) & mask) != 0;
 }
 
 void ContextScope::SetFlagAt(intptr_t scope_index,
-                             intptr_t mask,
+                             intptr_t bit_index,
                              bool value) const {
+  const intptr_t mask = 1 << bit_index;
   intptr_t flags = Smi::Value(untag()->flags_at(scope_index));
   untag()->set_flags_at(scope_index,
                         Smi::New(value ? flags | mask : flags & ~mask));
 }
 
-bool ContextScope::IsFinalAt(intptr_t scope_index) const {
+#define DEFINE_FLAG_ACCESSORS(Name)                                            \
-  return GetFlagAt(scope_index, UntaggedContextScope::VariableDesc::kIsFinal);
+  bool ContextScope::Is##Name##At(intptr_t scope_index) const {                \
-}
+    return GetFlagAt(scope_index,                                              \
+                     UntaggedContextScope::VariableDesc::kIs##Name);           \
+  }                                                                            \
+                                                                               \
+  void ContextScope::SetIs##Name##At(intptr_t scope_index, bool value) const { \
+    SetFlagAt(scope_index, UntaggedContextScope::VariableDesc::kIs##Name,      \
+              value);                                                          \
+  }
 
-void ContextScope::SetIsFinalAt(intptr_t scope_index, bool is_final) const {
+CONTEXT_SCOPE_VARIABLE_DESC_FLAG_LIST(DEFINE_FLAG_ACCESSORS)
-  SetFlagAt(scope_index, UntaggedContextScope::VariableDesc::kIsFinal,
+#undef DEFINE_FLAG_ACCESSORS
-            is_final);
-}
-
-bool ContextScope::IsLateAt(intptr_t scope_index) const {
-  return GetFlagAt(scope_index, UntaggedContextScope::VariableDesc::kIsLate);
-}
-
-void ContextScope::SetIsLateAt(intptr_t scope_index, bool is_late) const {
-  SetFlagAt(scope_index, UntaggedContextScope::VariableDesc::kIsLate, is_late);
-}
-
-bool ContextScope::IsConstAt(intptr_t scope_index) const {
-  return GetFlagAt(scope_index, UntaggedContextScope::VariableDesc::kIsConst);
-}
-
-void ContextScope::SetIsConstAt(intptr_t scope_index, bool is_const) const {
-  SetFlagAt(scope_index, UntaggedContextScope::VariableDesc::kIsConst,
-            is_const);
-}
-
-bool ContextScope::IsInvisibleAt(intptr_t scope_index) const {
-  return GetFlagAt(scope_index,
-                   UntaggedContextScope::VariableDesc::kIsInvisible);
-}
-
-void ContextScope::SetIsInvisibleAt(intptr_t scope_index,
-                                    bool is_invisible) const {
-  SetFlagAt(scope_index, UntaggedContextScope::VariableDesc::kIsInvisible,
-            is_invisible);
-}
 
 intptr_t ContextScope::LateInitOffsetAt(intptr_t scope_index) const {
   return Smi::Value(untag()->late_init_offset_at(scope_index));
@@ -27016,6 +27050,19 @@ static void PrintSymbolicStackFrame(Zone* zone,
   PrintSymbolicStackFrameBody(buffer, function_name, url, line, column);
 }
 
+static bool IsVisibleAsFutureListener(const Function& function) {
+  if (function.is_visible()) {
+    return true;
+  }
+
+  if (function.IsImplicitClosureFunction()) {
+    return function.parent_function() == Function::null() ||
+           Function::is_visible(function.parent_function());
+  }
+
+  return false;
+}
+
 const char* StackTrace::ToCString() const {
   auto const T = Thread::Current();
   auto const zone = T->zone();
@@ -27134,35 +27181,23 @@ const char* StackTrace::ToCString() const {
       }
       const uword pc = code.PayloadStart() + pc_offset;
 
-      // If the function is not to be shown, skip.
+      const bool is_future_listener =
-      if (!FLAG_show_invisible_frames && !function.IsNull() &&
+          pc_offset == StackTraceUtils::kFutureListenerPcOffset;
-          !function.is_visible()) {
-        continue;
-      }
 
       // A visible frame ends any gap we might be in.
       in_gap = false;
 
-      // Zero pc_offset can only occur in the frame produced by the async
-      // unwinding and it corresponds to the next future listener in the
-      // chain. This function is not yet called (it will be called when
-      // the future completes) hence pc_offset is set to 0. This frame
-      // is very different from other frames which have pc_offsets
-      // corresponding to call- or yield-sites in the generated code and
-      // should be handled specially.
-      const bool is_future_listener = pc_offset == 0;
-
 #if defined(DART_PRECOMPILED_RUNTIME)
       // When printing non-symbolic frames, we normally print call
       // addresses, not return addresses, by subtracting one from the PC to
       // get an address within the preceding instruction.
       //
       // The one exception is a normal closure registered as a listener on a
-      // future. In this case, the returned pc_offset is 0, as the closure
+      // future. In this case, the returned pc_offset will be pointing to the
-      // is invoked with the value of the resolved future. Thus, we must
+      // entry pooint of the function, which will be invoked when the future
-      // report the return address, as returning a value before the closure
+      // completes. To make things more uniform stack unwinding code offets
-      // payload will cause failures to decode the frame using DWARF info.
+      // pc_offset by 1 for such cases.
-      const uword call_addr = is_future_listener ? pc : pc - 1;
+      const uword call_addr = pc - 1;
 
       if (FLAG_dwarf_stack_traces_mode) {
         if (have_footnote_callback) {
@@ -27196,14 +27231,18 @@ const char* StackTrace::ToCString() const {
         // Note: In AOT mode EmitFunctionEntrySourcePositionDescriptorIfNeeded
         // will take care of emitting a descriptor that would allow us to
         // symbolize stack frame with 0 offset.
-        code.GetInlinedFunctionsAtReturnAddress(pc_offset, &inlined_functions,
+        code.GetInlinedFunctionsAtReturnAddress(
-                                                &inlined_token_positions);
+            is_future_listener ? 0 : pc_offset, &inlined_functions,
+            &inlined_token_positions);
         ASSERT(inlined_functions.length() >= 1);
         for (intptr_t j = inlined_functions.length() - 1; j >= 0; j--) {
-          const auto& inlined = *inlined_functions[j];
+          function = inlined_functions[j]->ptr();
           auto const pos = inlined_token_positions[j];
-          if (FLAG_show_invisible_frames || inlined.is_visible()) {
+          if (is_future_listener && function.IsImplicitClosureFunction()) {
-            PrintSymbolicStackFrame(zone, &buffer, inlined, pos, frame_index,
+            function = function.parent_function();
+          }
+          if (FLAG_show_invisible_frames || function.is_visible()) {
+            PrintSymbolicStackFrame(zone, &buffer, function, pos, frame_index,
                                     /*is_line=*/FLAG_precompiled_mode);
             frame_index++;
           }
@@ -27211,10 +27250,13 @@ const char* StackTrace::ToCString() const {
         continue;
       }
 
-      auto const pos = is_future_listener ? function.token_pos()
+      if (FLAG_show_invisible_frames || function.is_visible() ||
-                                          : code.GetTokenIndexOfPC(pc);
+          (is_future_listener && IsVisibleAsFutureListener(function))) {
-      PrintSymbolicStackFrame(zone, &buffer, function, pos, frame_index);
+        auto const pos = is_future_listener ? function.token_pos()
-      frame_index++;
+                                            : code.GetTokenIndexOfPC(pc);
+        PrintSymbolicStackFrame(zone, &buffer, function, pos, frame_index);
+        frame_index++;
+      }
     }
 
     // Follow the link.

runtime/vm/object.h

@@ -11,6 +11,7 @@
 
 #include <limits>
 #include <tuple>
+#include <utility>
 
 #include "include/dart_api.h"
 #include "platform/assert.h"
@@ -457,6 +458,7 @@ class Object {
   V(RecordType, null_record_type)                                              \
   V(TypeArguments, null_type_arguments)                                        \
   V(CompressedStackMaps, null_compressed_stackmaps)                            \
+  V(Closure, null_closure)                                                     \
   V(TypeArguments, empty_type_arguments)                                       \
   V(Array, empty_array)                                                        \
   V(Array, empty_instantiations_cache_array)                                   \
@@ -3079,6 +3081,22 @@ class Function : public Object {
   ContextScopePtr context_scope() const;
   void set_context_scope(const ContextScope& value) const;
 
+  struct AwaiterLink {
+    // Context depth at which the `@pragma('vm:awaiter-link')` variable
+    // is located.
+    uint8_t depth = UntaggedClosureData::kNoAwaiterLinkDepth;
+    // Context index at which the `@pragma('vm:awaiter-link')` variable
+    // is located.
+    uint8_t index = -1;
+  };
+
+  AwaiterLink awaiter_link() const;
+  void set_awaiter_link(AwaiterLink link) const;
+  bool HasAwaiterLink() const {
+    return IsClosureFunction() &&
+           (awaiter_link().depth != UntaggedClosureData::kNoAwaiterLinkDepth);
+  }
+
   // Enclosing function of this local function.
   FunctionPtr parent_function() const;
 
@@ -3999,6 +4017,10 @@ class Function : public Object {
   FOR_EACH_FUNCTION_KIND_BIT(DEFINE_ACCESSORS)
 #undef DEFINE_ACCESSORS
 
+  static bool is_visible(FunctionPtr f) {
+    return f.untag()->kind_tag_.Read<VisibleBit>();
+  }
+
 #define DEFINE_ACCESSORS(name, accessor_name)                                  \
   void set_##accessor_name(bool value) const {                                 \
     untag()->kind_tag_.UpdateBool<name##Bit>(value);                           \
@@ -4127,17 +4149,29 @@ class ClosureData : public Object {
     return RoundedAllocationSize(sizeof(UntaggedClosureData));
   }
 
-  static intptr_t default_type_arguments_kind_offset() {
+  static intptr_t packed_fields_offset() {
-    return OFFSET_OF(UntaggedClosureData, default_type_arguments_kind_);
+    return OFFSET_OF(UntaggedClosureData, packed_fields_);
   }
 
   using DefaultTypeArgumentsKind =
       UntaggedClosureData::DefaultTypeArgumentsKind;
+  using PackedDefaultTypeArgumentsKind =
+      UntaggedClosureData::PackedDefaultTypeArgumentsKind;
+
+  static constexpr uint8_t kNoAwaiterLinkDepth =
+      UntaggedClosureData::kNoAwaiterLinkDepth;
 
  private:
   ContextScopePtr context_scope() const { return untag()->context_scope(); }
   void set_context_scope(const ContextScope& value) const;
 
+  void set_packed_fields(uint32_t value) const {
+    untag()->packed_fields_ = value;
+  }
+
+  Function::AwaiterLink awaiter_link() const;
+  void set_awaiter_link(Function::AwaiterLink link) const;
+
   // Enclosing function of this local function.
   PRECOMPILER_WSR_FIELD_DECLARATION(Function, parent_function)
 
@@ -7252,21 +7286,16 @@ class ContextScope : public Object {
 
   void ClearFlagsAt(intptr_t scope_index) const;
 
-  bool IsFinalAt(intptr_t scope_index) const;
-  void SetIsFinalAt(intptr_t scope_index, bool is_final) const;
-
-  bool IsLateAt(intptr_t scope_index) const;
-  void SetIsLateAt(intptr_t scope_index, bool is_late) const;
-
   intptr_t LateInitOffsetAt(intptr_t scope_index) const;
   void SetLateInitOffsetAt(intptr_t scope_index,
                            intptr_t late_init_offset) const;
 
-  bool IsConstAt(intptr_t scope_index) const;
+#define DECLARE_FLAG_ACCESSORS(Name)                                           \
-  void SetIsConstAt(intptr_t scope_index, bool is_const) const;
+  bool Is##Name##At(intptr_t scope_index) const;                               \
+  void SetIs##Name##At(intptr_t scope_index, bool value) const;
 
-  bool IsInvisibleAt(intptr_t scope_index) const;
+  CONTEXT_SCOPE_VARIABLE_DESC_FLAG_LIST(DECLARE_FLAG_ACCESSORS)
-  void SetIsInvisibleAt(intptr_t scope_index, bool is_invisible) const;
+#undef DECLARE_FLAG_ACCESSORS
 
   AbstractTypePtr TypeAt(intptr_t scope_index) const;
   void SetTypeAt(intptr_t scope_index, const AbstractType& type) const;
@@ -7323,8 +7352,8 @@ class ContextScope : public Object {
     return untag()->VariableDescAddr(index);
   }
 
-  bool GetFlagAt(intptr_t scope_index, intptr_t mask) const;
+  bool GetFlagAt(intptr_t scope_index, intptr_t bit_index) const;
-  void SetFlagAt(intptr_t scope_index, intptr_t mask, bool value) const;
+  void SetFlagAt(intptr_t scope_index, intptr_t bit_index, bool value) const;
 
   FINAL_HEAP_OBJECT_IMPLEMENTATION(ContextScope, Object);
   friend class Class;
@@ -13487,6 +13516,12 @@ void DumpTypeTable(Isolate* isolate);
 void DumpTypeParameterTable(Isolate* isolate);
 void DumpTypeArgumentsTable(Isolate* isolate);
 
+bool FindPragmaInMetadata(Thread* T,
+                          const Object& metadata_obj,
+                          const String& pragma_name,
+                          bool multiple = false,
+                          Object* options = nullptr);
+
 EntryPointPragma FindEntryPointPragma(IsolateGroup* isolate_group,
                                       const Array& metadata,
                                       Field* reusable_field_handle,

runtime/vm/object_store.h

@@ -199,8 +199,6 @@ class ObjectPointerVisitor;
   RW(Field, sync_star_iterator_current)                                        \
   RW(Field, sync_star_iterator_state)                                          \
   RW(Field, sync_star_iterator_yield_star_iterable)                            \
-  ARW_RELAXED(Smi, future_timeout_future_index)                                \
-  ARW_RELAXED(Smi, future_wait_future_index)                                   \
   RW(CompressedStackMaps, canonicalized_stack_map_entries)                     \
   RW(ObjectPool, global_object_pool)                                           \
   RW(Array, unique_dynamic_targets)                                            \

runtime/vm/object_test.cc

@@ -2636,8 +2636,9 @@ ISOLATE_UNIT_TEST_CASE(ContextScope) {
   EXPECT(found_captured_vars);
 
   const intptr_t local_scope_context_level = 5;
-  const ContextScope& context_scope = ContextScope::Handle(
+  const ContextScope& context_scope =
-      local_scope->PreserveOuterScope(local_scope_context_level));
+      ContextScope::Handle(local_scope->PreserveOuterScope(
+          Function::null_function(), local_scope_context_level));
   LocalScope* outer_scope = LocalScope::RestoreOuterScope(context_scope);
   EXPECT_EQ(3, outer_scope->num_variables());
 

runtime/vm/parser.cc

@@ -203,6 +203,7 @@ void ParsedFunction::AllocateVariables() {
           variable->declaration_token_pos(), variable->token_pos(), tmp,
           variable->type(), variable->kernel_offset(),
           variable->parameter_type(), variable->parameter_value());
+      raw_parameter->set_annotations_offset(variable->annotations_offset());
       if (variable->is_explicit_covariant_parameter()) {
         raw_parameter->set_is_explicit_covariant_parameter();
       }

runtime/vm/raw_object.h

@@ -1398,7 +1398,21 @@ class UntaggedClosureData : public UntaggedObject {
                     compiler::target::kSmiBits,
                 "Default type arguments kind must fit in a Smi");
 
-  DefaultTypeArgumentsKind default_type_arguments_kind_;
+  static constexpr uint8_t kNoAwaiterLinkDepth = 0xFF;
+
+  AtomicBitFieldContainer<uint32_t> packed_fields_;
+
+  using PackedDefaultTypeArgumentsKind =
+      BitField<decltype(packed_fields_), DefaultTypeArgumentsKind, 0, 8>;
+  using PackedAwaiterLinkDepth =
+      BitField<decltype(packed_fields_),
+               uint8_t,
+               PackedDefaultTypeArgumentsKind::kNextBit,
+               8>;
+  using PackedAwaiterLinkIndex = BitField<decltype(packed_fields_),
+                                          uint8_t,
+                                          PackedAwaiterLinkDepth::kNextBit,
+                                          8>;
 
   friend class Function;
   friend class UnitDeserializationRoots;
@@ -2326,6 +2340,13 @@ class UntaggedContext : public UntaggedObject {
                                 ObjectPtr);  // num_variables_
 };
 
+#define CONTEXT_SCOPE_VARIABLE_DESC_FLAG_LIST(V)                               \
+  V(Final)                                                                     \
+  V(Const)                                                                     \
+  V(Late)                                                                      \
+  V(Invisible)                                                                 \
+  V(AwaiterLink)
+
 class UntaggedContextScope : public UntaggedObject {
   RAW_HEAP_OBJECT_IMPLEMENTATION(ContextScope);
 
@@ -2336,10 +2357,11 @@ class UntaggedContextScope : public UntaggedObject {
     CompressedSmiPtr token_pos;
     CompressedStringPtr name;
     CompressedSmiPtr flags;
-    static constexpr intptr_t kIsFinal = 1 << 0;
+    enum FlagBits {
-    static constexpr intptr_t kIsConst = 1 << 1;
+#define DECLARE_BIT(Name) kIs##Name,
-    static constexpr intptr_t kIsLate = 1 << 2;
+      CONTEXT_SCOPE_VARIABLE_DESC_FLAG_LIST(DECLARE_BIT)
-    static constexpr intptr_t kIsInvisible = 1 << 3;
+#undef DECLARE_BIT
+    };
     CompressedSmiPtr late_init_offset;
     union {
       CompressedAbstractTypePtr type;

runtime/vm/scopes.cc

@@ -6,6 +6,7 @@
 #include "vm/scopes.h"
 
 #include "vm/compiler/backend/slot.h"
+#include "vm/kernel.h"
 #include "vm/object.h"
 #include "vm/object_store.h"
 #include "vm/stack_frame.h"
@@ -146,48 +147,13 @@ VariableIndex LocalScope::AllocateVariables(const Function& function,
   VariableIndex next_index =
       first_parameter_index;  // Current free frame index.
 
-  LocalVariable* chained_future = nullptr;
   LocalVariable* suspend_state_var = nullptr;
   for (intptr_t i = 0; i < num_variables(); i++) {
     LocalVariable* variable = VariableAt(i);
-    if (variable->owner() == this) {
+    if (variable->owner() == this &&
-      if (variable->is_captured()) {
+        variable->name().Equals(Symbols::SuspendStateVar())) {
-        if (variable->is_chained_future()) {
+      ASSERT(!variable->is_captured());
-          chained_future = variable;
+      suspend_state_var = variable;
-        }
-      } else {
-        if (variable->name().Equals(Symbols::SuspendStateVar())) {
-          suspend_state_var = variable;
-        }
-      }
-    }
-  }
-  if (chained_future != nullptr) {
-    AllocateContextVariable(chained_future, &context_owner);
-    *found_captured_variables = true;
-    // Remember context indices of _future variables in _Future.timeout and
-    // Future.wait. They are used while collecting async stack traces.
-    if (function.recognized_kind() == MethodRecognizer::kFutureTimeout) {
-#ifdef DEBUG
-      auto old_value = IsolateGroup::Current()
-                           ->object_store()
-                           ->future_timeout_future_index();
-      ASSERT(old_value == Object::null() ||
-             Smi::Value(old_value) == chained_future->index().value());
-#endif  // DEBUG
-      IsolateGroup::Current()->object_store()->set_future_timeout_future_index(
-          Smi::Handle(Smi::New(chained_future->index().value())));
-    } else if (function.recognized_kind() == MethodRecognizer::kFutureWait) {
-#ifdef DEBUG
-      auto old_value =
-          IsolateGroup::Current()->object_store()->future_wait_future_index();
-      ASSERT(old_value == Object::null() ||
-             Smi::Value(old_value) == chained_future->index().value());
-#endif  // DEBUG
-      IsolateGroup::Current()->object_store()->set_future_wait_future_index(
-          Smi::Handle(Smi::New(chained_future->index().value())));
-    } else {
-      UNREACHABLE();
     }
   }
 
@@ -220,23 +186,21 @@ VariableIndex LocalScope::AllocateVariables(const Function& function,
   // No overlapping of parameters and locals.
   ASSERT(next_index.value() >= first_local_index.value());
   next_index = first_local_index;
-  while (pos < num_variables()) {
+  for (; pos < num_variables(); pos++) {
     LocalVariable* variable = VariableAt(pos);
+    if (variable == suspend_state_var) {
+      continue;
+    }
+
     if (variable->owner() == this) {
       if (variable->is_captured()) {
-        // Skip the variables already pre-allocated above.
+        AllocateContextVariable(variable, &context_owner);
-        if (variable != chained_future) {
+        *found_captured_variables = true;
-          AllocateContextVariable(variable, &context_owner);
-          *found_captured_variables = true;
-        }
       } else {
-        if (variable != suspend_state_var) {
+        variable->set_index(next_index);
-          variable->set_index(next_index);
+        next_index = VariableIndex(next_index.value() - 1);
-          next_index = VariableIndex(next_index.value() - 1);
-        }
       }
     }
-    pos++;
   }
   // Allocate variables of all children.
   VariableIndex min_index = next_index;
@@ -453,7 +417,13 @@ int LocalScope::NumCapturedVariables() const {
 }
 
 ContextScopePtr LocalScope::PreserveOuterScope(
-    int current_context_level) const {
+    const Function& function,
+    intptr_t current_context_level) const {
+  Zone* zone = Thread::Current()->zone();
+  auto& library = Library::Handle(
+      zone, function.IsNull()
+                ? Library::null()
+                : Class::Handle(zone, function.Owner()).library());
   // Since code generation for nested functions is postponed until first
   // invocation, the function level of the closure scope can only be 1.
   ASSERT(function_level() == 1);
@@ -465,6 +435,8 @@ ContextScopePtr LocalScope::PreserveOuterScope(
   const ContextScope& context_scope =
       ContextScope::Handle(ContextScope::New(num_captured_vars, false));
 
+  LocalVariable* awaiter_link = nullptr;
+
   // Create a descriptor for each referenced captured variable of enclosing
   // functions to preserve its name and its context allocation information.
   int captured_idx = 0;
@@ -494,14 +466,40 @@ ContextScopePtr LocalScope::PreserveOuterScope(
       // Adjust the context level relative to the current context level,
       // since the context of the current scope will be at level 0 when
       // compiling the nested function.
-      int adjusted_context_level =
+      intptr_t adjusted_context_level =
           variable->owner()->context_level() - current_context_level;
       context_scope.SetContextLevelAt(captured_idx, adjusted_context_level);
       context_scope.SetKernelOffsetAt(captured_idx, variable->kernel_offset());
+
+      // Handle async frame link.
+      const bool is_awaiter_link = variable->ComputeIfIsAwaiterLink(library);
+      context_scope.SetIsAwaiterLinkAt(captured_idx, is_awaiter_link);
+      if (is_awaiter_link) {
+        awaiter_link = variable;
+      }
       captured_idx++;
     }
   }
   ASSERT(context_scope.num_variables() == captured_idx);  // Verify count.
+
+  if (awaiter_link != nullptr) {
+    const intptr_t depth =
+        current_context_level - awaiter_link->owner()->context_level();
+    const intptr_t index = awaiter_link->index().value();
+    if (Utils::IsUint(8, depth) && Utils::IsUint(8, index)) {
+      function.set_awaiter_link(
+          {static_cast<uint8_t>(depth), static_cast<uint8_t>(index)});
+    } else if (FLAG_precompiled_mode) {
+      OS::PrintErr(
+          "Warning: @pragma('vm:awaiter-link') marked variable %s is visible "
+          "from the function %s but the link {%" Pd ", %" Pd
+          "} can't be encoded\n",
+          awaiter_link->name().ToCString(),
+          function.IsNull() ? "<?>" : function.ToFullyQualifiedCString(), depth,
+          index);
+    }
+  }
+
   return context_scope.ptr();
 }
 
@@ -528,6 +526,7 @@ LocalScope* LocalScope::RestoreOuterScope(const ContextScope& context_scope) {
                             AbstractType::ZoneHandle(context_scope.TypeAt(i)),
                             context_scope.KernelOffsetAt(i));
     }
+    variable->set_is_awaiter_link(context_scope.IsAwaiterLinkAt(i));
     variable->set_is_captured();
     variable->set_index(VariableIndex(context_scope.ContextIndexAt(i)));
     if (context_scope.IsFinalAt(i)) {
@@ -597,6 +596,20 @@ ContextScopePtr LocalScope::CreateImplicitClosureScope(const Function& func) {
   return context_scope.ptr();
 }
 
+bool LocalVariable::ComputeIfIsAwaiterLink(const Library& library) {
+  if (is_awaiter_link_ == IsAwaiterLink::kUnknown) {
+    RELEASE_ASSERT(annotations_offset_ != kNoKernelOffset);
+    Thread* T = Thread::Current();
+    Zone* Z = T->zone();
+    const auto& metadata = Object::Handle(
+        Z, kernel::EvaluateMetadata(library, annotations_offset_,
+                                    /* is_annotations_offset = */ true));
+    set_is_awaiter_link(
+        FindPragmaInMetadata(T, metadata, Symbols::vm_awaiter_link()));
+  }
+  return is_awaiter_link_ == IsAwaiterLink::kLink;
+}
+
 bool LocalVariable::Equals(const LocalVariable& other) const {
   if (HasIndex() && other.HasIndex() && (index() == other.index())) {
     if (is_captured() == other.is_captured()) {

runtime/vm/scopes.h

@@ -85,6 +85,7 @@ class LocalVariable : public ZoneAllocated {
         token_pos_(token_pos),
         name_(name),
         kernel_offset_(kernel_offset),
+        annotations_offset_(kNoKernelOffset),
         owner_(nullptr),
         type_(type),
         parameter_type_(parameter_type),
@@ -97,10 +98,10 @@ class LocalVariable : public ZoneAllocated {
         is_forced_stack_(false),
         covariance_mode_(kNotCovariant),
         is_late_(false),
-        is_chained_future_(false),
         late_init_offset_(0),
         type_check_mode_(kDoTypeCheck),
-        index_() {
+        index_(),
+        is_awaiter_link_(IsAwaiterLink::kNotLink) {
     DEBUG_ASSERT(type.IsNotTemporaryScopedHandle());
     ASSERT(type.IsFinalized());
     ASSERT(name.IsSymbol());
@@ -113,12 +114,19 @@ class LocalVariable : public ZoneAllocated {
   TokenPosition declaration_token_pos() const { return declaration_pos_; }
   const String& name() const { return name_; }
   intptr_t kernel_offset() const { return kernel_offset_; }
+  intptr_t annotations_offset() const { return annotations_offset_; }
   LocalScope* owner() const { return owner_; }
   void set_owner(LocalScope* owner) {
     ASSERT(owner_ == nullptr);
     owner_ = owner;
   }
 
+  void set_annotations_offset(intptr_t offset) {
+    annotations_offset_ = offset;
+    is_awaiter_link_ = (offset == kNoKernelOffset) ? IsAwaiterLink::kNotLink
+                                                   : IsAwaiterLink::kUnknown;
+  }
+
   const AbstractType& type() const { return type_; }
 
   CompileType* parameter_type() const { return parameter_type_; }
@@ -130,6 +138,11 @@ class LocalVariable : public ZoneAllocated {
   bool is_captured() const { return is_captured_; }
   void set_is_captured() { is_captured_ = true; }
 
+  bool ComputeIfIsAwaiterLink(const Library& library);
+  void set_is_awaiter_link(bool value) {
+    is_awaiter_link_ = value ? IsAwaiterLink::kLink : IsAwaiterLink::kNotLink;
+  }
+
   // Variables marked as forced to stack are skipped and not captured by
   // CaptureLocalVariables - which iterates scope chain between two scopes
   // and indiscriminately marks all variables as captured.
@@ -140,9 +153,6 @@ class LocalVariable : public ZoneAllocated {
   bool is_late() const { return is_late_; }
   void set_is_late() { is_late_ = true; }
 
-  bool is_chained_future() const { return is_chained_future_; }
-  void set_is_chained_future() { is_chained_future_ = true; }
-
   intptr_t late_init_offset() const { return late_init_offset_; }
   void set_late_init_offset(intptr_t late_init_offset) {
     late_init_offset_ = late_init_offset;
@@ -230,6 +240,7 @@ class LocalVariable : public ZoneAllocated {
   const TokenPosition token_pos_;
   const String& name_;
   const intptr_t kernel_offset_;
+  intptr_t annotations_offset_;
   LocalScope* owner_;  // Local scope declaring this variable.
 
   const AbstractType& type_;  // Declaration type of local variable.
@@ -247,11 +258,17 @@ class LocalVariable : public ZoneAllocated {
   bool is_forced_stack_;
   CovarianceMode covariance_mode_;
   bool is_late_;
-  bool is_chained_future_;
   intptr_t late_init_offset_;
   TypeCheckMode type_check_mode_;
   VariableIndex index_;
 
+  enum class IsAwaiterLink {
+    kUnknown,
+    kNotLink,
+    kLink,
+  };
+  IsAwaiterLink is_awaiter_link_;
+
   friend class LocalScope;
   DISALLOW_COPY_AND_ASSIGN(LocalVariable);
 };
@@ -402,7 +419,8 @@ class LocalScope : public ZoneAllocated {
 
   // Create a ContextScope object describing all captured variables referenced
   // from this scope and belonging to outer scopes.
-  ContextScopePtr PreserveOuterScope(int current_context_level) const;
+  ContextScopePtr PreserveOuterScope(const Function& function,
+                                     intptr_t current_context_level) const;
 
   // Mark all local variables that are accessible from this scope up to
   // top_scope (included) as captured unless they are marked as forced to stack.

runtime/vm/stack_trace.cc

@@ -11,6 +11,8 @@
 
 namespace dart {
 
+namespace {
+
 // Keep in sync with:
 // - sdk/lib/async/stream_controller.dart:_StreamController._STATE_SUBSCRIBED.
 const intptr_t k_StreamController__STATE_SUBSCRIBED = 1;
@@ -21,331 +23,8 @@ const intptr_t k_FutureListener_stateCatchError = 2;
 // - sdk/lib/async/future_impl.dart:_FutureListener.stateWhenComplete.
 const intptr_t k_FutureListener_stateWhenComplete = 8;
 
-// Keep in sync with sdk/lib/async/future_impl.dart:_FutureListener.handleValue.
+bool WasPreviouslySuspended(const Function& function,
-const intptr_t kNumArgsFutureListenerHandleValue = 1;
+                            const Object& suspend_state_var) {
-
-// Instance caches library and field references.
-// This way we don't have to do the look-ups for every frame in the stack.
-CallerClosureFinder::CallerClosureFinder(Zone* zone)
-    : closure_(Closure::Handle(zone)),
-      receiver_context_(Context::Handle(zone)),
-      receiver_function_(Function::Handle(zone)),
-      parent_function_(Function::Handle(zone)),
-      suspend_state_(SuspendState::Handle(zone)),
-      context_entry_(Object::Handle(zone)),
-      future_(Object::Handle(zone)),
-      listener_(Object::Handle(zone)),
-      callback_(Object::Handle(zone)),
-      controller_(Object::Handle(zone)),
-      state_(Object::Handle(zone)),
-      var_data_(Object::Handle(zone)),
-      callback_instance_(Object::Handle(zone)),
-      future_impl_class(Class::Handle(zone)),
-      future_listener_class(Class::Handle(zone)),
-      async_star_stream_controller_class(Class::Handle(zone)),
-      stream_controller_class(Class::Handle(zone)),
-      sync_stream_controller_class(Class::Handle(zone)),
-      controller_subscription_class(Class::Handle(zone)),
-      buffering_stream_subscription_class(Class::Handle(zone)),
-      stream_iterator_class(Class::Handle(zone)),
-      future_result_or_listeners_field(Field::Handle(zone)),
-      callback_field(Field::Handle(zone)),
-      future_listener_state_field(Field::Handle(zone)),
-      future_listener_result_field(Field::Handle(zone)),
-      controller_controller_field(Field::Handle(zone)),
-      var_data_field(Field::Handle(zone)),
-      state_field(Field::Handle(zone)),
-      on_data_field(Field::Handle(zone)),
-      state_data_field(Field::Handle(zone)),
-      has_value_field(Field::Handle(zone)) {
-  const auto& async_lib = Library::Handle(zone, Library::AsyncLibrary());
-  // Look up classes:
-  // - async:
-  future_impl_class = async_lib.LookupClassAllowPrivate(Symbols::FutureImpl());
-  ASSERT(!future_impl_class.IsNull());
-  future_listener_class =
-      async_lib.LookupClassAllowPrivate(Symbols::_FutureListener());
-  ASSERT(!future_listener_class.IsNull());
-  // - async*:
-  async_star_stream_controller_class =
-      async_lib.LookupClassAllowPrivate(Symbols::_AsyncStarStreamController());
-  ASSERT(!async_star_stream_controller_class.IsNull());
-  stream_controller_class =
-      async_lib.LookupClassAllowPrivate(Symbols::_StreamController());
-  ASSERT(!stream_controller_class.IsNull());
-  sync_stream_controller_class =
-      async_lib.LookupClassAllowPrivate(Symbols::_SyncStreamController());
-  ASSERT(!sync_stream_controller_class.IsNull());
-  controller_subscription_class =
-      async_lib.LookupClassAllowPrivate(Symbols::_ControllerSubscription());
-  ASSERT(!controller_subscription_class.IsNull());
-  buffering_stream_subscription_class = async_lib.LookupClassAllowPrivate(
-      Symbols::_BufferingStreamSubscription());
-  ASSERT(!buffering_stream_subscription_class.IsNull());
-  stream_iterator_class =
-      async_lib.LookupClassAllowPrivate(Symbols::_StreamIterator());
-  ASSERT(!stream_iterator_class.IsNull());
-
-  // Look up fields:
-  // - async:
-  future_result_or_listeners_field =
-      future_impl_class.LookupFieldAllowPrivate(Symbols::_resultOrListeners());
-  ASSERT(!future_result_or_listeners_field.IsNull());
-  callback_field =
-      future_listener_class.LookupFieldAllowPrivate(Symbols::callback());
-  ASSERT(!callback_field.IsNull());
-  future_listener_state_field =
-      future_listener_class.LookupFieldAllowPrivate(Symbols::state());
-  ASSERT(!future_listener_state_field.IsNull());
-  future_listener_result_field =
-      future_listener_class.LookupFieldAllowPrivate(Symbols::result());
-  ASSERT(!future_listener_result_field.IsNull());
-  // - async*:
-  controller_controller_field =
-      async_star_stream_controller_class.LookupFieldAllowPrivate(
-          Symbols::controller());
-  ASSERT(!controller_controller_field.IsNull());
-  state_field =
-      stream_controller_class.LookupFieldAllowPrivate(Symbols::_state());
-  ASSERT(!state_field.IsNull());
-  var_data_field =
-      stream_controller_class.LookupFieldAllowPrivate(Symbols::_varData());
-  ASSERT(!var_data_field.IsNull());
-  on_data_field = buffering_stream_subscription_class.LookupFieldAllowPrivate(
-      Symbols::_onData());
-  ASSERT(!on_data_field.IsNull());
-  state_data_field =
-      stream_iterator_class.LookupFieldAllowPrivate(Symbols::_stateData());
-  ASSERT(!state_data_field.IsNull());
-  has_value_field =
-      stream_iterator_class.LookupFieldAllowPrivate(Symbols::_hasValue());
-  ASSERT(!has_value_field.IsNull());
-}
-
-ClosurePtr CallerClosureFinder::GetCallerInFutureImpl(const Object& future) {
-  ASSERT(!future.IsNull());
-  ASSERT(future.GetClassId() == future_impl_class.id());
-  // Since this function is recursive, we have to keep a local ref.
-  auto& listener = Object::Handle(GetFutureFutureListener(future));
-  if (listener.IsNull()) {
-    return Closure::null();
-  }
-  return GetCallerInFutureListener(listener);
-}
-
-ClosurePtr CallerClosureFinder::FindCallerInAsyncStarStreamController(
-    const Object& async_star_stream_controller) {
-  ASSERT(async_star_stream_controller.IsInstance());
-  ASSERT(async_star_stream_controller.GetClassId() ==
-         async_star_stream_controller_class.id());
-
-  controller_ = Instance::Cast(async_star_stream_controller)
-                    .GetField(controller_controller_field);
-  ASSERT(!controller_.IsNull());
-  ASSERT(controller_.GetClassId() == sync_stream_controller_class.id());
-
-  // Get the _StreamController._state field.
-  state_ = Instance::Cast(controller_).GetField(state_field);
-  ASSERT(state_.IsSmi());
-  if (Smi::Cast(state_).Value() != k_StreamController__STATE_SUBSCRIBED) {
-    return Closure::null();
-  }
-
-  // Get the _StreamController._varData field.
-  var_data_ = Instance::Cast(controller_).GetField(var_data_field);
-  ASSERT(var_data_.GetClassId() == controller_subscription_class.id());
-
-  // _ControllerSubscription<T>/_BufferingStreamSubscription.<T>_onData
-  callback_ = Instance::Cast(var_data_).GetField(on_data_field);
-  ASSERT(callback_.IsClosure());
-
-  // If this is not the "_StreamIterator._onData" tear-off, we return the
-  // callback we found.
-  receiver_function_ = Closure::Cast(callback_).function();
-  if (!receiver_function_.IsImplicitInstanceClosureFunction() ||
-      receiver_function_.Owner() != stream_iterator_class.ptr()) {
-    return Closure::Cast(callback_).ptr();
-  }
-
-  // All implicit closure functions (tear-offs) have the "this" receiver
-  // captured.
-  receiver_context_ = Closure::Cast(callback_).context();
-  ASSERT(receiver_context_.num_variables() == 1);
-  callback_instance_ = receiver_context_.At(0);
-  ASSERT(callback_instance_.IsInstance());
-
-  // If the async* stream is await-for'd:
-  if (callback_instance_.GetClassId() == stream_iterator_class.id()) {
-    // If `_hasValue` is true then the `StreamIterator._stateData` field
-    // contains the iterator's value. In that case we cannot unwind anymore.
-    //
-    // Notice: With correct async* semantics this may never be true: The async*
-    // generator should only be invoked to produce a value if there's an
-    // in-progress `await streamIterator.moveNext()` call. Once such call has
-    // finished the async* generator should be paused/yielded until the next
-    // such call - and being paused/yielded means it should not appear in stack
-    // traces.
-    //
-    // See dartbug.com/48695.
-    const auto& stream_iterator = Instance::Cast(callback_instance_);
-    if (stream_iterator.GetField(has_value_field) ==
-        Object::bool_true().ptr()) {
-      return Closure::null();
-    }
-
-    // If we have an await'er for `await streamIterator.moveNext()` we continue
-    // unwinding there.
-    //
-    // Notice: With correct async* semantics this may always contain a Future
-    // See also comment above as well as dartbug.com/48695.
-    future_ = stream_iterator.GetField(state_data_field);
-    if (future_.GetClassId() == future_impl_class.id()) {
-      return GetCallerInFutureImpl(future_);
-    }
-    return Closure::null();
-  }
-
-  UNREACHABLE();  // If no onData is found we have a bug.
-}
-
-ClosurePtr CallerClosureFinder::GetCallerInFutureListener(
-    const Object& future_listener) {
-  auto value = GetFutureListenerState(future_listener);
-
-  // If the _FutureListener is a `then`, `catchError`, or `whenComplete`
-  // listener, follow the Future being completed, `result`, instead of the
-  // dangling whenComplete `callback`.
-  if (value == k_FutureListener_stateThen ||
-      value == k_FutureListener_stateCatchError ||
-      value == k_FutureListener_stateWhenComplete) {
-    future_ = GetFutureListenerResult(future_listener);
-    return GetCallerInFutureImpl(future_);
-  }
-
-  // If no chained futures, fall back on _FutureListener.callback.
-  return GetFutureListenerCallback(future_listener);
-}
-
-ClosurePtr CallerClosureFinder::FindCallerFromSuspendState(
-    const SuspendState& suspend_state) {
-  context_entry_ = suspend_state.function_data();
-  if (context_entry_.GetClassId() == future_impl_class.id()) {
-    return GetCallerInFutureImpl(context_entry_);
-  } else if (context_entry_.GetClassId() ==
-             async_star_stream_controller_class.id()) {
-    return FindCallerInAsyncStarStreamController(context_entry_);
-  } else {
-    UNREACHABLE();
-  }
-}
-
-bool CallerClosureFinder::IsAsyncCallback(const Function& function) {
-  parent_function_ = function.parent_function();
-  auto kind = parent_function_.recognized_kind();
-  return (kind == MethodRecognizer::kSuspendState_createAsyncCallbacks) ||
-         (kind == MethodRecognizer::kSuspendState_createAsyncStarCallback);
-}
-
-SuspendStatePtr CallerClosureFinder::GetSuspendStateFromAsyncCallback(
-    const Closure& closure) {
-  ASSERT(IsAsyncCallback(Function::Handle(closure.function())));
-  // Async/async* handler only captures the receiver (SuspendState).
-  receiver_context_ = closure.context();
-  RELEASE_ASSERT(receiver_context_.num_variables() == 1);
-  return SuspendState::RawCast(receiver_context_.At(0));
-}
-
-ClosurePtr CallerClosureFinder::FindCaller(const Closure& receiver_closure) {
-  receiver_function_ = receiver_closure.function();
-  receiver_context_ = receiver_closure.context();
-
-  if (IsAsyncCallback(receiver_function_)) {
-    suspend_state_ = GetSuspendStateFromAsyncCallback(receiver_closure);
-    return FindCallerFromSuspendState(suspend_state_);
-  }
-
-  if (receiver_function_.HasParent()) {
-    parent_function_ = receiver_function_.parent_function();
-    if (parent_function_.recognized_kind() ==
-        MethodRecognizer::kFutureTimeout) {
-      ASSERT(IsolateGroup::Current()
-                 ->object_store()
-                 ->future_timeout_future_index() != Object::null());
-      const intptr_t future_index =
-          Smi::Value(IsolateGroup::Current()
-                         ->object_store()
-                         ->future_timeout_future_index());
-      context_entry_ = receiver_context_.At(future_index);
-      return GetCallerInFutureImpl(context_entry_);
-    }
-
-    if (parent_function_.recognized_kind() == MethodRecognizer::kFutureWait) {
-      receiver_context_ = receiver_context_.parent();
-      ASSERT(!receiver_context_.IsNull());
-      ASSERT(
-          IsolateGroup::Current()->object_store()->future_wait_future_index() !=
-          Object::null());
-      const intptr_t future_index = Smi::Value(
-          IsolateGroup::Current()->object_store()->future_wait_future_index());
-      context_entry_ = receiver_context_.At(future_index);
-      return GetCallerInFutureImpl(context_entry_);
-    }
-  }
-
-  return Closure::null();
-}
-
-ObjectPtr CallerClosureFinder::GetFutureFutureListener(const Object& future) {
-  ASSERT(future.GetClassId() == future_impl_class.id());
-  auto& listener = Object::Handle(
-      Instance::Cast(future).GetField(future_result_or_listeners_field));
-  // This field can either hold a _FutureListener, Future, or the Future result.
-  if (listener.GetClassId() != future_listener_class.id()) {
-    return Closure::null();
-  }
-  return listener.ptr();
-}
-
-intptr_t CallerClosureFinder::GetFutureListenerState(
-    const Object& future_listener) {
-  ASSERT(future_listener.GetClassId() == future_listener_class.id());
-  state_ =
-      Instance::Cast(future_listener).GetField(future_listener_state_field);
-  return Smi::Cast(state_).Value();
-}
-
-ClosurePtr CallerClosureFinder::GetFutureListenerCallback(
-    const Object& future_listener) {
-  ASSERT(future_listener.GetClassId() == future_listener_class.id());
-  return Closure::RawCast(
-      Instance::Cast(future_listener).GetField(callback_field));
-}
-
-ObjectPtr CallerClosureFinder::GetFutureListenerResult(
-    const Object& future_listener) {
-  ASSERT(future_listener.GetClassId() == future_listener_class.id());
-  return Instance::Cast(future_listener).GetField(future_listener_result_field);
-}
-
-bool CallerClosureFinder::HasCatchError(const Object& future_listener) {
-  ASSERT(future_listener.GetClassId() == future_listener_class.id());
-  listener_ = future_listener.ptr();
-  Object& result = Object::Handle();
-  // Iterate through any `.then()` chain.
-  while (!listener_.IsNull()) {
-    if (GetFutureListenerState(listener_) == k_FutureListener_stateCatchError) {
-      return true;
-    }
-    result = GetFutureListenerResult(listener_);
-    RELEASE_ASSERT(!result.IsNull());
-    listener_ = GetFutureFutureListener(result);
-  }
-  return false;
-}
-
-bool CallerClosureFinder::WasPreviouslySuspended(
-    const Function& function,
-    const Object& suspend_state_var) {
   if (!suspend_state_var.IsSuspendState()) {
     return false;
   }
@@ -362,189 +41,489 @@ bool CallerClosureFinder::WasPreviouslySuspended(
   }
 }
 
-ClosurePtr StackTraceUtils::ClosureFromFrameFunction(
+// Unwinder which starts by unwinding the synchronous portion of the stack
-    Zone* zone,
+// until it reaches a frame which has an asynchronous awaiter (e.g. an
-    CallerClosureFinder* caller_closure_finder,
+// activation of the async function which has a listener attached to the
-    const DartFrameIterator& frames,
+// corresponding Future object) and then unwinds through the chain
-    StackFrame* frame,
+// of awaiters.
-    bool* skip_frame,
+class AsyncAwareStackUnwinder : public ValueObject {
-    bool* is_async) {
+ public:
-  auto& function = Function::Handle(zone);
+  explicit AsyncAwareStackUnwinder(Thread* thread)
+      : zone_(thread->zone()),
+        sync_frames_(thread, StackFrameIterator::kNoCrossThreadIteration),
+        sync_frame_(nullptr),
+        awaiter_frame_{Closure::Handle(zone_), Object::Handle(zone_)},
+        closure_(Closure::Handle(zone_)),
+        code_(Code::Handle(zone_)),
+        context_(Context::Handle(zone_)),
+        function_(Function::Handle(zone_)),
+        parent_function_(Function::Handle(zone_)),
+        object_(Object::Handle(zone_)),
+        suspend_state_(SuspendState::Handle(zone_)),
+        controller_(Object::Handle(zone_)),
+        subscription_(Object::Handle(zone_)),
+        stream_iterator_(Object::Handle(zone_)),
+        async_lib_(Library::Handle(zone_, Library::AsyncLibrary())),
+        null_closure_(Closure::Handle(zone_)) {}
 
-  function = frame->LookupDartFunction();
+  bool Unwind(intptr_t skip_frames,
-  if (function.IsNull()) {
+              std::function<void(const StackTraceUtils::Frame&)> handle_frame);
-    return Closure::null();
+
+ private:
+  bool HandleSynchronousFrame();
+
+  void UnwindAwaiterFrame();
+
+  // Returns either the `onData` or the Future awaiter.
+  ObjectPtr FindCallerInAsyncStarStreamController(
+      const Object& async_star_stream_controller);
+
+  void InitializeAwaiterFrameFromSuspendState();
+
+  void InitializeAwaiterFrameFromFutureListener(const Object& listener);
+
+  void UnwindToAwaiter();
+
+  // |frame.next| is a |_Future| instance. Unwind to the next frame.
+  void UnwindFrameToFutureListener();
+
+  // |frame.next| is an |_AsyncStarStreamController| instance corresponding to
+  // an async* function. Unwind to the next frame.
+  void UnwindFrameToStreamListener();
+
+  ObjectPtr GetReceiver() const;
+
+#define USED_CLASS_LIST(V)                                                     \
+  V(_AsyncStarStreamController)                                                \
+  V(_BufferingStreamSubscription)                                              \
+  V(_Completer)                                                                \
+  V(_AsyncCompleter)                                                           \
+  V(_SyncCompleter)                                                            \
+  V(_ControllerSubscription)                                                   \
+  V(_Future)                                                                   \
+  V(_FutureListener)                                                           \
+  V(_StreamController)                                                         \
+  V(_StreamIterator)                                                           \
+  V(_SyncStreamController)
+
+  enum ClassId {
+#define DECLARE_CONSTANT(symbol) k##symbol,
+    USED_CLASS_LIST(DECLARE_CONSTANT)
+#undef DECLARE_CONSTANT
+  };
+
+#define USED_FIELD_LIST(V)                                                     \
+  V(_AsyncStarStreamController, controller)                                    \
+  V(_BufferingStreamSubscription, _onData)                                     \
+  V(_Completer, future)                                                        \
+  V(_Future, _resultOrListeners)                                               \
+  V(_FutureListener, callback)                                                 \
+  V(_FutureListener, result)                                                   \
+  V(_FutureListener, state)                                                    \
+  V(_StreamController, _state)                                                 \
+  V(_StreamController, _varData)                                               \
+  V(_StreamIterator, _hasValue)                                                \
+  V(_StreamIterator, _stateData)
+
+  enum FieldId {
+#define DECLARE_CONSTANT(class_symbol, field_symbol)                           \
+  k##class_symbol##_##field_symbol,
+    USED_FIELD_LIST(DECLARE_CONSTANT)
+#undef DECLARE_CONSTANT
+  };
+
+#define PLUS_ONE(...) +1
+  static constexpr intptr_t kUsedClassCount = 0 USED_CLASS_LIST(PLUS_ONE);
+  static constexpr intptr_t kUsedFieldCount = 0 USED_FIELD_LIST(PLUS_ONE);
+#undef PLUS_ONE
+
+#define DECLARE_GETTER(symbol)                                                 \
+  const Class& symbol() {                                                      \
+    auto& cls = classes_[k##symbol];                                           \
+    if (cls == nullptr) {                                                      \
+      cls = &Class::Handle(                                                    \
+          zone_, async_lib_.LookupClassAllowPrivate(Symbols::symbol()));       \
+      ASSERT(!cls->IsNull());                                                  \
+    }                                                                          \
+    return *cls;                                                               \
+  }
+  USED_CLASS_LIST(DECLARE_GETTER)
+#undef DECLARE_GETTER
+
+#define DECLARE_GETTER(class_symbol, field_symbol)                             \
+  ObjectPtr Get##class_symbol##_##field_symbol(const Object& obj) {            \
+    auto& field = fields_[k##class_symbol##_##field_symbol];                   \
+    if (field == nullptr) {                                                    \
+      field = &Field::Handle(zone_, class_symbol().LookupFieldAllowPrivate(    \
+                                        Symbols::field_symbol()));             \
+      ASSERT(!field->IsNull());                                                \
+    }                                                                          \
+    return Instance::Cast(obj).GetField(*field);                               \
+  }
+  USED_FIELD_LIST(DECLARE_GETTER)
+#undef DECLARE_GETTER
+
+  struct AwaiterFrame {
+    Closure& closure;
+    Object& next;
+    bool has_catch_error;
+  };
+
+  Zone* zone_;
+  DartFrameIterator sync_frames_;
+
+  StackFrame* sync_frame_;
+  AwaiterFrame awaiter_frame_;
+
+  Closure& closure_;
+  Code& code_;
+  Context& context_;
+  Function& function_;
+  Function& parent_function_;
+  Object& object_;
+  SuspendState& suspend_state_;
+
+  Object& controller_;
+  Object& subscription_;
+  Object& stream_iterator_;
+
+  const Library& async_lib_;
+  Class* classes_[kUsedClassCount] = {};
+  Field* fields_[kUsedFieldCount] = {};
+
+  const Closure& null_closure_;
+
+  DISALLOW_COPY_AND_ASSIGN(AsyncAwareStackUnwinder);
+};
+
+bool AsyncAwareStackUnwinder::Unwind(
+    intptr_t skip_frames,
+    std::function<void(const StackTraceUtils::Frame&)> handle_frame) {
+  // First skip the given number of synchronous frames.
+  sync_frame_ = sync_frames_.NextFrame();
+  while (skip_frames > 0 && sync_frame_ != nullptr) {
+    sync_frame_ = sync_frames_.NextFrame();
+    skip_frames--;
   }
 
-  if (function.IsAsyncFunction() || function.IsAsyncGenerator()) {
+  // Continue unwinding synchronous portion of the stack looking for
-    auto& suspend_state = Object::Handle(
+  // a synchronous frame which has an awaiter.
-        zone, *reinterpret_cast<ObjectPtr*>(LocalVarAddress(
+  while (sync_frame_ != nullptr && awaiter_frame_.closure.IsNull()) {
-                  frame->fp(), runtime_frame_layout.FrameSlotForVariableIndex(
+    const bool was_handled = HandleSynchronousFrame();
-                                   SuspendState::kSuspendStateVarIndex))));
+    if (!was_handled) {
-    if (caller_closure_finder->WasPreviouslySuspended(function,
+      code_ = sync_frame_->LookupDartCode();
-                                                      suspend_state)) {
+      const uword pc_offset = sync_frame_->pc() - code_.PayloadStart();
-      *is_async = true;
+      handle_frame({sync_frame_, code_, pc_offset, null_closure_, false});
-      return caller_closure_finder->FindCallerFromSuspendState(
-          SuspendState::Cast(suspend_state));
     }
-
+    sync_frame_ = sync_frames_.NextFrame();
-    // Still running the sync part before the first await.
-    return Closure::null();
   }
 
-  // May have been called from `_FutureListener.handleValue`, which means its
+  // Traverse awaiter frames.
-  // receiver holds the Future chain.
+  bool any_async = false;
-  DartFrameIterator future_frames(frames);
+  for (; !awaiter_frame_.closure.IsNull(); UnwindToAwaiter()) {
-  if (function.recognized_kind() == MethodRecognizer::kRootZoneRunUnary) {
+    function_ = awaiter_frame_.closure.function();
-    frame = future_frames.NextFrame();
+    if (function_.IsNull()) {
-    function = frame->LookupDartFunction();
-    if (function.recognized_kind() !=
-        MethodRecognizer::kFutureListenerHandleValue) {
-      return Closure::null();
-    }
-  }
-  if (function.recognized_kind() ==
-      MethodRecognizer::kFutureListenerHandleValue) {
-    *is_async = true;
-    *skip_frame = true;
-
-    // The _FutureListener receiver is at the top of the previous frame, right
-    // before the arguments to the call.
-    Object& receiver =
-        Object::Handle(*(reinterpret_cast<ObjectPtr*>(frame->GetCallerSp()) +
-                         kNumArgsFutureListenerHandleValue));
-    if (receiver.ptr() == Object::optimized_out().ptr()) {
-      // In the very rare case that _FutureListener.handleValue has deoptimized
-      // it may override the receiver slot in the caller frame with "<optimized
-      // out>" due to the `this` no longer being needed.
-      return Closure::null();
-    }
-
-    return caller_closure_finder->GetCallerInFutureListener(receiver);
-  }
-
-  return Closure::null();
-}
-
-void StackTraceUtils::UnwindAwaiterChain(
-    Zone* zone,
-    const GrowableObjectArray& code_array,
-    GrowableArray<uword>* pc_offset_array,
-    CallerClosureFinder* caller_closure_finder,
-    const Closure& leaf_closure) {
-  auto& code = Code::Handle(zone);
-  auto& function = Function::Handle(zone);
-  auto& closure = Closure::Handle(zone, leaf_closure.ptr());
-  auto& pc_descs = PcDescriptors::Handle(zone);
-  auto& suspend_state = SuspendState::Handle(zone);
-
-  // Inject async suspension marker.
-  code_array.Add(StubCode::AsynchronousGapMarker());
-  pc_offset_array->Add(0);
-
-  // Traverse the trail of async futures all the way up.
-  for (; !closure.IsNull();
-       closure = caller_closure_finder->FindCaller(closure)) {
-    function = closure.function();
-    if (function.IsNull()) {
       continue;
     }
-    if (caller_closure_finder->IsAsyncCallback(function)) {
+
-      suspend_state =
+    any_async = true;
-          caller_closure_finder->GetSuspendStateFromAsyncCallback(closure);
+    if (awaiter_frame_.next.IsSuspendState()) {
-      const uword pc = suspend_state.pc();
+      const uword pc = SuspendState::Cast(awaiter_frame_.next).pc();
       if (pc == 0) {
         // Async function is already resumed.
         continue;
       }
-      code = suspend_state.GetCodeObject();
-      code_array.Add(code);
-      const uword pc_offset = pc - code.PayloadStart();
-      ASSERT(pc_offset > 0 && pc_offset <= code.Size());
-      pc_offset_array->Add(pc_offset);
-    } else {
-      // In hot-reload-test-mode we sometimes have to do this:
-      code = function.EnsureHasCode();
-      RELEASE_ASSERT(!code.IsNull());
-      code_array.Add(code);
-      pc_descs = code.pc_descriptors();
-      // We reached a non-async closure receiving the yielded value.
-      pc_offset_array->Add(0);
-    }
 
-    // Inject async suspension marker.
+      code_ = SuspendState::Cast(awaiter_frame_.next).GetCodeObject();
-    code_array.Add(StubCode::AsynchronousGapMarker());
+      const uword pc_offset = pc - code_.PayloadStart();
-    pc_offset_array->Add(0);
+      handle_frame({nullptr, code_, pc_offset, awaiter_frame_.closure,
+                    awaiter_frame_.has_catch_error});
+    } else {
+      // This is an asynchronous continuation represented by a closure which
+      // will handle successful completion. This function is not yet executing
+      // so we have to use artificial marker offset (1).
+      code_ = function_.EnsureHasCode();
+      RELEASE_ASSERT(!code_.IsNull());
+      const uword pc_offset =
+          (function_.entry_point() + StackTraceUtils::kFutureListenerPcOffset) -
+          code_.PayloadStart();
+      handle_frame({nullptr, code_, pc_offset, awaiter_frame_.closure,
+                    awaiter_frame_.has_catch_error});
+    }
   }
+  return any_async;
+}
+
+ObjectPtr AsyncAwareStackUnwinder::GetReceiver() const {
+  return *(reinterpret_cast<ObjectPtr*>(sync_frame_->GetCallerSp()) +
+           function_.num_fixed_parameters() - 1);
+}
+
+bool AsyncAwareStackUnwinder::HandleSynchronousFrame() {
+  function_ = sync_frame_->LookupDartFunction();
+  if (function_.IsNull()) {
+    return false;
+  }
+
+  // This is an invocation of an `async` or `async*` function.
+  if (function_.IsAsyncFunction() || function_.IsAsyncGenerator()) {
+    InitializeAwaiterFrameFromSuspendState();
+    return false;
+  }
+
+  // This is an invocation of a closure which has a variable marked
+  // with `@pragma('vm:awaiter-link')` which points to the awaiter.
+  if (function_.HasAwaiterLink()) {
+    object_ = GetReceiver();
+    if (object_.IsClosure() &&
+        StackTraceUtils::GetSuspendState(Closure::Cast(object_),
+                                         &awaiter_frame_.next)) {
+      awaiter_frame_.closure ^= object_.ptr();
+      return true;  // Hide this frame from the stack trace.
+    }
+  }
+
+  // This is `_FutureListener.handleValue(...)` invocation.
+  if (function_.recognized_kind() ==
+      MethodRecognizer::kFutureListenerHandleValue) {
+    object_ = GetReceiver();
+    InitializeAwaiterFrameFromFutureListener(object_);
+    UnwindToAwaiter();
+    return true;  // Hide this frame from the stack trace.
+  }
+
+  return false;
+}
+
+void AsyncAwareStackUnwinder::InitializeAwaiterFrameFromSuspendState() {
+  if (function_.IsAsyncFunction() || function_.IsAsyncGenerator()) {
+    // Check if we reached a resumed asynchronous function. In this case we
+    // are going to start following async frames after we emit this frame.
+    object_ = *reinterpret_cast<ObjectPtr*>(LocalVarAddress(
+        sync_frame_->fp(), runtime_frame_layout.FrameSlotForVariableIndex(
+                               SuspendState::kSuspendStateVarIndex)));
+
+    awaiter_frame_.closure = Closure::null();
+    if (WasPreviouslySuspended(function_, object_)) {
+      awaiter_frame_.next = object_.ptr();
+      UnwindToAwaiter();
+    }
+  }
+}
+
+void AsyncAwareStackUnwinder::UnwindToAwaiter() {
+  awaiter_frame_.has_catch_error = false;
+  do {
+    UnwindAwaiterFrame();
+  } while (awaiter_frame_.closure.IsNull() && !awaiter_frame_.next.IsNull());
+}
+
+void AsyncAwareStackUnwinder::UnwindAwaiterFrame() {
+  if (awaiter_frame_.next.IsSuspendState()) {
+    awaiter_frame_.next =
+        SuspendState::Cast(awaiter_frame_.next).function_data();
+  } else if (awaiter_frame_.next.GetClassId() == _SyncCompleter().id() ||
+             awaiter_frame_.next.GetClassId() == _AsyncCompleter().id()) {
+    awaiter_frame_.next = Get_Completer_future(awaiter_frame_.next);
+  }
+
+  if (awaiter_frame_.next.GetClassId() == _Future().id()) {
+    UnwindFrameToFutureListener();
+  } else if (awaiter_frame_.next.GetClassId() ==
+             _AsyncStarStreamController().id()) {
+    UnwindFrameToStreamListener();
+  } else {
+    awaiter_frame_.closure = Closure::null();
+    awaiter_frame_.next = Object::null();
+    return;
+  }
+
+  while (!awaiter_frame_.closure.IsNull()) {
+    function_ = awaiter_frame_.closure.function();
+    context_ = awaiter_frame_.closure.context();
+
+    const auto awaiter_link = function_.awaiter_link();
+    if (awaiter_link.depth != ClosureData::kNoAwaiterLinkDepth) {
+      intptr_t depth = awaiter_link.depth;
+      while (depth-- > 0) {
+        context_ = context_.parent();
+      }
+
+      const Object& object = Object::Handle(context_.At(awaiter_link.index));
+      if (object.IsClosure()) {
+        awaiter_frame_.closure ^= object.ptr();
+        continue;
+      } else {
+        awaiter_frame_.next = object.ptr();
+        return;
+      }
+    }
+    break;
+  }
+}
+
+void AsyncAwareStackUnwinder::UnwindFrameToFutureListener() {
+  object_ = Get_Future__resultOrListeners(awaiter_frame_.next);
+  if (object_.GetClassId() == _FutureListener().id()) {
+    InitializeAwaiterFrameFromFutureListener(object_);
+  } else {
+    awaiter_frame_.closure = Closure::null();
+    awaiter_frame_.next = Object::null();
+  }
+}
+
+void AsyncAwareStackUnwinder::InitializeAwaiterFrameFromFutureListener(
+    const Object& listener) {
+  if (listener.GetClassId() != _FutureListener().id()) {
+    awaiter_frame_.closure = Closure::null();
+    awaiter_frame_.next = Object::null();
+    return;
+  }
+
+  const auto state =
+      Smi::Value(Smi::RawCast(Get_FutureListener_state(listener)));
+  if (state == k_FutureListener_stateThen ||
+      state == k_FutureListener_stateCatchError ||
+      state == k_FutureListener_stateWhenComplete ||
+      state ==
+          (k_FutureListener_stateThen | k_FutureListener_stateCatchError)) {
+    awaiter_frame_.next = Get_FutureListener_result(listener);
+  } else {
+    awaiter_frame_.next = Object::null();
+  }
+  awaiter_frame_.closure =
+      Closure::RawCast(Get_FutureListener_callback(listener));
+  if (state == k_FutureListener_stateCatchError) {
+    awaiter_frame_.has_catch_error = true;
+  }
+}
+
+void AsyncAwareStackUnwinder::UnwindFrameToStreamListener() {
+  controller_ = Get_AsyncStarStreamController_controller(awaiter_frame_.next);
+
+  // Clear the frame.
+  awaiter_frame_.closure = Closure::null();
+  awaiter_frame_.next = Object::null();
+
+  const auto state =
+      Smi::Value(Smi::RawCast(Get_StreamController__state(controller_)));
+  if (state != k_StreamController__STATE_SUBSCRIBED) {
+    return;
+  }
+
+  subscription_ = Get_StreamController__varData(controller_);
+  closure_ ^= Get_BufferingStreamSubscription__onData(subscription_);
+
+  // If this is not the "_StreamIterator._onData" tear-off, we return the
+  // callback we found.
+  function_ = closure_.function();
+  if (!function_.IsImplicitInstanceClosureFunction() ||
+      function_.Owner() != _StreamIterator().ptr()) {
+    awaiter_frame_.closure = closure_.ptr();
+    return;
+  }
+
+  // All implicit closure functions (tear-offs) have the "this" receiver
+  // captured.
+  context_ = closure_.context();
+  ASSERT(context_.num_variables() == 1);
+  stream_iterator_ = context_.At(0);
+  ASSERT(stream_iterator_.IsInstance());
+
+  if (stream_iterator_.GetClassId() != _StreamIterator().id()) {
+    UNREACHABLE();
+  }
+
+  // If `_hasValue` is true then the `StreamIterator._stateData` field
+  // contains the iterator's value. In that case we cannot unwind anymore.
+  //
+  // Notice: With correct async* semantics this may never be true: The async*
+  // generator should only be invoked to produce a value if there's an
+  // in-progress `await streamIterator.moveNext()` call. Once such call has
+  // finished the async* generator should be paused/yielded until the next
+  // such call - and being paused/yielded means it should not appear in stack
+  // traces.
+  //
+  // See dartbug.com/48695.
+  if (Get_StreamIterator__hasValue(stream_iterator_) ==
+      Object::bool_true().ptr()) {
+    return;
+  }
+
+  // If we have an await'er for `await streamIterator.moveNext()` we continue
+  // unwinding there.
+  //
+  // Notice: With correct async* semantics this may always contain a Future
+  // See also comment above as well as dartbug.com/48695.
+  object_ = Get_StreamIterator__stateData(stream_iterator_);
+  if (object_.GetClassId() == _Future().id()) {
+    awaiter_frame_.next = object_.ptr();
+  }
+}
+
+}  // namespace
+
+bool StackTraceUtils::IsNeededForAsyncAwareUnwinding(const Function& function) {
+  // If this is a closure function which specifies an awaiter-link then
+  // we need to retain both function and the corresponding code.
+  if (function.HasAwaiterLink()) {
+    return true;
+  }
+
+  if (function.recognized_kind() ==
+      MethodRecognizer::kFutureListenerHandleValue) {
+    return true;
+  }
+
+  return false;
 }
 
 void StackTraceUtils::CollectFrames(
     Thread* thread,
-    const GrowableObjectArray& code_array,
-    GrowableArray<uword>* pc_offset_array,
     int skip_frames,
-    std::function<void(StackFrame*)>* on_sync_frames,
+    const std::function<void(const StackTraceUtils::Frame&)>& handle_frame) {
-    bool* has_async) {
+  const Closure& null_closure = Closure::Handle(thread->zone());
-  if (has_async != nullptr) {
-    *has_async = false;
-  }
-  Zone* zone = thread->zone();
-  DartFrameIterator frames(thread, StackFrameIterator::kNoCrossThreadIteration);
-  StackFrame* frame = frames.NextFrame();
 
-  // If e.g. the isolate is paused before executing anything, we might not get
+  const Frame gap_frame = {nullptr, StubCode::AsynchronousGapMarker(),
-  // any frames at all. Bail:
+                           /*pc_offset=*/0, null_closure, false};
-  if (frame == nullptr) {
-    return;
-  }
 
-  auto& code = Code::Handle(zone);
+  const Frame gap_frame_with_catch = {nullptr,
-  auto& closure = Closure::Handle(zone);
+                                      StubCode::AsynchronousGapMarker(),
+                                      /*pc_offset=*/0, null_closure, true};
 
-  CallerClosureFinder caller_closure_finder(zone);
+  AsyncAwareStackUnwinder it(thread);
-
+  const bool any_async = it.Unwind(skip_frames, [&](const Frame& frame) {
-  // Start by traversing the sync. part of the stack.
+    if (frame.frame == nullptr) {
-  for (; frame != nullptr; frame = frames.NextFrame()) {
+      handle_frame(frame.has_async_catch_error ? gap_frame_with_catch
-    if (skip_frames > 0) {
+                                               : gap_frame);
-      skip_frames--;
-      continue;
     }
 
-    // If we encounter a known part of the async/Future mechanism, unwind the
+    handle_frame(frame);
-    // awaiter chain from the closures.
+  });
-    bool skip_frame = false;
-    bool is_async = false;
-    closure = ClosureFromFrameFunction(zone, &caller_closure_finder, frames,
-                                       frame, &skip_frame, &is_async);
 
-    // This isn't a special (async) frame we should skip.
+  if (any_async) {
-    if (!skip_frame) {
+    handle_frame(gap_frame);
-      // Add the current synchronous frame.
+  }
-      code = frame->LookupDartCode();
+}
-      code_array.Add(code);
+
-      const uword pc_offset = frame->pc() - code.PayloadStart();
+bool StackTraceUtils::GetSuspendState(const Closure& closure,
-      ASSERT(pc_offset > 0 && pc_offset <= code.Size());
+                                      Object* suspend_state) {
-      pc_offset_array->Add(pc_offset);
+  const Function& function = Function::Handle(closure.function());
-      // Callback for sync frame.
+  const auto awaiter_link = function.awaiter_link();
-      if (on_sync_frames != nullptr) {
+  if (awaiter_link.depth != ClosureData::kNoAwaiterLinkDepth) {
-        (*on_sync_frames)(frame);
+    Context& context = Context::Handle(closure.context());
-      }
+    intptr_t depth = awaiter_link.depth;
+    while (depth-- > 0) {
+      context = context.parent();
     }
 
-    // This frame is running async.
+    const Object& link = Object::Handle(context.At(awaiter_link.index));
-    // Note: The closure might still be null in case it's an unawaited future.
+    if (link.IsSuspendState()) {
-    if (is_async) {
+      *suspend_state = link.ptr();
-      UnwindAwaiterChain(zone, code_array, pc_offset_array,
+      return true;
-                         &caller_closure_finder, closure);
-      if (has_async != nullptr) {
-        *has_async = true;
-      }
-      // Ignore the rest of the stack; already unwound all async calls.
-      return;
     }
   }
-
+  return false;
-  return;
 }
 
 }  // namespace dart

runtime/vm/stack_trace.h

@@ -14,130 +14,52 @@
 
 namespace dart {
 
-// Helper class for finding the closure of the caller.
-class CallerClosureFinder {
- public:
-  explicit CallerClosureFinder(Zone* zone);
-
-  // Recursively follow any `_FutureListener.result`.
-  // If no `result`, then return (bottom) `_FutureListener.callback`
-  ClosurePtr GetCallerInFutureImpl(const Object& future_);
-
-  // Get caller closure from _FutureListener.
-  // Returns closure found either via the `result` Future, or the `callback`.
-  ClosurePtr GetCallerInFutureListener(const Object& future_listener);
-
-  // Find caller closure from an _AsyncStarStreamController instance
-  // corresponding to async* function.
-  // Returns either the `onData` or the Future awaiter.
-  ClosurePtr FindCallerInAsyncStarStreamController(
-      const Object& async_star_stream_controller);
-
-  // Find caller closure from a function receiver closure.
-  // For async* functions, async functions, `Future.timeout` and `Future.wait`,
-  // we can do this by finding and following their awaited Futures.
-  ClosurePtr FindCaller(const Closure& receiver_closure);
-
-  // Find caller closure from a SuspendState of a resumed async function.
-  ClosurePtr FindCallerFromSuspendState(const SuspendState& suspend_state);
-
-  // Returns true if given closure function is a Future callback
-  // corresponding to an async/async* function or async* body callback.
-  bool IsAsyncCallback(const Function& function);
-
-  // Returns SuspendState from the given callback which corresponds
-  // to an async/async* function.
-  SuspendStatePtr GetSuspendStateFromAsyncCallback(const Closure& closure);
-
-  // Get sdk/lib/async/future_impl.dart:_FutureListener.state.
-  intptr_t GetFutureListenerState(const Object& future_listener);
-
-  // Get sdk/lib/async/future_impl.dart:_FutureListener.callback.
-  ClosurePtr GetFutureListenerCallback(const Object& future_listener);
-
-  // Get sdk/lib/async/future_impl.dart:_FutureListener.result.
-  ObjectPtr GetFutureListenerResult(const Object& future_listener);
-
-  // Get sdk/lib/async/future_impl.dart:_Future._resultOrListeners.
-  ObjectPtr GetFutureFutureListener(const Object& future);
-
-  bool HasCatchError(const Object& future_listener);
-
-  // Tests if given [function] with given value of :suspend_state variable
-  // was suspended at least once and running asynchronously.
-  static bool WasPreviouslySuspended(const Function& function,
-                                     const Object& suspend_state_var);
-
- private:
-  Closure& closure_;
-  Context& receiver_context_;
-  Function& receiver_function_;
-  Function& parent_function_;
-  SuspendState& suspend_state_;
-
-  Object& context_entry_;
-  Object& future_;
-  Object& listener_;
-  Object& callback_;
-  Object& controller_;
-  Object& state_;
-  Object& var_data_;
-  Object& callback_instance_;
-
-  Class& future_impl_class;
-  Class& future_listener_class;
-  Class& async_star_stream_controller_class;
-  Class& stream_controller_class;
-  Class& sync_stream_controller_class;
-  Class& controller_subscription_class;
-  Class& buffering_stream_subscription_class;
-  Class& stream_iterator_class;
-
-  Field& future_result_or_listeners_field;
-  Field& callback_field;
-  Field& future_listener_state_field;
-  Field& future_listener_result_field;
-  Field& controller_controller_field;
-  Field& var_data_field;
-  Field& state_field;
-  Field& on_data_field;
-  Field& state_data_field;
-  Field& has_value_field;
-
-  DISALLOW_COPY_AND_ASSIGN(CallerClosureFinder);
-};
-
 class StackTraceUtils : public AllStatic {
  public:
-  static ClosurePtr ClosureFromFrameFunction(
+  static constexpr uword kFutureListenerPcOffset = 1;
-      Zone* zone,
-      CallerClosureFinder* caller_closure_finder,
-      const DartFrameIterator& frames,
-      StackFrame* frame,
-      bool* skip_frame,
-      bool* is_async);
 
-  static void UnwindAwaiterChain(Zone* zone,
+  struct Frame {
-                                 const GrowableObjectArray& code_array,
+    // Corresponding on stack frame or |nullptr| if this is an awaiter frame
-                                 GrowableArray<uword>* pc_offset_array,
+    // or a gap.
-                                 CallerClosureFinder* caller_closure_finder,
+    StackFrame* frame;
-                                 const Closure& leaf_closure);
+
+    // Code object corresponding to this frame.
+    const Code& code;
+
+    // Offset into the code object corresponding to this frame.
+    //
+    // Will be set to |kFutureListenerPcOffset| if this frame corresponds to
+    // future listener that is not yet executing.
+    uword pc_offset;
+
+    // Closure corresponding to the awaiter frame or |null| if this is
+    // a synchronous frame or a gap.
+    const Closure& closure;
+
+    // |true| if an asynchronous exception would be caught by a |catchError|
+    // listener somewhere between the previous frame and this frame.
+    bool has_async_catch_error;
+  };
+
+  // Returns |true| if this function is needed to correctly unwind through
+  // awaiter chains. This means AOT compiler should retain |Function| object,
+  // its signature and the corresponding |Code| object (so that we could
+  // perform the reverse lookup).
+  static bool IsNeededForAsyncAwareUnwinding(const Function& function);
 
   /// Collects all frames on the current stack until an async/async* frame is
   /// hit which has yielded before (i.e. is not in sync-async case).
   ///
   /// From there on finds the closure of the async/async* frame and starts
   /// traversing the listeners.
-  ///
-  /// If [on_sync_frames] is non-null, it will be called for every
-  /// synchronous frame which is collected.
   static void CollectFrames(
       Thread* thread,
-      const GrowableObjectArray& code_array,
-      GrowableArray<uword>* pc_offset_array,
       int skip_frames,
-      std::function<void(StackFrame*)>* on_sync_frames = nullptr,
+      const std::function<void(const Frame&)>& handle_frame);
-      bool* has_async = nullptr);
+
+  // If |closure| has an awaiter-link pointing to the |SuspendState|
+  // the return that object.
+  static bool GetSuspendState(const Closure& closure, Object* suspend_state);
 };
 
 }  // namespace dart

runtime/vm/symbols.h

@@ -27,7 +27,6 @@ class ObjectPointerVisitor;
   V(StateError, "StateError")                                                  \
   V(AssertionError, "_AssertionError")                                         \
   V(AssignIndexToken, "[]=")                                                   \
-  V(AsyncStarMoveNextHelper, "_asyncStarMoveNextHelper")                       \
   V(Bool, "bool")                                                              \
   V(BooleanExpression, "boolean expression")                                   \
   V(BoundsCheckForPartialInstantiation, "_boundsCheckForPartialInstantiation") \
@@ -41,6 +40,9 @@ class ObjectPointerVisitor;
   V(Code, "Code")                                                              \
   V(CodeSourceMap, "CodeSourceMap")                                            \
   V(ColonMatcher, ":matcher")                                                  \
+  V(_Completer, "_Completer")                                                  \
+  V(_AsyncCompleter, "_AsyncCompleter")                                        \
+  V(_SyncCompleter, "_SyncCompleter")                                          \
   V(Compound, "_Compound")                                                     \
   V(CompressedStackMaps, "CompressedStackMaps")                                \
   V(Context, "Context")                                                        \
@@ -144,7 +146,7 @@ class ObjectPointerVisitor;
   V(FunctionResult, "function result")                                         \
   V(FunctionTypeArgumentsVar, ":function_type_arguments_var")                  \
   V(Future, "Future")                                                          \
-  V(FutureImpl, "_Future")                                                     \
+  V(_Future, "_Future")                                                        \
   V(FutureOr, "FutureOr")                                                      \
   V(FutureValue, "Future.value")                                               \
   V(GetCall, "get:call")                                                       \
@@ -422,6 +424,7 @@ class ObjectPointerVisitor;
   V(_classRangeCheck, "_classRangeCheck")                                      \
   V(_current, "_current")                                                      \
   V(_ensureScheduleImmediate, "_ensureScheduleImmediate")                      \
+  V(future, "future")                                                          \
   V(_future, "_future")                                                        \
   V(_getRegisters, "_getRegisters")                                            \
   V(_growBacktrackingStack, "_growBacktrackingStack")                          \
@@ -518,6 +521,7 @@ class ObjectPointerVisitor;
   V(vm_ffi_struct_fields, "vm:ffi:struct-fields")                              \
   V(vm_invisible, "vm:invisible")                                              \
   V(vm_isolate_unsendable, "vm:isolate-unsendable")                            \
+  V(vm_awaiter_link, "vm:awaiter-link")                                        \
   V(vm_never_inline, "vm:never-inline")                                        \
   V(vm_non_nullable_result_type, "vm:non-nullable-result-type")                \
   V(vm_notify_debugger_on_exception, "vm:notify-debugger-on-exception")        \

sdk/lib/_internal/vm/lib/async_patch.dart

@@ -17,7 +17,7 @@ part "timer_patch.dart";
 @pragma("vm:external-name", "DartAsync_fatal")
 external _fatal(msg);
 
-@pragma("vm:entry-point", "call")
+// This function is used when lowering `await for` statements.
 void _asyncStarMoveNextHelper(var stream) {
   if (stream is! _StreamImpl) {
     return;
@@ -188,16 +188,18 @@ class _SuspendState {
   }
 
   @pragma("vm:invisible")
-  @pragma("vm:recognized", "other")
   void _createAsyncCallbacks() {
+    @pragma('vm:awaiter-link')
+    final suspendState = this;
+
     @pragma("vm:invisible")
     thenCallback(value) {
-      _resume(value, null, null);
+      suspendState._resume(value, null, null);
     }
 
     @pragma("vm:invisible")
     errorCallback(Object exception, StackTrace stackTrace) {
-      _resume(null, exception, stackTrace);
+      suspendState._resume(null, exception, stackTrace);
     }
 
     final currentZone = Zone._current;
@@ -373,10 +375,12 @@ class _SuspendState {
   }
 
   @pragma("vm:invisible")
-  @pragma("vm:recognized", "other")
   _createAsyncStarCallback(_AsyncStarStreamController controller) {
+    @pragma('vm:awaiter-link')
+    final suspendState = this;
+
     controller.asyncStarBody = (value) {
-      _resume(value, null, null);
+      suspendState._resume(value, null, null);
     };
   }
 

sdk/lib/async/future.dart

@@ -478,11 +478,9 @@ abstract interface class Future<T> {
   ///   return 'result';
   /// }
   /// ```
-  @pragma("vm:recognized", "other")
   static Future<List<T>> wait<T>(Iterable<Future<T>> futures,
       {bool eagerError = false, void cleanUp(T successValue)?}) {
-    // This is a VM recognised method, and the _future variable is deliberately
+    @pragma('vm:awaiter-link')
-    // allocated in a specific slot in the closure context for stack unwinding.
     final _Future<List<T>> _future = _Future<List<T>>();
     List<T?>? values; // Collects the values. Set to null on error.
     int remaining = 0; // How many futures are we waiting for.
@@ -1068,6 +1066,7 @@ extension FutureExtensions<T> on Future<T> {
       }
       return handleError(error, stackTrace);
     }
+
     if (this is _Future<Object?>) {
       // Internal method working like `catchError`,
       // but allows specifying a different result future type.

sdk/lib/async/future_impl.dart

@@ -6,6 +6,7 @@ part of dart.async;
 
 abstract class _Completer<T> implements Completer<T> {
   @pragma("wasm:entry-point")
+  @pragma("vm:entry-point")
   final _Future<T> future = new _Future<T>();
 
   // Overridden by either a synchronous or asynchronous implementation.
@@ -35,6 +36,7 @@ abstract class _Completer<T> implements Completer<T> {
 }
 
 /// Completer which completes future asynchronously.
+@pragma("vm:entry-point")
 class _AsyncCompleter<T> extends _Completer<T> {
   @pragma("wasm:entry-point")
   void complete([FutureOr<T>? value]) {
@@ -50,6 +52,7 @@ class _AsyncCompleter<T> extends _Completer<T> {
 /// Completer which completes future synchronously.
 ///
 /// Created by [Completer.sync]. Use with caution.
+@pragma("vm:entry-point")
 class _SyncCompleter<T> extends _Completer<T> {
   void complete([FutureOr<T>? value]) {
     if (!future._mayComplete) throw new StateError("Future already completed");
@@ -914,12 +917,10 @@ class _Future<T> implements Future<T> {
     }
   }
 
-  @pragma("vm:recognized", "other")
   @pragma("vm:entry-point")
   Future<T> timeout(Duration timeLimit, {FutureOr<T> onTimeout()?}) {
     if (_isComplete) return new _Future.immediate(this);
-    // This is a VM recognised method, and the _future variable is deliberately
+    @pragma('vm:awaiter-link')
-    // allocated in a specific slot in the closure context for stack unwinding.
     _Future<T> _future = new _Future<T>();
     Timer timer;
     if (onTimeout == null) {

sdk/lib/async/zone.dart

@@ -1656,7 +1656,7 @@ base class _RootZone extends _Zone {
     return _rootRun(null, null, this, f);
   }
 
-  @pragma("vm:recognized", "other")
+  @pragma('vm:invisible')
   R runUnary<R, T>(R f(T arg), T arg) {
     if (identical(Zone._current, _rootZone)) return f(arg);
     return _rootRunUnary(null, null, this, f, arg);