[analyzer] Cache deserialized unlinked units, don't deserialize the same units over and over again

TL;DR: In the scenario outlined below this CL makes the analyzer use
~50MB less ram after initial analysis, answer the first implements query
~1.2 seconds (~25%) faster and use ~300MB less ram after the first
implements query.

Details:

The below measurements are taken with 8 folders from the sdk open that
overlaps: `pkg/analysis_server`, `pkg/analysis_server_client`,
`pkg/analyzer`, `pkg/analyzer_cli`, `pkg/analyzer_plugin`,
`pkg/analyzer_utilities`, `pkg/_fe_analyzer_shared`, `pkg/front_end`).

The query run is a find all implementations query for the class
`ToJsonable` in
`pkg/analysis_server/lib/lsp_protocol/protocol_special.dart`.

An initial run not shown here was made to let the analyzer cache results as needed.

BEFORE:

isAnalyzing is now done after 0:00:06.661205
heap 524.6MB of 546.8MB
Got answer to query in 0:00:05.285724
heap 943.8MB of 990.1MB

isAnalyzing is now done after 0:00:06.766694
heap 525.0MB of 547.0MB
Got answer to query in 0:00:04.932227
heap 941.9MB of 990.5MB

isAnalyzing is now done after 0:00:06.806875
heap 522.4MB of 547.6MB
Got answer to query in 0:00:05.227859
heap 941.9MB of 990.5MB

isAnalyzing is now done after 0:00:06.663847
heap 507.4MB of 550.0MB
Got answer to query in 0:00:05.107593
heap 941.9MB of 991.0MB

isAnalyzing is now done after 0:00:06.649731
heap 507.4MB of 549.8MB
Got answer to query in 0:00:05.055295
heap 941.9MB of 990.5MB


WITH CL:

isAnalyzing is now done after 0:00:06.575444
heap 461.7MB of 489.2MB
Got answer to query in 0:00:03.758280
heap 628.0MB of 671.6MB

isAnalyzing is now done after 0:00:06.630200
heap 471.2MB of 488.9MB
Got answer to query in 0:00:03.786806
heap 643.3MB of 671.6MB

isAnalyzing is now done after 0:00:06.437794
heap 465.0MB of 485.6MB
Got answer to query in 0:00:03.995996
heap 628.0MB of 671.3MB

isAnalyzing is now done after 0:00:06.706091
heap 457.4MB of 487.2MB
Got answer to query in 0:00:03.748728
heap 640.8MB of 671.1MB

isAnalyzing is now done after 0:00:06.610832
heap 467.7MB of 486.2MB
Got answer to query in 0:00:03.877194
heap 643.9MB of 671.1MB


Initial analysis:
No difference proven at 95.0% confidence

After initial analysis, heap, utilized:
Difference at 95.0% confidence
        -52.76 +/- 10.9161
        -10.1979% +/- 2.10996%
        (Student's t, pooled s = 7.48475)

After initial analysis, heap, capacity:
Difference at 95.0% confidence
        -60.82 +/- 2.29212
        -11.0937% +/- 0.418087%
        (Student's t, pooled s = 1.57162)

First implements query:
Difference at 95.0% confidence
        -1.28834 +/- 0.18012
        -25.1543% +/- 3.51677%
        (Student's t, pooled s = 0.123502)

After first implements query, heap, utilized:
Difference at 95.0% confidence
        -305.48 +/- 8.41655
        -32.4192% +/- 0.893211%
        (Student's t, pooled s = 5.77092)

After first implements query, heap, capacity:
Difference at 95.0% confidence
        -319.18 +/- 0.418906
        -32.2235% +/- 0.0422915%
        (Student's t, pooled s = 0.287228)

Change-Id: Ice2bd0e8c90285c2097afe2589d19f79762d3837
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/275201
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>

pkg/analysis_server/lib/src/analysis_server.dart

@@ -45,6 +45,7 @@ import 'package:analyzer/src/dart/analysis/file_byte_store.dart'
 import 'package:analyzer/src/dart/analysis/file_content_cache.dart';
 import 'package:analyzer/src/dart/analysis/performance_logger.dart';
 import 'package:analyzer/src/dart/analysis/results.dart';
+import 'package:analyzer/src/dart/analysis/unlinked_unit_store.dart';
 import 'package:analyzer/src/dart/ast/element_locator.dart';
 import 'package:analyzer/src/dart/ast/utilities.dart';
 import 'package:analyzer/src/dartdoc/dartdoc_directive_info.dart';
@@ -95,6 +96,8 @@ abstract class AnalysisServer {
 
   late FileContentCache fileContentCache;
 
+  final UnlinkedUnitStore unlinkedUnitStore = UnlinkedUnitStoreImpl();
+
   late analysis.AnalysisDriverScheduler analysisDriverScheduler;
 
   DeclarationsTracker? declarationsTracker;
@@ -235,6 +238,7 @@ abstract class AnalysisServer {
       options.enabledExperiments,
       byteStore,
       fileContentCache,
+      unlinkedUnitStore,
       analysisPerformanceLogger,
       analysisDriverScheduler,
       instrumentationService,

pkg/analysis_server/lib/src/context_manager.dart

@@ -19,6 +19,7 @@ import 'package:analyzer/src/dart/analysis/driver.dart';
 import 'package:analyzer/src/dart/analysis/driver_based_analysis_context.dart';
 import 'package:analyzer/src/dart/analysis/file_content_cache.dart';
 import 'package:analyzer/src/dart/analysis/performance_logger.dart';
+import 'package:analyzer/src/dart/analysis/unlinked_unit_store.dart';
 import 'package:analyzer/src/generated/sdk.dart';
 import 'package:analyzer/src/generated/source.dart';
 import 'package:analyzer/src/lint/linter.dart';
@@ -157,6 +158,9 @@ class ContextManagerImpl implements ContextManager {
   /// The cache of file contents shared between context of the collection.
   final FileContentCache _fileContentCache;
 
+  /// The cache of already deserialized unlinked units.
+  final UnlinkedUnitStore _unlinkedUnitStore;
+
   /// The logger used to create analysis contexts.
   final PerformanceLog _performanceLog;
 
@@ -230,6 +234,7 @@ class ContextManagerImpl implements ContextManager {
       this._enabledExperiments,
       this._byteStore,
       this._fileContentCache,
+      this._unlinkedUnitStore,
       this._performanceLog,
       this._scheduler,
       this._instrumentationService,
@@ -472,6 +477,7 @@ class ContextManagerImpl implements ContextManager {
           sdkPath: sdkManager.defaultSdkDirectory,
           packagesFile: packagesFile,
           fileContentCache: _fileContentCache,
+          unlinkedUnitStore: _unlinkedUnitStore,
           updateAnalysisOptions2: ({
             required analysisOptions,
             required contextRoot,

pkg/analyzer/lib/src/dart/analysis/analysis_context_collection.dart

@@ -16,6 +16,7 @@ import 'package:analyzer/src/dart/analysis/driver.dart';
 import 'package:analyzer/src/dart/analysis/driver_based_analysis_context.dart';
 import 'package:analyzer/src/dart/analysis/file_content_cache.dart';
 import 'package:analyzer/src/dart/analysis/performance_logger.dart';
+import 'package:analyzer/src/dart/analysis/unlinked_unit_store.dart';
 import 'package:analyzer/src/generated/engine.dart' show AnalysisOptionsImpl;
 import 'package:analyzer/src/generated/sdk.dart';
 import 'package:analyzer/src/summary2/kernel_compilation_service.dart';
@@ -55,6 +56,7 @@ class AnalysisContextCollectionImpl implements AnalysisContextCollection {
     String? sdkSummaryPath,
     AnalysisDriverScheduler? scheduler,
     FileContentCache? fileContentCache,
+    UnlinkedUnitStore? unlinkedUnitStore,
     @Deprecated('Use updateAnalysisOptions2, which must be a function that '
         'accepts a second parameter')
         void Function(AnalysisOptionsImpl)? updateAnalysisOptions,
@@ -106,6 +108,7 @@ class AnalysisContextCollectionImpl implements AnalysisContextCollection {
         updateAnalysisOptions: updateAnalysisOptions,
         updateAnalysisOptions2: updateAnalysisOptions2,
         fileContentCache: fileContentCache,
+        unlinkedUnitStore: unlinkedUnitStore ?? UnlinkedUnitStoreImpl(),
         macroKernelBuilder: macroKernelBuilder,
         macroExecutor: macroExecutor,
       );

pkg/analyzer/lib/src/dart/analysis/context_builder.dart

@@ -20,6 +20,7 @@ import 'package:analyzer/src/dart/analysis/driver_based_analysis_context.dart';
 import 'package:analyzer/src/dart/analysis/file_content_cache.dart';
 import 'package:analyzer/src/dart/analysis/performance_logger.dart'
     show PerformanceLog;
+import 'package:analyzer/src/dart/analysis/unlinked_unit_store.dart';
 import 'package:analyzer/src/dart/sdk/sdk.dart';
 import 'package:analyzer/src/generated/engine.dart' show AnalysisOptionsImpl;
 import 'package:analyzer/src/generated/sdk.dart' show DartSdk;
@@ -68,6 +69,7 @@ class ContextBuilderImpl implements ContextBuilder {
     })?
         updateAnalysisOptions2,
     FileContentCache? fileContentCache,
+    UnlinkedUnitStore? unlinkedUnitStore,
     MacroKernelBuilder? macroKernelBuilder,
     macro.MultiMacroExecutor? macroExecutor,
   }) {
@@ -142,6 +144,7 @@ class ContextBuilderImpl implements ContextBuilder {
       externalSummaries: summaryData,
       retainDataForTesting: retainDataForTesting,
       fileContentCache: fileContentCache,
+      unlinkedUnitStore: unlinkedUnitStore,
       macroKernelBuilder: macroKernelBuilder,
       macroExecutor: macroExecutor,
       declaredVariables: declaredVariables,

pkg/analyzer/lib/src/dart/analysis/driver.dart

@@ -30,6 +30,7 @@ import 'package:analyzer/src/dart/analysis/search.dart';
 import 'package:analyzer/src/dart/analysis/session.dart';
 import 'package:analyzer/src/dart/analysis/status.dart';
 import 'package:analyzer/src/dart/analysis/testing_data.dart';
+import 'package:analyzer/src/dart/analysis/unlinked_unit_store.dart';
 import 'package:analyzer/src/dart/element/element.dart';
 import 'package:analyzer/src/diagnostic/diagnostic.dart';
 import 'package:analyzer/src/error/codes.dart';
@@ -115,6 +116,10 @@ class AnalysisDriver implements AnalysisDriverGeneric {
   /// the content from the file.
   final FileContentCache _fileContentCache;
 
+  /// The already loaded unlinked units,  consulted before deserializing
+  /// from file again.
+  final UnlinkedUnitStore _unlinkedUnitStore;
+
   late final StoredFileContentStrategy _fileContentStrategy;
 
   /// The analysis options to analyze with.
@@ -265,6 +270,7 @@ class AnalysisDriver implements AnalysisDriverGeneric {
     this.macroExecutor,
     this.analysisContext,
     FileContentCache? fileContentCache,
+    UnlinkedUnitStore? unlinkedUnitStore,
     this.enableIndex = false,
     SummaryDataStore? externalSummaries,
     DeclaredVariables? declaredVariables,
@@ -275,6 +281,7 @@ class AnalysisDriver implements AnalysisDriverGeneric {
         _byteStore = byteStore,
         _fileContentCache =
             fileContentCache ?? FileContentCache.ephemeral(resourceProvider),
+        _unlinkedUnitStore = unlinkedUnitStore ?? UnlinkedUnitStoreImpl(),
         _analysisOptions = analysisOptions,
         _logger = logger,
         _packages = packages,
@@ -1488,6 +1495,7 @@ class AnalysisDriver implements AnalysisDriverGeneric {
       _saltForElements,
       featureSetProvider,
       fileContentStrategy: _fileContentStrategy,
+      unlinkedUnitStore: _unlinkedUnitStore,
       prefetchFiles: null,
       isGenerated: (_) => false,
       testData: testView?.fileSystem,

pkg/analyzer/lib/src/dart/analysis/file_state.dart

@@ -22,6 +22,7 @@ import 'package:analyzer/src/dart/analysis/performance_logger.dart';
 import 'package:analyzer/src/dart/analysis/referenced_names.dart';
 import 'package:analyzer/src/dart/analysis/unlinked_api_signature.dart';
 import 'package:analyzer/src/dart/analysis/unlinked_data.dart';
+import 'package:analyzer/src/dart/analysis/unlinked_unit_store.dart';
 import 'package:analyzer/src/dart/ast/ast.dart';
 import 'package:analyzer/src/dart/scanner/reader.dart';
 import 'package:analyzer/src/dart/scanner/scanner.dart';
@@ -509,6 +510,7 @@ class FileState {
     _fileContent = rawFileState;
 
     // Prepare the unlinked bundle key.
+    var previousUnlinkedKey = _unlinkedKey;
     {
       var signature = ApiSignature();
       signature.addUint32List(_fsState._saltForUnlinked);
@@ -523,7 +525,7 @@ class FileState {
     }
 
     // Prepare the unlinked unit.
-    _driverUnlinkedUnit = _getUnlinkedUnit();
+    _driverUnlinkedUnit = _getUnlinkedUnit(previousUnlinkedKey);
     _unlinked2 = _driverUnlinkedUnit!.unit;
     _lineInfo = LineInfo(_unlinked2!.lineStarts);
 
@@ -642,14 +644,30 @@ class FileState {
     return _fsState.getFileForUri(absoluteUri);
   }
 
-  /// Return the unlinked unit, from bytes or new.
-  AnalysisDriverUnlinkedUnit _getUnlinkedUnit() {
+  /// Return the unlinked unit, freshly deserialized from bytes,
+  /// previously deserialized from bytes, or new.
+  AnalysisDriverUnlinkedUnit _getUnlinkedUnit(String? previousUnlinkedKey) {
+    if (previousUnlinkedKey != null) {
+      if (previousUnlinkedKey != _unlinkedKey) {
+        _fsState.unlinkedUnitStore.release(previousUnlinkedKey);
+      } else {
+        return _driverUnlinkedUnit!;
+      }
+    }
+
     final testData = _fsState.testData?.forFile(resource, uri);
+    var fromStore = _fsState.unlinkedUnitStore.get(_unlinkedKey!);
+    if (fromStore != null) {
+      testData?.unlinkedKeyGet.add(unlinkedKey);
+      return fromStore;
+    }
 
     var bytes = _fsState._byteStore.get(_unlinkedKey!);
     if (bytes != null && bytes.isNotEmpty) {
       testData?.unlinkedKeyGet.add(unlinkedKey);
-      return AnalysisDriverUnlinkedUnit.fromBytes(bytes);
+      var result = AnalysisDriverUnlinkedUnit.fromBytes(bytes);
+      _fsState.unlinkedUnitStore.put(_unlinkedKey!, result);
+      return result;
     }
 
     var unit = parse();
@@ -1121,6 +1139,7 @@ class FileSystemState {
   int fileStamp = 0;
 
   final FileContentStrategy fileContentStrategy;
+  final UnlinkedUnitStore unlinkedUnitStore;
 
   /// A function that fetches the given list of files. This function can be used
   /// to batch file reads in systems where file fetches are expensive.
@@ -1146,6 +1165,7 @@ class FileSystemState {
     this._saltForElements,
     this.featureSetProvider, {
     required this.fileContentStrategy,
+    required this.unlinkedUnitStore,
     required this.prefetchFiles,
     required this.isGenerated,
     required this.testData,
@@ -1421,6 +1441,8 @@ class FileSystemState {
     _pathToFile.clear();
     _subtypedNameToFiles.clear();
     _libraryNameToFiles.clear();
+    // TODO(jensj): If we use finalizers we shouldn't clear.
+    unlinkedUnitStore.clear();
   }
 
   FileState _newFile(File resource, String path, Uri uri) {

pkg/analyzer/lib/src/dart/analysis/unlinked_unit_store.dart

@@ -0,0 +1,56 @@
+// Copyright (c) 2022, the Dart project authors. Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import 'package:analyzer/src/dart/analysis/unlinked_data.dart';
+
+abstract class UnlinkedUnitStore {
+  void clear();
+  AnalysisDriverUnlinkedUnit? get(String key);
+  void put(String key, AnalysisDriverUnlinkedUnit value);
+  void release(String key);
+}
+
+class UnlinkedUnitStoreImpl implements UnlinkedUnitStore {
+  // TODO(jensj): Could we use finalizers and automatically clean up
+  // this map?
+  final Map<String, _UnlinkedUnitStoreData> _deserializedUnlinked = {};
+
+  @override
+  void clear() {
+    _deserializedUnlinked.clear();
+  }
+
+  @override
+  AnalysisDriverUnlinkedUnit? get(String key) {
+    var lookup = _deserializedUnlinked[key];
+    if (lookup != null) {
+      lookup.usageCount++;
+      return lookup.value.target;
+    }
+    return null;
+  }
+
+  @override
+  void put(String key, AnalysisDriverUnlinkedUnit value) {
+    _deserializedUnlinked[key] = _UnlinkedUnitStoreData(WeakReference(value));
+  }
+
+  @override
+  void release(String key) {
+    var lookup = _deserializedUnlinked[key];
+    if (lookup != null) {
+      lookup.usageCount--;
+      if (lookup.usageCount <= 0) {
+        _deserializedUnlinked.remove(key);
+      }
+    }
+  }
+}
+
+class _UnlinkedUnitStoreData {
+  final WeakReference<AnalysisDriverUnlinkedUnit> value;
+  int usageCount = 1;
+
+  _UnlinkedUnitStoreData(this.value);
+}

pkg/analyzer/lib/src/dart/micro/resolve_file.dart

@@ -24,6 +24,7 @@ import 'package:analyzer/src/dart/analysis/library_context.dart';
 import 'package:analyzer/src/dart/analysis/performance_logger.dart';
 import 'package:analyzer/src/dart/analysis/results.dart';
 import 'package:analyzer/src/dart/analysis/search.dart';
+import 'package:analyzer/src/dart/analysis/unlinked_unit_store.dart';
 import 'package:analyzer/src/dart/micro/analysis_context.dart';
 import 'package:analyzer/src/dart/micro/utils.dart';
 import 'package:analyzer/src/generated/engine.dart' show AnalysisOptionsImpl;
@@ -737,6 +738,7 @@ class FileResolver {
         prefetchFiles: prefetchFiles,
         isGenerated: isGenerated,
         testData: testData?.fileSystem,
+        unlinkedUnitStore: UnlinkedUnitStoreImpl(),
       );
     }
 

pkg/analyzer/test/src/dart/analysis/file_state_test.dart

@@ -14,6 +14,7 @@ import 'package:analyzer/src/dart/analysis/feature_set_provider.dart';
 import 'package:analyzer/src/dart/analysis/file_content_cache.dart';
 import 'package:analyzer/src/dart/analysis/file_state.dart';
 import 'package:analyzer/src/dart/analysis/performance_logger.dart';
+import 'package:analyzer/src/dart/analysis/unlinked_unit_store.dart';
 import 'package:analyzer/src/dart/sdk/sdk.dart';
 import 'package:analyzer/src/generated/source.dart';
 import 'package:analyzer/src/source/package_map_resolver.dart';
@@ -5749,6 +5750,7 @@ class FileSystemStateTest with ResourceProviderMixin {
       prefetchFiles: null,
       isGenerated: (_) => false,
       testData: null,
+      unlinkedUnitStore: UnlinkedUnitStoreImpl(),
     );
   }