dart-sdk/pkg/analyzer
Paul Berry ef57b15de7 Analyzer: initial infrastructure for logging details of type inference.
This is the first in a planned series of commits that will give the
analyzer and the CFE the ability to instrument their type inference
functionality. The eventual goal is to have an easy way for developers
of the Dart SDK (and perhaps even curious customers) to supply some
code to the analyzer and/or CFE, and see a trace of the relevant type
inference events (e.g. when each statement, expression, pattern, or
collection element was type inferred, the context supplied to type
inference of each expression, the static type of each expression, the
steps involved in inferring generic type parameters, significant flow
analysis events, etc.).

For now, the only functionality that is implemented is to track when
the `ResolverVisitor` enters and exits each expression. This is done
by adding the null-aware calls
`inferenceLogWriter?.enterExpression(...)` and
`inferenceLogWriter?.exitExpression(...)` to each expression `visit`
method in the `ResolverVisitor`.

In normal operation, `inferenceLogWriter` returns `null`, so there is
no effect. But when type inference logging is active, this call
triggers information to be recorded in memory. This information can be
printed to standard output in two scenarios: (1) if the user requests
type inference logging (not yet implemented), or (2) if the type
inference logging mechanism detects that an important invariant has
not been satisfied (e.g. improper nesting of `enterExpression` and
`exitExpression` calls).

This mechanism of using null-aware calls for instrumentation is very
low overhead when inference logging is inactive; I've verified using
Golem that the overhead is well below the noise level of all our
benchmarks (this Golem run was done using a private branch in which
inference logging is more completely fleshed out, to try to provoke
the most overhead possible).  Details can be found here:
https://golem.corp.goog/Comparison?repository=dart#targetA%3Ddart-analysis-server%3BmachineTypeA%3Dlinux-x64%3BrevisionA%3D110445%3BpatchA%3Dpaulberry-inference_log%3BtargetB%3Ddart-analysis-server%3BmachineTypeB%3Dlinux-x64%3BrevisionB%3D110443%3BpatchB%3DNone

Note that the expression visit methods in `ResolverVisitor` are
invoked in two ways depending on the surrounding code: via the
abstract function `ExpressionImpl.resolveExpression` (when supplying a
context) and via the standard visitor mechanism (typically using
`AstNode.visitChildren`). So there is no single unique function
involved in all expression type inference. This means that in order to
fully instrument expression inference, every single expression visit
method in `ResolverVisitor` must call `enterExpression` and
`exitExpression`. It would be very easy to get this wrong, and forget
to instrument an important expression type.

Two mitigation strategies were used to reduce the risk of an
expression type getting forgotten. Firstly, the
`ResolverVisitor.dispatchExpression` method (which is a
frequently-used entry point for type inferring an expression) calls
`inferenceLogWriter?.assertExpressionWasRecorded` after visiting the
expression; this checks that `enterExpression` was properly
called. Secondly, each time `enterExpression` is called, the inference
logging mechanism looks through the ancestors in the AST, and checks
that `enterExpression` was also appropriately called for the innermost
enclosing expression.

To make sure that the analyzer really satisfies the invariants that
these checks are checking, the type inference logging mechanism is
turned on when assertions are enabled (but it is turned on in a way
where it only dumps to standard out if something goes wrong). This
ensures that the checks will be verified when running analyzer unit
tests on the trybots, but doesn't introduce any extra overhead the
production analyzer (beyond the null-aware calls mentioned earlier).

In follow-up commits I intend to add logic to do the following:
- Track the context and static type of each expression.
- Track type inference of statements, patterns, and collection
  elements.
- Track the computations involved in solving for generic type
  parameters.
- Add a command-line mechanism that developers of the Dart SDK can use
  to analyze some code with type inference logging enabled.

Change-Id: I9757bdd2b3cb996fc98b615d87991de16674e53b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/369788
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
2024-06-10 15:16:06 +00:00
..
doc [analyzer] Update new_language_feature.dart with more details. 2024-05-13 18:17:50 +00:00
example Fix 'unnecessary_final' in analyzer/. 2024-04-16 21:46:15 +00:00
lib Analyzer: initial infrastructure for logging details of type inference. 2024-06-10 15:16:06 +00:00
test Add a validator for pubspec.yaml workspace field. 2024-06-10 14:46:20 +00:00
tool Add a validator for pubspec.yaml workspace field. 2024-06-10 14:46:20 +00:00
.gitignore
analysis_options.yaml analyzer: Add a test-directory-specific analysis options file 2024-05-10 15:52:03 +00:00
CHANGELOG.md CQ. Make FileSource API, move to lib/source. 2024-06-03 00:36:59 +00:00
LICENSE
messages.yaml Add a validator for pubspec.yaml workspace field. 2024-06-10 14:46:20 +00:00
OWNERS [infra] Add OWNERS to the Dart SDK 2022-02-14 14:06:34 +00:00
PRESUBMIT.py Fix NodeTextExpectationsCollector.updatingIsEnabled and add presubmit check. 2023-11-22 16:55:30 +00:00
pubspec.yaml Macro. Serialization for MacroExecutionResult(s). 2024-06-06 16:12:31 +00:00
README.md Link to dart.dev for linter rules 2023-06-02 18:26:13 +00:00
TRIAGE.md Remove migration engine hooks from pkg/analyzer and pkg/_fe_analyzer_shared. 2023-11-20 23:20:27 +00:00

pub package package publisher

This package provides a library that performs static analysis of Dart code. It is useful for tool integration and embedding.

End-users should use the dart analyze command-line tool to analyze their Dart code.

Integrators that want to add Dart support to their editor should use the Dart Analysis Server. The Analysis Server API Specification is available. If you are adding Dart support to an editor or IDE, please let us know by emailing our list.

Configuring the analyzer

Both dart analyze and Dart Analysis Server can be configured with an analysis_options.yaml file (using an .analysis_options file is deprecated). This YAML file can control which files and paths are analyzed, which lints are applied, and more.

If you are embedding the analyzer library in your project, you are responsible for finding the analysis options file, parsing it, and configuring the analyzer.

The analysis options file should live at the root of your project (for example, next to your pubspec.yaml). Different embedders of analyzer, such as dart analyze or Dart Analysis Server, may choose to find the file in various different ways. Consult their documentation to learn more.

Here is an example file that instructs the analyzer to ignore two files:

analyzer:
  exclude:
    - test/_data/p4/lib/lib1.dart
    - test/_data/p5/p5.dart
    - test/_data/bad*.dart
    - test/_brokendata/**

Note that you can use globs, as defined by the glob package.

Here is an example file that enables two lint rules:

linter:
  rules:
    - camel_case_types
    - empty_constructor_bodies

Check out all the available Dart lint rules.

You can combine the analyzer section and the linter section into a single configuration. Here is an example:

analyzer:
  exclude:
    - test/_data/p4/lib/lib1.dart
linter:
  rules:
    - camel_case_types

For more information, see the docs for customizing static analysis.

Who uses this library?

Many tools embed this library, such as:

Support

Post issues and feature requests at https://github.com/dart-lang/sdk/issues. These will be triaged according to the analyzer triage priorities.

Questions and discussions are welcome at the Dart Analyzer Discussion Group.

Background

The APIs in this package were originally machine generated by a translator and were based on an earlier Java implementation. Several of the API's still look like their Java predecessors rather than clean Dart APIs.

In addition, there is currently no clean distinction between public and internal APIs. We plan to address this issue but doing so will, unfortunately, require a large number of breaking changes. We will try to minimize the pain this causes for our clients, but some pain is inevitable.

License

See the LICENSE file.