cargdev/angular
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
angular/packages/compiler-cli/src/ngtsc/core
History
Alex Rickabaugh 57a518a36d perf(compiler-cli): fix memory leak in retained incremental state (#37835) 
Incremental compilation allows for the output state of one compilation to be
reused as input to the next compilation. This involves retaining references
to instances from prior compilations, which must be done carefully to avoid
memory leaks.

This commit fixes such a leak with a complicated retention chain:

* `TrackedIncrementalBuildStrategy` unnecessarily hangs on to the previous
  `IncrementalDriver` (state of the previous compilation) once the current
  compilation completes.

  In general this is unnecessary, but should be safe as long as the chain
  only goes back one level - if the `IncrementalDriver` doesn't retain any
  previous `TrackedIncrementalBuildStrategy` instances. However, this does
  happen:

* `NgCompiler` indirectly causes retention of previous `NgCompiler`
  instances (and thus previous `TrackedIncrementalBuildStrategy` instances)
  through accidental capture of the `this` context in a closure created in
  its constructor. This closure is wrapped in a `ts.ModuleResolutionCache`
  used to create a `ModuleResolver` class, which is passed to the program's
  `TraitCompiler` on construction.

* The `IncrementalDriver` retains a reference to the `TraitCompiler` of the
  previous compilation, completing the reference chain.

The final retention chain thus looks like:

* `TrackedIncrementalBuildStrategy` of current program
* `.previous`: `IncrementalDriver` of previous program
* `.lastGood.traitCompiler`: `TraitCompiler`
* `.handlers[..].moduleResolver.moduleResolutionCache`: cache
* (via `getCanonicalFileName` closure): `NgCompiler`
* `.incrementalStrategy`: `TrackedIncrementalBuildStrategy` of previous
  program.

The closure link is the "real" leak here. `NgCompiler` is creating a closure
for `getCanonicalFileName`, delegating to its
`this.adapter.getCanonicalFileName`, for the purposes of creating a
`ts.ModuleResolutionCache`. The fact that the closure references
`NgCompiler` thus eventually causes previous `NgCompiler` iterations to be
retained. This is also potentially problematic due to the shared nature of
`ts.ModuleResolutionCache`, which is potentially retained across multiple
compilations intentionally.

This commit fixes the first two links in the retention chain: the build
strategy is patched to not retain a `previous` pointer, and the `NgCompiler`
is patched to not create a closure in the first place, but instead pass a
bound function. This ensures that the `NgCompiler` does not retain previous
instances of itself in the first place, even if the build strategy does
end up retaining the previous incremental state unnecessarily.

The third link (`IncrementalDriver` unnecessarily retaining the whole
`TraitCompiler`) is not addressed in this commit as it's a more
architectural problem that will require some refactoring. However, the leak
potential of this retention is eliminated thanks to fixing the first two
issues.

PR Close #37835
2020-06-29 16:34:52 -07:00
..
api
docs: update file header to be correct (#37425)
2020-06-03 15:31:29 -07:00
src
perf(compiler-cli): fix memory leak in retained incremental state (#37835)
2020-06-29 16:34:52 -07:00
test
refactor(compiler-cli): make IncrementalBuild strategy configurable (#37339)
2020-06-15 09:50:09 -07:00
BUILD.bazel
refactor(compiler-cli): extract NgCompilerAdapter interface (#37118)
2020-06-03 13:29:45 -07:00
index.ts
build: update license headers to reference Google LLC (#37205)
2020-05-26 14:27:01 -04:00
README.md
feat(compiler-cli): implement NgTscPlugin on top of the NgCompiler API (#34792)
2020-02-06 15:27:34 -08:00

README.md

What is the 'core' package?

This package contains the core functionality of the Angular compiler. It provides APIs for the implementor of a TypeScript compiler to provide Angular compilation as well.

It supports the 'ngc' command-line tool and the Angular CLI (via the NgtscProgram), as well as an experimental integration with tsc_wrapped and the ts_library Bazel rule via NgTscPlugin.

Angular compilation

Angular compilation involves the translation of Angular decorators into static definition fields. At build time, this is done during the overall process of TypeScript compilation, where TypeScript code is type-checked and then downleveled to JavaScript code. Along the way, diagnostics specific to Angular can also be produced.

Compilation flow

Any use of the TypeScript compiler APIs follows a multi-step process:

  1. A ts.CompilerHost is created.
  2. That ts.CompilerHost, plus a set of "root files", is used to create a ts.Program.
  3. The ts.Program is used to gather various kinds of diagnostics.
  4. Eventually, the ts.Program is asked to emit, and JavaScript code is produced.

A compiler which integrates Angular compilation into this process follows a very similar flow, with a few extra steps:

  1. A ts.CompilerHost is created.
  2. That ts.CompilerHost is wrapped in an NgCompilerHost, which adds Angular specific files to the compilation.
  3. A ts.Program is created from the NgCompilerHost and its augmented set of root files.
  4. An NgCompiler is created using the ts.Program.
  5. Diagnostics can be gathered from the ts.Program as normal, as well as from the NgCompiler.
  6. Prior to emit, NgCompiler.prepareEmit is called to retrieve the Angular transformers which need to be fed to ts.Program.emit.
  7. emit is called on the ts.Program with the Angular transformers from above, which produces JavaScript code with Angular extensions.

Asynchronous compilation

In some compilation environments (such as the Webpack-driven compilation inside the Angular CLI), various inputs to the compilation are only producible in an asynchronous fashion. For example, SASS compilation of styleUrls that link to SASS files requires spawning a child Webpack compilation. To support this, Angular has an asynchronous interface for loading such resources.

If this interface is used, an additional asynchronous step after NgCompiler creation is to call NgCompiler.analyzeAsync and await its Promise. After this operation completes, all resources have been loaded and the rest of the NgCompiler API can be used synchronously.

Wrapping the ts.CompilerHost

Angular compilation generates a number of synthetic files (files which did not exist originally as inputs), depending on configuration. Such files can include:

  • .ngfactory shim files, if requested.
  • .ngsummary shim files, if requested.
  • A flat module index file, if requested.
  • The __ng_typecheck__.ts file, which supports template type-checking code.

These files don't exist on disk, but need to appear as such to the ts.Program. This is accomplished by wrapping the ts.CompilerHost (which abstracts the outside world to the ts.Program) in an implementation which provides these synthetic files. This is the primary function of NgCompilerHost.

API definitions

The core package contains separate API definitions, which are used across the compiler. Of note is the interface NgCompilerOptions, which unifies various supported compilation options across Angular and TypeScript itself. It's assignable to ts.CompilerOptions, and implemented by the legacy CompilerOptions type in //packages/compiler-cli/src/transformers/api.ts.

The various types of options are split out into distinct interfaces according to their purpose and level of support.