perf(ngcc): process tasks in parallel in async mode (#32427)

`ngcc` supports both synchronous and asynchronous execution. The default
mode when using `ngcc` programmatically (which is how `@angular/cli` is
using it) is synchronous. When running `ngcc` from the command line
(i.e. via the `ivy-ngcc` script), it runs in async mode.

Previously, the work would be executed in the same way in both modes.

This commit improves the performance of `ngcc` in async mode by
processing tasks in parallel on multiple processes. It uses the Node.js
built-in [`cluster` module](https://nodejs.org/api/cluster.html) to
launch a cluster of Node.js processes and take advantage of multi-core
systems.

Preliminary comparisons indicate a 1.8x to 2.6x speed improvement when
processing the angular.io app (apparently depending on the OS, number of
available cores, system load, etc.). Further investigation is needed to
better understand these numbers and identify potential areas of
improvement.

Inspired by/Based on @alxhub's prototype: alxhub/angular@cb631bdb1
Original design doc: https://hackmd.io/uYG9CJrFQZ-6FtKqpnYJAA?view

Jira issue: [FW-1460](https://angular-team.atlassian.net/browse/FW-1460)

PR Close #32427
This commit is contained in:
George Kalpakas
2019-08-29 18:47:54 +03:00
committed by Matias Niemelä
parent f4e4bb2085
commit e36e6c85ef
17 changed files with 1174 additions and 19 deletions

View File

@ -0,0 +1,57 @@
/**
* @license
* Copyright Google Inc. All Rights Reserved.
*
* Use of this source code is governed by an MIT-style license that can be
* found in the LICENSE file at https://angular.io/license
*/
/// <reference types="node" />
import * as cluster from 'cluster';
import {AbsoluteFsPath} from '../../../../src/ngtsc/file_system';
import {JsonObject} from '../../packages/entry_point';
import {PackageJsonChange, PackageJsonUpdate, PackageJsonUpdater, applyChange} from '../../writing/package_json_updater';
import {sendMessageToMaster} from './utils';
/**
* A `PackageJsonUpdater` that can safely handle update operations on multiple processes.
*/
export class ClusterPackageJsonUpdater implements PackageJsonUpdater {
constructor(private delegate: PackageJsonUpdater) {}
createUpdate(): PackageJsonUpdate {
return new PackageJsonUpdate((...args) => this.writeChanges(...args));
}
writeChanges(
changes: PackageJsonChange[], packageJsonPath: AbsoluteFsPath,
preExistingParsedJson?: JsonObject): void {
if (cluster.isMaster) {
// This is the master process:
// Actually apply the changes to the file on disk.
return this.delegate.writeChanges(changes, packageJsonPath, preExistingParsedJson);
}
// This is a worker process:
// Apply the changes in-memory (if necessary) and send a message to the master process.
if (preExistingParsedJson) {
for (const [propPath, value] of changes) {
if (propPath.length === 0) {
throw new Error(`Missing property path for writing value to '${packageJsonPath}'.`);
}
applyChange(preExistingParsedJson, propPath, value);
}
}
sendMessageToMaster({
type: 'update-package-json',
packageJsonPath,
changes,
});
}
}