Previously, ngcc would only be able to match an ngcc configuration to
packages that were located inside the project's top-level
`node_modules/`. However, if there are multiple versions of a package in
a project (e.g. as a transitive dependency of other packages), multiple
copies of a package (at different versions) may exist in nested
`node_modules/` directories. For example, one at
`<project-root>/node_modules/some-package/` and one at
`<project-root>/node_modules/other-package/node_modules/some-package/`.
In such cases, ngcc was only able to detect the config for the first
copy but not for the second.
This commit fixes this by returning a new instance of
`ProcessedNgccPackageConfig` for each different package path (even if
they refer to the same package name). In these
`ProcessedNgccPackageConfig`, the `entryPoints` paths have been
processed to take the package path into account.
PR Close#37040
This commit adds a `packageName` property to the `EntryPoint` interface.
In a subsequent commit this will be used to retrieve the correct ngcc
configuration for each package, regardless of its path.
PR Close#37040
In order to retrieve the ngcc configuration (if any) for an entry-point,
ngcc has to detect the containing package's version.
Previously, ngcc would try to read the version from the entry-point's
`package.json` file, which was different than the package's top-level
`package.json` for secondary entry-points. For example, it would try to
read it from `node_modules/@angular/common/http/package.json` for
entry-point `@angular/common/http`. However, the `package.json` files
for secondary entry-points are not guaranteed to include a `version`
property.
This commit fixes this by first trying to read the version from the
_package's_ `package.json` (falling back to the entry-point's
`package.json`). For example, it will first try to read it from
`@angular/common/package.json` for entry-point `@angular/common/http`.
PR Close#37040
Rename the `package` property to `packagePath` on the `EntryPoint`
interface. This makes it more clear that the `packagePath` property
holds the absolute path to the containing package (similar to how `path`
holds the path to the entry-point). This will also align with the
`packageName` property that will be added in a subsequent commit.
This commit also re-orders the `EntryPoint` properties to group related
properties together and to match the order of properties on instances
with that on the interface.
PR Close#37040
Previously, when an entry-point was ignored via an ngcc config, ngcc
would scan sub-directories for sub-entry-points, but would not use the
correct `packagePath`. For example, if `@angular/common` was ignored, it
would look at `@angular/common/http` but incorrectly use
`.../@angular/common/http` as the `packagePath` (instead of
`.../@angular/common`). As a result, it would not retrieve the correct
ngcc config for the actual package.
This commit fixes it by ensuring the correct `packagePath` is used, even
if the primary entry-point corresponding to that path is ignored. In
order to do this, a new return value for `getEntryPointInfo()` is added:
`IGNORED_ENTRY_POINT`. This is used to differentiate between directories
that correspond to no or an incompatible entry-point and those that
correspond to an entry-point that could otherwise be valid but is
explicitly ignored. Consumers of `getEntryPointInfo()` can then use this
info to discard ignored entry-points, but still use the correct
`packagePath` when scanning their sub-directories for secondary
entry-points.
PR Close#37040
As of version 10, libraries following the APF will no longer contain
ESM5 output. Hence, tests in ngcc need to be updated as they currently
rely on the release output of `@angular/core`.
Additionally, we'd need to support in ngcc that the `module`
property of entry-points no longer necessarily refers to
`esm5` output, but instead can also target `esm2015`.
We currently achieve this by checking the path the `module`
property points to. We can do this because as per APF, the
folder name is known for the esm2015 output. Long-term for
more coverage, we want to sniff the format by looking for
known ES2015 constructs in the file `module` refers to.
PR Close#36944
The commit adds support to the ngcc.config.js file for setting the
`retryAttempts` and `retryDelay` options for the `AsyncLocker`.
An integration test adds a new check for a timeout and actually uses the
ngcc.config.js to reduce the timeout time to prevent the test from taking
too long to complete.
PR Close#36838
Strictly this method only returns config for packages. So this commit
renames it to `getPackageConfig()`, which frees us up to add other
"getXxxxConfig()` methods later.
PR Close#36838
This test is basically duplicated (and slightly enhanced) in the
following test. So it is superfluous. (I suspect it was the result
of a broken rebase.)
PR Close#36838
The base path for package and entry-points is known so there is
no need to store these in the file. Also this commit avoids storing
empty arrays unnecessarily.
PR Close#36486
Previously, even if an entry-point did not need to be processed,
ngcc would always parse the files of the entry-point to compute
its dependencies. This can take a lot of time for large node_modules.
Now these dependencies are cached in the entry-point manifest,
and read from there rather than computing them every time.
See https://github.com/angular/angular/issues/36414\#issuecomment-608401834
FW-2047
PR Close#36486
1. update jasmine to 3.5
2. update @types/jasmine to 3.5
3. update @types/jasminewd2 to 2.0.8
Also fix several cases, the new jasmine 3 will help to create test cases correctly,
such as in the `jasmine 2.x` version, the following case will pass
```
expect(1 == 2);
```
But in jsamine 3, the case will need to be
```
expect(1 == 2).toBeTrue();
```
PR Close#34625
Recently we added support for ignoring specified deep-import
warnings by providing sets of regular expressions within the
`ngcc.config.js` file. But this was only working for the project
level configuration.
This commit fixes ngcc so that it will also read these regular
expressions from package level configuration too.
Fixes#35750
PR Close#36423
The `browser` package.json property is now supported to the same
level as `main` - i.e. it is sniffed for UMD, ESM5 and CommonJS.
The `browser` property can also contain an object with file overrides
but this is not supported by ngcc.
Fixes#36062
PR Close#36396
Previously, `main` was only checked for `umd` or `commonjs`
formats. Now if there are `import` or `export` statements in the
source file it will be deemed to be in `esm5` format.
Fixes#35788
PR Close#36396
Ngcc supports providing a project-level configuration to affect how
certain dependencies are processed and also has a built-in fallback
configuration for some unmaintained packages. Each entry in these
configurations could be scoped to specific versions of a package by
providing a version range. If no version range is provided for a
package, it defaults to `*` (with the intention of matching any
version).
Previously, the installed version of a package was tested against the
version range using the [semver][1] package's `satisfies()` function
with the default options. By default, `satisfies()` does not match
pre-releases (see [here][2] for more details on reasoning). While this
makes sense when determining what version of a dependency to install
(trying to avoid unexpected breaking changes), it is not desired in the
case of ngcc.
This commit fixes it by explicitly specifying that pre-release versions
should be matched normally.
[1]: https://www.npmjs.com/package/semver
[2]: https://github.com/npm/node-semver#prerelease-tags
PR Close#36370
The new `EntryPointManifest` class can read and write a
manifest file that contains all the paths to the entry-points
that have been found in a node_modules folder.
This can be used to speed up finding entry-points in
subsequent runs.
The manifest file stores the ngcc version and hashes of
the package lock-file and project config, since if these
change the manifest will need to be recomputed.
PR Close#35931
This reduces the time that `findEntryPoints` takes from 9701.143ms to 4177.278ms, by reducing the file operations done.
Reference: #35717
PR Close#35756
Now `hasBeenProcessed()` will no longer throw if there
is an entry-point that has been built with an outdated
version of ngcc.
Instead it just returns `false`, which will include it in this
processing run.
This is a precursor to adding functionality that will
automatically revert outdate build artifacts.
PR Close#35079
Previously the `rootDir` was set to the entry-point path but
this is incorrect if the source files are stored in a directory outside
the entry-point path. This is the case in the latest versions of the
Angular CDK.
Instead the `rootDir` should be the containing package path, which is
guaranteed to include all the source for the entry-point.
---
A symptom of this is an error when ngcc is trying to process the source of
an entry-point format after the entry-point's typings have already been
processed by a previous processing run.
During processing the `_toR3Reference()` function gets called which in turn
makes a call to `ReflectionHost.getDtsDeclaration()`. If the typings files
are also being processed this returns the node from the dts typings files.
But if we have already processed the typings files and are now processing
only an entry-point format without typings, the call to
`ReflectionHost.getDtsDeclaration()` returns `null`.
When this value is `null`, a JS `valueRef` is passed through as the DTS
`typeRef` to the `ReferenceEmitter`. In this case, the `ReferenceEmitter`
fails during `emit()` because no `ReferenceEmitStrategy` is able to provide
an emission:
1) The `LocalIdentifierStrategy` is not able help because in this case
`ImportMode` is `ForceNewImport`.
2) The `LogicalProjectStrategy` cannot find the JS file below the `rootDir`.
The second strategy failure is fixed by this PR.
Fixes https://github.com/angular/ngcc-validation/issues/495
PR Close#34212
In a package.json file, the "typings" or "types" field could be an array
of typings files. ngcc would previously crash unexpectedly for such
packages, as it assumed that the typings field would be a string. This
commit lets ngcc skip over such packages, as having multiple typing
entry-points is not supported for Angular packages so it is safe to
ignore them.
Fixes#33646
PR Close#33973
Some declaration files may not be referenced from an entry-point's
main typings file, as it may declare types that are only used internally.
ngcc has logic to include declaration files based on all source files,
to ensure internal declaration files are available.
For packages following APF layout, however, this logic was insufficient.
Consider an entry-point with base path of `/esm2015/testing` and typings
residing in `/testing`, the file
`/esm2015/testing/src/nested/internal.js` has its typings file at
`/testing/src/nested/internal.d.ts`. Previously, the declaration was
assumed to be located at `/esm2015/testing/testing/internal.d.ts` (by
means of `/esm2015/testing/src/nested/../../testing/internal.d.ts`)
which is not where the declaration file can be found. This commit
resolves the issue by looking in the correct directory.
PR Close#33875
This commit adapts the private NgModule re-export system (using aliasing) to
ngcc. Not all ngcc compilations are compatible with these re-exports, as
they assume a 1:1 correspondence between .js and .d.ts files. The primary
concern here is supporting them for commonjs-only packages.
PR Close#33177
Normally, when ngcc encounters a package with missing dependencies while
attempting to determine a compilation ordering, it will ignore that package.
This commit adds a configuration for a flag to tell ngcc to compile the
package anyway, regardless of any missing dependencies.
FW-1931 #resolve
PR Close#33192
It is now possible to include a set of default ngcc configurations
that ship with ngcc out of the box. This allows ngcc to handle a
set of common packages, which are unlikely to be fixed, without
requiring the application developer to write their own configuration
for them.
Any packages that are configured at the package or project level
will override these default configurations. This allows a reasonable
level of control at the package and user level.
PR Close#33008
To persist some of its state, `ngcc` needs to update `package.json`
files (both in memory and on disk).
This refactoring abstracts these operations behind the
`PackageJsonUpdater` interface, making it easier to orchestrate them
from different contexts (e.g. when running tasks in parallel on multiple
processes).
Inspired by/Based on @alxhub's prototype: alxhub/angular@cb631bdb1
PR Close#32427
In order to prevent `ngcc`'d packages (e.g. libraries) from getting
accidentally published, `ngcc` overwrites the `prepublishOnly` npm
script to log a warning and exit with an error. In case we want to
restore the original script (e.g. "undo" `ngcc` processing), we keep a
backup of the original `prepublishOnly` script.
Previously, running `ngcc` a second time (e.g. for a different format)
would create a backup of the overwritten `prepublishOnly` script (if
there was originally no `prepublishOnly` script). As a result, if we
ever tried to "undo" `ngcc` processing and restore the original
`prepublishOnly` script, the error-throwing script would be restored
instead.
This commit fixes it by ensuring that we only back up a `prepublishOnly`
script, iff it is not the one we created ourselves (i.e. the
error-throwing one).
PR Close#32427
If a project has nested projects that contain node_modules folders
that get processed by ngcc, it can be confusing when the ngcc
version changes since the error message is very generic:
```
The ngcc compiler has changed since the last ngcc build.
Please completely remove `node_modules` and try again.
```
This commit augments the error message with the path of
the entry-point that failed so that it is more obvious which
node_modules folder to remove.
BREAKING CHANGE:
This commit removes the public export of `hasBeenProcessed()`.
This was exported to be availble to the CLI integration but was never
used. The change to the function signature is a breaking change in itself
so we remove the function altogether to simplify and lower the public
API surface going forward.
PR Close#32396
Publishing of NGCC packages should not be allowed. It is easy for a user to publish an NGCC'd version of a library they have workspace libraries which are being used in a workspace application.
If a users builds a library and afterwards the application, the library will be transformed with NGCC and since NGCC taints the distributed files that should be published.
With this change we use the npm/yarn `prepublishOnly` hook to display and error and abort the process with a non zero error code when a user tries to publish an NGCC version of the package.
More info: https://docs.npmjs.com/misc/scripts
PR Close#32031
Remove the `formatProperty` property from the `EntryPointBundle`
interface, because the property is not directly related to that type.
It was only used in one place, when calling `fileWriter.writeBundle()`,
but we can pass `formatProperty` directrly to `writeBundle()`.
PR Close#32052
In commit 7b55ba58b (part of PR #29092), the implementation of
`makeEntryPointBundle()` was changed such that it now always return
`EntryPointBundle` (and not `null`).
However, the return type was not updated and as result we continued to
unnecessarily handle `null` as a potential return value in some places.
This commit fixes the return type to reflect the implementation and
removes the redundant code that was dealing with `null`.
PR Close#32052
Previously, when `ngcc` needed to mark multiple properties as processed
(e.g. a processed format property and `typings` or all supported
properties for a non-Angular entry-point), it would update each one
separately and write the file to disk multiple times.
This commit changes this, so that multiple properties can be updated at
once with one file-write operation. While this theoretically improves
performance (reducing the I/O operations), it is not expected to have
any noticeable impact in practice, since these operations are a tiny
fraction of `ngcc`'s work.
This change will be useful for a subsequent change to mark all
properties that map to the same `formatPath` as processed, once it is
processed the first time.
PR Close#32003
There are two places in the ngcc processing where it needs to load the
content of a file given by a general path:
* when determining the format of an entry-point.
To do this ngcc uses the value of the relevant property in package.json.
But in the case of `main` it must parse the contents of the entry-point
file to decide whether the format is UMD or CommonJS.
* when parsing the source files for dependencies to determine the order in
which compilation must occur. The relative imports in each file are parsed
and followed recursively, looking for external imports.
Previously, we naively assumed that the path would match the file name exactly.
But actually we must consider the standard module resolution conventions.
E.g. the extension (.js) may be missing, or the path may refer to a directory
containing an index.js file.
This commit fixes both places.
This commit now requires the `DependencyHost` instances to check
the existence of more files than before (at worst all the different possible
post-fixes). This should not create a significant performance reduction for
ngcc. Since the results of the checks will be cached, and similar work is
done inside the TS compiler, so what we lose in doing it here, is saved later
in the processing. The main performance loss would be where there are lots
of files that need to be parsed for dependencies that do not end up being
processed by TS. But compared to the main ngcc processing this dependency
parsing is a small proportion of the work done and so should not impact
much on the overall performance of ngcc.
// FW-1444
PR Close#31509
When determining if a `main` path points to a UMD or CommonJS
format, the contents of the file need to be loaded and parsed.
Previously, it was assumed that the path referred to the exact filename,
but did not account for normal module resolution semantics, where the
path may be missing an extension or refer to a directory containing an
`index.js` file.
// FW-1444
PR Close#31509
Previously, ngcc had to walk the entire `node_modules` tree looking for
entry-points, even if it only needed to process a single target entry-point
and its dependencies.
This added up to a few seconds to each execution of ngcc, which is noticeable
when being run via the CLI integration.
Now, if an entry-point target is provided, only that target and its entry-points
are considered rather than the whole folder tree.
PR Close#30525
If a package delcares a class internally on an NgModule, ngcc
needs to be able to add a public export to this class's type.
Previously, if the typing file for the declared is not imported
from the typings entry-point file, then ngcc cannot find it.
Now we try to guess the .d.ts files from the equivalent .js
files.
PR Close#31411
Some packages do not actually provide a `typings` field in their
package.json. But TypeScript naturally infers the typings file from
the location of the JavaScript source file.
This commit modifies ngcc to do a similar inference when finding
entry-points to process.
Fixes#28603 (FW-1299)
PR Close#30591
There are scenarios where it is not possible for ngcc to guess the format
or configuration of an entry-point just from the files on disk.
Such scenarios include:
1) Unwanted entry-points: A spurious package.json makes ngcc think
there is an entry-point when there should not be one.
2) Deep-import entry-points: some packages allow deep-imports but do not
provide package.json files to indicate to ngcc that the imported path is
actually an entry-point to be processed.
3) Invalid/missing package.json properties: For example, an entry-point
that does not provide a valid property to a required format.
The configuration is provided by one or more `ngcc.config.js` files:
* If placed at the root of the project, this file can provide configuration
for named packages (and their entry-points) that have been npm installed
into the project.
* If published as part of a package, the file can provide configuration
for entry-points of the package.
The configured of a package at the project level will override any
configuration provided by the package itself.
PR Close#30591
Previously each test relied on large shared mock file-systems, which
makes it difficult to reason about what is actually being tested.
This commit breaks up these big mock file-systems into smaller more
focused chunks.
PR Close#30591
