Previously, in order to remain as deterministic as possible, the
Dockerfile for the preview server Docker image had all dependencies
pinned to specific versions. It turns out that some packages (such as
`nginx`, `nodejs`, and `openssl` - potentially others too) make older
versions unavailable on the repositories once a newer version is
available.
See for example:
- https://github.com/nodesource/distributions/issues/33
- https://askubuntu.com/questions/715104/how-can-i-downgrade-openssl-via-apt-get
This commit, therefore, removes the exact versions for these packages.
The latest versions will be installed everytime the Docker image is
built (subject to Docker caching).
PR Close#36837
In order to ease local development, self-signed SSL/TLS certificates are
created when building the preview server Docker image. These
certificates are valid for 365 days. Thus, it is possible for an old
certificate to be re-used past its expiration date due to Docker's
caching intermediate layers.
Previously, this would lead to hard-to-debug failures in the
`aio-health-check` and `aio-verify-setup` checks. Even after finding out
that the failures were caused by an expired certificate, it was not
obvious why that would be the case.
This commit adds an additional check to the `aio-health-check` command
that checks the certificates' expiration dates. This helps surface such
errors. It also prints a more helpful message, prompting the user to
build the Docker image with the `--no-cache` option to fix the problem
with self-signed certificates.
PR Close#36837
Previously, the preview server Docker image was based on Debian 9
(stretch).
This commit upgrades the preview server Docker image to Debian 10
(buster) and also upgrades all dependencies to latest versions
(including upgrading Node.js from v10 to v12).
(The GCE VM running the preview server Docker container was also
upgraded from Debian 9 to 10 on 2020-04-27.)
---
Other changes:
- Pinned the installed version of `curl` to make the `aio-health-check`
and `aio-verify-setup` checks (which use `curl`) more deterministic.
- Dropped the `*-backports` Debian repositories, since they are no
longer needed. The `*-backports` repositories were introduced in
593fe5ed259d5b118d41e2b961b462e922762700 to install `nginx` from, but
became obsolete in 2f1a862b831f44b9b923dcc52f94af69d3b40aa6, which
switched to installing `nginx` from the regular repositories again.
- Added `vim` to the list of installed dependencies (for convenience
during debugging).
PR Close#36837
This commit upgrades all dependencies in `scripts-js/` to latest
versions and also includes all necessary code changes to ensure the
tests are passing with the new dependency versions.
PR Close#36837
The current benchmark for transplanted views only exercises the path
when the declaration location is dirty and the insertion is not. This
test adds a benchmark for when both insertion and declaration are dirty.
PR Close#36722
This commit fixes 2 separate issues related to root nodes retrieval from
embedded views with `<ng-content>`:
1) we did not account for the case where there were no projectable nodes
for a given `<ng-content>`;
2) we did not account for the case where projectable nodes for a given
`<ng-content>` were represented as an array of native nodes (happens in
the case of dynamically created components with projectable nodes);
Fixes#35967
PR Close#36051
A caching mechanism was put in place to prevent repeated calls to
the Github API. As the CI setup no longer relies on calls to the
Github API, this caching is no longer necessary.
It was discovered that this caching was causing a contention issue
for saucelabs testing as the same tunnel was being reused for
multiple jobs simultaneously. With this caching mechanism removed
the jobs will once again run via separate tunnels.
PR Close#36936
The `elements` tests were disabled on Saucelabs, because they were failing on IE10. The problem was that we were loading an es2015 file from npm directly, causing a syntax error. These changes transpile the file to es5.
PR Close#36929
Disables Bazel runfile tree creation. Only if a given
execution strategy relies on runfile tree creation, the
runfile tree is created lazily. This helps as currently
Bazel spends unnecessary time on CI building runfile trees
for tests which are cached and aren't re-run.
The goal is to spend less time on CI for cached test/build
targets. We can't measure impact yet as there are targets
for the integration tests that hide the potential benefits.
on the components repo a 80% time reduction could be observed.
PR Close#36914
The `getLocation()` method was not working as there were typos in the
properties it was reading. This was not picked up because there were
neither typings for these properties nor unit tests to check it worked.
PR Close#36920
Previously we were passing a string form of the value to pluralize
to the `getLocalePluralCase()` function that is extracted from the
locale data. But some locales have functions that rely upon this
value being a number not a string.
Now we convert the value to a number before passing it to the
locale data function.
Fixes#36888
PR Close#36901
This commit also updates the projects to more closely match what a newly
generated app would look like with the exception of `tslint.json` files,
which would create too many linting failures. These will be updated in a
follow-up PR.
PR Close#36145
Previously, in the `test_aio` CI job, we ran ngcc before building the
app with `yarn build`. This was supposed to have the benefit of taking
advantage of the parallel capabilities of standalone ngcc (vs implicitly
running it via `ng build`).
It turns out that the work done by the standalone ngcc was thrown away
before the `ng build`, resulting in `ng build` having to run ngcc all
over again. This happened because the `yarn build` script (run after the
standalone ngcc step) also runs `yarn install`, which essentially cleans
up `node_modules/`, thus discarding all the work already done by ngcc.
Here is an [example CI job][1], where this can be seen in action:
One can see the "Compiling <some-package> : es2015 as esm2015" logs in
the `yarn --cwd aio ngcc --properties es2015` step (as the standalone
ngcc processes the various entry-points) and then see the same logs in
the `yarn --cwd aio build --progress=false` step (as ngcc has to process
the entry-points all over again).
This commit removes the redundant standalone ngcc run and lets the CLI
handle ngcc via `ng build`. It is possible to instrument the build
process in a way that we can run the standalone ngcc after
`yarn install` and thus take advantage of the performance gains in
parallel mode, but the latest version of the CLI can already run ngcc in
parallel mode as a pre-build step, so this is unnecessary.
[1]: https://circleci.com/gh/angular/angular/658691
PR Close#36145
Update the Angular CLI and Angular framework packages to latest `@next`
versions. Also, update the app to look more closely to how a newly
generated app with the latest CLI would look like.
PR Close#36145
This commit updates all payload sizes for angular.io to make it easier
to compare payload size changes as a result of upgrading Angular
packages and other dependencies in subsequent commits.
PR Close#36145
This commit adds a method `overrideInlineTemplate` to the
`MockTypescriptHost`. This allows us to override an inline template
in a Component without changing the TypeScript parts. This methods works
in a similar way as `MockTypescriptHost.override()`, which is used for
overriding external template.
PR Close#36890
This function needs to deduplicate the paths that are found from the
paths mappings. Previously this deduplication was not linear and also
called the expensive `relative()` function many times.
This commit, suggested by @JoostK, reduces the complexity of the deduplication
by using a tree structure built from the segments of each path.
PR Close#36881
Previously the `basePaths` were computed when the finder was instantiated.
This was a waste of effort in the case that the targeted entry-point is already
processed.
This change makes the computation of `basePaths` lazy, so that the work is
only done if they are actually needed.
Fixes#36874
PR Close#36881
In TS 3.9 the compiler will start to wrap ES2015 classes in an IIFE to help with
tree-shaking when the class has "associated" statements.
E.g.
```ts
let PlatformLocation = /** @class */ (() => {
...
class PlatformLocation {
}
...
return PlatformLocation;
})();
```
This commit updates `Esm2015ReflectionHost` to support this format.
PR Close#36884
Previously the setup-rbe.sh script did not allow accounts that did
have domains of angular.io or google.com. Since we add emails from
other domains into everyone@angular.io, we are unable to be certain
in the script that the account is not actually a member of the
required group. This change adds the option to choose to continue
with an email account logged in which we cannot verify by domain.
PR Close#36846
Migrates away from inline searching for files and running buildifier
directly, instead using ng-dev for formatting. Additionally, provides
a deprecation message for any usages of the previous commands.
PR Close#36842
In an effort to centralize formatting and linting enforcement into one
location, buildifier is being added as a formatter for ng-dev's format
command. Allowing for format enforcement for all .bzl, .bazel, WORKSPACE
and BUILD files.
PR Close#36842
For testing on IE, shims must be served in es5. Because the shims
served in these tests come from node_modules and are not part of
the angular/angular source code, they must be transpiled to es5 if
they are published as es6. This macro allows for a uniform method
for running this transpilation during the build process.
PR Close#36802
This commit makes the test project a filegroup so that it could be
shared with the Ivy tests.
Also removed `project/foo.ts` since it is no longer used.
PR Close#36865
Create a common environment stamping script in the ng-dev tooling
as to be used in common release tasks. This is the first step in
consolidating pieces of the release scripting process into a single
location to develop a release tool.
PR Close#36844
Only refresh transplanted views at the insertion location in Ivy.
Previously, Ivy would check transplanted views at both the insertion and
declaration points. This is achieved by adding a marker to the insertion
tree when we encounter a transplanted view that needs to be refreshed at
its declaration. We use this marker as an extra indication that we still
need to descend and refresh those transplanted views at their insertion
locations even if the insertion view and/or its parents are not dirty.
This change fixes several issues:
* Transplanted views refreshed twice if both insertion and declaration
are dirty. This could be an error if the insertion component changes
result in data not being available to the transplanted view because it
is slated to be removed.
* CheckAlways transplanted views not refreshed if shielded by
non-dirty OnPush (fixes#35400)
* Transplanted views still refreshed when insertion tree is detached
(fixes#21324)
PR Close#35968
Now that `ngcc/src/ngcc_options` imports `FileWriter` type, there is a
circular dependency detected by the `ts-circular-deps:check` lint check:
```
ngcc/src/ngcc_options.ts
→ ngcc/src/writing/file_writer.ts
→ ngcc/src/packages/entry_point_bundle.ts
→ ngcc/src/ngcc_options.ts
```
This commit moves the `PathMappings` type (and related helpers) to a
separate file to avoid the circular dependency.
NOTE:
The circular dependency was only with taking types into account. There
was no circular dependency for the actual (JS) code.
PR Close#36626
When running in parallel mode, worker processes forward errors thrown
during task processing to the master process, which in turn exits with
an error.
However, there are cases where the error is not directly related to
processing the entry-point. One such case is when there is not enough
memory (for example, due to all the other tasks being processed
simultaneously).
Previously, an `ENOMEM` error thrown on a worker process would propagate
to the master process, eventually causing ngcc to exit with an error.
Example failure: https://circleci.com/gh/angular/angular/682198
This commit improves handling of these low-memory situations by
detecting `ENOMEM` errors and killing the worker process, thus allowing
the master process to decide how to handle that. The master process will
put the task back into the tasks queue and continue processing tasks
with the rest of the worker processes (and thus with lower memory
pressure).
PR Close#36626
Previously, when the last worker process crashed, the master process
would try to re-spawn it indefinitely. This could lead to an infinite
loop (if for some reason the worker process kept crashing).
This commit avoids this by limiting the number of re-spawn attempts to
3, after which ngcc will exit with an error.
PR Close#36626
Previously, when running in parallel mode and a worker process crashed
while processing a task, it was not possible for ngcc to continue
without risking ending up with a corrupted entry-point and therefore it
exited with an error. This, for example, could happen when a worker
process received a `SIGKILL` signal, which was frequently observed in CI
environments. This was probably the result of Docker killing processes
due to increased memory pressure.
One factor that amplifies the problem under Docker (which is often used
in CI) is that it is not possible to distinguish between the available
CPU cores on the host machine and the ones made available to Docker
containers, thus resulting in ngcc spawning too many worker processes.
This commit addresses these issues in the following ways:
1. We take advantage of the fact that files are written to disk only
after an entry-point has been fully analyzed/compiled. The master
process can now determine whether a worker process has not yet
started writing files to disk (even if it was in the middle of
processing a task) and just put the task back into the tasks queue if
the worker process crashes.
2. The master process keeps track of the transformed files that a worker
process will attempt to write to disk. If the worker process crashes
while writing files, the master process can revert any changes and
put the task back into the tasks queue (without risking corruption).
3. When a worker process crashes while processing a task (which can be a
result of increased memory pressure or too many worker processes),
the master process will not try to re-spawn it. This way the number
or worker processes is gradually adjusted to a level that can be
accomodated by the system's resources.
Examples of ngcc being able to recover after a worker process crashed:
- While idling: https://circleci.com/gh/angular/angular/682197
- While compiling: https://circleci.com/gh/angular/angular/682209
- While writing files: https://circleci.com/gh/angular/angular/682267
Jira issue: [FW-2008](https://angular-team.atlassian.net/browse/FW-2008)
Fixes#36278
PR Close#36626
This commit adds a `revertFile()` method to `FileWriter`, which can
revert a transformed file (and its backup - if any) written by the
`FileWriter`.
In a subsequent commit, this will be used to allow ngcc to recover
when a worker process crashes in the middle of processing a task.
PR Close#36626
With this commit, the master process will keep track of the transformed
files that each worker process is intending to write to disk.
In a subsequent commit, this info will be used to allow ngcc to recover
when a worker process crashes in the middle of processing a task.
PR Close#36626
With this commit, worker processes will notify the master process about
the transformed files they are about to write to disk before starting
writing them.
In a subsequent commit, this will be used to allow ngcc to recover when
a worker process crashes in the middle of processing a task.
PR Close#36626
This commit enhances the `CompileFn`, which is used to process each
entry-point, to support running a passed-in callback (and wait for it to
complete) before proceeding with writing the transformed files to disk.
This functionality is currently not used. In a subsequent commit, it
will be used for passing info from worker processes to the master
process that will allow ngcc to recover when a worker process crashes in
the middle of processing a task.
PR Close#36626
Rename the `markTaskCompleted()` method to be consistent with the other
similar methods of `TaskQueue` (`markAsFailed()` and
`markAsUnprocessed()`).
PR Close#36626
This commit adds support for stopping processing an in-progress task
and moving it back to the list of pending tasks.
In a subsequent commit, this will be used to allow ngcc to recover when
a worker process crashes in the middle of processing a task.
PR Close#36626
Previously, ngcc would run in parallel mode (using the
`ClusterExecutor`) when there were at least 2 CPU cores (and all other
requirements where met). On systems with just 2 CPU cores, this meant
there would only be one worker process (since one CPU core is always
reserved for the master process). In these cases, the tasks would still
be processed serially (on the one worker process), but we would also pay
the overhead of communicating between the master and worker processes.
This commit fixes this by only running in parallel mode if there are
more than 2 CPU cores (i.e. at least 2 worker processes).
PR Close#36626
Previously, the "Compiling <entryPoint>" log message was printed before
starting to analyze and transform files, but after creating the
`EntryPointBundle` (which includes creating the TS program).
Since creating the `EntryPointBundle` involves some work, it is more
accurate to move the log message before creating the bundle.
PR Close#36626
Previously the exec command was used, however the exec command would
exit the original calling script regardless of the whether exit was
called. This caused the release script to always exit after the
pre-check phase.
PR Close#36862
