The html parser already normalizes line endings (converting `\r\n` to `\n`)
for most text in templates but it was missing the expressions of ICU expansions.
In ViewEngine backticked literal strings, used to define inline templates,
were already normalized by the TypeScript parser.
In Ivy we are parsing the raw text of the source file directly so the line
endings need to be manually normalized.
This change ensures that inline templates have the line endings of ICU
expression normalized correctly, which matches the ViewEngine.
In ViewEngine external templates, defined in HTML files, the behavior was
different, since TypeScript was not normalizing the line endings.
Specifically, ICU expansion "expressions" are not being normalized.
This is a problem because it means that i18n message ids can be different on
different machines that are setup with different line ending handling,
or if the developer moves a template from inline to external or vice versa.
The goal is always to normalize line endings, whether inline or external.
But this would be a breaking change since it would change i18n message ids
that have been previously computed. Therefore this commit aligns the ivy
template parsing to have the same "buggy" behavior for external templates.
There is now a compiler option `i18nNormalizeLineEndingsInICUs`, which
if set to `true` will ensure the correct non-buggy behavior. For the time
being this option defaults to `false` to ensure backward compatibility while
allowing opt-in to the desired behavior. This option's default will be
flipped in a future breaking change release.
Further, when this option is set to `false`, any ICU expression tokens,
which have not been normalized, are added to the `ParseResult` from the
`HtmlParser.parse()` method. In the future, this collection of tokens could
be used to diagnose and encourage developers to migrate their i18n message
ids. See FW-2106.
Closes#36725
PR Close#36741
Move the creation of the results objects into the wrapper functions.
This makes it easier to reason about what the parser and lexer classes
are responsible for - you create a new object for each tokenization or
parsing activity and they hold the state of the activity.
PR Close#36741
Previously, an expansion case could only start with an alpha numeric character.
This commit fixes this by allowing an expansion case to start with any character
except `}`.
The [ICU spec](http://userguide.icu-project.org/formatparse/messages) is pretty vague:
> Use a "select" argument to select sub-messages via a fixed set of keywords.
It does not specify what can be a "keyword" but from looking at the surrounding syntax it
appears that it can indeed be any string that does not contain a `}` character.
Closes#31586
PR Close#36123
On a large compilation unit with big templates, the total time spent in
the `PlainCharacterCursor` constructor was 470ms. This commit applies
two optimizations to reduce this time:
1. Avoid the object spread operator within the constructor, as the
generated `__assign` helper in the emitted UMD bundle (ES5) does not
optimize well compared to a hardcoded object literal. This results in a
significant performance improvement. Because of the straight-forward
object literal, the VM is now much better able to optimize the memory
allocations which makes a significant difference as the
`PlainCharacterCursor` constructor is called in tight loops.
2. Reduce the number of `CharacterCursor` clones. Although cloning
itself is now much faster because of the optimization above, several
clone operations were not necessary.
Combined, these changes reduce the total time spent in the
`PlainCharacterCursor` constructor to just 10ms.
PR Close#34332
This is a potential fix for https://github.com/angular/vscode-ng-language-service/issues/235
suggested by @andrius-pra in
47696136e3.
Currently, CRLF line endings are converted to LFs and this causes the
diagnostics span to be off in templates that use CRLF. The line endings
must be preserved in order to maintain correct span offset. The solution
is to add an option to the Tokenizer to indicate such preservation.
PR Close#33241
Similar to interpolation, we do not want to completely remove whitespace
nodes that are siblings of an expansion.
For example, the following template
```html
<div>
<strong>items left<strong> {count, plural, =1 {item} other {items}}
</div>
```
was being collapsed to
```html
<div><strong>items left<strong>{count, plural, =1 {item} other {items}}</div>
```
which results in the text looking like
```
items left4
```
instead it should be collapsed to
```html
<div><strong>items left<strong> {count, plural, =1 {item} other {items}}</div>
```
which results in the text looking like
```
items left 4
```
---
**Analysis of the code and manual testing has shown that this does not cause
the generated ids to change, so there is no breaking change here.**
PR Close#31962
As part of FW-1265, the `@angular/compiler` package is made compatible
with the TypeScript `--strict` flag. This already unveiled a few bugs,
so the strictness flag seems to help with increasing the overall code health.
Read more about the strict flag [here](https://www.typescriptlang.org/docs/handbook/compiler-options.html)
PR Close#30993
Leading trivia, such as whitespace or comments, is
confusing for developers looking at source-mapped
templates, since they expect the source-map segment
to start after the trivia.
This commit adds skipping trivial characters to the lexer;
and then implements that in the template parser.
PR Close#30095
This PR alligns markup language lexer with the previous behaviour in version 7.x:
https://stackblitz.com/edit/angular-iancj2
While this behaviour is not perfect (we should be giving users an error message
here about invalid HTML instead of assuming text node) this is probably best we
can do without more substential re-write of lexing / parsing infrastructure.
This PR just fixes#29231 and restores VE behaviour - a more elaborate fix will
be done in a separate PR as it requries non-trivial rewrites.
PR Close#29328
This PR alligns markup language lexer with the previous behaviour in version 7.x:
https://stackblitz.com/edit/angular-iancj2
While this behaviour is not perfect (we should be giving users an error message
here about invalid HTML instead of assuming text node) this is probably best we
can do without more substential re-write of lexing / parsing infrastructure.
This PR just fixes#29231 and restores VE behaviour - a more elaborate fix will
be done in a separate PR as it requries non-trivial rewrites.
PR Close#29328
BREAKING CHANGE:
Certain elements (like `<tr>` or `<col>`) require parent elements to be of a certain type by the HTML specification
(ex. <tr> can only be inside <tbody> / <thead>). Before this change Angular template parser was auto-correcting
"invalid" HTML using the following rules:
- `<tr>` would be wrapped in `<tbody>` if not inside `<tbody>`, `<tfoot>` or `<thead>`;
- `<col>` would be wrapped in `<colgroup>` if not inside `<colgroup>`.
This meachanism of automatic wrapping / auto-correcting was problematic for several reasons:
- it is non-obvious and arbitrary (ex. there are more HTML elements that has rules for parent type);
- it is incorrect for cases where `<tr>` / `<col>` are at the root of a component's content, ex.:
```html
<projecting-tr-inside-tbody>
<tr>...</tr>
</projecting-tr-inside-tbody>
```
In the above example the `<projecting-tr-inside-tbody>` component culd be "surprised" to see additional
`<tbody>` elements inserted by Angular HTML parser.
PR Close#29219
Previously the start of a character indicated by an escape sequence
was being incorrectly computed by the lexer, which caused tokens
to include the start of the escaped character sequence in the
preceding token. In particular this affected the name extracted
from opening tags if the name was terminated by an escape sequence.
For example, `<t\n>` would have the name `t\` rather than `t`.
This fix refactors the lexer to use a "cursor" object to iterate over
the characters in the template source. There are two cursor implementations,
one expects a simple string, the other expects a string that contains
JavaScript escape sequences that need to be unescaped.
PR Close#28978
The parts of a token are supposed to be an array of not-null strings,
but we were using `null` for tags that had no prefix. This has been
fixed to use the empty string in such cases, which allows the `null !`
hack to be removed.
PR Close#28978
When tokenizing markup (e.g. HTML) element attributes
can have quoted or unquoted values (e.g. `a=b` or `a="b"`).
The `ATTR_VALUE` tokens were capturing the quotes, which
was inconsistent and also affected source-mapping.
Now the tokenizer captures additional `ATTR_QUOTE` tokens,
which the HTML related parsers understand and factor into their
token parsing.
PR Close#28055
In order to support source mapping of templates, we need
to be able to tokenize the template in its original context.
When the template is defined inline as a JavaScript string
in a TS/JS source file, the tokenizer must be able to handle
string escape sequences, such as `\n` and `\"` as they
appear in the original source file.
This commit teaches the lexer how to unescape these
sequences, but only when the `escapedString` option is
set to true. Otherwise there is no change to the tokenizing
behaviour.
PR Close#28055
The lexer that does the tokenizing can now process only a part the source
string, by passing a `range` property in the `options` argument. The
locations of the nodes that are tokenized will now take into account the
position of the span in the context of the original source string.
This `range` option is, in turn, exposed from the template parser as well.
Being able to process parts of files helps to enable SourceMap support
when compiling inline component templates.
PR Close#28055
When we added the strict null checks, the lexer had some `!`
operators added to prevent the compilation from failing.
This commit resolves this problem correctly and removes the
hacks.
Also the comment
```
// Note: this is always lowercase!
```
has been removed as it is no longer true.
See #24571
PR Close#28055
This commit consolidates the options that can modify the
parsing of text (e.g. HTML, Angular templates, CSS, i18n)
into an AST for further processing into a single `options`
hash.
This makes the code cleaner and more readable, but also
enables us to support further options to parsing without
triggering wide ranging changes to code that should not
be affected by these new options. Specifically, it will let
us pass information about the placement of a template
that is being parsed in its containing file, which is essential
for accurate SourceMap processing.
PR Close#28055
By pulling in `compiler` into `core` the `compiler` was not
100% tree-shakable and about 8KB of code was retained
when tree-shaken with closure.
PR Close#25531
All errors for existing fields have been detected and suppressed with a
`!` assertion.
Issue/24571 is tracking proper clean up of those instances.
One-line change required in ivy/compilation.ts, because it appears that
the new syntax causes tsickle emitted node to no longer track their
original sourceFiles.
PR Close#24572
This commit builds out enough of the JIT compiler to render
//packages/core/test/bundling/todo, and allows the tests to run in
JIT mode.
To play with the app, run:
bazel run --define=compile=jit //packages/core/test/bundling/todo:prodserver
PR Close#24138
BREAKING CHANGE:
The `<template>` tag was deprecated in Angular v4 to avoid collisions (i.e. when
using Web Components).
This commit removes support for `<template>`. `<ng-template>` should be used
instead.
BEFORE:
<!-- html template -->
<template>some template content</template>
# tsconfig.json
{
# ...
"angularCompilerOptions": {
# ...
# This option is no more supported and will have no effect
"enableLegacyTemplate": [true|false]
}
}
AFTER:
<!-- html template -->
<ng-template>some template content</ng-template>
PR Close#22783
After this, neither @angular/compiler nor @angular/comnpiler-cli depend
on @angular/core.
This add a duplication of some interfaces and enums which is stored
in @angular/compiler/src/core.ts
BREAKING CHANGE:
- `@angular/platform-server` now additionally depends on
`@angular/platform-browser-dynamic` as a peer dependency.
PR Close#18683
In previous version of tsickle abstract class methods were materialized.
The change resulted in 6Kb savings in angular.io bundle.
This change also required the removal of `@private` and `@return` type
annotation as it is explicitly dissalowed by tsickle.
NOTE: removed casts in front of `makeDecorator` due to:
https://github.com/angular/devkit/issues/45
```
14938 Jul 19 13:16 0.b19e913fbdd6507d346b.chunk.js
1535 Jul 19 13:16 inline.d8e019ea3cfdd86c2bd0.bundle.js
589178 Jul 19 13:16 main.54c97bcb6f254776b678.bundle.js
34333 Jul 19 13:16 polyfills.4a3c9ca9481d53803157.bundle.js
14938 Jul 18 16:55 0.b19e913fbdd6507d346b.chunk.js
1535 Jul 18 16:55 inline.0c83abb44fad9a2768a7.bundle.js
582786 Jul 18 16:55 main.ea290db71b051813e156.bundle.js
34333 Jul 18 16:55 polyfills.4a3c9ca9481d53803157.bundle.js
main savings: 589178 - 582786 = 6,392
```
PR Close#18236
