Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make TS InstantiationExpr parsing more permissive #2188

Merged
merged 4 commits into from
Apr 19, 2022
Merged

Make TS InstantiationExpr parsing more permissive #2188

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
46f959b
Make TS InstantiationExpr parsing more permissive
g-plane Apr 18, 2022
a505195
Merge branch 'master' into fix-permissive-ts-instantiation-expr
evanw Apr 19, 2022
0f9ba22
handle the `import` keyword
evanw Apr 19, 2022
0efcf5d
changelog entry
evanw Apr 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,22 @@

The upcoming version of TypeScript adds the `moduleSuffixes` field to `tsconfig.json` that introduces more rules to import path resolution. Setting `moduleSuffixes` to `[".ios", ".native", ""]` will try to look at the the relative files `./foo.ios.ts`, `./foo.native.ts`, and finally `./foo.ts` for an import path of `./foo`. Note that the empty string `""` in `moduleSuffixes` is necessary for TypeScript to also look-up `./foo.ts`. This was announced in the [TypeScript 4.7 beta blog post](https://devblogs.microsoft.com/typescript/announcing-typescript-4-7-beta/#resolution-customization-with-modulesuffixes).

* Match the new ASI behavior from TypeScript nightly builds
* Match the new ASI behavior from TypeScript nightly builds ([#2188](https://github.com/evanw/esbuild/pull/2188))

This release updates esbuild to match some very recent behavior changes in the TypeScript parser regarding automatic semicolon insertion. For more information, see the following issues:
This release updates esbuild to match some very recent behavior changes in the TypeScript parser regarding automatic semicolon insertion. For more information, see TypeScript issues #48711 and #48654 (I'm not linking to them directly to avoid Dependabot linkback spam on these issues due to esbuild's popularity). The result is that the following TypeScript code is now considered valid TypeScript syntax:

* https://github.com/microsoft/TypeScript/issues/48711
```ts
class A<T> {}
new A<number> /* ASI now happens here */
if (0) {}

interface B {
(a: number): typeof a /* ASI now happens here */
<T>(): void
}
```

This fix was contributed by [@g-plane](https://github.com/g-plane).

## 0.14.36

Expand Down
74 changes: 48 additions & 26 deletions internal/js_parser/ts_parser.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -856,6 +856,19 @@ func (p *parser) isTSArrowFnJSX() (isTSArrowFn bool) {
return
}

func (p *parser) nextTokenIsOpenParenOrLessThanOrDot() (result bool) {
oldLexer := p.lexer
p.lexer.Next()

result = p.lexer.Token == js_lexer.TOpenParen ||
p.lexer.Token == js_lexer.TLessThan ||
p.lexer.Token == js_lexer.TDot

// Restore the lexer
p.lexer = oldLexer
return
}

// This function is taken from the official TypeScript compiler source code:
// https://github.com/microsoft/TypeScript/blob/master/src/compiler/parser.ts
func (p *parser) canFollowTypeArgumentsInExpression() bool {
Expand All @@ -867,35 +880,44 @@ func (p *parser) canFollowTypeArgumentsInExpression() bool {
js_lexer.TTemplateHead: // foo<T> `...${100}...`
return true

// Consider something a type argument list only if the following token can't start an expression.
case
// These tokens can't follow in a call expression, nor can they start an
// expression. So, consider the type argument list part of an instantiation
// expression.
js_lexer.TComma, // foo<x>,
js_lexer.TDot, // foo<x>.
js_lexer.TQuestionDot, // foo<x>?.
js_lexer.TCloseParen, // foo<x>)
js_lexer.TCloseBracket, // foo<x>]
js_lexer.TColon, // foo<x>:
js_lexer.TSemicolon, // foo<x>;
js_lexer.TQuestion, // foo<x>?
js_lexer.TEqualsEquals, // foo<x> ==
js_lexer.TEqualsEqualsEquals, // foo<x> ===
js_lexer.TExclamationEquals, // foo<x> !=
js_lexer.TExclamationEqualsEquals, // foo<x> !==
js_lexer.TAmpersandAmpersand, // foo<x> &&
js_lexer.TBarBar, // foo<x> ||
js_lexer.TQuestionQuestion, // foo<x> ??
js_lexer.TCaret, // foo<x> ^
js_lexer.TAmpersand, // foo<x> &
js_lexer.TBar, // foo<x> |
js_lexer.TCloseBrace, // foo<x> }
js_lexer.TEndOfFile: // foo<x>
return true
// From "isStartOfExpression()"
js_lexer.TPlus,
js_lexer.TMinus,
js_lexer.TTilde,
js_lexer.TExclamation,
js_lexer.TDelete,
js_lexer.TTypeof,
js_lexer.TVoid,
js_lexer.TPlusPlus,
js_lexer.TMinusMinus,
js_lexer.TLessThan,

// From "isStartOfLeftHandSideExpression()"
js_lexer.TThis,
js_lexer.TSuper,
js_lexer.TNull,
js_lexer.TTrue,
js_lexer.TFalse,
js_lexer.TNumericLiteral,
js_lexer.TBigIntegerLiteral,
js_lexer.TStringLiteral,
js_lexer.TOpenBracket,
js_lexer.TOpenBrace,
js_lexer.TFunction,
js_lexer.TClass,
js_lexer.TNew,
js_lexer.TSlash,
js_lexer.TSlashEquals,
js_lexer.TIdentifier:
return false
Copy link
Owner

@evanw evanw Apr 18, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where did these come from? This doesn't appear to follow what Microsoft changed in their parser. For example, consider the following code:

f<T>
import "x"

f<T>
import("x")

After their PR, TypeScript parses it like this:

f;
import "x";

f < T > import("x");

But with your PR, esbuild parses it like this:

f;
import "x";
f;
import("x");

I tested this case with import because some of the code Microsoft is now using looks like this:

function isStartOfLeftHandSideExpression(): boolean {
  switch (token()) {
    case SyntaxKind.ThisKeyword:
    case SyntaxKind.SuperKeyword:
    case SyntaxKind.NullKeyword:
    case SyntaxKind.TrueKeyword:
    case SyntaxKind.FalseKeyword:
    case SyntaxKind.NumericLiteral:
    case SyntaxKind.BigIntLiteral:
    case SyntaxKind.StringLiteral:
    case SyntaxKind.NoSubstitutionTemplateLiteral:
    case SyntaxKind.TemplateHead:
    case SyntaxKind.OpenParenToken:
    case SyntaxKind.OpenBracketToken:
    case SyntaxKind.OpenBraceToken:
    case SyntaxKind.FunctionKeyword:
    case SyntaxKind.ClassKeyword:
    case SyntaxKind.NewKeyword:
    case SyntaxKind.SlashToken:
    case SyntaxKind.SlashEqualsToken:
    case SyntaxKind.Identifier:
      return true;
    case SyntaxKind.ImportKeyword:
      return lookAhead(nextTokenIsOpenParenOrLessThanOrDot);
    default:
      return isIdentifier();
  }
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I forgot to handle import token, and I will add it.

They changed to use an existing function isStartOfExpression.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can I do something similar to lookAhead in esbuild?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's kind of hacky. I stash a copy of p.lexer on the stack and then overwrite it again after backtracking has finished. I can take this PR from here and add the backtracking.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I hope we can have a more common method to do backtracking.


case js_lexer.TImport:
return !p.nextTokenIsOpenParenOrLessThanOrDot()

default:
// Anything else treat as an expression.
return false
return true
}
}

Expand Down
16 changes: 16 additions & 0 deletions internal/js_parser/ts_parser_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1913,6 +1913,22 @@ func TestTSInstantiationExpression(t *testing.T) {
expectParseErrorTSX(t, "type x = y\n<number>\nz", "<stdin>: ERROR: Unexpected end of file\n")
expectParseErrorTS(t, "type x = typeof y\n<number>\nz\n</number>", "<stdin>: ERROR: Unterminated regular expression\n")
expectParseErrorTSX(t, "type x = typeof y\n<number>\nz", "<stdin>: ERROR: Unexpected end of file\n")

// See: https://github.com/microsoft/TypeScript/issues/48654
expectPrintedTS(t, "x<true>\ny", "x < true > y;\n")
expectPrintedTS(t, "x<true>\nif (y) {}", "x;\nif (y) {\n}\n")
expectPrintedTS(t, "x<true>\nimport 'y'", "x;\nimport \"y\";\n")
expectPrintedTS(t, "x<true>\nimport('y')", "x < true > import(\"y\");\n")
expectPrintedTS(t, "x<true>\nimport.meta", "x < true > import.meta;\n")
expectPrintedTS(t, "new x<number>\ny", "new x() < number > y;\n")
expectPrintedTS(t, "new x<number>\nif (y) {}", "new x();\nif (y) {\n}\n")
expectPrintedTS(t, "new x<true>\nimport 'y'", "new x();\nimport \"y\";\n")
expectPrintedTS(t, "new x<true>\nimport('y')", "new x() < true > import(\"y\");\n")
expectPrintedTS(t, "new x<true>\nimport.meta", "new x() < true > import.meta;\n")

// See: https://github.com/microsoft/TypeScript/issues/48759
expectParseErrorTS(t, "x<true>\nimport<T>('y')", "<stdin>: ERROR: Expected \"(\" but found \"<\"\n")
expectParseErrorTS(t, "new x<true>\nimport<T>('y')", "<stdin>: ERROR: Expected \"(\" but found \"<\"\n")
}

func TestTSExponentiation(t *testing.T) {
Expand Down