Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Less template literal and string literal reduction #56165

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

PranavSenthilnathan
Copy link
Member

@PranavSenthilnathan PranavSenthilnathan commented Oct 20, 2023

Checking perf impact of the naive change.

Fixes #56081

@typescript-bot typescript-bot added the For Uncommitted Bug PR for untriaged, rejected, closed or missing bug label Oct 20, 2023
@PranavSenthilnathan
Copy link
Member Author

@typescript-bot test this
@typescript-bot perf test this
@typescript-bot user test this

@jakebailey
Copy link
Member

@typescript-bot test this
@typescript-bot perf test this
@typescript-bot user test this

I don't think you're on the author list so that doesn't work, though maybe it's some other list that does it.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 20, 2023

Heya @jakebailey, I've started to run the regular perf test suite on this PR at 8e76d16. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 20, 2023

Heya @jakebailey, I've started to run the diff-based user code test suite on this PR at 8e76d16. You can monitor the build here.

Update: The results are in!

@jakebailey
Copy link
Member

What test case are you looking at for this? #52345?

@typescript-bot typescript-bot added For Milestone Bug PRs that fix a bug with a specific milestone and removed For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Oct 20, 2023
@PranavSenthilnathan
Copy link
Member Author

Updated comment. It's #56081

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the user test suite comparing main and refs/pull/56165/merge:

There were infrastructure failures potentially unrelated to your change:

  • 3 instances of "Package install failed"
  • 1 instance of "Unknown failure"

Otherwise...

Everything looks good!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,123k (± 0.01%) 295,097k (± 0.01%) ~ 295,046k 295,160k p=0.199 n=6
Parse Time 2.63s (± 0.57%) 2.63s (± 0.29%) ~ 2.62s 2.64s p=0.503 n=6
Bind Time 0.84s (± 0.90%) 0.84s (± 1.30%) ~ 0.83s 0.85s p=0.865 n=6
Check Time 8.06s (± 0.31%) 8.06s (± 0.29%) ~ 8.03s 8.10s p=0.935 n=6
Emit Time 7.08s (± 0.12%) 7.06s (± 0.29%) ~ 7.03s 7.08s p=0.119 n=6
Total Time 18.61s (± 0.14%) 18.58s (± 0.19%) ~ 18.53s 18.63s p=0.076 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 192,610k (± 1.56%) 193,587k (± 1.64%) ~ 190,674k 196,533k p=0.471 n=6
Parse Time 1.35s (± 1.30%) 1.36s (± 1.27%) ~ 1.34s 1.38s p=0.278 n=6
Bind Time 0.73s (± 0.00%) 0.73s (± 0.00%) ~ 0.73s 0.73s p=1.000 n=6
Check Time 9.18s (± 0.17%) 9.19s (± 0.34%) ~ 9.13s 9.21s p=0.413 n=6
Emit Time 2.65s (± 0.52%) 2.63s (± 0.40%) ~ 2.62s 2.65s p=0.161 n=6
Total Time 13.90s (± 0.22%) 13.91s (± 0.17%) ~ 13.88s 13.93s p=0.569 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,301k (± 0.00%) 347,299k (± 0.00%) ~ 347,276k 347,314k p=0.630 n=6
Parse Time 2.46s (± 0.33%) 2.46s (± 0.60%) ~ 2.44s 2.48s p=0.508 n=6
Bind Time 0.94s (± 0.43%) 0.94s (± 0.00%) ~ 0.94s 0.94s p=0.405 n=6
Check Time 6.92s (± 0.33%) 6.92s (± 0.32%) ~ 6.90s 6.96s p=1.000 n=6
Emit Time 4.02s (± 0.44%) 4.02s (± 0.27%) ~ 4.01s 4.04s p=0.806 n=6
Total Time 14.34s (± 0.17%) 14.34s (± 0.27%) ~ 14.30s 14.40s p=0.935 n=6
TFS - node (v18.15.0, x64)
Memory used 302,535k (± 0.01%) 302,526k (± 0.01%) ~ 302,486k 302,541k p=0.688 n=6
Parse Time 2.00s (± 0.73%) 2.00s (± 0.52%) ~ 1.99s 2.02s p=1.000 n=6
Bind Time 1.00s (± 0.00%) 1.01s (± 1.46%) ~ 0.99s 1.03s p=0.295 n=6
Check Time 6.26s (± 0.42%) 6.25s (± 0.54%) ~ 6.22s 6.31s p=0.573 n=6
Emit Time 3.56s (± 0.58%) 3.56s (± 0.50%) ~ 3.53s 3.58s p=0.570 n=6
Total Time 12.81s (± 0.17%) 12.82s (± 0.17%) ~ 12.79s 12.85s p=0.571 n=6
material-ui - node (v18.15.0, x64)
Memory used 470,481k (± 0.00%) 470,483k (± 0.00%) ~ 470,453k 470,504k p=0.688 n=6
Parse Time 2.58s (± 0.53%) 2.58s (± 0.62%) ~ 2.56s 2.61s p=0.315 n=6
Bind Time 1.00s (± 1.21%) 0.99s (± 1.22%) ~ 0.98s 1.01s p=0.680 n=6
Check Time 16.62s (± 0.34%) 16.59s (± 0.29%) ~ 16.52s 16.66s p=0.470 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.19s (± 0.38%) 20.17s (± 0.28%) ~ 20.08s 20.25s p=0.575 n=6
xstate - node (v18.15.0, x64)
Memory used 512,617k (± 0.01%) 512,621k (± 0.01%) ~ 512,558k 512,726k p=0.936 n=6
Parse Time 3.27s (± 0.42%) 3.28s (± 0.32%) ~ 3.26s 3.29s p=0.241 n=6
Bind Time 1.55s (± 0.26%) 1.55s (± 0.53%) ~ 1.53s 1.55s p=0.218 n=6
Check Time 2.88s (± 1.49%) 2.86s (± 1.01%) ~ 2.82s 2.89s p=0.574 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=1.000 n=6
Total Time 7.77s (± 0.68%) 7.75s (± 0.38%) ~ 7.70s 7.78s p=0.521 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,376ms (± 1.44%) 2,360ms (± 1.26%) ~ 2,335ms 2,412ms p=0.575 n=6
Req 2 - geterr 5,339ms (± 1.13%) 5,446ms (± 1.13%) +107ms (+ 2.01%) 5,322ms 5,483ms p=0.016 n=6
Req 3 - references 326ms (± 0.19%) 327ms (± 0.81%) ~ 325ms 332ms p=0.557 n=6
Req 4 - navto 278ms (± 1.26%) 273ms (± 1.29%) ~ 271ms 280ms p=0.115 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 78ms (± 4.93%) 86ms (± 7.24%) ~ 74ms 90ms p=0.075 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,490ms (± 1.06%) 2,488ms (± 1.07%) ~ 2,455ms 2,533ms p=0.810 n=6
Req 2 - geterr 4,113ms (± 2.06%) 4,127ms (± 1.78%) ~ 4,055ms 4,219ms p=0.378 n=6
Req 3 - references 340ms (± 1.61%) 341ms (± 1.74%) ~ 333ms 348ms p=0.934 n=6
Req 4 - navto 284ms (± 0.29%) 285ms (± 0.48%) ~ 283ms 287ms p=0.865 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 84ms (± 6.63%) 84ms (± 6.77%) ~ 77ms 89ms p=1.000 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,592ms (± 0.97%) 2,595ms (± 0.63%) ~ 2,569ms 2,619ms p=0.748 n=6
Req 2 - geterr 1,672ms (± 2.68%) 1,711ms (± 1.18%) ~ 1,686ms 1,734ms p=0.173 n=6
Req 3 - references 113ms (± 9.51%) 121ms (± 8.93%) ~ 105ms 128ms p=0.220 n=6
Req 4 - navto 359ms (± 0.45%) 362ms (± 0.77%) ~ 359ms 367ms p=0.101 n=6
Req 5 - completionInfo count 2,073 (± 0.00%) 2,073 (± 0.00%) ~ 2,073 2,073 p=1.000 n=6
Req 5 - completionInfo 302ms (± 2.18%) 304ms (± 1.93%) ~ 297ms 313ms p=0.520 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 152.53ms (± 0.18%) 152.36ms (± 0.19%) -0.17ms (- 0.11%) 151.08ms 156.17ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 227.29ms (± 0.15%) 227.18ms (± 0.16%) -0.10ms (- 0.05%) 225.95ms 232.25ms p=0.001 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 228.45ms (± 0.17%) 228.35ms (± 0.15%) ~ 226.47ms 232.80ms p=0.077 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 228.73ms (± 0.15%) 228.79ms (± 0.15%) ~ 227.44ms 231.94ms p=0.092 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@PranavSenthilnathan
Copy link
Member Author

So this is not the solution since this was an intentional feature introduced in #41276. But I think there should probably be some middle ground between reducing eagerly during union creation and lazily when subtype reduction is performed. Maybe just doing subtype reduction when checking variable declaration nodes?

@jakebailey
Copy link
Member

The main problem is that if we don't always reduce, then two unions that are actually the same thing can be present at once, which means you can't fast check for equality and such (maybe some other invariants).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
For Milestone Bug PRs that fix a bug with a specific milestone
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Memory Leak / Infinite Recursion. TS hangs, TSC never finishes, VSCode and Intellisense dies.
3 participants