Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Experiment - monomorphic node/type/signature #58928

Draft
wants to merge 14 commits into
base: main
Choose a base branch
from
Draft

Experiment - monomorphic node/type/signature #58928

wants to merge 14 commits into from
+4,274 −1,908

Conversation

dragomirtitian
Copy link
Contributor

@dragomirtitian dragomirtitian commented Jun 19, 2024 •
edited
Loading

Experiment to make several objects monomorphic by keeping the common properties in a single object shape and moving all other properties to a separate object and using accessors to preserve compatibility with existing code.

Local improvements seem promising, building TSC/compiler: -20% in time with +10% in memory (See bellow for results)

Code is rough around the edges, just testing out as a proof of concept.

Note: This is the result of me and @acutmore bouncing ideas about improving access to Node.kind

Update

On Node, the improvements with everything in this PR are:

  • a total time win of 16.0% (ranging 10.5% to 23.7%)
  • a checker time win of 20.8% (ranging 16.6% to 26.8%)
  • a memory cost of 2.7% (ranging 1.9% to 4.4%)

Which is an exchange rate of 5.8 on total time, and 7.6 on checker time.

Breaking down where the performance win comes from (from the separate PRs):

AST Node monomorphism:

  • total time win: 3.3% - 9%
  • Check Time win: 3.7% - 11%
  • Mem cost: ~0-1%

Types monomorphism:

  • total time win: 4.1% - 20%
  • Check Time win: 5.3% - 22.5%
  • Mem cost: 0.75%-1.2%

Signatures monomorphism:

  • total time win: 0.45% - 1.72%
  • Check Time win: 0.56% - 2.04%
  • Mem cost: ~0%

The AST nodes win is smaller than I remember in the first experiments. This is probably due to the reduction in memory which meant some relatively common properties are now in data not in Node.

@typescript-bot typescript-bot added the For Uncommitted Bug PR for untriaged, rejected, closed or missing bug label Jun 19, 2024
@jakebailey
Copy link
Member

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 19, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 194,582k (± 0.91%) 217,957k (± 0.03%) 🔻+23,375k (+12.01%) 217,873k 218,049k p=0.005 n=6
Parse Time 1.57s (± 0.33%) 1.67s (± 0.49%) 🔻+0.09s (+ 5.93%) 1.66s 1.68s p=0.004 n=6
Bind Time 0.86s (± 1.55%) 0.81s (± 1.04%) 🟩-0.05s (- 6.21%) 0.80s 0.82s p=0.004 n=6
Check Time 11.20s (± 0.36%) 8.67s (± 0.93%) 🟩-2.53s (-22.60%) 8.57s 8.78s p=0.005 n=6
Emit Time 3.26s (± 0.69%) 2.87s (± 0.51%) 🟩-0.39s (-12.10%) 2.85s 2.89s p=0.005 n=6
Total Time 16.89s (± 0.29%) 14.01s (± 0.59%) 🟩-2.88s (-17.07%) 13.92s 14.14s p=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,051 407,051 ~ ~ ~ p=1.000 n=6
Memory used 1,218,316k (± 0.00%) 1,387,661k (± 0.00%) 🔻+169,345k (+13.90%) 1,387,617k 1,387,712k p=0.005 n=6
Parse Time 7.96s (± 0.46%) 8.42s (± 0.48%) 🔻+0.46s (+ 5.76%) 8.37s 8.49s p=0.005 n=6
Bind Time 2.23s (± 0.63%) 1.91s (± 0.39%) 🟩-0.32s (-14.28%) 1.90s 1.92s p=0.005 n=6
Check Time 35.82s (± 0.40%) 27.74s (± 0.22%) 🟩-8.07s (-22.54%) 27.63s 27.81s p=0.005 n=6
Emit Time 16.18s (± 0.53%) 14.05s (± 0.29%) 🟩-2.12s (-13.12%) 14.01s 14.12s p=0.005 n=6
Total Time 62.18s (± 0.26%) 52.13s (± 0.17%) 🟩-10.05s (-16.17%) 51.97s 52.21s p=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,133,539 2,133,539 ~ ~ ~ p=1.000 n=6
Types 926,551 926,551 ~ ~ ~ p=1.000 n=6
Memory used 2,115,789k (± 0.01%) 2,268,208k (± 0.00%) 🔻+152,419k (+ 7.20%) 2,268,182k 2,268,228k p=0.005 n=6
Parse Time 7.88s (± 0.23%) 9.27s (± 0.28%) 🔻+1.39s (+17.64%) 9.24s 9.31s p=0.005 n=6
Bind Time 2.74s (± 0.38%) 2.01s (± 0.87%) 🟩-0.74s (-26.85%) 1.99s 2.04s p=0.004 n=6
Check Time 84.05s (± 0.27%) 61.65s (± 0.24%) 🟩-22.40s (-26.65%) 61.46s 61.80s p=0.005 n=6
Emit Time 0.16s (± 3.95%) 0.17s (± 3.32%) ~ 0.16s 0.17s p=0.201 n=6
Total Time 94.84s (± 0.25%) 73.09s (± 0.20%) 🟩-21.74s (-22.93%) 72.91s 73.21s p=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,761 1,240,272 +9,511 (+ 0.77%) ~ ~ p=0.001 n=6
Types 261,189 261,013 -176 (- 0.07%) ~ ~ p=0.001 n=6
Memory used 2,356,206k (± 1.03%) 2,678,211k (± 0.01%) 🔻+322,005k (+13.67%) 2,677,639k 2,678,592k p=0.005 n=6
Parse Time 5.97s (± 0.40%) 6.43s (± 0.99%) 🔻+0.46s (+ 7.65%) 6.37s 6.52s p=0.005 n=6
Bind Time 2.26s (± 1.02%) 2.03s (± 0.96%) 🟩-0.23s (-10.32%) 2.01s 2.06s p=0.005 n=6
Check Time 39.74s (± 0.61%) 32.39s (± 0.18%) 🟩-7.36s (-18.51%) 32.33s 32.45s p=0.005 n=6
Emit Time 3.15s (± 4.19%) 3.08s (± 0.87%) ~ 3.03s 3.11s p=0.470 n=6
Total Time 51.12s (± 0.68%) 43.93s (± 0.21%) 🟩-7.19s (-14.06%) 43.83s 44.04s p=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,761 1,240,272 +9,511 (+ 0.77%) ~ ~ p=0.001 n=6
Types 261,189 261,013 -176 (- 0.07%) ~ ~ p=0.001 n=6
Memory used 2,432,527k (± 0.98%) 2,764,869k (± 0.02%) 🔻+332,342k (+13.66%) 2,764,038k 2,765,515k p=0.005 n=6
Parse Time 6.20s (± 0.57%) 6.54s (± 0.83%) 🔻+0.34s (+ 5.43%) 6.45s 6.60s p=0.005 n=6
Bind Time 2.03s (± 1.22%) 2.03s (± 0.51%) ~ 2.01s 2.04s p=1.000 n=6
Check Time 40.03s (± 0.32%) 32.37s (± 0.20%) 🟩-7.66s (-19.13%) 32.29s 32.45s p=0.005 n=6
Emit Time 3.22s (± 3.43%) 3.08s (± 2.76%) 🟩-0.14s (- 4.29%) 3.00s 3.21s p=0.031 n=6
Total Time 51.50s (± 0.30%) 44.04s (± 0.29%) 🟩-7.46s (-14.49%) 43.86s 44.18s p=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,573 259,522 +949 (+ 0.37%) ~ ~ p=0.001 n=6
Types 104,819 104,829 +10 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 428,188k (± 0.00%) 495,625k (± 0.02%) 🔻+67,437k (+15.75%) 495,502k 495,768k p=0.005 n=6
Parse Time 4.11s (± 0.70%) 4.27s (± 0.67%) 🔻+0.17s (+ 4.02%) 4.24s 4.32s p=0.005 n=6
Bind Time 1.63s (± 0.32%) 1.48s (± 0.35%) 🟩-0.14s (- 8.81%) 1.48s 1.49s p=0.004 n=6
Check Time 21.92s (± 0.29%) 17.01s (± 0.45%) 🟩-4.91s (-22.41%) 16.87s 17.07s p=0.005 n=6
Emit Time 1.72s (± 1.63%) 1.68s (± 1.23%) -0.04s (- 2.22%) 1.65s 1.71s p=0.029 n=6
Total Time 29.38s (± 0.21%) 24.44s (± 0.40%) 🟩-4.94s (-16.80%) 24.27s 24.51s p=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,444k (± 0.02%) 425,551k (± 0.01%) 🔻+56,107k (+15.19%) 425,509k 425,599k p=0.005 n=6
Parse Time 2.76s (± 1.11%) 3.12s (± 0.98%) 🔻+0.36s (+12.96%) 3.09s 3.17s p=0.005 n=6
Bind Time 1.58s (± 0.87%) 1.29s (± 0.40%) 🟩-0.30s (-18.82%) 1.28s 1.29s p=0.004 n=6
Check Time 15.45s (± 0.44%) 12.30s (± 0.34%) 🟩-3.15s (-20.40%) 12.25s 12.37s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.81s (± 0.39%) 16.71s (± 0.36%) 🟩-3.10s (-15.64%) 16.65s 16.82s p=0.005 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=NaN n=0
Symbols 0 0 ~ ~ ~ p=NaN n=0
Types 0 0 ~ ~ ~ p=NaN n=0
Memory used 0k 0k ~ ~ ~ p=NaN n=0
Parse Time 0s 0s ~ ~ ~ p=NaN n=0
Bind Time 0s 0s ~ ~ ~ p=NaN n=0
Check Time 0s 0s ~ ~ ~ p=NaN n=0
Emit Time 0s 0s ~ ~ ~ p=NaN n=0
Total Time 0s 0s ~ ~ ~ p=NaN n=0
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 266,926 266,926 ~ ~ ~ p=1.000 n=6
Types 108,691 108,691 ~ ~ ~ p=1.000 n=6
Memory used 411,270k (± 0.01%) 464,212k (± 0.01%) 🔻+52,942k (+12.87%) 464,168k 464,328k p=0.005 n=6
Parse Time 3.78s (± 0.22%) 3.82s (± 0.67%) +0.04s (+ 1.10%) 3.78s 3.86s p=0.010 n=6
Bind Time 1.68s (± 0.31%) 1.53s (± 0.55%) 🟩-0.15s (- 9.05%) 1.51s 1.53s p=0.004 n=6
Check Time 16.77s (± 0.29%) 14.04s (± 0.20%) 🟩-2.73s (-16.28%) 14.00s 14.07s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.22s (± 0.20%) 19.38s (± 0.24%) 🟩-2.84s (-12.78%) 19.33s 19.46s p=0.005 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 523,765 523,765 ~ ~ ~ p=1.000 n=6
Types 178,055 178,055 ~ ~ ~ p=1.000 n=6
Memory used 461,853k (± 0.09%) 509,764k (± 0.07%) 🔻+47,911k (+10.37%) 509,413k 510,186k p=0.005 n=6
Parse Time 2.64s (± 0.57%) 2.90s (± 0.84%) 🔻+0.26s (+10.06%) 2.87s 2.94s p=0.005 n=6
Bind Time 0.98s (± 0.56%) 0.85s (± 0.48%) 🟩-0.13s (-13.54%) 0.85s 0.86s p=0.003 n=6
Check Time 15.18s (± 0.41%) 13.04s (± 0.29%) 🟩-2.14s (-14.07%) 12.99s 13.10s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.80s (± 0.35%) 16.79s (± 0.30%) 🟩-2.01s (-10.68%) 16.73s 16.86s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,279ms (± 0.67%) 2,330ms (± 0.82%) +50ms (+ 2.21%) 2,293ms 2,349ms p=0.008 n=6
Req 2 - geterr 5,069ms (± 0.77%) 4,282ms (± 1.34%) 🟩-787ms (-15.52%) 4,166ms 4,314ms p=0.005 n=6
Req 3 - references 265ms (± 0.37%) 267ms (± 0.70%) ~ 265ms 270ms p=0.082 n=6
Req 4 - navto 227ms (± 1.08%) 227ms (± 0.77%) ~ 224ms 229ms p=0.493 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 90ms (± 3.06%) 84ms (± 7.70%) ~ 75ms 92ms p=0.165 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,394ms (± 0.46%) 2,505ms (± 1.16%) 🔻+111ms (+ 4.65%) 2,462ms 2,546ms p=0.005 n=6
Req 2 - geterr 3,791ms (± 0.28%) 3,283ms (± 3.01%) 🟩-509ms (-13.42%) 3,189ms 3,385ms p=0.005 n=6
Req 3 - references 275ms (± 0.15%) 303ms (±10.57%) 🔻+28ms (+ 9.99%) 281ms 345ms p=0.004 n=6
Req 4 - navto 227ms (± 0.23%) 230ms (± 2.71%) ~ 226ms 242ms p=0.242 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 69ms (± 0.75%) 73ms (±10.64%) ~ 65ms 82ms p=0.367 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,189ms (± 0.44%) 5,435ms (± 0.33%) 🔻+246ms (+ 4.73%) 5,412ms 5,462ms p=0.005 n=6
Req 2 - geterr 1,123ms (± 0.98%) 985ms (± 0.63%) 🟩-138ms (-12.33%) 978ms 994ms p=0.005 n=6
Req 3 - references 76ms (± 0.99%) 88ms (± 4.72%) 🔻+12ms (+15.38%) 83ms 95ms p=0.004 n=6
Req 4 - navto 451ms (± 1.02%) 331ms (± 0.76%) 🟩-120ms (-26.63%) 328ms 335ms p=0.005 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 837ms (± 1.47%) 816ms (± 2.39%) ~ 801ms 854ms p=0.065 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.49ms (± 0.16%) 159.01ms (± 0.19%) +1.52ms (+ 0.97%) 157.99ms 164.16ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 241.31ms (± 0.13%) 234.00ms (± 0.15%) 🟩-7.31ms (- 3.03%) 232.43ms 240.65ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 236.15ms (± 0.15%) 228.82ms (± 0.16%) 🟩-7.33ms (- 3.11%) 227.29ms 235.52ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 236.21ms (± 0.14%) 228.64ms (± 0.16%) 🟩-7.57ms (- 3.21%) 227.36ms 233.70ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey jakebailey mentioned this pull request Jun 19, 2024
Merged
@Andarist
Copy link
Contributor

This is the most green thing I've seen this week. Nice!!

@DanielRosenwasser DanielRosenwasser changed the title Experiment - monomorphic node/type/sigature Experiment - monomorphic node/type/signature Jun 19, 2024
@MichaelMitchell-at
Copy link

Unfortunately I didn't see a noticeable perf improvement in the Airtable code base, but did see a 5-10% increase in peak memory usage.

@dragomirtitian
Copy link
Contributor Author

Unfortunately I didn't see a noticeable perf improvement in the Airtable code base, but did see a 5-10% increase in peak memory usage.

@MichaelMitchell-at i find this a bit surprising. While the % of difference varies, we've generally noticed improvements across the board. Is this a public repo? If it not, could you run the tsc on this branch through dexnode to get a report for the inline caches?

Also how did you measure the times? What is the output with - - diagnostics?

@MichaelMitchell-at
Copy link

Unfortunately I didn't see a noticeable perf improvement in the Airtable code base, but did see a 5-10% increase in peak memory usage.

@MichaelMitchell-at i find this a bit surprising. While the % of difference varies, we've generally noticed improvements across the board. Is this a public repo? If it not, could you run the tsc on this branch through dexnode to get a report for the inline caches?

Also how did you measure the times? What is the output with - - diagnostics?

Actually there seems to be an error with my methodology so throw out the previous result I mentioned. I can't seem to apply the patch cleanly at the moment. I'll just have to wait for this to reach nightly.

@MichaelMitchell-at
Copy link

MichaelMitchell-at commented Jun 20, 2024 •
edited
Loading

Ok, I looked into it a bit more. Our custom runner uses the compiler API and that code path hasn't been updated to support the new node structure yet.

> const ts = require('./lib/typescript.js')
> ts.readConfigFile('src/tsconfig.json', ts.sys.readFile)
Uncaught Error: Debug Failure. False expression.
    at parseUpdateExpression (TypeScript/lib/typescript.js:37704:11)
    at parseUnaryExpressionOrHigher (/TypeScript/lib/typescript.js:37622:32)
    at parseBinaryExpressionOrHigher (TypeScript/lib/typescript.js:37544:25)
    at parseAssignmentExpressionOrHigher (TypeScript/lib/typescript.js:37215:18)

I should be able to get past this if I can swap out this part with a lib that can parse JSON with comments and trailing commas.

Edit: ts.transpileDeclaration has the same problem, so I just tried comparing build times without leveraging isolated declarations. Performance still seems about the same or even worse overall, particularly on smaller projects, < 10s to build. I'm running these in our CI pipeline for more consistent results. Anyway, I think I should just wait for this PR to be ready to preview and can try it out again then.

@jakebailey
Copy link
Member

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 22, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,880k (± 0.74%) 218,478k (± 0.63%) 🔻+25,599k (+13.27%) 217,750k 221,281k p=0.005 n=6
Parse Time 1.57s (± 0.52%) 1.63s (± 0.92%) +0.05s (+ 3.39%) 1.60s 1.64s p=0.005 n=6
Bind Time 0.85s (± 0.98%) 0.81s (± 1.00%) 🟩-0.04s (- 4.87%) 0.80s 0.82s p=0.004 n=6
Check Time 11.20s (± 0.56%) 8.59s (± 1.04%) 🟩-2.61s (-23.30%) 8.47s 8.71s p=0.005 n=6
Emit Time 3.30s (± 1.09%) 2.81s (± 0.27%) 🟩-0.49s (-14.75%) 2.80s 2.82s p=0.005 n=6
Total Time 16.93s (± 0.36%) 13.84s (± 0.66%) 🟩-3.09s (-18.26%) 13.73s 13.96s p=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,051 407,051 ~ ~ ~ p=1.000 n=6
Memory used 1,218,304k (± 0.00%) 1,387,391k (± 0.00%) 🔻+169,088k (+13.88%) 1,387,359k 1,387,431k p=0.005 n=6
Parse Time 7.98s (± 0.55%) 8.05s (± 0.50%) +0.08s (+ 0.98%) 7.99s 8.10s p=0.024 n=6
Bind Time 2.22s (± 0.53%) 1.91s (± 0.51%) 🟩-0.31s (-13.82%) 1.90s 1.92s p=0.005 n=6
Check Time 35.68s (± 0.55%) 27.28s (± 0.32%) 🟩-8.40s (-23.55%) 27.16s 27.40s p=0.005 n=6
Emit Time 16.02s (± 0.80%) 13.85s (± 0.49%) 🟩-2.17s (-13.52%) 13.79s 13.98s p=0.005 n=6
Total Time 61.89s (± 0.28%) 51.10s (± 0.16%) 🟩-10.80s (-17.44%) 51.02s 51.20s p=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,134,511 2,134,511 ~ ~ ~ p=1.000 n=6
Types 927,068 927,068 ~ ~ ~ p=1.000 n=6
Memory used 2,115,981k (± 0.01%) 2,268,286k (± 0.00%) 🔻+152,305k (+ 7.20%) 2,268,243k 2,268,330k p=0.005 n=6
Parse Time 7.85s (± 0.47%) 8.97s (± 0.57%) 🔻+1.12s (+14.26%) 8.92s 9.05s p=0.005 n=6
Bind Time 2.76s (± 0.81%) 2.04s (± 0.78%) 🟩-0.72s (-25.98%) 2.03s 2.07s p=0.005 n=6
Check Time 83.64s (± 0.46%) 60.66s (± 0.47%) 🟩-22.98s (-27.48%) 60.36s 61.18s p=0.005 n=6
Emit Time 0.16s (± 4.75%) 0.16s (± 3.16%) ~ 0.16s 0.17s p=0.241 n=6
Total Time 94.41s (± 0.41%) 71.83s (± 0.36%) 🟩-22.58s (-23.91%) 71.60s 72.31s p=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,804 1,246,592 +15,788 (+ 1.28%) ~ ~ p=0.001 n=6
Types 261,202 263,541 +2,339 (+ 0.90%) ~ ~ p=0.001 n=6
Memory used 2,345,838k (± 0.05%) 2,680,576k (± 0.01%) 🔻+334,738k (+14.27%) 2,680,221k 2,681,092k p=0.005 n=6
Parse Time 6.01s (± 0.40%) 6.19s (± 0.94%) +0.17s (+ 2.88%) 6.10s 6.27s p=0.005 n=6
Bind Time 2.26s (± 0.90%) 2.06s (± 1.14%) 🟩-0.20s (- 9.06%) 2.02s 2.09s p=0.005 n=6
Check Time 39.77s (± 0.39%) 31.89s (± 0.43%) 🟩-7.89s (-19.83%) 31.68s 32.05s p=0.005 n=6
Emit Time 3.24s (± 5.62%) 3.07s (± 1.40%) 🟩-0.17s (- 5.24%) 3.03s 3.14s p=0.037 n=6
Total Time 51.30s (± 0.24%) 43.20s (± 0.48%) 🟩-8.09s (-15.78%) 42.96s 43.54s p=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,804 1,246,592 +15,788 (+ 1.28%) ~ ~ p=0.001 n=6
Types 261,202 263,541 +2,339 (+ 0.90%) ~ ~ p=0.001 n=6
Memory used 2,422,778k (± 0.01%) 2,777,844k (± 0.97%) 🔻+355,066k (+14.66%) 2,765,961k 2,832,701k p=0.005 n=6
Parse Time 6.18s (± 0.77%) 6.35s (± 0.89%) +0.17s (+ 2.81%) 6.29s 6.44s p=0.005 n=6
Bind Time 2.02s (± 0.40%) 2.04s (± 0.40%) +0.03s (+ 1.32%) 2.03s 2.05s p=0.006 n=6
Check Time 40.12s (± 0.14%) 31.84s (± 0.39%) 🟩-8.29s (-20.65%) 31.60s 31.92s p=0.005 n=6
Emit Time 3.19s (± 1.81%) 3.12s (± 2.29%) ~ 3.06s 3.24s p=0.108 n=6
Total Time 51.51s (± 0.26%) 43.36s (± 0.32%) 🟩-8.15s (-15.83%) 43.11s 43.48s p=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,574 265,812 +7,238 (+ 2.80%) ~ ~ p=0.001 n=6
Types 104,825 107,348 +2,523 (+ 2.41%) ~ ~ p=0.001 n=6
Memory used 428,178k (± 0.01%) 499,567k (± 0.01%) 🔻+71,389k (+16.67%) 499,538k 499,615k p=0.005 n=6
Parse Time 4.08s (± 0.34%) 4.17s (± 1.06%) +0.09s (+ 2.12%) 4.11s 4.24s p=0.005 n=6
Bind Time 1.60s (± 0.83%) 1.49s (± 0.69%) 🟩-0.10s (- 6.57%) 1.48s 1.51s p=0.004 n=6
Check Time 21.94s (± 0.38%) 16.74s (± 0.48%) 🟩-5.19s (-23.67%) 16.64s 16.84s p=0.005 n=6
Emit Time 1.69s (± 1.45%) 1.67s (± 1.58%) ~ 1.65s 1.72s p=0.062 n=6
Total Time 29.31s (± 0.29%) 24.08s (± 0.51%) 🟩-5.23s (-17.85%) 23.92s 24.24s p=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,562k (± 0.05%) 425,386k (± 0.02%) 🔻+55,825k (+15.11%) 425,266k 425,443k p=0.005 n=6
Parse Time 2.77s (± 1.29%) 3.00s (± 1.32%) 🔻+0.23s (+ 8.24%) 2.95s 3.05s p=0.005 n=6
Bind Time 1.58s (± 0.65%) 1.28s (± 0.81%) 🟩-0.31s (-19.37%) 1.26s 1.29s p=0.004 n=6
Check Time 15.47s (± 0.23%) 11.99s (± 0.35%) 🟩-3.48s (-22.50%) 11.91s 12.03s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.83s (± 0.25%) 16.27s (± 0.31%) 🟩-3.56s (-17.96%) 16.20s 16.33s p=0.005 n=6
vscode - node (v18.15.0, x64)
Errors 139 139 ~ ~ ~ p=1.000 n=6
Symbols 2,854,562 2,854,562 ~ ~ ~ p=1.000 n=6
Types 976,042 976,042 ~ ~ ~ p=1.000 n=6
Memory used 3,045,619k (± 0.00%) 3,455,715k (± 0.01%) 🔻+410,097k (+13.47%) 3,455,517k 3,456,128k p=0.005 n=6
Parse Time 16.68s (± 0.11%) 17.33s (± 0.38%) +0.66s (+ 3.94%) 17.23s 17.42s p=0.005 n=6
Bind Time 5.18s (± 0.31%) 4.31s (± 0.35%) 🟩-0.87s (-16.87%) 4.28s 4.32s p=0.005 n=6
Check Time 89.80s (± 0.21%) 74.27s (± 0.40%) 🟩-15.53s (-17.30%) 73.77s 74.53s p=0.005 n=6
Emit Time 29.26s (± 0.29%) 27.54s (± 1.38%) 🟩-1.72s (- 5.88%) 26.77s 27.76s p=0.005 n=6
Total Time 140.91s (± 0.13%) 123.44s (± 0.31%) 🟩-17.47s (-12.40%) 122.94s 123.86s p=0.005 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,507k (± 0.01%) 464,374k (± 0.01%) 🔻+52,867k (+12.85%) 464,315k 464,477k p=0.005 n=6
Parse Time 3.19s (± 0.38%) 3.13s (± 0.27%) -0.06s (- 1.94%) 3.11s 3.13s p=0.004 n=6
Bind Time 1.41s (± 0.29%) 1.29s (± 0.63%) 🟩-0.12s (- 8.38%) 1.28s 1.30s p=0.003 n=6
Check Time 14.22s (± 0.14%) 11.93s (± 0.53%) 🟩-2.29s (-16.07%) 11.87s 12.03s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.82s (± 0.12%) 16.35s (± 0.32%) 🟩-2.46s (-13.10%) 16.29s 16.42s p=0.005 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,883k (± 0.06%) 510,721k (± 0.06%) 🔻+47,839k (+10.33%) 510,404k 511,148k p=0.005 n=6
Parse Time 3.17s (± 0.53%) 3.35s (± 1.01%) 🔻+0.19s (+ 5.84%) 3.31s 3.40s p=0.005 n=6
Bind Time 1.17s 1.07s (± 0.76%) 🟩-0.10s (- 8.26%) 1.06s 1.08s p=0.003 n=6
Check Time 17.93s (± 0.39%) 14.91s (± 0.29%) 🟩-3.02s (-16.84%) 14.85s 14.98s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.27s (± 0.34%) 19.34s (± 0.30%) 🟩-2.93s (-13.14%) 19.24s 19.40s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,295ms (± 0.74%) 2,267ms (± 0.34%) -28ms (- 1.20%) 2,256ms 2,278ms p=0.031 n=6
Req 2 - geterr 5,081ms (± 0.40%) 4,202ms (± 2.63%) 🟩-879ms (-17.30%) 4,036ms 4,289ms p=0.005 n=6
Req 3 - references 264ms (± 0.32%) 266ms (± 0.67%) +3ms (+ 0.95%) 264ms 269ms p=0.010 n=6
Req 4 - navto 229ms (± 0.53%) 229ms (± 0.92%) ~ 227ms 232ms p=1.000 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 85ms (± 6.42%) 84ms (± 7.95%) ~ 73ms 90ms p=0.567 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,423ms (± 0.58%) 2,435ms (± 0.11%) ~ 2,432ms 2,439ms p=0.261 n=6
Req 2 - geterr 3,792ms (± 0.35%) 3,235ms (± 2.73%) 🟩-557ms (-14.69%) 3,149ms 3,324ms p=0.005 n=6
Req 3 - references 278ms (± 0.30%) 311ms (±11.03%) 🔻+34ms (+12.19%) 278ms 344ms p=0.009 n=6
Req 4 - navto 226ms (± 0.48%) 232ms (± 2.08%) +6ms (+ 2.73%) 227ms 240ms p=0.008 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 73ms (± 8.47%) 71ms (±10.26%) ~ 65ms 82ms p=0.371 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,182ms (± 0.44%) 5,278ms (± 0.65%) +95ms (+ 1.84%) 5,223ms 5,326ms p=0.005 n=6
Req 2 - geterr 1,126ms (± 0.94%) 961ms (± 0.57%) 🟩-165ms (-14.63%) 952ms 966ms p=0.005 n=6
Req 3 - references 77ms (± 3.85%) 89ms (± 4.52%) 🔻+12ms (+15.37%) 86ms 94ms p=0.004 n=6
Req 4 - navto 458ms (± 0.97%) 331ms (± 0.59%) 🟩-127ms (-27.72%) 329ms 334ms p=0.005 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 841ms (± 0.65%) 813ms (± 1.56%) 🟩-28ms (- 3.37%) 797ms 827ms p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.10ms (± 0.17%) 158.58ms (± 0.18%) +1.47ms (+ 0.94%) 157.45ms 162.49ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 232.57ms (± 0.14%) 234.34ms (± 0.15%) +1.77ms (+ 0.76%) 232.98ms 240.10ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 227.61ms (± 0.17%) 229.21ms (± 0.13%) +1.60ms (+ 0.71%) 227.88ms 231.89ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 225.93ms (± 0.17%) 227.60ms (± 0.15%) +1.67ms (+ 0.74%) 226.24ms 234.18ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@dragomirtitian
Copy link
Contributor Author

Latest perf results seemed to have clawed back most of the parser perf loss.

@jakebailey
Copy link
Member

Half a gig of memory for vscode is a bit scary, though, given how close that is to the max memory limit...

@dragomirtitian
Copy link
Contributor Author

dragomirtitian commented Jun 24, 2024 •
edited
Loading

@jakebailey I did some extra work. I was too eager to make everything monomorphic, a bit of polymorphism is fine. There should be 3 shapes for nodes now: Token, Identifier and Node. There is a 4th one Token with original that I might look into removing, but it doesn't seem to have much impact for now.

With these changes, at least locally the increase in mem usage goes form 13% to about 5%, while keeping the same performance profile.

@jakebailey
Copy link
Member

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 24, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,924k (± 1.03%) 204,029k (± 0.91%) 🔻+10,104k (+ 5.21%) 202,782k 206,426k p=0.005 n=6
Parse Time 1.58s (± 1.03%) 1.62s (± 0.64%) +0.03s (+ 2.11%) 1.60s 1.63s p=0.007 n=6
Bind Time 0.86s (± 1.20%) 0.86s (± 0.60%) ~ 0.86s 0.87s p=0.793 n=6
Check Time 11.22s (± 0.24%) 8.94s (± 1.08%) 🟩-2.28s (-20.33%) 8.87s 9.11s p=0.005 n=6
Emit Time 3.27s (± 0.63%) 3.16s (± 0.97%) 🟩-0.11s (- 3.41%) 3.13s 3.21s p=0.005 n=6
Total Time 16.93s (± 0.27%) 14.58s (± 0.61%) 🟩-2.35s (-13.91%) 14.48s 14.74s p=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,051 407,051 ~ ~ ~ p=1.000 n=6
Memory used 1,218,281k (± 0.00%) 1,295,371k (± 0.00%) 🔻+77,089k (+ 6.33%) 1,295,310k 1,295,431k p=0.005 n=6
Parse Time 7.94s (± 0.30%) 8.31s (± 0.97%) 🔻+0.37s (+ 4.64%) 8.22s 8.44s p=0.005 n=6
Bind Time 2.23s (± 0.34%) 2.07s (± 0.59%) 🟩-0.15s (- 6.88%) 2.05s 2.08s p=0.003 n=6
Check Time 35.77s (± 0.27%) 29.04s (± 0.36%) 🟩-6.72s (-18.79%) 28.86s 29.13s p=0.005 n=6
Emit Time 16.11s (± 0.56%) 14.93s (± 0.49%) 🟩-1.18s (- 7.31%) 14.84s 15.03s p=0.005 n=6
Total Time 62.04s (± 0.21%) 54.37s (± 0.37%) 🟩-7.68s (-12.37%) 54.01s 54.58s p=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,135,122 2,135,122 ~ ~ ~ p=1.000 n=6
Types 927,182 927,182 ~ ~ ~ p=1.000 n=6
Memory used 2,117,373k (± 0.01%) 2,199,492k (± 0.00%) +82,119k (+ 3.88%) 2,199,411k 2,199,584k p=0.005 n=6
Parse Time 7.92s (± 0.35%) 8.29s (± 2.32%) 🔻+0.37s (+ 4.67%) 8.19s 8.68s p=0.005 n=6
Bind Time 2.77s (± 0.70%) 2.71s (± 8.48%) ~ 2.24s 2.82s p=0.332 n=6
Check Time 83.86s (± 0.39%) 62.40s (± 0.60%) 🟩-21.46s (-25.59%) 62.04s 63.08s p=0.005 n=6
Emit Time 0.15s (± 3.53%) 0.16s (± 3.16%) 🔻+0.01s (+ 5.38%) 0.16s 0.17s p=0.038 n=6
Total Time 94.71s (± 0.31%) 73.56s (± 0.50%) 🟩-21.15s (-22.33%) 73.23s 74.24s p=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,804 1,246,046 +15,242 (+ 1.24%) ~ ~ p=0.001 n=6
Types 261,202 263,618 +2,416 (+ 0.92%) ~ ~ p=0.001 n=6
Memory used 2,356,275k (± 1.03%) 2,474,466k (± 0.03%) 🔻+118,192k (+ 5.02%) 2,473,593k 2,475,488k p=0.005 n=6
Parse Time 5.95s (± 0.85%) 6.47s (± 1.03%) 🔻+0.52s (+ 8.77%) 6.37s 6.55s p=0.005 n=6
Bind Time 2.25s (± 0.83%) 2.25s (± 0.76%) ~ 2.23s 2.27s p=0.936 n=6
Check Time 39.81s (± 0.32%) 33.41s (± 0.36%) 🟩-6.39s (-16.06%) 33.30s 33.57s p=0.005 n=6
Emit Time 3.13s (± 1.38%) 3.58s (± 1.59%) 🔻+0.45s (+14.35%) 3.52s 3.66s p=0.005 n=6
Total Time 51.15s (± 0.36%) 45.73s (± 0.30%) 🟩-5.42s (-10.59%) 45.54s 45.90s p=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,804 1,246,046 +15,242 (+ 1.24%) ~ ~ p=0.001 n=6
Types 261,202 263,618 +2,416 (+ 0.92%) ~ ~ p=0.001 n=6
Memory used 2,422,330k (± 0.05%) 2,577,288k (± 0.03%) 🔻+154,958k (+ 6.40%) 2,576,002k 2,578,462k p=0.005 n=6
Parse Time 7.77s (± 0.89%) 8.17s (± 0.65%) 🔻+0.41s (+ 5.24%) 8.08s 8.24s p=0.005 n=6
Bind Time 2.52s (± 1.06%) 2.82s (± 0.76%) 🔻+0.31s (+12.12%) 2.80s 2.86s p=0.005 n=6
Check Time 49.53s (± 0.39%) 41.40s (± 0.28%) 🟩-8.14s (-16.43%) 41.19s 41.52s p=0.005 n=6
Emit Time 3.92s (± 3.67%) 4.52s (± 3.31%) 🔻+0.60s (+15.21%) 4.34s 4.72s p=0.005 n=6
Total Time 63.76s (± 0.48%) 56.92s (± 0.39%) 🟩-6.84s (-10.72%) 56.68s 57.25s p=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,574 265,896 +7,322 (+ 2.83%) ~ ~ p=0.001 n=6
Types 104,825 107,377 +2,552 (+ 2.43%) ~ ~ p=0.001 n=6
Memory used 428,212k (± 0.02%) 459,220k (± 0.01%) 🔻+31,008k (+ 7.24%) 459,199k 459,271k p=0.005 n=6
Parse Time 3.31s (± 0.60%) 3.46s (± 0.75%) 🔻+0.15s (+ 4.53%) 3.43s 3.50s p=0.005 n=6
Bind Time 1.31s (± 1.67%) 1.30s (± 0.93%) ~ 1.28s 1.31s p=0.324 n=6
Check Time 17.74s (± 0.27%) 14.40s (± 0.55%) 🟩-3.34s (-18.84%) 14.31s 14.52s p=0.005 n=6
Emit Time 1.36s (± 2.21%) 1.65s (± 2.02%) 🔻+0.28s (+20.63%) 1.58s 1.67s p=0.004 n=6
Total Time 23.73s (± 0.33%) 20.80s (± 0.34%) 🟩-2.93s (-12.35%) 20.73s 20.91s p=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,410k (± 0.01%) 390,898k (± 0.01%) 🔻+21,488k (+ 5.82%) 390,854k 390,974k p=0.005 n=6
Parse Time 2.77s (± 1.06%) 3.05s (± 0.88%) 🔻+0.28s (+10.17%) 3.03s 3.10s p=0.005 n=6
Bind Time 1.58s (± 0.98%) 1.38s (± 1.74%) 🟩-0.20s (-12.55%) 1.37s 1.43s p=0.004 n=6
Check Time 15.41s (± 0.23%) 12.68s (± 0.15%) 🟩-2.73s (-17.69%) 12.66s 12.72s p=0.004 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.76s (± 0.31%) 17.12s (± 0.19%) 🟩-2.64s (-13.37%) 17.09s 17.16s p=0.005 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,880,886 2,880,886 ~ ~ ~ p=1.000 n=6
Types 975,501 975,501 ~ ~ ~ p=1.000 n=6
Memory used 3,042,328k (± 0.00%) 3,227,789k (± 0.00%) 🔻+185,461k (+ 6.10%) 3,227,714k 3,227,862k p=0.005 n=6
Parse Time 16.73s (± 0.68%) 17.61s (± 0.44%) 🔻+0.89s (+ 5.29%) 17.50s 17.69s p=0.005 n=6
Bind Time 5.10s (± 0.26%) 4.61s (± 1.15%) 🟩-0.49s (- 9.64%) 4.52s 4.67s p=0.005 n=6
Check Time 88.86s (± 0.39%) 78.07s (± 0.23%) 🟩-10.79s (-12.14%) 77.85s 78.35s p=0.005 n=6
Emit Time 29.21s (± 0.50%) 23.23s (± 0.37%) 🟩-5.98s (-20.47%) 23.16s 23.38s p=0.005 n=6
Total Time 139.90s (± 0.35%) 123.52s (± 0.12%) 🟩-16.38s (-11.71%) 123.30s 123.73s p=0.005 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,548k (± 0.01%) 434,675k (± 0.01%) 🔻+23,126k (+ 5.62%) 434,608k 434,733k p=0.005 n=6
Parse Time 3.18s (± 0.65%) 3.27s (± 0.23%) +0.10s (+ 2.99%) 3.26s 3.28s p=0.005 n=6
Bind Time 1.41s (± 0.53%) 1.44s (± 0.52%) +0.03s (+ 1.89%) 1.43s 1.45s p=0.004 n=6
Check Time 14.19s (± 0.34%) 12.72s (± 0.33%) 🟩-1.47s (-10.34%) 12.66s 12.79s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.77s (± 0.26%) 17.43s (± 0.22%) 🟩-1.34s (- 7.16%) 17.38s 17.49s p=0.005 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,712k (± 0.09%) 484,265k (± 0.06%) 🔻+21,553k (+ 4.66%) 484,094k 484,872k p=0.005 n=6
Parse Time 3.17s (± 0.61%) 3.26s (± 0.71%) +0.09s (+ 2.84%) 3.23s 3.29s p=0.005 n=6
Bind Time 1.17s (± 1.16%) 1.15s (± 0.45%) -0.02s (- 1.70%) 1.15s 1.16s p=0.006 n=6
Check Time 17.86s (± 0.50%) 15.47s (± 0.23%) 🟩-2.38s (-13.35%) 15.44s 15.53s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.20s (± 0.38%) 19.88s (± 0.23%) 🟩-2.31s (-10.41%) 19.82s 19.94s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,290ms (± 0.81%) 2,375ms (± 0.75%) +85ms (+ 3.71%) 2,353ms 2,397ms p=0.005 n=6
Req 2 - geterr 5,068ms (± 0.29%) 4,364ms (± 0.26%) 🟩-704ms (-13.89%) 4,351ms 4,379ms p=0.005 n=6
Req 3 - references 264ms (± 0.71%) 273ms (± 0.43%) +9ms (+ 3.54%) 271ms 274ms p=0.005 n=6
Req 4 - navto 228ms (± 1.30%) 238ms (± 0.65%) 🔻+11ms (+ 4.62%) 237ms 241ms p=0.004 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 87ms (± 5.20%) 88ms (± 8.94%) ~ 78ms 94ms p=0.459 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,418ms (± 0.70%) 2,517ms (± 0.72%) 🔻+99ms (+ 4.10%) 2,495ms 2,549ms p=0.005 n=6
Req 2 - geterr 3,801ms (± 0.34%) 3,355ms (± 0.12%) 🟩-446ms (-11.73%) 3,349ms 3,361ms p=0.005 n=6
Req 3 - references 278ms (± 0.38%) 290ms (± 0.29%) 🔻+12ms (+ 4.32%) 289ms 291ms p=0.004 n=6
Req 4 - navto 225ms 235ms (± 0.23%) 🔻+10ms (+ 4.22%) 234ms 235ms p=0.002 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 73ms (± 2.05%) 71ms 🟩-2ms (- 3.18%) ~ ~ p=0.003 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,201ms (± 0.50%) 5,336ms (± 0.40%) +135ms (+ 2.60%) 5,318ms 5,375ms p=0.005 n=6
Req 2 - geterr 1,115ms (± 0.94%) 974ms (± 0.34%) 🟩-142ms (-12.70%) 970ms 978ms p=0.005 n=6
Req 3 - references 76ms 98ms (± 3.70%) 🔻+22ms (+28.51%) 96ms 105ms p=0.002 n=6
Req 4 - navto 456ms (± 0.18%) 372ms (± 0.15%) 🟩-84ms (-18.47%) 371ms 372ms p=0.004 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 836ms (± 1.23%) 822ms (± 1.36%) -14ms (- 1.71%) 810ms 835ms p=0.045 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.09ms (± 0.18%) 158.79ms (± 0.17%) +1.70ms (+ 1.08%) 157.67ms 162.61ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 232.76ms (± 0.14%) 234.79ms (± 0.13%) +2.03ms (+ 0.87%) 233.60ms 239.68ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 226.59ms (± 0.15%) 228.50ms (± 0.14%) +1.91ms (+ 0.84%) 227.22ms 231.79ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 226.60ms (± 0.15%) 228.70ms (± 0.13%) +2.09ms (+ 0.92%) 227.38ms 230.95ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member

Much better, though emit is now much slower on certain benchmarks. Maybe a case or two aren't monomorphic for those benchmarks?

@dragomirtitian
Copy link
Contributor Author

For the extra parse time I have a pretty good idea. TextRange has too many shapes. Beyond the 3 AST nodes, we also have at least 2 if not more that come from createNodeArray making setting of positions much more expensive in some cases.

Will look into it a bit more.

@dragomirtitian
Copy link
Contributor Author

I moved some stuff around between node and node data. This seems to reduce some more of the extra memory that was being used.

I also experimented with reducing the size of Types, but don't think there are any easy wins there.

@jakebailey When you get a chance, can you run the tests again please?

@jakebailey
Copy link
Member

@typescript-bot perf test this
@typescript-bot perf test this faster host=bun@1.1.17

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 28, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results
perf test this faster host=bun@1.1.17 ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - bun (v1.1.17, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 245,591k (± 4.51%) 271,787k (± 1.75%) 🔻+26,197k (+10.67%) 266,725k 280,687k p=0.008 n=6
Parse Time 1.08s (± 1.13%) 1.07s (± 1.15%) -0.02s (- 1.84%) 1.06s 1.09s p=0.024 n=6
Bind Time 0.82s (± 0.50%) 0.83s (± 2.48%) ~ 0.81s 0.86s p=0.405 n=6
Check Time 8.68s (± 0.64%) 8.33s (± 1.56%) 🟩-0.36s (- 4.11%) 8.12s 8.51s p=0.005 n=6
Emit Time 2.89s (± 1.17%) 3.00s (± 2.74%) +0.11s (+ 3.80%) 2.95s 3.16s p=0.005 n=6
Total Time 13.49s (± 0.45%) 13.23s (± 0.54%) -0.26s (- 1.90%) 13.16s 13.37s p=0.005 n=6
angular-1 - bun (v1.1.17, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,274,831k (± 0.54%) 1,435,731k (± 0.17%) 🔻+160,900k (+12.62%) 1,432,805k 1,439,052k p=0.005 n=6
Parse Time 4.75s (± 0.18%) 4.73s (± 0.28%) -0.02s (- 0.35%) 4.71s 4.74s p=0.040 n=6
Bind Time 1.95s (± 1.00%) 1.91s (± 2.96%) ~ 1.81s 1.97s p=0.196 n=6
Check Time 29.67s (± 0.51%) 25.10s (± 0.51%) 🟩-4.57s (-15.42%) 24.93s 25.32s p=0.005 n=6
Emit Time 11.37s (± 1.50%) 11.25s (± 1.44%) ~ 11.08s 11.46s p=0.261 n=6
Total Time 47.73s (± 0.56%) 42.99s (± 0.37%) 🟩-4.74s (- 9.94%) 42.67s 43.09s p=0.005 n=6
mui-docs - bun (v1.1.17, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,133,039 2,133,039 ~ ~ ~ p=1.000 n=6
Types 926,444 926,444 ~ ~ ~ p=1.000 n=6
Memory used 2,312,288k (± 0.44%) 2,442,773k (± 0.16%) 🔻+130,485k (+ 5.64%) 2,438,826k 2,446,599k p=0.005 n=6
Parse Time 5.46s (± 0.47%) 5.80s (± 1.00%) 🔻+0.33s (+ 6.13%) 5.75s 5.91s p=0.005 n=6
Bind Time 1.51s (± 1.42%) 1.60s (± 0.39%) 🔻+0.09s (+ 6.08%) 1.59s 1.61s p=0.004 n=6
Check Time 64.89s (± 0.59%) 52.64s (± 0.47%) 🟩-12.26s (-18.89%) 52.30s 52.90s p=0.005 n=6
Emit Time 0.15s 0.15s (± 2.69%) ~ 0.15s 0.16s p=0.405 n=6
Total Time 72.01s (± 0.54%) 60.19s (± 0.47%) 🟩-11.82s (-16.42%) 59.82s 60.57s p=0.005 n=6
self-build-src - bun (v1.1.17, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,224,331 1,242,584 +18,253 (+ 1.49%) ~ ~ p=0.001 n=6
Types 261,007 265,198 +4,191 (+ 1.61%) ~ ~ p=0.001 n=6
Memory used 8,894,363k (± 1.89%) 10,126,558k (± 2.96%) 🔻+1,232,196k (+13.85%) 9,682,006k 10,561,465k p=0.005 n=6
Parse Time 5.41s (± 1.30%) 5.42s (± 1.07%) ~ 5.36s 5.52s p=0.684 n=6
Bind Time 2.48s (± 1.44%) 2.61s (± 2.27%) 🔻+0.13s (+ 5.31%) 2.53s 2.71s p=0.005 n=6
Check Time 43.90s (± 3.70%) 39.95s (± 4.72%) 🟩-3.95s (- 8.99%) 38.67s 42.46s p=0.031 n=6
Emit Time 2.61s (± 2.75%) 2.81s (± 1.38%) 🔻+0.19s (+ 7.39%) 2.74s 2.84s p=0.005 n=6
Total Time 54.39s (± 2.98%) 50.79s (± 3.72%) 🟩-3.60s (- 6.62%) 49.42s 53.43s p=0.031 n=6
self-build-src-public-api - bun (v1.1.17, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,224,331 1,242,584 +18,253 (+ 1.49%) ~ ~ p=0.001 n=6
Types 261,007 265,198 +4,191 (+ 1.61%) ~ ~ p=0.001 n=6
Memory used 5,162,965k (±33.11%) 5,459,368k (±12.12%) ~ 4,742,646k 6,167,166k p=0.066 n=6
Parse Time 3.97s (± 1.76%) 3.87s (± 1.55%) -0.10s (- 2.60%) 3.79s 3.94s p=0.019 n=6
Bind Time 1.78s (± 0.94%) 1.92s (± 2.61%) 🔻+0.14s (+ 7.58%) 1.87s 2.00s p=0.005 n=6
Check Time 33.06s (± 0.90%) 31.44s (± 4.52%) 🟩-1.62s (- 4.91%) 29.50s 32.40s p=0.005 n=6
Emit Time 1.62s (± 4.56%) 1.97s (± 2.31%) 🔻+0.35s (+21.73%) 1.92s 2.04s p=0.005 n=6
Total Time 40.42s (± 0.74%) 39.18s (± 3.50%) 🟩-1.24s (- 3.08%) 37.25s 40.21s p=0.030 n=6
self-compiler - bun (v1.1.17, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,166 268,004 +9,838 (+ 3.81%) ~ ~ p=0.001 n=6
Types 104,689 109,058 🔻+4,369 (+ 4.17%) ~ ~ p=0.001 n=6
Memory used 421,994k (± 0.96%) 492,939k (± 0.25%) 🔻+70,945k (+16.81%) 491,737k 495,383k p=0.005 n=6
Parse Time 3.28s (± 1.04%) 3.22s (± 0.44%) -0.06s (- 1.88%) 3.20s 3.24s p=0.006 n=6
Bind Time 1.65s (± 1.97%) 1.60s (± 3.06%) ~ 1.54s 1.65s p=0.090 n=6
Check Time 20.73s (± 0.53%) 19.43s (± 0.38%) 🟩-1.30s (- 6.27%) 19.35s 19.53s p=0.005 n=6
Emit Time 1.28s (± 1.62%) 1.52s (± 1.95%) 🔻+0.24s (+19.06%) 1.49s 1.57s p=0.005 n=6
Total Time 26.93s (± 0.41%) 25.76s (± 0.52%) 🟩-1.16s (- 4.32%) 25.60s 25.92s p=0.005 n=6
ts-pre-modules - bun (v1.1.17, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 354,877k (± 0.04%) 404,915k (± 0.57%) 🔻+50,037k (+14.10%) 402,847k 409,482k p=0.005 n=6
Parse Time 2.34s (± 1.36%) 2.25s (± 0.36%) 🟩-0.08s (- 3.64%) 2.24s 2.26s p=0.005 n=6
Bind Time 1.45s (± 0.75%) 1.47s (± 0.70%) +0.02s (+ 1.15%) 1.45s 1.48s p=0.032 n=6
Check Time 14.40s (± 0.49%) 13.30s (± 0.72%) 🟩-1.10s (- 7.63%) 13.21s 13.44s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.19s (± 0.49%) 17.02s (± 0.56%) 🟩-1.17s (- 6.43%) 16.92s 17.16s p=0.005 n=6
vscode - bun (v1.1.17, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,878,940 2,878,940 ~ ~ ~ p=1.000 n=6
Types 975,074 975,074 ~ ~ ~ p=1.000 n=6
Memory used 3,107,604k (± 0.35%) 3,565,274k (± 0.14%) 🔻+457,670k (+14.73%) 3,559,160k 3,569,963k p=0.005 n=6
Parse Time 11.15s (± 0.37%) 10.71s (± 1.93%) 🟩-0.43s (- 3.87%) 10.52s 10.94s p=0.005 n=6
Bind Time 3.70s (± 0.99%) 3.60s (± 5.95%) ~ 3.34s 3.81s p=1.000 n=6
Check Time 65.19s (± 0.35%) 52.50s (± 0.50%) 🟩-12.69s (-19.47%) 52.19s 52.78s p=0.005 n=6
Emit Time 19.75s (± 0.89%) 19.49s (± 0.75%) -0.27s (- 1.35%) 19.30s 19.64s p=0.030 n=6
Total Time 99.79s (± 0.30%) 86.29s (± 0.37%) 🟩-13.50s (-13.53%) 85.83s 86.68s p=0.005 n=6
webpack - bun (v1.1.17, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 429,087k (± 0.18%) 463,714k (± 0.53%) 🔻+34,627k (+ 8.07%) 460,293k 466,592k p=0.005 n=6
Parse Time 3.07s (± 1.17%) 3.06s (± 0.71%) ~ 3.03s 3.09s p=0.466 n=6
Bind Time 1.38s (± 0.85%) 1.51s (± 2.35%) 🔻+0.13s (+ 9.65%) 1.48s 1.58s p=0.005 n=6
Check Time 13.77s (± 0.42%) 12.73s (± 0.63%) 🟩-1.04s (- 7.54%) 12.63s 12.84s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.23s (± 0.55%) 17.30s (± 0.57%) 🟩-0.93s (- 5.10%) 17.19s 17.44s p=0.005 n=6
xstate-main - bun (v1.1.17, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,061k (± 0.29%) 453,485k (± 0.19%) -8,576k (- 1.86%) 452,663k 455,076k p=0.005 n=6
Parse Time 2.72s (± 1.48%) 2.96s (± 4.71%) 🔻+0.23s (+ 8.63%) 2.88s 3.24s p=0.005 n=6
Bind Time 1.04s (± 5.73%) 0.98s (± 1.78%) ~ 0.96s 1.01s p=0.065 n=6
Check Time 15.86s (± 0.41%) 15.70s (± 0.45%) -0.16s (- 1.01%) 15.62s 15.79s p=0.005 n=6
Emit Time 0.00s 0.01s 🔻+0.01s (+ ∞%) ~ ~ p=0.001 n=6
Total Time 19.63s (± 0.56%) 19.64s (± 0.65%) ~ 19.51s 19.86s p=1.000 n=6
System info unknown
Hosts
  • bun (v1.1.17, x64)
Scenarios
  • Compiler-Unions - bun (v1.1.17, x64)
  • angular-1 - bun (v1.1.17, x64)
  • mui-docs - bun (v1.1.17, x64)
  • self-build-src - bun (v1.1.17, x64)
  • self-build-src-public-api - bun (v1.1.17, x64)
  • self-compiler - bun (v1.1.17, x64)
  • ts-pre-modules - bun (v1.1.17, x64)
  • vscode - bun (v1.1.17, x64)
  • webpack - bun (v1.1.17, x64)
  • xstate-main - bun (v1.1.17, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 194,647k (± 0.92%) 200,834k (± 0.95%) +6,188k (+ 3.18%) 199,561k 203,385k p=0.005 n=6
Parse Time 1.58s (± 0.48%) 1.59s (± 1.26%) ~ 1.56s 1.62s p=0.188 n=6
Bind Time 0.86s (± 0.73%) 0.81s (± 1.00%) 🟩-0.05s (- 5.43%) 0.80s 0.82s p=0.004 n=6
Check Time 11.19s (± 0.21%) 8.65s (± 0.54%) 🟩-2.54s (-22.69%) 8.58s 8.70s p=0.005 n=6
Emit Time 3.28s (± 0.42%) 3.04s (± 0.62%) 🟩-0.24s (- 7.41%) 3.02s 3.07s p=0.005 n=6
Total Time 16.91s (± 0.17%) 14.09s (± 0.27%) 🟩-2.82s (-16.67%) 14.04s 14.15s p=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,381k (± 0.00%) 1,265,388k (± 0.00%) +47,008k (+ 3.86%) 1,265,338k 1,265,434k p=0.005 n=6
Parse Time 7.96s (± 0.48%) 7.72s (± 0.44%) 🟩-0.24s (- 3.01%) 7.68s 7.77s p=0.005 n=6
Bind Time 2.21s (± 0.66%) 1.87s (± 2.07%) 🟩-0.34s (-15.30%) 1.85s 1.95s p=0.004 n=6
Check Time 35.71s (± 0.23%) 27.69s (± 0.55%) 🟩-8.02s (-22.45%) 27.53s 27.93s p=0.005 n=6
Emit Time 16.13s (± 0.28%) 14.90s (± 0.27%) 🟩-1.23s (- 7.61%) 14.85s 14.95s p=0.005 n=6
Total Time 62.01s (± 0.06%) 52.19s (± 0.27%) 🟩-9.82s (-15.84%) 51.99s 52.34s p=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,133,039 2,133,039 ~ ~ ~ p=1.000 n=6
Types 926,444 926,444 ~ ~ ~ p=1.000 n=6
Memory used 2,115,366k (± 0.00%) 2,167,316k (± 0.01%) +51,950k (+ 2.46%) 2,167,035k 2,167,491k p=0.005 n=6
Parse Time 7.93s (± 0.33%) 7.92s (± 0.43%) ~ 7.87s 7.96s p=0.627 n=6
Bind Time 2.77s (± 0.49%) 2.66s (± 1.05%) 🟩-0.12s (- 4.21%) 2.62s 2.69s p=0.005 n=6
Check Time 84.20s (± 0.94%) 61.73s (± 0.08%) 🟩-22.46s (-26.68%) 61.67s 61.80s p=0.005 n=6
Emit Time 0.16s (± 3.95%) 0.17s (± 3.32%) ~ 0.16s 0.17s p=0.201 n=6
Total Time 95.06s (± 0.85%) 72.48s (± 0.09%) 🟩-22.58s (-23.75%) 72.36s 72.56s p=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,224,331 1,242,584 +18,253 (+ 1.49%) ~ ~ p=0.001 n=6
Types 261,007 265,198 +4,191 (+ 1.61%) ~ ~ p=0.001 n=6
Memory used 2,339,130k (± 0.02%) 2,424,730k (± 0.02%) +85,601k (+ 3.66%) 2,423,980k 2,425,137k p=0.005 n=6
Parse Time 6.02s (± 0.58%) 5.97s (± 1.09%) ~ 5.89s 6.06s p=0.470 n=6
Bind Time 2.25s (± 1.22%) 1.90s (± 0.98%) 🟩-0.34s (-15.28%) 1.88s 1.93s p=0.005 n=6
Check Time 40.12s (± 0.35%) 33.14s (± 0.21%) 🟩-6.98s (-17.40%) 33.03s 33.25s p=0.005 n=6
Emit Time 3.13s (± 1.21%) 3.36s (± 2.63%) 🔻+0.23s (+ 7.23%) 3.22s 3.44s p=0.005 n=6
Total Time 51.54s (± 0.24%) 44.40s (± 0.27%) 🟩-7.14s (-13.86%) 44.24s 44.56s p=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,224,331 1,242,584 +18,253 (+ 1.49%) ~ ~ p=0.001 n=6
Types 261,007 265,198 +4,191 (+ 1.61%) ~ ~ p=0.001 n=6
Memory used 2,412,607k (± 0.04%) 2,511,845k (± 0.03%) 🔻+99,239k (+ 4.11%) 2,511,025k 2,512,668k p=0.005 n=6
Parse Time 5.19s (± 0.76%) 5.07s (± 0.79%) -0.12s (- 2.34%) 5.01s 5.12s p=0.005 n=6
Bind Time 1.70s (± 0.48%) 1.62s (± 0.50%) 🟩-0.07s (- 4.32%) 1.61s 1.63s p=0.004 n=6
Check Time 34.56s (± 0.28%) 28.29s (± 0.12%) 🟩-6.27s (-18.15%) 28.23s 28.32s p=0.005 n=6
Emit Time 2.75s (± 2.47%) 2.89s (± 1.22%) 🔻+0.14s (+ 5.15%) 2.85s 2.93s p=0.005 n=6
Total Time 44.21s (± 0.35%) 37.91s (± 0.17%) 🟩-6.30s (-14.24%) 37.83s 38.01s p=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,166 268,004 +9,838 (+ 3.81%) ~ ~ p=0.001 n=6
Types 104,689 109,058 🔻+4,369 (+ 4.17%) ~ ~ p=0.001 n=6
Memory used 427,315k (± 0.01%) 451,209k (± 0.01%) 🔻+23,894k (+ 5.59%) 451,177k 451,253k p=0.005 n=6
Parse Time 4.07s (± 0.44%) 4.14s (± 0.48%) +0.07s (+ 1.60%) 4.12s 4.16s p=0.005 n=6
Bind Time 1.63s (± 1.06%) 1.47s (± 0.51%) 🟩-0.16s (- 9.62%) 1.46s 1.48s p=0.004 n=6
Check Time 21.97s (± 0.48%) 17.71s (± 0.37%) 🟩-4.26s (-19.39%) 17.66s 17.84s p=0.005 n=6
Emit Time 1.72s (± 1.45%) 1.78s (± 0.55%) +0.05s (+ 3.09%) 1.76s 1.79s p=0.004 n=6
Total Time 29.40s (± 0.36%) 25.10s (± 0.28%) 🟩-4.30s (-14.62%) 25.03s 25.23s p=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,521k (± 0.03%) 382,750k (± 0.01%) +13,229k (+ 3.58%) 382,708k 382,842k p=0.005 n=6
Parse Time 2.31s (± 0.60%) 2.31s (± 0.71%) ~ 2.29s 2.33s p=0.410 n=6
Bind Time 1.33s (± 1.48%) 1.18s (± 2.10%) 🟩-0.15s (-11.49%) 1.15s 1.22s p=0.005 n=6
Check Time 13.16s (± 0.24%) 10.46s (± 0.24%) 🟩-2.70s (-20.52%) 10.44s 10.51s p=0.004 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 16.80s (± 0.26%) 13.96s (± 0.29%) 🟩-2.84s (-16.92%) 13.91s 14.00s p=0.005 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,878,940 2,878,940 ~ ~ ~ p=1.000 n=6
Types 975,074 975,074 ~ ~ ~ p=1.000 n=6
Memory used 3,041,776k (± 0.00%) 3,152,755k (± 0.00%) +110,980k (+ 3.65%) 3,152,697k 3,152,866k p=0.005 n=6
Parse Time 13.67s (± 0.42%) 13.36s (± 0.36%) -0.31s (- 2.28%) 13.30s 13.42s p=0.005 n=6
Bind Time 4.23s (± 1.79%) 3.40s (± 0.73%) 🟩-0.82s (-19.52%) 3.37s 3.43s p=0.005 n=6
Check Time 73.45s (± 0.21%) 62.02s (± 0.13%) 🟩-11.43s (-15.56%) 61.92s 62.13s p=0.005 n=6
Emit Time 24.23s (± 0.54%) 25.68s (± 0.64%) 🔻+1.44s (+ 5.94%) 25.45s 25.85s p=0.005 n=6
Total Time 115.58s (± 0.13%) 104.46s (± 0.18%) 🟩-11.12s (- 9.62%) 104.22s 104.72s p=0.005 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,564k (± 0.01%) 424,477k (± 0.01%) +12,913k (+ 3.14%) 424,428k 424,589k p=0.005 n=6
Parse Time 3.18s (± 0.31%) 3.09s (± 0.17%) -0.09s (- 2.78%) 3.09s 3.10s p=0.004 n=6
Bind Time 1.41s (± 0.94%) 1.27s (± 0.59%) 🟩-0.14s (- 9.70%) 1.26s 1.28s p=0.004 n=6
Check Time 14.19s (± 0.33%) 11.96s (± 0.25%) 🟩-2.23s (-15.74%) 11.92s 12.00s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.78s (± 0.25%) 16.32s (± 0.17%) 🟩-2.46s (-13.11%) 16.28s 16.36s p=0.005 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,790k (± 0.08%) 475,510k (± 0.01%) +12,720k (+ 2.75%) 475,472k 475,586k p=0.005 n=6
Parse Time 3.19s (± 0.44%) 3.16s (± 0.92%) ~ 3.12s 3.21s p=0.064 n=6
Bind Time 1.17s (± 0.35%) 1.15s (± 0.65%) -0.02s (- 1.99%) 1.14s 1.16s p=0.003 n=6
Check Time 17.95s (± 0.24%) 15.29s (± 0.35%) 🟩-2.66s (-14.84%) 15.20s 15.35s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.31s (± 0.21%) 19.59s (± 0.39%) 🟩-2.72s (-12.18%) 19.51s 19.72s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,212ms (±10.48%) 3,449ms (± 0.64%) ~ 3,409ms 3,469ms p=0.054 n=6
Req 2 - geterr 7,238ms (± 8.32%) 6,175ms (± 0.52%) ~ 6,114ms 6,208ms p=0.066 n=6
Req 3 - references 392ms (± 0.67%) 391ms (± 0.40%) ~ 389ms 393ms p=0.746 n=6
Req 4 - navto 339ms (± 0.81%) 342ms (± 0.40%) +4ms (+ 1.08%) 341ms 345ms p=0.042 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 124ms (±11.78%) 122ms (±10.55%) ~ 110ms 147ms p=0.293 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,588ms (± 1.31%) 3,538ms (± 1.34%) ~ 3,477ms 3,597ms p=0.093 n=6
Req 2 - geterr 5,537ms (± 0.49%) 4,725ms (± 0.27%) 🟩-812ms (-14.66%) 4,705ms 4,736ms p=0.005 n=6
Req 3 - references 416ms (± 0.20%) 422ms (± 0.20%) +6ms (+ 1.40%) 421ms 423ms p=0.004 n=6
Req 4 - navto 338ms (± 0.68%) 357ms (± 2.91%) 🔻+19ms (+ 5.47%) 347ms 369ms p=0.005 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 108ms (± 2.51%) 115ms (± 5.37%) 🔻+7ms (+ 6.16%) 105ms 122ms p=0.043 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,207ms (± 0.66%) 5,195ms (± 0.96%) ~ 5,135ms 5,281ms p=0.689 n=6
Req 2 - geterr 1,123ms (± 0.34%) 984ms (± 0.45%) 🟩-139ms (-12.41%) 977ms 990ms p=0.005 n=6
Req 3 - references 77ms (± 3.56%) 83ms (± 3.65%) 🔻+6ms (+ 7.61%) 80ms 88ms p=0.019 n=6
Req 4 - navto 459ms (± 0.79%) 378ms (± 0.74%) 🟩-81ms (-17.70%) 373ms 380ms p=0.004 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 826ms (± 1.81%) 872ms (± 0.90%) 🔻+45ms (+ 5.47%) 856ms 878ms p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 188.88ms (± 0.17%) 190.60ms (± 0.17%) +1.72ms (+ 0.91%) 188.88ms 196.42ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 284.73ms (± 0.27%) 286.94ms (± 0.29%) +2.21ms (+ 0.78%) 279.96ms 294.76ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 226.07ms (± 0.14%) 227.92ms (± 0.18%) +1.85ms (+ 0.82%) 226.45ms 233.75ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 276.26ms (± 0.30%) 278.51ms (± 0.28%) +2.25ms (+ 0.81%) 271.34ms 282.31ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@robpalme
Copy link

Good stuff. On Node the mean averages for tsc are:

  • a total time win of 15.1% (ranging 9.6% to 23.7%)
  • a memory cost of 3.6% (ranging 2.5% to 5.6%)

...which is an exchange rate of 4.2

@MichaelMitchell-at
Copy link

Good stuff. On Node the mean averages for tsc are:

  • a total time win of 15.1% (ranging 9.6% to 23.7%)
  • a memory cost of 3.6% (ranging 2.5% to 5.6%)

...which is an exchange rate of 4.2

It's worth keeping in mind that if TSServer requires more than 4096 MB of memory then VSCode can no longer use its prebuilt version of Node with pointer compression, which has a significant effect on interactivity performance. So it might be worth re-measuring that exchange rate on projects that are close to reaching that threshold. I plan to do this on the Airtable monorepo (which at this point has already blown past the 4GB memory limit, but we hope to eventually bring it down), once the PR is stable.

@dragomirtitian
Copy link
Contributor Author

@jakebailey Can you please run the perf tests one more time?

I hope this latest version will bring the mem usage a bit more, either way I think this is at the limit of what I can do to reduce memory usage.

I can cleanup the PR if there is interest in merging this based on these perf metrics.

Beside performance, we should also consider the issue of this being a breaking change for API consumers. Anyone manually iterating over the properties of nodes (either with Object.keys or for .. in) will observe the difference.

Possible future improvements: Actually use data instead of the accessors (both in the compiler and transition API consumers as well). While the accessors seems to perform well, the fact that a lot of them end up being megamorphic hints at the fact that direct access would be faster. Direct access would be monomorphic/polymorphic in many places (although probably not all). More experimentation would be needed to decide if this change would be worth the DX pain.

@jakebailey
Copy link
Member

I'll rerun it, though I do think there are still concerns about this particular approach.

@typescript-bot perf test this
@typescript-bot perf test this faster host=bun@1.1.17

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 1, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results
perf test this faster host=bun@1.1.17 ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - bun (v1.1.17, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 242,764k (± 1.20%) 270,728k (± 1.11%) 🔻+27,963k (+11.52%) 266,203k 273,638k p=0.005 n=6
Parse Time 1.09s (± 1.16%) 1.07s (± 0.78%) -0.02s (- 1.38%) 1.07s 1.09s p=0.038 n=6
Bind Time 0.82s (± 0.92%) 0.81s (± 1.21%) ~ 0.80s 0.83s p=0.167 n=6
Check Time 8.69s (± 0.60%) 7.97s (± 0.52%) 🟩-0.73s (- 8.36%) 7.89s 8.01s p=0.005 n=6
Emit Time 2.87s (± 1.48%) 2.98s (± 1.52%) +0.11s (+ 3.89%) 2.93s 3.04s p=0.005 n=6
Total Time 13.47s (± 0.21%) 12.83s (± 0.64%) 🟩-0.64s (- 4.74%) 12.70s 12.94s p=0.005 n=6
angular-1 - bun (v1.1.17, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,275,656k (± 0.68%) 1,453,136k (± 0.07%) 🔻+177,480k (+13.91%) 1,452,066k 1,454,609k p=0.005 n=6
Parse Time 4.75s (± 0.22%) 4.75s (± 0.44%) ~ 4.72s 4.78s p=0.869 n=6
Bind Time 1.96s (± 0.62%) 1.87s (± 1.12%) 🟩-0.09s (- 4.43%) 1.85s 1.91s p=0.005 n=6
Check Time 29.73s (± 0.53%) 23.98s (± 0.39%) 🟩-5.75s (-19.33%) 23.87s 24.14s p=0.005 n=6
Emit Time 11.35s (± 1.70%) 11.14s (± 1.53%) ~ 10.94s 11.45s p=0.066 n=6
Total Time 47.78s (± 0.62%) 41.74s (± 0.43%) 🟩-6.04s (-12.65%) 41.54s 42.02s p=0.005 n=6
mui-docs - bun (v1.1.17, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,136,980 2,136,980 ~ ~ ~ p=1.000 n=6
Types 929,117 929,117 ~ ~ ~ p=1.000 n=6
Memory used 2,312,597k (± 0.28%) 2,464,135k (± 0.11%) 🔻+151,539k (+ 6.55%) 2,459,266k 2,466,462k p=0.005 n=6
Parse Time 5.47s (± 0.88%) 5.79s (± 0.46%) 🔻+0.32s (+ 5.82%) 5.76s 5.83s p=0.005 n=6
Bind Time 1.51s (± 1.94%) 1.66s (± 1.88%) 🔻+0.15s (+ 9.80%) 1.63s 1.70s p=0.005 n=6
Check Time 64.89s (± 0.82%) 51.14s (± 0.32%) 🟩-13.76s (-21.20%) 50.93s 51.33s p=0.005 n=6
Emit Time 0.15s (± 2.75%) 0.15s ~ ~ ~ p=0.405 n=6
Total Time 72.03s (± 0.80%) 58.74s (± 0.26%) 🟩-13.29s (-18.45%) 58.58s 58.94s p=0.005 n=6
self-build-src - bun (v1.1.17, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,260 1,243,897 +18,637 (+ 1.52%) ~ ~ p=0.001 n=6
Types 261,341 265,541 +4,200 (+ 1.61%) ~ ~ p=0.001 n=6
Memory used 9,070,572k (± 1.58%) 10,385,546k (± 0.33%) 🔻+1,314,974k (+14.50%) 10,349,092k 10,444,220k p=0.005 n=6
Parse Time 4.05s (± 0.53%) 4.00s (± 0.61%) -0.04s (- 1.03%) 3.98s 4.04s p=0.030 n=6
Bind Time 1.71s (± 1.21%) 1.87s (± 3.05%) 🔻+0.17s (+ 9.87%) 1.80s 1.96s p=0.005 n=6
Check Time 31.64s (± 0.50%) 26.48s (± 0.44%) 🟩-5.16s (-16.32%) 26.36s 26.66s p=0.005 n=6
Emit Time 1.77s (± 0.92%) 1.81s (± 1.08%) +0.04s (+ 2.26%) 1.79s 1.83s p=0.011 n=6
Total Time 39.18s (± 0.45%) 34.17s (± 0.27%) 🟩-5.01s (-12.79%) 34.09s 34.35s p=0.005 n=6
self-build-src-public-api - bun (v1.1.17, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,260 1,243,897 +18,637 (+ 1.52%) ~ ~ p=0.001 n=6
Types 261,341 265,541 +4,200 (+ 1.61%) ~ ~ p=0.001 n=6
Memory used 5,828,138k (±38.89%) 8,850,265k (±24.77%) 🔻+3,022,127k (+51.85%) 5,809,658k 10,285,605k p=0.031 n=6
Parse Time 4.77s (± 1.23%) 4.88s (± 1.09%) +0.11s (+ 2.24%) 4.81s 4.95s p=0.013 n=6
Bind Time 2.14s (± 3.64%) 2.15s (± 3.80%) ~ 2.08s 2.30s p=0.810 n=6
Check Time 39.24s (± 3.79%) 34.52s (± 2.09%) 🟩-4.73s (-12.04%) 34.08s 35.97s p=0.005 n=6
Emit Time 1.97s (± 6.60%) 2.23s (± 3.92%) 🔻+0.25s (+12.83%) 2.15s 2.40s p=0.005 n=6
Total Time 48.12s (± 3.22%) 43.76s (± 1.81%) 🟩-4.36s (- 9.06%) 43.26s 45.35s p=0.005 n=6
self-compiler - bun (v1.1.17, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,194 268,439 +10,245 (+ 3.97%) ~ ~ p=0.001 n=6
Types 104,737 109,115 🔻+4,378 (+ 4.18%) ~ ~ p=0.001 n=6
Memory used 419,866k (± 1.25%) 483,535k (± 1.50%) 🔻+63,669k (+15.16%) 477,256k 494,647k p=0.005 n=6
Parse Time 3.26s (± 1.24%) 3.21s (± 0.92%) ~ 3.18s 3.26s p=0.061 n=6
Bind Time 1.71s (± 4.26%) 1.62s (± 1.44%) 🟩-0.08s (- 4.88%) 1.60s 1.65s p=0.008 n=6
Check Time 20.63s (± 0.91%) 18.46s (± 0.41%) 🟩-2.18s (-10.54%) 18.38s 18.56s p=0.005 n=6
Emit Time 1.25s (± 3.86%) 1.47s (± 3.05%) 🔻+0.23s (+18.21%) 1.39s 1.52s p=0.005 n=6
Total Time 26.84s (± 0.47%) 24.77s (± 0.40%) 🟩-2.08s (- 7.74%) 24.65s 24.90s p=0.005 n=6
ts-pre-modules - bun (v1.1.17, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 354,957k (± 0.13%) 409,212k (± 0.59%) 🔻+54,255k (+15.28%) 407,048k 412,365k p=0.005 n=6
Parse Time 2.31s (± 0.84%) 2.25s (± 0.72%) -0.06s (- 2.52%) 2.23s 2.28s p=0.005 n=6
Bind Time 1.45s (± 1.55%) 1.47s (± 1.27%) ~ 1.44s 1.49s p=0.123 n=6
Check Time 14.38s (± 0.63%) 12.58s (± 0.62%) 🟩-1.80s (-12.52%) 12.46s 12.70s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.13s (± 0.53%) 16.30s (± 0.53%) 🟩-1.84s (-10.14%) 16.21s 16.45s p=0.005 n=6
vscode - bun (v1.1.17, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,879,347 2,879,347 ~ ~ ~ p=1.000 n=6
Types 975,216 975,216 ~ ~ ~ p=1.000 n=6
Memory used 3,099,929k (± 0.16%) 3,618,962k (± 0.55%) 🔻+519,033k (+16.74%) 3,604,828k 3,647,106k p=0.005 n=6
Parse Time 12.85s (±10.27%) 13.45s (± 1.27%) ~ 13.31s 13.69s p=0.689 n=6
Bind Time 4.32s (±10.26%) 4.25s (± 0.49%) ~ 4.23s 4.29s p=0.377 n=6
Check Time 74.75s (± 9.78%) 62.01s (± 0.92%) 🟩-12.74s (-17.04%) 61.26s 62.67s p=0.005 n=6
Emit Time 22.64s (±10.14%) 22.93s (± 8.14%) ~ 20.46s 24.42s p=0.936 n=6
Total Time 114.56s (± 9.91%) 102.64s (± 1.23%) ~ 100.93s 103.91s p=0.378 n=6
webpack - bun (v1.1.17, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 429,397k (± 0.21%) 465,418k (± 0.66%) 🔻+36,022k (+ 8.39%) 460,961k 468,633k p=0.005 n=6
Parse Time 3.10s (± 0.24%) 3.10s (± 1.07%) ~ 3.05s 3.14s p=0.805 n=6
Bind Time 1.38s (± 1.36%) 1.51s (± 1.43%) 🔻+0.14s (+10.18%) 1.49s 1.55s p=0.005 n=6
Check Time 13.78s (± 0.29%) 12.30s (± 0.44%) 🟩-1.48s (-10.77%) 12.24s 12.38s p=0.005 n=6
Emit Time 0.00s 0.00s (±244.70%) ~ 0.00s 0.01s p=0.405 n=6
Total Time 18.26s (± 0.14%) 16.92s (± 0.31%) 🟩-1.34s (- 7.36%) 16.84s 17.00s p=0.005 n=6
xstate-main - bun (v1.1.17, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 454,859k (± 4.91%) 455,861k (± 0.51%) ~ 451,557k 457,939k p=0.066 n=6
Parse Time 2.71s (± 1.20%) 2.87s (± 0.53%) 🔻+0.15s (+ 5.59%) 2.84s 2.88s p=0.005 n=6
Bind Time 0.98s (± 6.94%) 1.10s (± 4.81%) 🔻+0.12s (+11.71%) 0.99s 1.13s p=0.012 n=6
Check Time 15.96s (± 0.76%) 14.45s (± 0.73%) 🟩-1.50s (- 9.41%) 14.35s 14.63s p=0.005 n=6
Emit Time 0.00s 0.01s (±48.94%) 🔻+0.01s (+ ∞%) 0.00s 0.01s p=0.007 n=6
Total Time 19.65s (± 0.53%) 18.43s (± 0.38%) 🟩-1.23s (- 6.26%) 18.33s 18.51s p=0.005 n=6
System info unknown
Hosts
  • bun (v1.1.17, x64)
Scenarios
  • Compiler-Unions - bun (v1.1.17, x64)
  • angular-1 - bun (v1.1.17, x64)
  • mui-docs - bun (v1.1.17, x64)
  • self-build-src - bun (v1.1.17, x64)
  • self-build-src-public-api - bun (v1.1.17, x64)
  • self-compiler - bun (v1.1.17, x64)
  • ts-pre-modules - bun (v1.1.17, x64)
  • vscode - bun (v1.1.17, x64)
  • webpack - bun (v1.1.17, x64)
  • xstate-main - bun (v1.1.17, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,293k (± 0.96%) 199,044k (± 0.96%) +5,751k (+ 2.98%) 197,768k 201,582k p=0.005 n=6
Parse Time 1.95s (± 1.11%) 2.04s (± 1.48%) 🔻+0.09s (+ 4.63%) 1.99s 2.08s p=0.005 n=6
Bind Time 1.06s (± 1.51%) 1.01s (± 0.63%) 🟩-0.05s (- 4.87%) 1.00s 1.02s p=0.004 n=6
Check Time 13.85s (± 0.32%) 10.46s (± 0.41%) 🟩-3.39s (-24.48%) 10.41s 10.52s p=0.005 n=6
Emit Time 4.07s (± 3.71%) 3.77s (± 1.29%) 🟩-0.30s (- 7.26%) 3.68s 3.81s p=0.005 n=6
Total Time 20.92s (± 0.82%) 17.27s (± 0.47%) 🟩-3.65s (-17.44%) 17.15s 17.39s p=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,384k (± 0.00%) 1,251,932k (± 0.00%) +33,547k (+ 2.75%) 1,251,878k 1,252,003k p=0.005 n=6
Parse Time 7.98s (± 0.86%) 7.79s (± 0.58%) -0.19s (- 2.38%) 7.73s 7.86s p=0.005 n=6
Bind Time 2.25s (± 1.80%) 1.91s (± 0.57%) 🟩-0.34s (-14.92%) 1.90s 1.93s p=0.005 n=6
Check Time 35.80s (± 0.37%) 27.31s (± 0.35%) 🟩-8.50s (-23.74%) 27.17s 27.41s p=0.005 n=6
Emit Time 16.13s (± 0.49%) 14.96s (± 0.30%) 🟩-1.18s (- 7.28%) 14.90s 15.02s p=0.005 n=6
Total Time 62.16s (± 0.17%) 51.96s (± 0.19%) 🟩-10.20s (-16.41%) 51.85s 52.10s p=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,136,980 2,136,980 ~ ~ ~ p=1.000 n=6
Types 929,117 929,117 ~ ~ ~ p=1.000 n=6
Memory used 2,117,752k (± 0.00%) 2,157,609k (± 0.00%) +39,857k (+ 1.88%) 2,157,555k 2,157,687k p=0.005 n=6
Parse Time 7.91s (± 0.35%) 7.97s (± 0.26%) +0.06s (+ 0.74%) 7.95s 8.01s p=0.006 n=6
Bind Time 2.76s (± 0.55%) 2.68s (± 0.55%) -0.08s (- 2.84%) 2.66s 2.70s p=0.005 n=6
Check Time 83.59s (± 0.89%) 61.19s (± 0.40%) 🟩-22.41s (-26.80%) 60.92s 61.54s p=0.005 n=6
Emit Time 0.16s (± 3.95%) 0.17s (± 3.10%) ~ 0.16s 0.17s p=0.091 n=6
Total Time 94.42s (± 0.77%) 72.00s (± 0.36%) 🟩-22.42s (-23.74%) 71.72s 72.35s p=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,260 1,243,897 +18,637 (+ 1.52%) ~ ~ p=0.001 n=6
Types 261,341 265,541 +4,200 (+ 1.61%) ~ ~ p=0.001 n=6
Memory used 2,340,931k (± 0.03%) 2,412,903k (± 1.02%) +71,972k (+ 3.07%) 2,401,115k 2,463,312k p=0.005 n=6
Parse Time 5.04s (± 1.06%) 4.96s (± 1.42%) ~ 4.90s 5.09s p=0.078 n=6
Bind Time 1.90s (± 0.77%) 1.65s (± 0.97%) 🟩-0.25s (-13.15%) 1.63s 1.67s p=0.005 n=6
Check Time 34.21s (± 0.37%) 27.70s (± 0.30%) 🟩-6.51s (-19.03%) 27.61s 27.80s p=0.005 n=6
Emit Time 2.53s (± 1.21%) 2.73s (± 3.37%) 🔻+0.19s (+ 7.70%) 2.60s 2.87s p=0.005 n=6
Total Time 43.71s (± 0.31%) 37.08s (± 0.45%) 🟩-6.64s (-15.18%) 36.84s 37.30s p=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,260 1,243,897 +18,637 (+ 1.52%) ~ ~ p=0.001 n=6
Types 261,341 265,541 +4,200 (+ 1.61%) ~ ~ p=0.001 n=6
Memory used 2,414,838k (± 0.07%) 2,492,479k (± 0.14%) +77,641k (+ 3.22%) 2,490,230k 2,499,661k p=0.005 n=6
Parse Time 6.24s (± 0.82%) 6.15s (± 1.12%) ~ 6.07s 6.24s p=0.128 n=6
Bind Time 2.03s (± 0.67%) 1.99s (± 0.84%) -0.04s (- 2.13%) 1.96s 2.01s p=0.005 n=6
Check Time 40.95s (± 0.62%) 32.80s (± 0.25%) 🟩-8.16s (-19.92%) 32.71s 32.95s p=0.005 n=6
Emit Time 3.09s (± 4.12%) 3.21s (± 5.54%) ~ 3.07s 3.55s p=0.149 n=6
Total Time 52.34s (± 0.51%) 44.17s (± 0.58%) 🟩-8.16s (-15.60%) 43.99s 44.65s p=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,194 268,439 +10,245 (+ 3.97%) ~ ~ p=0.001 n=6
Types 104,737 109,115 🔻+4,378 (+ 4.18%) ~ ~ p=0.001 n=6
Memory used 427,501k (± 0.02%) 446,329k (± 0.01%) 🔻+18,829k (+ 4.40%) 446,293k 446,414k p=0.005 n=6
Parse Time 4.07s (± 0.55%) 4.12s (± 0.61%) +0.05s (+ 1.15%) 4.09s 4.15s p=0.019 n=6
Bind Time 1.63s (± 0.87%) 1.47s (± 1.65%) 🟩-0.15s (- 9.51%) 1.44s 1.51s p=0.005 n=6
Check Time 21.98s (± 0.22%) 17.35s (± 0.42%) 🟩-4.64s (-21.08%) 17.26s 17.43s p=0.005 n=6
Emit Time 1.55s (± 1.03%) 1.56s (± 1.59%) ~ 1.53s 1.60s p=0.624 n=6
Total Time 29.23s (± 0.14%) 24.50s (± 0.25%) 🟩-4.74s (-16.20%) 24.44s 24.58s p=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,449k (± 0.03%) 378,436k (± 0.01%) +8,987k (+ 2.43%) 378,405k 378,467k p=0.005 n=6
Parse Time 2.77s (± 1.44%) 2.80s (± 0.84%) ~ 2.77s 2.82s p=0.056 n=6
Bind Time 1.58s (± 0.86%) 1.44s (± 0.98%) 🟩-0.14s (- 9.05%) 1.42s 1.46s p=0.004 n=6
Check Time 15.49s (± 0.31%) 12.05s (± 0.36%) 🟩-3.43s (-22.18%) 11.97s 12.10s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.84s (± 0.40%) 16.30s (± 0.32%) 🟩-3.54s (-17.85%) 16.21s 16.37s p=0.005 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,879,347 2,879,347 ~ ~ ~ p=1.000 n=6
Types 975,216 975,216 ~ ~ ~ p=1.000 n=6
Memory used 3,042,393k (± 0.00%) 3,124,829k (± 0.01%) +82,436k (+ 2.71%) 3,124,509k 3,125,279k p=0.005 n=6
Parse Time 11.45s (± 0.39%) 11.23s (± 0.35%) -0.22s (- 1.91%) 11.17s 11.29s p=0.005 n=6
Bind Time 3.60s (± 2.64%) 2.93s (± 0.50%) 🟩-0.67s (-18.66%) 2.91s 2.95s p=0.005 n=6
Check Time 64.47s (± 2.78%) 52.97s (± 0.21%) 🟩-11.49s (-17.83%) 52.82s 53.10s p=0.005 n=6
Emit Time 19.29s (± 7.42%) 21.29s (± 1.92%) 🔻+2.00s (+10.38%) 20.97s 21.90s p=0.005 n=6
Total Time 98.81s (± 0.44%) 88.42s (± 0.49%) 🟩-10.38s (-10.51%) 88.06s 89.05s p=0.005 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,582k (± 0.02%) 420,166k (± 0.01%) +8,584k (+ 2.09%) 420,103k 420,218k p=0.005 n=6
Parse Time 3.82s (± 0.54%) 3.83s (± 0.60%) ~ 3.80s 3.87s p=0.684 n=6
Bind Time 1.68s (± 0.92%) 1.52s (± 1.02%) 🟩-0.16s (- 9.52%) 1.51s 1.55s p=0.004 n=6
Check Time 16.71s (± 0.43%) 13.95s (± 0.12%) 🟩-2.77s (-16.55%) 13.93s 13.97s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.22s (± 0.18%) 19.30s (± 0.14%) 🟩-2.92s (-13.14%) 19.27s 19.33s p=0.005 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,921k (± 0.07%) 471,998k (± 0.08%) +9,077k (+ 1.96%) 471,588k 472,469k p=0.005 n=6
Parse Time 3.93s (± 0.58%) 3.92s (± 0.53%) ~ 3.90s 3.95s p=0.683 n=6
Bind Time 1.45s (± 0.81%) 1.40s (± 0.58%) 🟩-0.04s (- 3.11%) 1.39s 1.41s p=0.004 n=6
Check Time 22.14s (± 0.19%) 18.43s (± 0.16%) 🟩-3.71s (-16.76%) 18.40s 18.47s p=0.005 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.52s (± 0.22%) 23.76s (± 0.14%) 🟩-3.76s (-13.68%) 23.71s 23.80s p=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,302ms (± 0.34%) 2,333ms (± 0.67%) +31ms (+ 1.32%) 2,311ms 2,356ms p=0.008 n=6
Req 2 - geterr 5,074ms (± 0.14%) 4,125ms (± 0.30%) 🟩-949ms (-18.70%) 4,103ms 4,139ms p=0.005 n=6
Req 3 - references 264ms (± 0.24%) 270ms (± 1.26%) +6ms (+ 2.27%) 266ms 274ms p=0.004 n=6
Req 4 - navto 228ms (± 0.18%) 230ms (± 0.18%) +2ms (+ 0.88%) 229ms 230ms p=0.003 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 88ms 80ms (± 2.95%) 🟩-9ms (- 9.66%) 78ms 84ms p=0.003 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,575ms (± 0.91%) 3,601ms (± 1.25%) ~ 3,565ms 3,677ms p=0.149 n=6
Req 2 - geterr 5,547ms (± 0.43%) 4,669ms (± 0.59%) 🟩-878ms (-15.83%) 4,623ms 4,697ms p=0.005 n=6
Req 3 - references 415ms (± 0.20%) 425ms (± 0.64%) +10ms (+ 2.29%) 422ms 428ms p=0.005 n=6
Req 4 - navto 338ms (± 0.66%) 354ms (± 2.49%) 🔻+15ms (+ 4.53%) 343ms 362ms p=0.004 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 108ms (± 3.49%) 120ms (± 2.38%) 🔻+12ms (+11.13%) 114ms 121ms p=0.003 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,208ms (± 0.78%) 5,209ms (± 0.43%) ~ 5,185ms 5,243ms p=0.810 n=6
Req 2 - geterr 1,126ms (± 0.92%) 956ms (± 1.46%) 🟩-171ms (-15.15%) 930ms 971ms p=0.005 n=6
Req 3 - references 77ms (± 4.19%) 84ms (± 4.21%) 🔻+8ms (+10.02%) 81ms 89ms p=0.018 n=6
Req 4 - navto 458ms (± 0.23%) 376ms (± 0.86%) 🟩-81ms (-17.78%) 372ms 380ms p=0.005 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 829ms (± 1.80%) 861ms (± 0.91%) +32ms (+ 3.86%) 852ms 872ms p=0.016 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 187.92ms (± 0.19%) 189.99ms (± 0.18%) +2.07ms (+ 1.10%) 187.96ms 193.65ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 283.75ms (± 0.28%) 285.86ms (± 0.28%) +2.11ms (+ 0.74%) 279.63ms 291.74ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 277.08ms (± 0.28%) 279.35ms (± 0.29%) +2.26ms (+ 0.82%) 272.24ms 289.38ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 226.88ms (± 0.14%) 228.64ms (± 0.16%) +1.76ms (+ 0.78%) 226.96ms 234.68ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@robpalme
Copy link

robpalme commented Jul 1, 2024

Wow that's even better. On Node the mean averages for tsc are now:

  • a total time win of 16.0% (ranging 10.5% to 23.7%)
  • a checker time win of 20.8% (ranging 16.6% to 26.8%)
  • a memory cost of 2.7% (ranging 1.9% to 4.4%)

...which is an exchange rate of 5.8 on total time, and 7.6 on checker time.

@DanielRosenwasser DanielRosenwasser mentioned this pull request Jul 3, 2024
Open
@dragomirtitian dragomirtitian marked this pull request as ready for review July 4, 2024 05:08
@typescript-bot
Copy link
Collaborator

This PR doesn't have any linked issues. Please open an issue that references this PR. From there we can discuss and prioritise.

@dragomirtitian dragomirtitian marked this pull request as draft July 4, 2024 05:09
This was referenced Jul 8, 2024
Draft
Draft
Draft
@dragomirtitian
Copy link
Contributor Author

Conclusions from breaking down where the performance win comes from:

AST Nodes:
- total time win: 3.3% - 9%
- Check Time win: 3.7% - 11%
- Mem cost: ~0-1%

Types:
- total time win: 4.1% - 20%
- Check Time win: 5.3% - 22.5%
- Mem cost: 0.75%-1.2%

Signatures:
- total time win: 0.45% - 1.72%
- Check Time win: 0.56% - 2.04%
- Mem cost: ~0%

The AST nodes win is smaller than I remember in the first experiments. This is probably due to the reduction in memory which meant some relatively common properties are now in data not in Node.

@dragomirtitian dragomirtitian mentioned this pull request Jul 9, 2024
Open
This was referenced Jul 11, 2024
Closed
Open
dmichon-msft
dmichon-msft reviewed Aug 1, 2024
View reviewed changes
src/compiler/utilities.ts
Comment on lines +9144 to +9146
get sourceText() {
return this.data?.sourceText;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might look a bit silly, but if any of these accessors are a particular hot path, you can make an accessor monomorphic by having a switch/case on this.kind and having several branches that all read e.g. this.data.sourceText but behind different case headers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried that. It did not improve performance in my experiment. And I tried several different versions.. switch, if - linear search, if-binary search. All performed worse.

@DanielRosenwasser DanielRosenwasser mentioned this pull request Sep 9, 2024
Open
@rbuckton rbuckton mentioned this pull request Sep 17, 2024
Draft
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dmichon-msft dmichon-msft dmichon-msft left review comments

@jakebailey jakebailey jakebailey left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@dragomirtitian @jakebailey @typescript-bot @Andarist @MichaelMitchell-at @Jarred-Sumner @robpalme @dmichon-msft