Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Defer type comparability check for assertions #53261

Merged
merged 12 commits into from
Mar 23, 2023
Merged

Defer type comparability check for assertions #53261

merged 12 commits into from
Mar 23, 2023

Conversation

gabritto
Copy link
Member

@gabritto gabritto commented Mar 14, 2023 •
edited
Loading

Fixes #52813.

This is the fix that is implemented as deferring the type comparability check we do for assertions, and it fixes the circularity case in the original issue.
However, we can still trigger circularity errors when computing variance, as one of the new tests shows, because we can't always defer the check that makes us compute variances.

I'll attempt a different fix next that involves "resetting" the type resolution stack when we start to compute variances.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Mar 14, 2023
@gabritto
Copy link
Member Author

@typescript-bot perf test this
@typescript-bot run DT
@typescript-bot test this
@typescript-bot user test this
@typescript-bot test top100

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 14, 2023 •
edited
Loading

Heya @gabritto, I'm starting to run the parallelized Definitely Typed test suite on this PR at 46c61b3. Hold tight - I'll update this comment with the log link once the build has been queued.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 14, 2023 •
edited
Loading

Heya @gabritto, I'm starting to run the diff-based user code test suite on this PR at 46c61b3. Hold tight - I'll update this comment with the log link once the build has been queued.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

Heya @gabritto, I'm starting to run the extended test suite on this PR at 46c61b3. Hold tight - I'll update this comment with the log link once the build has been queued.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 14, 2023 •
edited
Loading

Heya @gabritto, I'm starting to run the diff-based top-repos suite on this PR at 46c61b3. Hold tight - I'll update this comment with the log link once the build has been queued.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 14, 2023 •
edited
Loading

Heya @gabritto, I'm starting to run the perf test suite on this PR at 46c61b3. Hold tight - I'll update this comment with the log link once the build has been queued.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the user test suite comparing main and refs/pull/53261/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

Heya @gabritto, I've run the RWC suite on this PR - assuming you're on the TS core team, you can view the resulting diff here.

@typescript-bot
Copy link
Collaborator

@gabritto
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..53261
Metric main 53261 Delta Best Worst p-value
Angular - node (v18.10.0, x64)
Memory used 362,142k (± 0.01%) 362,132k (± 0.01%) ~ 362,099k 362,181k p=0.470 n=6
Parse Time 3.38s (± 0.52%) 3.38s (± 0.16%) ~ 3.38s 3.39s p=0.508 n=6
Bind Time 1.11s (± 0.37%) 1.12s (± 0.98%) ~ 1.11s 1.14s p=0.100 n=6
Check Time 8.63s (± 0.47%) 8.66s (± 0.43%) ~ 8.62s 8.71s p=0.420 n=6
Emit Time 7.40s (± 0.36%) 7.43s (± 0.56%) ~ 7.38s 7.49s p=0.225 n=6
Total Time 20.52s (± 0.27%) 20.60s (± 0.35%) ~ 20.51s 20.72s p=0.109 n=6
Compiler-Unions - node (v18.10.0, x64)
Memory used 192,877k (± 1.52%) 190,757k (± 0.02%) -2,120k (- 1.10%) 190,714k 190,797k p=0.005 n=6
Parse Time 1.51s (± 0.68%) 1.52s (± 1.38%) ~ 1.50s 1.55s p=0.263 n=6
Bind Time 0.77s (± 0.53%) 0.77s (± 0.67%) ~ 0.77s 0.78s p=0.114 n=6
Check Time 9.43s (± 0.41%) 9.39s (± 0.55%) ~ 9.33s 9.46s p=0.228 n=6
Emit Time 2.73s (± 0.92%) 2.79s (± 1.06%) +0.06s (+ 2.32%) 2.74s 2.82s p=0.012 n=6
Total Time 14.43s (± 0.36%) 14.47s (± 0.59%) ~ 14.40s 14.60s p=0.419 n=6
Monaco - node (v18.10.0, x64)
Memory used 347,450k (± 0.01%) 347,524k (± 0.01%) +74k (+ 0.02%) 347,489k 347,543k p=0.008 n=6
Parse Time 2.60s (± 0.62%) 2.61s (± 0.75%) ~ 2.58s 2.64s p=0.676 n=6
Bind Time 1.00s (± 0.81%) 1.01s (± 1.02%) ~ 1.00s 1.03s p=0.070 n=6
Check Time 7.07s (± 0.49%) 7.10s (± 0.62%) ~ 7.02s 7.14s p=0.295 n=6
Emit Time 4.23s (± 0.97%) 4.25s (± 0.60%) ~ 4.21s 4.28s p=0.221 n=6
Total Time 14.90s (± 0.50%) 14.97s (± 0.34%) ~ 14.90s 15.05s p=0.148 n=6
TFS - node (v18.10.0, x64)
Memory used 300,806k (± 0.01%) 300,802k (± 0.01%) ~ 300,769k 300,822k p=0.873 n=6
Parse Time 2.06s (± 1.39%) 2.07s (± 1.06%) ~ 2.03s 2.09s p=0.737 n=6
Bind Time 1.13s (± 0.48%) 1.14s (± 1.20%) ~ 1.13s 1.17s p=0.138 n=6
Check Time 6.53s (± 0.55%) 6.50s (± 0.58%) ~ 6.43s 6.53s p=0.335 n=6
Emit Time 3.85s (± 0.54%) 3.85s (± 0.75%) ~ 3.81s 3.87s p=0.935 n=6
Total Time 13.57s (± 0.45%) 13.56s (± 0.39%) ~ 13.50s 13.63s p=0.810 n=6
material-ui - node (v18.10.0, x64)
Memory used 478,385k (± 0.01%) 478,374k (± 0.01%) ~ 478,322k 478,427k p=0.689 n=6
Parse Time 3.04s (± 1.97%) 2.96s (± 2.01%) ~ 2.90s 3.07s p=0.106 n=6
Bind Time 0.94s (± 7.39%) 1.03s (± 6.28%) ~ 0.90s 1.06s p=0.271 n=6
Check Time 17.09s (± 0.46%) 17.02s (± 0.65%) ~ 16.89s 17.16s p=0.297 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.07s (± 0.35%) 21.02s (± 0.61%) ~ 20.87s 21.19s p=0.378 n=6
xstate - node (v18.10.0, x64)
Memory used 553,201k (± 0.03%) 556,483k (± 0.03%) +3,281k (+ 0.59%) 556,296k 556,723k p=0.005 n=6
Parse Time 3.77s (± 0.74%) 3.78s (± 0.78%) ~ 3.73s 3.81s p=0.419 n=6
Bind Time 1.68s (± 1.21%) 1.68s (± 0.82%) ~ 1.67s 1.70s p=0.871 n=6
Check Time 2.77s (± 0.64%) 3.13s (± 0.91%) +0.37s (+13.26%) 3.10s 3.16s p=0.005 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=1.000 n=6
Total Time 8.29s (± 0.28%) 8.68s (± 0.55%) +0.38s (+ 4.60%) 8.59s 8.72s p=0.005 n=6
Angular - node (v16.17.1, x64)
Memory used 361,458k (± 0.01%) 361,427k (± 0.01%) ~ 361,377k 361,464k p=0.066 n=6
Parse Time 3.51s (± 0.58%) 3.51s (± 0.39%) ~ 3.49s 3.53s p=0.743 n=6
Bind Time 1.18s (± 0.69%) 1.18s (± 0.35%) ~ 1.17s 1.18s p=0.584 n=6
Check Time 9.46s (± 0.54%) 9.45s (± 0.22%) ~ 9.42s 9.48s p=0.565 n=6
Emit Time 7.91s (± 0.80%) 7.91s (± 0.39%) ~ 7.88s 7.96s p=0.521 n=6
Total Time 22.05s (± 0.28%) 22.05s (± 0.19%) ~ 22.00s 22.13s p=0.572 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,215k (± 0.76%) 192,505k (± 0.03%) ~ 192,445k 192,597k p=0.066 n=6
Parse Time 1.57s (± 1.25%) 1.57s (± 1.30%) ~ 1.54s 1.60s p=0.682 n=6
Bind Time 0.82s (± 1.25%) 0.82s (± 0.77%) ~ 0.81s 0.83s p=0.654 n=6
Check Time 10.07s (± 0.39%) 10.07s (± 0.43%) ~ 10.01s 10.11s p=0.809 n=6
Emit Time 2.99s (± 0.52%) 3.03s (± 0.98%) +0.04s (+ 1.34%) 2.98s 3.06s p=0.043 n=6
Total Time 15.44s (± 0.26%) 15.49s (± 0.39%) ~ 15.41s 15.57s p=0.172 n=6
Monaco - node (v16.17.1, x64)
Memory used 346,709k (± 0.00%) 346,747k (± 0.01%) +39k (+ 0.01%) 346,711k 346,774k p=0.020 n=6
Parse Time 2.72s (± 0.49%) 2.72s (± 0.28%) ~ 2.71s 2.73s p=1.000 n=6
Bind Time 1.09s (± 0.47%) 1.09s (± 0.00%) ~ 1.09s 1.09s p=0.174 n=6
Check Time 7.74s (± 0.28%) 7.75s (± 0.43%) ~ 7.70s 7.80s p=0.466 n=6
Emit Time 4.46s (± 0.52%) 4.45s (± 0.66%) ~ 4.41s 4.50s p=0.869 n=6
Total Time 15.99s (± 0.31%) 16.01s (± 0.29%) ~ 15.97s 16.09s p=0.331 n=6
TFS - node (v16.17.1, x64)
Memory used 300,126k (± 0.01%) 300,146k (± 0.01%) ~ 300,107k 300,175k p=0.229 n=6
Parse Time 2.17s (± 0.69%) 2.18s (± 0.84%) ~ 2.16s 2.21s p=0.251 n=6
Bind Time 1.23s (± 0.80%) 1.24s (± 1.02%) ~ 1.22s 1.25s p=0.115 n=6
Check Time 7.16s (± 0.29%) 7.18s (± 0.52%) ~ 7.14s 7.24s p=0.332 n=6
Emit Time 4.34s (± 0.74%) 4.33s (± 0.66%) ~ 4.29s 4.37s p=0.629 n=6
Total Time 14.89s (± 0.45%) 14.93s (± 0.56%) ~ 14.85s 15.07s p=0.521 n=6
material-ui - node (v16.17.1, x64)
Memory used 477,697k (± 0.01%) 477,730k (± 0.01%) ~ 477,675k 477,777k p=0.423 n=6
Parse Time 3.22s (± 0.32%) 3.21s (± 0.70%) ~ 3.17s 3.23s p=0.867 n=6
Bind Time 0.96s (± 0.57%) 0.96s (± 1.44%) ~ 0.94s 0.98s p=0.663 n=6
Check Time 18.08s (± 0.46%) 17.98s (± 0.20%) -0.10s (- 0.55%) 17.95s 18.05s p=0.019 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.25s (± 0.38%) 22.15s (± 0.14%) -0.10s (- 0.44%) 22.12s 22.21s p=0.010 n=6
xstate - node (v16.17.1, x64)
Memory used 550,691k (± 0.02%) 553,949k (± 0.01%) +3,258k (+ 0.59%) 553,889k 554,038k p=0.005 n=6
Parse Time 3.94s (± 0.41%) 3.95s (± 0.39%) ~ 3.93s 3.97s p=0.413 n=6
Bind Time 1.80s (± 0.57%) 1.79s (± 0.86%) ~ 1.77s 1.81s p=0.458 n=6
Check Time 3.00s (± 0.49%) 3.40s (± 0.34%) +0.40s (+13.34%) 3.38s 3.41s p=0.005 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=1.000 n=6
Total Time 8.83s (± 0.14%) 9.23s (± 0.22%) +0.40s (+ 4.51%) 9.21s 9.26s p=0.005 n=6
Angular - node (v14.15.1, x64)
Memory used 355,316k (± 0.00%) 355,301k (± 0.01%) ~ 355,276k 355,361k p=0.199 n=6
Parse Time 3.59s (± 0.41%) 3.60s (± 0.38%) ~ 3.57s 3.61s p=0.437 n=6
Bind Time 1.23s (± 0.61%) 1.23s (± 0.61%) ~ 1.22s 1.24s p=0.487 n=6
Check Time 9.73s (± 0.49%) 9.75s (± 0.19%) ~ 9.73s 9.78s p=0.334 n=6
Emit Time 8.29s (± 0.25%) 8.38s (± 1.08%) +0.09s (+ 1.05%) 8.27s 8.54s p=0.043 n=6
Total Time 22.84s (± 0.27%) 22.96s (± 0.41%) +0.12s (+ 0.53%) 22.84s 23.11s p=0.016 n=6
Compiler-Unions - node (v14.15.1, x64)
Memory used 187,929k (± 0.01%) 187,819k (± 0.01%) -110k (- 0.06%) 187,802k 187,847k p=0.005 n=6
Parse Time 1.59s (± 0.26%) 1.60s (± 0.39%) +0.01s (+ 0.52%) 1.59s 1.61s p=0.033 n=6
Bind Time 0.84s (± 0.99%) 0.84s (± 0.61%) ~ 0.84s 0.85s p=0.923 n=6
Check Time 10.15s (± 0.52%) 10.18s (± 0.60%) ~ 10.08s 10.24s p=0.335 n=6
Emit Time 3.11s (± 0.60%) 3.13s (± 0.78%) ~ 3.10s 3.15s p=0.165 n=6
Total Time 15.71s (± 0.32%) 15.75s (± 0.47%) ~ 15.67s 15.83s p=0.377 n=6
Monaco - node (v14.15.1, x64)
Memory used 341,684k (± 0.01%) 341,687k (± 0.01%) ~ 341,655k 341,733k p=0.873 n=6
Parse Time 2.82s (± 0.22%) 2.83s (± 0.36%) +0.01s (+ 0.47%) 2.82s 2.85s p=0.028 n=6
Bind Time 1.09s (± 0.47%) 1.09s (± 0.50%) ~ 1.09s 1.10s p=0.640 n=6
Check Time 8.12s (± 0.69%) 8.16s (± 0.24%) ~ 8.13s 8.18s p=0.072 n=6
Emit Time 4.66s (± 0.40%) 4.66s (± 0.34%) ~ 4.63s 4.67s p=0.366 n=6
Total Time 16.69s (± 0.45%) 16.75s (± 0.22%) ~ 16.70s 16.80s p=0.077 n=6
TFS - node (v14.15.1, x64)
Memory used 295,230k (± 0.00%) 295,261k (± 0.00%) +31k (+ 0.01%) 295,252k 295,271k p=0.005 n=6
Parse Time 2.38s (± 0.35%) 2.40s (± 0.59%) +0.02s (+ 1.05%) 2.38s 2.42s p=0.011 n=6
Bind Time 1.06s (± 0.38%) 1.07s (± 0.51%) ~ 1.06s 1.07s p=0.282 n=6
Check Time 7.47s (± 0.41%) 7.48s (± 0.41%) ~ 7.43s 7.52s p=0.518 n=6
Emit Time 4.27s (± 0.77%) 4.30s (± 0.93%) ~ 4.27s 4.37s p=0.124 n=6
Total Time 15.18s (± 0.21%) 15.25s (± 0.40%) ~ 15.16s 15.32s p=0.052 n=6
material-ui - node (v14.15.1, x64)
Memory used 473,296k (± 0.01%) 473,281k (± 0.00%) ~ 473,247k 473,306k p=0.574 n=6
Parse Time 3.33s (± 0.23%) 3.34s (± 0.57%) ~ 3.32s 3.37s p=0.282 n=6
Bind Time 1.00s (± 0.75%) 1.00s (± 0.75%) ~ 0.99s 1.01s p=1.000 n=6
Check Time 18.99s (± 0.49%) 19.01s (± 0.60%) ~ 18.84s 19.15s p=0.810 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 23.32s (± 0.40%) 23.35s (± 0.47%) ~ 23.19s 23.47s p=0.630 n=6
xstate - node (v14.15.1, x64)
Memory used 539,308k (± 0.01%) 542,544k (± 0.00%) +3,236k (+ 0.60%) 542,524k 542,587k p=0.005 n=6
Parse Time 4.24s (± 0.47%) 4.24s (± 0.29%) ~ 4.23s 4.26s p=0.806 n=6
Bind Time 1.66s (± 0.62%) 1.67s (± 0.31%) ~ 1.66s 1.67s p=0.437 n=6
Check Time 3.16s (± 0.99%) 3.54s (± 0.75%) +0.37s (+11.80%) 3.51s 3.57s p=0.005 n=6
Emit Time 0.09s (± 0.00%) 0.10s (± 5.76%) ~ 0.09s 0.10s p=0.071 n=6
Total Time 9.16s (± 0.24%) 9.54s (± 0.34%) +0.38s (+ 4.09%) 9.50s 9.58s p=0.005 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 53261 6
Baseline main 6

TSServer

Comparison Report - main..53261
Metric main 53261 Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,384ms (± 0.59%) 2,388ms (± 0.36%) ~ 2,377ms 2,403ms p=0.810 n=6
Req 2 - geterr 5,321ms (± 0.17%) 5,402ms (± 0.42%) +81ms (+ 1.53%) 5,378ms 5,434ms p=0.005 n=6
Req 3 - references 336ms (± 1.17%) 337ms (± 0.73%) ~ 335ms 340ms p=0.284 n=6
Req 4 - navto 279ms (± 0.27%) 277ms (± 0.38%) -3ms (- 0.96%) 275ms 278ms p=0.006 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 81ms (± 3.06%) 85ms (± 1.37%) +4ms (+ 4.93%) 83ms 86ms p=0.008 n=6
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,523ms (± 1.04%) 2,512ms (± 1.01%) ~ 2,470ms 2,548ms p=0.575 n=6
Req 2 - geterr 4,007ms (± 0.75%) 3,980ms (± 0.58%) ~ 3,954ms 4,013ms p=0.173 n=6
Req 3 - references 351ms (± 0.51%) 345ms (± 0.56%) -6ms (- 1.76%) 342ms 347ms p=0.005 n=6
Req 4 - navto 291ms (± 1.42%) 291ms (± 0.60%) ~ 288ms 292ms p=1.000 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 81ms (± 7.91%) 71ms (± 3.45%) 🟩-10ms (-12.16%) 69ms 74ms p=0.019 n=6
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,022ms (± 0.43%) 3,036ms (± 0.50%) ~ 3,018ms 3,060ms p=0.149 n=6
Req 2 - geterr 1,597ms (± 0.90%) 1,726ms (± 1.13%) +129ms (+ 8.09%) 1,703ms 1,749ms p=0.005 n=6
Req 3 - references 107ms (± 1.15%) 108ms (± 1.97%) ~ 105ms 110ms p=0.163 n=6
Req 4 - navto 356ms (± 0.59%) 354ms (± 0.47%) ~ 351ms 356ms p=0.098 n=6
Req 5 - completionInfo count 2,862 (± 0.00%) 2,862 (± 0.00%) ~ 2,862 2,862 p=1.000 n=6
Req 5 - completionInfo 385ms (± 1.25%) 385ms (± 1.31%) ~ 378ms 390ms p=1.000 n=6
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,507ms (± 0.44%) 2,508ms (± 0.77%) ~ 2,496ms 2,545ms p=0.470 n=6
Req 2 - geterr 5,740ms (± 0.49%) 5,788ms (± 0.31%) +49ms (+ 0.85%) 5,771ms 5,815ms p=0.013 n=6
Req 3 - references 347ms (± 1.01%) 356ms (± 1.38%) +9ms (+ 2.69%) 348ms 362ms p=0.010 n=6
Req 4 - navto 278ms (± 0.84%) 279ms (± 0.35%) ~ 278ms 280ms p=0.137 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 73ms (± 0.70%) 75ms (± 1.08%) +2ms (+ 2.73%) 74ms 76ms p=0.006 n=6
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,672ms (± 0.50%) 2,690ms (± 0.99%) ~ 2,672ms 2,743ms p=0.148 n=6
Req 2 - geterr 4,355ms (± 0.42%) 4,335ms (± 0.50%) ~ 4,303ms 4,352ms p=0.128 n=6
Req 3 - references 360ms (± 1.68%) 363ms (± 0.99%) ~ 360ms 370ms p=0.228 n=6
Req 4 - navto 290ms (± 0.52%) 288ms (± 0.47%) ~ 286ms 290ms p=0.210 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 78ms (± 3.07%) 76ms (± 3.46%) ~ 73ms 81ms p=0.198 n=6
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,168ms (± 0.33%) 3,164ms (± 0.29%) ~ 3,156ms 3,178ms p=0.571 n=6
Req 2 - geterr 1,752ms (± 0.86%) 1,884ms (± 0.90%) +132ms (+ 7.54%) 1,868ms 1,907ms p=0.005 n=6
Req 3 - references 117ms (± 1.08%) 116ms (± 1.60%) ~ 114ms 118ms p=0.557 n=6
Req 4 - navto 341ms (± 0.64%) 340ms (± 0.42%) ~ 338ms 342ms p=0.462 n=6
Req 5 - completionInfo count 2,862 (± 0.00%) 2,862 (± 0.00%) ~ 2,862 2,862 p=1.000 n=6
Req 5 - completionInfo 403ms (± 5.19%) 391ms (± 0.46%) ~ 388ms 393ms p=0.053 n=6
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,597ms (± 0.36%) 2,604ms (± 0.18%) ~ 2,596ms 2,608ms p=0.229 n=6
Req 2 - geterr 6,137ms (± 0.35%) 6,277ms (± 1.10%) +140ms (+ 2.28%) 6,199ms 6,378ms p=0.005 n=6
Req 3 - references 368ms (± 0.48%) 367ms (± 1.59%) ~ 359ms 374ms p=1.000 n=6
Req 4 - navto 280ms (± 0.98%) 281ms (± 0.89%) ~ 278ms 284ms p=0.567 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 101ms (± 2.24%) 97ms (± 6.27%) ~ 89ms 101ms p=0.122 n=6
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,810ms (± 0.33%) 2,808ms (± 0.24%) ~ 2,802ms 2,819ms p=0.936 n=6
Req 2 - geterr 4,450ms (± 0.49%) 4,518ms (± 2.56%) ~ 4,401ms 4,646ms p=0.936 n=6
Req 3 - references 419ms (± 5.16%) 387ms (± 6.65%) 🟩-32ms (- 7.68%) 367ms 421ms p=0.013 n=6
Req 4 - navto 288ms (± 1.45%) 288ms (± 0.72%) ~ 285ms 290ms p=0.451 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 95ms (± 6.37%) 91ms (±12.09%) ~ 81ms 104ms p=0.686 n=6
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,443ms (± 0.23%) 3,455ms (± 0.52%) ~ 3,439ms 3,489ms p=0.093 n=6
Req 2 - geterr 1,837ms (± 0.81%) 1,850ms (± 0.82%) ~ 1,833ms 1,870ms p=0.378 n=6
Req 3 - references 126ms (± 1.12%) 125ms (± 2.42%) ~ 122ms 130ms p=0.167 n=6
Req 4 - navto 369ms (± 0.20%) 370ms (± 0.68%) ~ 368ms 375ms p=0.241 n=6
Req 5 - completionInfo count 2,862 (± 0.00%) 2,862 (± 0.00%) ~ 2,862 2,862 p=1.000 n=6
Req 5 - completionInfo 411ms (± 0.95%) 412ms (± 1.30%) ~ 407ms 420ms p=0.809 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 53261 6
Baseline main 6

Startup

Comparison Report - main..53261
Metric main 53261 Delta Best Worst p-value
tsc-startup - node (v16.17.1, x64)
Execution time 141.08ms (± 0.17%) 141.78ms (± 0.23%) +0.69ms (+ 0.49%) 140.89ms 146.98ms p=0.000 n=600
tsserver-startup - node (v16.17.1, x64)
Execution time 225.76ms (± 0.13%) 227.52ms (± 0.25%) +1.75ms (+ 0.78%) 226.24ms 236.58ms p=0.000 n=600
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 227.84ms (± 0.15%) 230.80ms (± 0.38%) +2.96ms (+ 1.30%) 227.92ms 236.39ms p=0.000 n=600
typescript-startup - node (v16.17.1, x64)
Execution time 208.69ms (± 0.13%) 212.68ms (± 0.33%) +3.98ms (+ 1.91%) 208.71ms 218.49ms p=0.000 n=600
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 53261 6
Baseline main 6

Developer Information:

Download Benchmark

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the top-repos suite comparing main and refs/pull/53261/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

Hey @gabritto, the results of running the DT tests are ready.
Everything looks the same!
You can check the log here.

@gabritto
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 16, 2023 •
edited
Loading

Heya @gabritto, I've started to run the abridged perf test suite on this PR at 45124b1. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@gabritto
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..53261

Metric main 53261 Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 361,346k (± 0.01%) 361,323k (± 0.01%) ~ 361,296k 361,362k p=0.173 n=6
Parse Time 3.52s (± 0.36%) 3.54s (± 0.65%) ~ 3.51s 3.56s p=0.182 n=6
Bind Time 1.18s (± 0.44%) 1.18s (± 0.44%) ~ 1.18s 1.19s p=1.000 n=6
Check Time 9.46s (± 0.50%) 9.46s (± 0.53%) ~ 9.40s 9.54s p=1.000 n=6
Emit Time 7.93s (± 0.48%) 7.93s (± 0.88%) ~ 7.87s 8.04s p=1.000 n=6
Total Time 22.09s (± 0.39%) 22.12s (± 0.51%) ~ 22.01s 22.28s p=1.000 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 192,575k (± 0.02%) 192,493k (± 0.04%) ~ 192,406k 192,627k p=0.066 n=6
Parse Time 1.58s (± 1.41%) 1.59s (± 1.42%) ~ 1.55s 1.61s p=0.743 n=6
Bind Time 0.83s (± 1.01%) 0.82s (± 0.63%) ~ 0.82s 0.83s p=0.923 n=6
Check Time 10.07s (± 0.88%) 10.11s (± 0.54%) ~ 10.04s 10.19s p=0.296 n=6
Emit Time 3.00s (± 0.63%) 3.04s (± 1.20%) +0.04s (+ 1.39%) 3.01s 3.11s p=0.024 n=6
Total Time 15.47s (± 0.42%) 15.56s (± 0.26%) +0.09s (+ 0.60%) 15.49s 15.60s p=0.024 n=6
Monaco - node (v16.17.1, x64)
Memory used 346,709k (± 0.00%) 346,708k (± 0.01%) ~ 346,688k 346,739k p=0.630 n=6
Parse Time 2.73s (± 0.43%) 2.73s (± 0.59%) ~ 2.71s 2.75s p=0.805 n=6
Bind Time 1.09s (± 0.90%) 1.09s (± 0.90%) ~ 1.08s 1.11s p=1.000 n=6
Check Time 7.74s (± 0.45%) 7.73s (± 0.39%) ~ 7.69s 7.77s p=0.571 n=6
Emit Time 4.46s (± 0.39%) 4.44s (± 0.54%) ~ 4.40s 4.46s p=0.366 n=6
Total Time 16.02s (± 0.38%) 15.98s (± 0.33%) ~ 15.92s 16.07s p=0.468 n=6
TFS - node (v16.17.1, x64)
Memory used 300,137k (± 0.01%) 300,151k (± 0.01%) ~ 300,076k 300,190k p=0.471 n=6
Parse Time 2.17s (± 0.54%) 2.17s (± 0.38%) ~ 2.16s 2.18s p=0.738 n=6
Bind Time 1.24s (± 0.97%) 1.24s (± 0.66%) ~ 1.23s 1.25s p=1.000 n=6
Check Time 7.20s (± 0.38%) 7.18s (± 0.47%) ~ 7.15s 7.24s p=0.466 n=6
Emit Time 4.33s (± 0.68%) 4.34s (± 0.60%) ~ 4.30s 4.37s p=0.629 n=6
Total Time 14.94s (± 0.31%) 14.94s (± 0.33%) ~ 14.88s 15.02s p=1.000 n=6
material-ui - node (v16.17.1, x64)
Memory used 477,719k (± 0.02%) 477,710k (± 0.00%) ~ 477,691k 477,726k p=0.423 n=6
Parse Time 3.21s (± 0.38%) 3.21s (± 0.61%) ~ 3.19s 3.24s p=0.806 n=6
Bind Time 0.96s (± 1.02%) 0.96s (± 0.43%) ~ 0.95s 0.96s p=0.673 n=6
Check Time 18.10s (± 0.62%) 18.07s (± 0.62%) ~ 17.91s 18.22s p=1.000 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.28s (± 0.51%) 22.24s (± 0.57%) ~ 22.06s 22.41s p=0.748 n=6
xstate - node (v16.17.1, x64)
Memory used 550,722k (± 0.02%) 553,975k (± 0.02%) +3,253k (+ 0.59%) 553,862k 554,106k p=0.005 n=6
Parse Time 3.94s (± 0.31%) 3.96s (± 0.31%) +0.03s (+ 0.72%) 3.95s 3.98s p=0.018 n=6
Bind Time 1.79s (± 0.23%) 1.79s (± 0.45%) ~ 1.78s 1.80s p=0.584 n=6
Check Time 3.05s (± 1.25%) 3.39s (± 0.51%) +0.33s (+10.97%) 3.37s 3.41s p=0.005 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=1.000 n=6
Total Time 8.87s (± 0.41%) 9.25s (± 0.23%) +0.37s (+ 4.23%) 9.22s 9.27s p=0.005 n=6

System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current 53261 6
Baseline main 6

Developer Information:

Download Benchmark

@gabritto
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 16, 2023 •
edited
Loading

Heya @gabritto, I've started to run the abridged perf test suite on this PR at d409b7f. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@gabritto
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..53261

Metric main 53261 Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 360,900k (± 0.01%) 360,870k (± 0.00%) ~ 360,851k 360,890k p=0.173 n=6
Parse Time 3.51s (± 0.61%) 3.53s (± 0.47%) ~ 3.50s 3.55s p=0.249 n=6
Bind Time 1.18s (± 0.64%) 1.18s (± 0.44%) ~ 1.18s 1.19s p=0.784 n=6
Check Time 9.46s (± 0.44%) 9.43s (± 0.19%) ~ 9.41s 9.46s p=0.092 n=6
Emit Time 7.93s (± 0.92%) 7.89s (± 0.35%) ~ 7.87s 7.93s p=0.517 n=6
Total Time 22.09s (± 0.37%) 22.03s (± 0.21%) ~ 21.96s 22.08s p=0.227 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,106k (± 0.69%) 192,518k (± 0.03%) ~ 192,468k 192,624k p=0.173 n=6
Parse Time 1.57s (± 1.47%) 1.58s (± 0.53%) ~ 1.58s 1.60s p=0.114 n=6
Bind Time 0.82s (± 0.00%) 0.82s (± 0.50%) ~ 0.82s 0.83s p=0.405 n=6
Check Time 10.04s (± 0.51%) 10.29s (± 0.63%) +0.26s (+ 2.56%) 10.22s 10.39s p=0.005 n=6
Emit Time 2.97s (± 0.98%) 3.00s (± 1.25%) ~ 2.96s 3.07s p=0.196 n=6
Total Time 15.40s (± 0.52%) 15.70s (± 0.38%) +0.30s (+ 1.98%) 15.63s 15.80s p=0.005 n=6
Monaco - node (v16.17.1, x64)
Memory used 345,435k (± 0.01%) 345,587k (± 0.00%) +153k (+ 0.04%) 345,563k 345,607k p=0.005 n=6
Parse Time 2.73s (± 0.50%) 2.72s (± 0.43%) ~ 2.71s 2.74s p=0.161 n=6
Bind Time 1.09s (± 0.75%) 1.09s (± 0.58%) ~ 1.08s 1.10s p=0.599 n=6
Check Time 7.67s (± 0.27%) 7.72s (± 0.34%) +0.05s (+ 0.59%) 7.68s 7.74s p=0.016 n=6
Emit Time 4.44s (± 0.74%) 4.48s (± 0.58%) ~ 4.44s 4.50s p=0.076 n=6
Total Time 15.94s (± 0.34%) 16.01s (± 0.32%) +0.07s (+ 0.44%) 15.94s 16.07s p=0.036 n=6
TFS - node (v16.17.1, x64)
Memory used 299,780k (± 0.00%) 300,015k (± 0.01%) +236k (+ 0.08%) 299,987k 300,035k p=0.005 n=6
Parse Time 2.18s (± 0.75%) 2.17s (± 0.54%) ~ 2.15s 2.18s p=0.171 n=6
Bind Time 1.24s (± 0.41%) 1.24s (± 1.10%) ~ 1.22s 1.25s p=0.541 n=6
Check Time 7.20s (± 0.56%) 7.15s (± 0.42%) -0.05s (- 0.74%) 7.12s 7.20s p=0.035 n=6
Emit Time 4.35s (± 0.69%) 4.35s (± 0.71%) ~ 4.30s 4.38s p=1.000 n=6
Total Time 14.97s (± 0.53%) 14.91s (± 0.39%) ~ 14.79s 14.96s p=0.108 n=6
material-ui - node (v16.17.1, x64)
Memory used 477,434k (± 0.01%) 477,438k (± 0.01%) ~ 477,337k 477,538k p=0.810 n=6
Parse Time 3.23s (± 0.32%) 3.22s (± 0.36%) ~ 3.20s 3.23s p=0.345 n=6
Bind Time 0.96s (± 0.54%) 0.96s (± 0.54%) ~ 0.95s 0.96s p=1.000 n=6
Check Time 18.08s (± 0.53%) 18.09s (± 0.71%) ~ 17.99s 18.29s p=0.873 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.26s (± 0.45%) 22.27s (± 0.60%) ~ 22.17s 22.47s p=0.810 n=6
xstate - node (v16.17.1, x64)
Memory used 550,637k (± 0.01%) 550,706k (± 0.01%) ~ 550,630k 550,789k p=0.066 n=6
Parse Time 3.95s (± 0.34%) 3.95s (± 0.25%) ~ 3.94s 3.96s p=0.928 n=6
Bind Time 1.80s (± 0.50%) 1.80s (± 0.82%) ~ 1.78s 1.82s p=0.804 n=6
Check Time 3.04s (± 1.14%) 3.09s (± 0.65%) +0.05s (+ 1.64%) 3.07s 3.12s p=0.030 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=1.000 n=6
Total Time 8.88s (± 0.44%) 8.94s (± 0.25%) +0.06s (+ 0.68%) 8.92s 8.97s p=0.009 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current 53261 6
Baseline main 6

Developer Information:

Download Benchmark

@gabritto
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 17, 2023 •
edited
Loading

Heya @gabritto, I've started to run the abridged perf test suite on this PR at 469f24d. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 20, 2023 •
edited
Loading

Heya @gabritto, I've started to run the diff-based top-repos suite (tsserver) on this PR at 7620056. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 20, 2023 •
edited
Loading

Heya @gabritto, I've started to run the perf test suite on this PR at 7620056. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 20, 2023 •
edited
Loading

Heya @gabritto, I've started to run the parallelized Definitely Typed test suite on this PR at 7620056. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 20, 2023 •
edited
Loading

Heya @gabritto, I've started to run the diff-based user code test suite on this PR at 7620056. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 20, 2023 •
edited
Loading

Heya @gabritto, I've started to run the diff-based top-repos suite on this PR at 7620056. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the user test suite comparing main and refs/pull/53261/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

Heya @gabritto, I've run the RWC suite on this PR - assuming you're on the TS core team, you can view the resulting diff here.

@typescript-bot
Copy link
Collaborator

@gabritto
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..53261
Metric main 53261 Delta Best Worst p-value
Angular - node (v18.10.0, x64)
Memory used 361,706k (± 0.01%) 361,853k (± 0.01%) +147k (+ 0.04%) 361,817k 361,884k p=0.005 n=6
Parse Time 3.38s (± 0.48%) 3.39s (± 0.51%) ~ 3.37s 3.41s p=0.673 n=6
Bind Time 1.12s (± 0.67%) 1.12s (± 0.49%) ~ 1.11s 1.12s p=0.137 n=6
Check Time 8.66s (± 0.42%) 8.71s (± 0.34%) +0.04s (+ 0.50%) 8.67s 8.73s p=0.043 n=6
Emit Time 7.48s (± 0.76%) 7.47s (± 0.78%) ~ 7.38s 7.54s p=0.936 n=6
Total Time 20.64s (± 0.34%) 20.68s (± 0.43%) ~ 20.57s 20.78s p=0.422 n=6
Compiler-Unions - node (v18.10.0, x64)
Memory used 193,702k (± 1.57%) 0k ~ 0k 0k p=1.000 n=6+0
Parse Time 1.51s (± 1.00%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Bind Time 0.77s (± 1.42%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Check Time 9.43s (± 0.92%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Emit Time 2.73s (± 0.60%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Total Time 14.44s (± 0.73%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Monaco - node (v18.10.0, x64)
Memory used 346,257k (± 0.01%) 346,389k (± 0.01%) +132k (+ 0.04%) 346,358k 346,411k p=0.005 n=6
Parse Time 2.60s (± 0.75%) 2.58s (± 0.90%) ~ 2.56s 2.62s p=0.103 n=6
Bind Time 1.01s (± 0.81%) 1.01s (± 0.81%) ~ 1.00s 1.02s p=1.000 n=6
Check Time 7.07s (± 0.54%) 7.05s (± 0.24%) ~ 7.03s 7.07s p=0.171 n=6
Emit Time 4.25s (± 1.07%) 4.25s (± 1.03%) ~ 4.20s 4.30s p=1.000 n=6
Total Time 14.93s (± 0.55%) 14.88s (± 0.51%) ~ 14.78s 14.96s p=0.293 n=6
TFS - node (v18.10.0, x64)
Memory used 300,462k (± 0.01%) 300,529k (± 0.00%) +67k (+ 0.02%) 300,510k 300,552k p=0.005 n=6
Parse Time 2.05s (± 0.57%) 2.06s (± 0.84%) ~ 2.03s 2.08s p=0.254 n=6
Bind Time 1.14s (± 0.90%) 1.14s (± 0.71%) ~ 1.14s 1.16s p=1.000 n=6
Check Time 6.50s (± 0.32%) 6.54s (± 0.44%) ~ 6.49s 6.57s p=0.053 n=6
Emit Time 3.88s (± 1.30%) 3.87s (± 0.63%) ~ 3.83s 3.89s p=0.871 n=6
Total Time 13.56s (± 0.32%) 13.60s (± 0.27%) ~ 13.54s 13.64s p=0.169 n=6
material-ui - node (v18.10.0, x64)
Memory used 477,134k (± 0.01%) 477,185k (± 0.01%) ~ 477,160k 477,231k p=0.128 n=6
Parse Time 3.02s (± 3.20%) 3.03s (± 1.38%) ~ 2.96s 3.07s p=1.000 n=6
Bind Time 0.99s (± 8.59%) 0.94s (± 6.92%) ~ 0.90s 1.07s p=0.413 n=6
Check Time 17.01s (± 0.75%) 16.89s (± 0.48%) ~ 16.78s 17.00s p=0.107 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.01s (± 0.65%) 20.86s (± 0.27%) ~ 20.81s 20.96s p=0.065 n=6
xstate - node (v18.10.0, x64)
Memory used 552,916k (± 0.01%) 552,964k (± 0.03%) ~ 552,801k 553,138k p=0.936 n=6
Parse Time 3.76s (± 0.58%) 3.75s (± 0.45%) ~ 3.73s 3.77s p=0.373 n=6
Bind Time 1.70s (± 0.30%) 1.70s (± 0.44%) ~ 1.69s 1.71s p=0.241 n=6
Check Time 2.75s (± 1.14%) 2.81s (± 0.32%) +0.06s (+ 2.06%) 2.80s 2.82s p=0.019 n=6
Emit Time 0.08s (± 4.99%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=0.405 n=6
Total Time 8.31s (± 0.46%) 8.35s (± 0.10%) ~ 8.34s 8.36s p=0.115 n=6
Angular - node (v16.17.1, x64)
Memory used 361,045k (± 0.01%) 361,190k (± 0.01%) +146k (+ 0.04%) 361,149k 361,248k p=0.005 n=6
Parse Time 3.52s (± 0.78%) 3.52s (± 0.88%) ~ 3.49s 3.56s p=0.809 n=6
Bind Time 1.19s (± 0.69%) 1.18s (± 0.35%) ~ 1.18s 1.19s p=0.248 n=6
Check Time 9.47s (± 0.48%) 9.53s (± 0.29%) +0.06s (+ 0.62%) 9.48s 9.56s p=0.036 n=6
Emit Time 7.93s (± 0.71%) 7.91s (± 0.10%) ~ 7.90s 7.92s p=0.625 n=6
Total Time 22.11s (± 0.43%) 22.14s (± 0.23%) ~ 22.08s 22.21s p=0.199 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,656k (± 0.92%) 0k ~ 0k 0k p=1.000 n=6+0
Parse Time 1.60s (± 0.56%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Bind Time 0.82s (± 0.63%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Check Time 10.17s (± 0.90%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Emit Time 3.00s (± 1.11%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Total Time 15.59s (± 0.60%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Monaco - node (v16.17.1, x64)
Memory used 345,573k (± 0.00%) 345,681k (± 0.01%) +108k (+ 0.03%) 345,654k 345,697k p=0.005 n=6
Parse Time 2.73s (± 0.78%) 2.71s (± 0.54%) ~ 2.69s 2.73s p=0.104 n=6
Bind Time 1.10s (± 1.06%) 1.08s (± 0.50%) -0.01s (- 1.21%) 1.08s 1.09s p=0.025 n=6
Check Time 7.71s (± 0.13%) 7.71s (± 0.24%) ~ 7.69s 7.74s p=0.935 n=6
Emit Time 4.47s (± 0.75%) 4.45s (± 0.58%) ~ 4.41s 4.48s p=0.225 n=6
Total Time 16.02s (± 0.19%) 15.96s (± 0.36%) ~ 15.88s 16.04s p=0.077 n=6
TFS - node (v16.17.1, x64)
Memory used 299,785k (± 0.01%) 299,884k (± 0.01%) +99k (+ 0.03%) 299,843k 299,911k p=0.005 n=6
Parse Time 2.16s (± 0.54%) 2.17s (± 0.35%) ~ 2.16s 2.18s p=0.054 n=6
Bind Time 1.23s (± 0.66%) 1.24s (± 0.98%) ~ 1.22s 1.25s p=0.677 n=6
Check Time 7.19s (± 0.56%) 7.19s (± 0.12%) ~ 7.18s 7.20s p=1.000 n=6
Emit Time 4.36s (± 1.21%) 4.33s (± 0.60%) ~ 4.30s 4.37s p=0.466 n=6
Total Time 14.94s (± 0.53%) 14.93s (± 0.24%) ~ 14.88s 14.98s p=1.000 n=6
material-ui - node (v16.17.1, x64)
Memory used 476,430k (± 0.01%) 476,408k (± 0.01%) ~ 476,373k 476,469k p=0.229 n=6
Parse Time 3.22s (± 0.57%) 3.22s (± 0.28%) ~ 3.21s 3.23s p=0.806 n=6
Bind Time 0.95s (± 0.54%) 0.96s (± 0.54%) ~ 0.95s 0.96s p=0.311 n=6
Check Time 17.93s (± 0.42%) 17.98s (± 0.62%) ~ 17.85s 18.15s p=0.470 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.10s (± 0.38%) 22.16s (± 0.51%) ~ 22.04s 22.32s p=0.376 n=6
xstate - node (v16.17.1, x64)
Memory used 550,520k (± 0.01%) 550,512k (± 0.01%) ~ 550,463k 550,572k p=0.936 n=6
Parse Time 3.95s (± 0.35%) 3.94s (± 0.37%) ~ 3.92s 3.95s p=0.465 n=6
Bind Time 1.81s (± 0.54%) 1.79s (± 0.65%) -0.02s (- 0.92%) 1.78s 1.81s p=0.036 n=6
Check Time 2.99s (± 0.68%) 3.04s (± 0.80%) +0.05s (+ 1.62%) 3.01s 3.08s p=0.010 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 4.45%) ~ 0.09s 0.10s p=0.405 n=6
Total Time 8.83s (± 0.38%) 8.86s (± 0.39%) ~ 8.83s 8.92s p=0.225 n=6
Angular - node (v14.15.1, x64)
Memory used 354,915k (± 0.01%) 355,077k (± 0.00%) +161k (+ 0.05%) 355,058k 355,101k p=0.005 n=6
Parse Time 3.61s (± 0.83%) 3.61s (± 1.12%) ~ 3.56s 3.67s p=0.746 n=6
Bind Time 1.24s (± 0.66%) 1.24s (± 0.44%) ~ 1.23s 1.24s p=0.859 n=6
Check Time 9.81s (± 0.47%) 9.93s (± 0.37%) +0.12s (+ 1.17%) 9.88s 9.97s p=0.008 n=6
Emit Time 8.39s (± 1.09%) 8.36s (± 0.52%) ~ 8.30s 8.41s p=0.687 n=6
Total Time 23.06s (± 0.58%) 23.13s (± 0.18%) ~ 23.06s 23.17s p=0.377 n=6
Compiler-Unions - node (v14.15.1, x64)
Memory used 187,777k (± 0.01%) 0k ~ 0k 0k p=1.000 n=6+0
Parse Time 1.60s (± 0.56%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Bind Time 0.85s (± 1.73%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Check Time 10.28s (± 0.54%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Emit Time 3.13s (± 0.56%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Total Time 15.86s (± 0.36%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Monaco - node (v14.15.1, x64)
Memory used 340,548k (± 0.01%) 340,667k (± 0.01%) +119k (+ 0.04%) 340,617k 340,691k p=0.005 n=6
Parse Time 2.83s (± 0.48%) 2.84s (± 0.79%) ~ 2.81s 2.86s p=0.808 n=6
Bind Time 1.10s (± 0.68%) 1.11s (± 1.05%) ~ 1.09s 1.12s p=0.279 n=6
Check Time 8.11s (± 0.26%) 8.13s (± 0.71%) ~ 8.02s 8.19s p=0.226 n=6
Emit Time 4.70s (± 0.58%) 4.67s (± 0.65%) ~ 4.64s 4.72s p=0.090 n=6
Total Time 16.75s (± 0.18%) 16.74s (± 0.54%) ~ 16.59s 16.84s p=1.000 n=6
TFS - node (v14.15.1, x64)
Memory used 294,890k (± 0.00%) 294,998k (± 0.00%) +108k (+ 0.04%) 294,988k 295,009k p=0.005 n=6
Parse Time 2.39s (± 0.57%) 2.39s (± 0.65%) ~ 2.37s 2.41s p=0.805 n=6
Bind Time 1.07s (± 0.78%) 1.07s (± 0.00%) ~ 1.07s 1.07s p=0.121 n=6
Check Time 7.54s (± 0.49%) 7.53s (± 0.32%) ~ 7.49s 7.56s p=0.572 n=6
Emit Time 4.28s (± 0.39%) 4.34s (± 1.21%) ~ 4.26s 4.40s p=0.075 n=6
Total Time 15.27s (± 0.40%) 15.32s (± 0.32%) ~ 15.25s 15.39s p=0.294 n=6
material-ui - node (v14.15.1, x64)
Memory used 471,958k (± 0.01%) 471,994k (± 0.00%) +36k (+ 0.01%) 471,973k 472,018k p=0.045 n=6
Parse Time 3.36s (± 0.44%) 3.35s (± 0.24%) ~ 3.33s 3.35s p=0.087 n=6
Bind Time 1.00s (± 0.83%) 1.00s (± 0.75%) ~ 0.99s 1.01s p=0.432 n=6
Check Time 18.91s (± 0.49%) 18.84s (± 0.62%) ~ 18.74s 19.00s p=0.261 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 23.28s (± 0.34%) 23.19s (± 0.49%) ~ 23.09s 23.36s p=0.199 n=6
xstate - node (v14.15.1, x64)
Memory used 539,126k (± 0.00%) 539,194k (± 0.00%) +69k (+ 0.01%) 539,151k 539,209k p=0.008 n=6
Parse Time 4.22s (± 0.49%) 4.21s (± 0.68%) ~ 4.19s 4.26s p=0.805 n=6
Bind Time 1.67s (± 0.31%) 1.67s (± 0.63%) ~ 1.65s 1.68s p=0.142 n=6
Check Time 3.15s (± 0.68%) 3.16s (± 0.65%) ~ 3.12s 3.18s p=0.517 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=1.000 n=6
Total Time 9.13s (± 0.23%) 9.13s (± 0.45%) ~ 9.07s 9.19s p=0.871 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 53261 6
Baseline main 6

TSServer

Comparison Report - main..53261
Metric main 53261 Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,401ms (± 0.61%) 2,410ms (± 0.26%) ~ 2,400ms 2,417ms p=0.294 n=6
Req 2 - geterr 5,395ms (± 0.49%) 4,719ms (± 0.60%) 🟩-677ms (-12.54%) 4,685ms 4,767ms p=0.005 n=6
Req 3 - references 337ms (± 0.31%) 357ms (± 0.62%) +20ms (+ 5.99%) 355ms 361ms p=0.005 n=6
Req 4 - navto 280ms (± 0.54%) 280ms (± 0.81%) ~ 276ms 282ms p=0.616 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 77ms (± 5.63%) 79ms (± 3.08%) ~ 74ms 81ms p=0.253 n=6
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,538ms (± 1.04%) 2,518ms (± 1.00%) ~ 2,470ms 2,536ms p=0.423 n=6
Req 2 - geterr 4,046ms (± 0.58%) 3,584ms (± 0.83%) 🟩-462ms (-11.42%) 3,557ms 3,630ms p=0.005 n=6
Req 3 - references 351ms (± 0.78%) 359ms (± 0.42%) +8ms (+ 2.37%) 358ms 361ms p=0.004 n=6
Req 4 - navto 293ms (± 0.97%) 287ms (± 0.41%) -6ms (- 2.05%) 285ms 288ms p=0.005 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 76ms (± 6.39%) 63ms (± 1.30%) 🟩-13ms (-17.54%) 62ms 64ms p=0.005 n=6
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,011ms (± 0.43%) 3,021ms (± 0.45%) ~ 3,009ms 3,047ms p=0.689 n=6
Req 2 - geterr 1,578ms (± 1.28%) 1,678ms (± 1.88%) +100ms (+ 6.35%) 1,621ms 1,711ms p=0.005 n=6
Req 3 - references 107ms (± 1.02%) 107ms (± 1.09%) ~ 106ms 109ms p=0.557 n=6
Req 4 - navto 355ms (± 0.29%) 354ms (± 0.40%) ~ 352ms 356ms p=0.410 n=6
Req 5 - completionInfo count 2,861 (± 0.00%) 2,861 (± 0.00%) ~ 2,861 2,861 p=1.000 n=6
Req 5 - completionInfo 394ms (± 4.43%) 379ms (± 2.17%) ~ 369ms 392ms p=0.053 n=6
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,498ms (± 0.62%) 2,499ms (± 0.55%) ~ 2,483ms 2,516ms p=0.936 n=6
Req 2 - geterr 5,807ms (± 0.53%) 5,040ms (± 0.66%) 🟩-767ms (-13.21%) 5,012ms 5,102ms p=0.005 n=6
Req 3 - references 349ms (± 0.65%) 366ms (± 0.56%) +18ms (+ 5.12%) 364ms 369ms p=0.005 n=6
Req 4 - navto 279ms (± 1.49%) 277ms (± 1.31%) ~ 272ms 281ms p=0.809 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 75ms (± 0.73%) 78ms (± 7.59%) ~ 74ms 86ms p=0.341 n=6
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,682ms (± 0.60%) 2,670ms (± 0.45%) ~ 2,654ms 2,688ms p=0.230 n=6
Req 2 - geterr 4,378ms (± 0.41%) 3,920ms (± 0.56%) 🟩-458ms (-10.45%) 3,889ms 3,938ms p=0.005 n=6
Req 3 - references 358ms (± 0.99%) 376ms (± 0.76%) +18ms (+ 5.07%) 372ms 381ms p=0.005 n=6
Req 4 - navto 290ms (± 0.75%) 283ms (± 0.39%) -7ms (- 2.25%) 282ms 285ms p=0.005 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 77ms (± 3.17%) 69ms (± 0.92%) 🟩-8ms (- 9.80%) 68ms 70ms p=0.004 n=6
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,164ms (± 0.60%) 3,163ms (± 0.19%) ~ 3,152ms 3,170ms p=0.471 n=6
Req 2 - geterr 1,749ms (± 0.81%) 1,823ms (± 1.80%) +74ms (+ 4.22%) 1,792ms 1,878ms p=0.005 n=6
Req 3 - references 114ms (± 1.65%) 113ms (± 0.87%) ~ 112ms 114ms p=0.565 n=6
Req 4 - navto 340ms (± 0.34%) 340ms (± 0.36%) ~ 338ms 341ms p=0.868 n=6
Req 5 - completionInfo count 2,861 (± 0.00%) 2,861 (± 0.00%) ~ 2,861 2,861 p=1.000 n=6
Req 5 - completionInfo 395ms (± 0.30%) 395ms (± 0.67%) ~ 391ms 399ms p=0.742 n=6
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,603ms (± 0.91%) 2,604ms (± 0.74%) ~ 2,583ms 2,638ms p=0.873 n=6
Req 2 - geterr 6,205ms (± 0.29%) 5,256ms (± 0.44%) 🟩-949ms (-15.29%) 5,227ms 5,292ms p=0.005 n=6
Req 3 - references 363ms (± 0.54%) 383ms (± 0.77%) +20ms (+ 5.51%) 380ms 388ms p=0.005 n=6
Req 4 - navto 277ms (± 0.42%) 282ms (± 0.52%) +5ms (+ 1.68%) 280ms 284ms p=0.005 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 102ms (± 0.88%) 84ms (± 0.48%) 🟩-18ms (-17.48%) 84ms 85ms p=0.003 n=6
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,822ms (± 0.71%) 2,827ms (± 0.81%) ~ 2,798ms 2,868ms p=0.936 n=6
Req 2 - geterr 4,486ms (± 0.65%) 4,036ms (± 0.52%) 🟩-450ms (-10.04%) 4,003ms 4,060ms p=0.005 n=6
Req 3 - references 410ms (± 6.65%) 384ms (± 1.08%) ~ 376ms 387ms p=0.295 n=6
Req 4 - navto 289ms (± 1.06%) 291ms (± 0.79%) ~ 289ms 294ms p=0.292 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 87ms (± 7.44%) 81ms (± 8.82%) ~ 76ms 95ms p=0.294 n=6
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,461ms (± 0.49%) 3,449ms (± 0.72%) ~ 3,422ms 3,496ms p=0.520 n=6
Req 2 - geterr 1,878ms (± 1.15%) 1,878ms (± 0.90%) ~ 1,856ms 1,895ms p=0.873 n=6
Req 3 - references 128ms (± 1.34%) 125ms (± 1.40%) -3ms (- 2.21%) 124ms 128ms p=0.034 n=6
Req 4 - navto 370ms (± 0.58%) 370ms (± 0.54%) ~ 368ms 373ms p=0.806 n=6
Req 5 - completionInfo count 2,861 (± 0.00%) 2,861 (± 0.00%) ~ 2,861 2,861 p=1.000 n=6
Req 5 - completionInfo 411ms (± 1.01%) 408ms (± 1.01%) ~ 404ms 415ms p=0.226 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 53261 6
Baseline main 6

Startup

Comparison Report - main..53261
Metric main 53261 Delta Best Worst p-value
tsc-startup - node (v16.17.1, x64)
Execution time 144.56ms (± 0.37%) 141.53ms (± 0.18%) -3.02ms (- 2.09%) 140.79ms 144.29ms p=0.000 n=600
tsserver-startup - node (v16.17.1, x64)
Execution time 232.00ms (± 0.25%) 226.11ms (± 0.19%) -5.89ms (- 2.54%) 225.00ms 231.04ms p=0.000 n=600
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 232.96ms (± 0.19%) 228.03ms (± 0.16%) -4.93ms (- 2.12%) 226.92ms 234.80ms p=0.000 n=600
typescript-startup - node (v16.17.1, x64)
Execution time 212.98ms (± 0.24%) 208.92ms (± 0.16%) -4.06ms (- 1.91%) 208.00ms 213.18ms p=0.000 n=600
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 53261 6
Baseline main 6

Developer Information:

Download Benchmark

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the top-repos suite comparing main and refs/pull/53261/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

Hey @gabritto, the results of running the DT tests are ready.
Everything looks the same!
You can check the log here.

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the top-repos suite comparing main and refs/pull/53261/merge:

Everything looks good!

@gabritto
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 21, 2023 •
edited
Loading

Heya @gabritto, I've started to run the abridged perf test suite on this PR at fd7d37a. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@gabritto
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..53261

Metric main 53261 Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 361,406k (± 0.01%) 361,432k (± 0.01%) ~ 361,385k 361,467k p=0.298 n=6
Parse Time 3.54s (± 0.92%) 3.52s (± 0.49%) ~ 3.50s 3.54s p=0.684 n=6
Bind Time 1.18s (± 0.69%) 1.18s (± 0.46%) ~ 1.18s 1.19s p=0.859 n=6
Check Time 9.54s (± 0.64%) 9.53s (± 0.17%) ~ 9.52s 9.56s p=0.686 n=6
Emit Time 7.94s (± 0.52%) 7.93s (± 0.22%) ~ 7.90s 7.95s p=0.414 n=6
Total Time 22.20s (± 0.53%) 22.17s (± 0.08%) ~ 22.15s 22.19s p=0.466 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,117k (± 0.72%) 193,041k (± 0.68%) ~ 192,215k 195,694k p=0.748 n=6
Parse Time 1.59s (± 1.15%) 1.59s (± 1.45%) ~ 1.55s 1.61s p=0.934 n=6
Bind Time 0.83s (± 0.62%) 0.82s (± 0.00%) -0.01s (- 0.81%) 0.82s 0.82s p=0.025 n=6
Check Time 10.23s (± 0.61%) 10.32s (± 0.50%) ~ 10.25s 10.37s p=0.054 n=6
Emit Time 3.03s (± 1.29%) 3.03s (± 3.76%) ~ 2.96s 3.26s p=0.170 n=6
Total Time 15.68s (± 0.44%) 15.76s (± 0.94%) ~ 15.58s 16.02s p=0.336 n=6
Monaco - node (v16.17.1, x64)
Memory used 345,571k (± 0.01%) 345,717k (± 0.00%) +146k (+ 0.04%) 345,702k 345,732k p=0.005 n=6
Parse Time 2.75s (± 0.93%) 2.73s (± 0.57%) ~ 2.72s 2.76s p=0.138 n=6
Bind Time 1.09s (± 0.58%) 1.09s (± 1.00%) ~ 1.08s 1.11s p=0.787 n=6
Check Time 7.78s (± 0.34%) 7.79s (± 0.45%) ~ 7.76s 7.85s p=0.290 n=6
Emit Time 4.50s (± 0.66%) 4.45s (± 0.87%) -0.05s (- 1.19%) 4.41s 4.52s p=0.044 n=6
Total Time 16.11s (± 0.33%) 16.06s (± 0.48%) ~ 15.99s 16.19s p=0.199 n=6
TFS - node (v16.17.1, x64)
Memory used 299,792k (± 0.01%) 300,022k (± 0.01%) +230k (+ 0.08%) 299,993k 300,065k p=0.005 n=6
Parse Time 2.18s (± 0.41%) 2.18s (± 0.93%) ~ 2.16s 2.22s p=0.801 n=6
Bind Time 1.23s (± 0.80%) 1.24s (± 1.10%) ~ 1.22s 1.26s p=0.111 n=6
Check Time 7.20s (± 0.77%) 7.22s (± 0.45%) ~ 7.18s 7.27s p=0.376 n=6
Emit Time 4.36s (± 1.02%) 4.35s (± 0.40%) ~ 4.32s 4.37s p=1.000 n=6
Total Time 14.97s (± 0.64%) 14.99s (± 0.42%) ~ 14.90s 15.09s p=0.572 n=6
material-ui - node (v16.17.1, x64)
Memory used 476,432k (± 0.00%) 476,428k (± 0.00%) ~ 476,412k 476,443k p=0.470 n=6
Parse Time 3.22s (± 0.97%) 3.21s (± 0.32%) ~ 3.19s 3.22s p=0.139 n=6
Bind Time 0.96s (± 1.82%) 0.96s (± 0.57%) ~ 0.95s 0.96s p=0.201 n=6
Check Time 18.24s (± 0.66%) 18.13s (± 0.18%) ~ 18.09s 18.18s p=0.065 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.42s (± 0.59%) 22.29s (± 0.14%) -0.13s (- 0.59%) 22.25s 22.33s p=0.024 n=6
xstate - node (v16.17.1, x64)
Memory used 550,623k (± 0.04%) 550,542k (± 0.01%) ~ 550,476k 550,655k p=0.936 n=6
Parse Time 3.95s (± 0.46%) 3.94s (± 0.30%) ~ 3.92s 3.95s p=0.570 n=6
Bind Time 1.80s (± 0.35%) 1.79s (± 0.84%) ~ 1.77s 1.81s p=0.080 n=6
Check Time 3.04s (± 0.54%) 3.09s (± 0.39%) +0.05s (+ 1.59%) 3.08s 3.11s p=0.005 n=6
Emit Time 0.09s (± 4.45%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=0.405 n=6
Total Time 8.88s (± 0.31%) 8.90s (± 0.22%) ~ 8.87s 8.93s p=0.091 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current 53261 6
Baseline main 6

Developer Information:

Download Benchmark

gabritto
gabritto commented Mar 22, 2023
View reviewed changes
tests/cases/compiler/classVarianceCircularity.ts
@@ -0,0 +1,16 @@
// @strict: true

// Issue #52813
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an example where this PR's approach doesn't solve the circularity problem.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume you mean the other test with the function calls rather than expressions, since that's the one that still has an error. :) If you want, you should file an issue about the other form, so the issue can still get backlog'd, if you want to consider looking into a broader fix in the future after this PR marks the original issue as fixed.

@gabritto gabritto marked this pull request as ready for review March 22, 2023 17:28
@gabritto gabritto requested a review from weswigham March 22, 2023 17:30
@gabritto
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 22, 2023 •
edited
Loading

Heya @gabritto, I've started to run the perf test suite on this PR at fd7d37a. You can monitor the build here.

Update: The results are in!

weswigham
weswigham approved these changes Mar 22, 2023
View reviewed changes
Copy link
Member

@weswigham weswigham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a small style suggestion to remove some duplication.

tests/cases/compiler/classVarianceCircularity.ts
@@ -0,0 +1,16 @@
// @strict: true

// Issue #52813
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume you mean the other test with the function calls rather than expressions, since that's the one that still has an error. :) If you want, you should file an issue about the other form, so the issue can still get backlog'd, if you want to consider looking into a broader fix in the future after this PR marks the original issue as fixed.

@typescript-bot
Copy link
Collaborator

@gabritto
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..53261
Metric main 53261 Delta Best Worst p-value
Angular - node (v18.10.0, x64)
Memory used 362,087k (± 0.00%) 362,080k (± 0.01%) ~ 362,042k 362,119k p=0.574 n=6
Parse Time 3.38s (± 0.58%) 3.39s (± 0.53%) ~ 3.37s 3.42s p=0.570 n=6
Bind Time 1.12s (± 0.56%) 1.12s (± 0.49%) ~ 1.11s 1.12s p=0.201 n=6
Check Time 8.71s (± 0.47%) 8.72s (± 0.33%) ~ 8.68s 8.76s p=0.332 n=6
Emit Time 7.49s (± 0.56%) 7.45s (± 0.28%) ~ 7.41s 7.47s p=0.121 n=6
Total Time 20.70s (± 0.31%) 20.67s (± 0.15%) ~ 20.63s 20.71s p=0.334 n=6
Compiler-Unions - node (v18.10.0, x64)
Memory used 190,945k (± 0.01%) 193,654k (± 1.59%) ~ 190,795k 196,468k p=1.000 n=6
Parse Time 1.50s (± 1.85%) 1.50s (± 0.54%) ~ 1.49s 1.51s p=0.511 n=6
Bind Time 0.76s (± 0.68%) 0.77s (± 0.67%) ~ 0.76s 0.77s p=0.311 n=6
Check Time 9.42s (± 0.42%) 9.61s (± 0.59%) +0.19s (+ 2.00%) 9.55s 9.68s p=0.005 n=6
Emit Time 2.73s (± 0.85%) 2.73s (± 0.65%) ~ 2.71s 2.76s p=0.745 n=6
Total Time 14.41s (± 0.43%) 14.59s (± 0.37%) +0.18s (+ 1.25%) 14.52s 14.66s p=0.006 n=6
Monaco - node (v18.10.0, x64)
Memory used 346,294k (± 0.01%) 346,442k (± 0.01%) +148k (+ 0.04%) 346,414k 346,470k p=0.005 n=6
Parse Time 2.57s (± 0.92%) 2.57s (± 0.96%) ~ 2.54s 2.60s p=0.936 n=6
Bind Time 1.01s (± 0.54%) 1.00s (± 0.54%) -0.01s (- 0.99%) 1.00s 1.01s p=0.024 n=6
Check Time 7.05s (± 0.48%) 7.11s (± 0.29%) +0.05s (+ 0.76%) 7.07s 7.13s p=0.012 n=6
Emit Time 4.24s (± 1.08%) 4.22s (± 0.78%) ~ 4.19s 4.27s p=0.467 n=6
Total Time 14.88s (± 0.66%) 14.90s (± 0.28%) ~ 14.87s 14.98s p=0.378 n=6
TFS - node (v18.10.0, x64)
Memory used 300,468k (± 0.01%) 300,672k (± 0.01%) +205k (+ 0.07%) 300,643k 300,728k p=0.005 n=6
Parse Time 2.06s (± 1.81%) 2.06s (± 0.91%) ~ 2.04s 2.09s p=0.808 n=6
Bind Time 1.14s (± 0.72%) 1.13s (± 0.48%) ~ 1.13s 1.14s p=0.441 n=6
Check Time 6.52s (± 0.47%) 6.52s (± 0.53%) ~ 6.49s 6.58s p=0.872 n=6
Emit Time 3.85s (± 0.75%) 3.85s (± 0.69%) ~ 3.82s 3.89s p=0.745 n=6
Total Time 13.56s (± 0.70%) 13.58s (± 0.47%) ~ 13.50s 13.65s p=0.628 n=6
material-ui - node (v18.10.0, x64)
Memory used 477,166k (± 0.02%) 477,097k (± 0.01%) ~ 477,029k 477,185k p=0.230 n=6
Parse Time 3.02s (± 2.41%) 2.98s (± 2.50%) ~ 2.92s 3.08s p=0.515 n=6
Bind Time 0.96s (± 8.83%) 1.01s (± 8.41%) ~ 0.90s 1.08s p=0.251 n=6
Check Time 17.09s (± 0.36%) 17.13s (± 0.69%) ~ 17.05s 17.36s p=0.629 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.07s (± 0.30%) 21.13s (± 0.67%) ~ 21.02s 21.40s p=0.422 n=6
xstate - node (v18.10.0, x64)
Memory used 552,847k (± 0.01%) 552,911k (± 0.02%) ~ 552,788k 553,027k p=0.173 n=6
Parse Time 3.78s (± 0.72%) 3.78s (± 0.68%) ~ 3.74s 3.82s p=0.746 n=6
Bind Time 1.71s (± 0.24%) 1.70s (± 0.64%) ~ 1.69s 1.72s p=0.090 n=6
Check Time 2.80s (± 0.83%) 2.85s (± 0.53%) +0.04s (+ 1.49%) 2.83s 2.87s p=0.016 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=1.000 n=6
Total Time 8.38s (± 0.46%) 8.41s (± 0.33%) ~ 8.37s 8.44s p=0.297 n=6
Angular - node (v16.17.1, x64)
Memory used 361,424k (± 0.01%) 361,410k (± 0.01%) ~ 361,387k 361,452k p=0.471 n=6
Parse Time 3.55s (± 0.47%) 3.55s (± 0.46%) ~ 3.52s 3.57s p=0.933 n=6
Bind Time 1.18s (± 0.35%) 1.18s (± 0.35%) ~ 1.18s 1.19s p=1.000 n=6
Check Time 9.51s (± 0.30%) 9.53s (± 0.35%) ~ 9.48s 9.57s p=0.419 n=6
Emit Time 7.91s (± 0.55%) 7.93s (± 0.59%) ~ 7.86s 8.00s p=0.747 n=6
Total Time 22.15s (± 0.29%) 22.19s (± 0.29%) ~ 22.11s 22.28s p=0.295 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,696k (± 0.87%) 193,143k (± 0.73%) ~ 192,527k 196,020k p=0.471 n=6
Parse Time 1.58s (± 1.04%) 1.58s (± 1.09%) ~ 1.56s 1.60s p=0.868 n=6
Bind Time 0.82s (± 0.77%) 0.82s (± 0.99%) ~ 0.82s 0.84s p=0.599 n=6
Check Time 10.11s (± 0.53%) 10.30s (± 0.65%) +0.19s (+ 1.91%) 10.22s 10.38s p=0.005 n=6
Emit Time 2.98s (± 0.73%) 2.98s (± 0.54%) ~ 2.96s 3.00s p=0.935 n=6
Total Time 15.49s (± 0.33%) 15.69s (± 0.34%) +0.20s (+ 1.26%) 15.64s 15.76s p=0.005 n=6
Monaco - node (v16.17.1, x64)
Memory used 345,553k (± 0.01%) 345,706k (± 0.01%) +153k (+ 0.04%) 345,669k 345,732k p=0.005 n=6
Parse Time 2.72s (± 0.60%) 2.72s (± 0.38%) ~ 2.71s 2.74s p=1.000 n=6
Bind Time 1.09s (± 1.00%) 1.09s (± 0.37%) ~ 1.08s 1.09s p=1.000 n=6
Check Time 7.77s (± 0.62%) 7.80s (± 0.69%) ~ 7.74s 7.88s p=0.421 n=6
Emit Time 4.47s (± 0.49%) 4.44s (± 0.52%) ~ 4.42s 4.48s p=0.064 n=6
Total Time 16.04s (± 0.43%) 16.05s (± 0.36%) ~ 15.98s 16.13s p=1.000 n=6
TFS - node (v16.17.1, x64)
Memory used 299,811k (± 0.01%) 300,032k (± 0.01%) +221k (+ 0.07%) 299,999k 300,060k p=0.005 n=6
Parse Time 2.17s (± 0.63%) 2.17s (± 0.61%) ~ 2.15s 2.19s p=0.673 n=6
Bind Time 1.23s (± 0.68%) 1.24s (± 0.94%) ~ 1.22s 1.25s p=0.547 n=6
Check Time 7.20s (± 0.65%) 7.21s (± 0.61%) ~ 7.16s 7.28s p=0.517 n=6
Emit Time 4.34s (± 0.49%) 4.34s (± 0.89%) ~ 4.30s 4.41s p=0.520 n=6
Total Time 14.95s (± 0.40%) 14.96s (± 0.45%) ~ 14.88s 15.06s p=1.000 n=6
material-ui - node (v16.17.1, x64)
Memory used 476,418k (± 0.00%) 476,411k (± 0.00%) ~ 476,398k 476,428k p=0.471 n=6
Parse Time 3.22s (± 0.50%) 3.20s (± 1.04%) ~ 3.16s 3.23s p=0.317 n=6
Bind Time 0.96s (± 0.57%) 0.97s (± 2.57%) ~ 0.95s 1.01s p=0.138 n=6
Check Time 18.14s (± 0.42%) 18.12s (± 0.69%) ~ 17.99s 18.32s p=0.575 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.31s (± 0.32%) 22.28s (± 0.54%) ~ 22.16s 22.49s p=0.520 n=6
xstate - node (v16.17.1, x64)
Memory used 550,505k (± 0.02%) 550,511k (± 0.02%) ~ 550,419k 550,754k p=0.936 n=6
Parse Time 3.94s (± 0.14%) 3.94s (± 0.23%) ~ 3.93s 3.95s p=0.342 n=6
Bind Time 1.79s (± 1.12%) 1.79s (± 0.65%) ~ 1.77s 1.80s p=1.000 n=6
Check Time 3.02s (± 1.13%) 3.10s (± 0.39%) +0.08s (+ 2.54%) 3.09s 3.12s p=0.005 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 4.62%) ~ 0.08s 0.09s p=0.405 n=6
Total Time 8.83s (± 0.46%) 8.91s (± 0.17%) +0.08s (+ 0.91%) 8.89s 8.93s p=0.005 n=6
Angular - node (v14.15.1, x64)
Memory used 355,280k (± 0.01%) 355,304k (± 0.00%) ~ 355,297k 355,318k p=0.298 n=6
Parse Time 3.60s (± 0.58%) 3.59s (± 0.48%) ~ 3.56s 3.61s p=0.192 n=6
Bind Time 1.22s (± 0.42%) 1.22s (± 1.09%) ~ 1.20s 1.24s p=0.928 n=6
Check Time 9.80s (± 0.40%) 9.88s (± 0.56%) +0.08s (+ 0.80%) 9.83s 9.98s p=0.020 n=6
Emit Time 8.36s (± 0.26%) 8.32s (± 0.54%) ~ 8.28s 8.41s p=0.064 n=6
Total Time 22.98s (± 0.23%) 23.01s (± 0.43%) ~ 22.94s 23.20s p=0.686 n=6
Compiler-Unions - node (v14.15.1, x64)
Memory used 188,308k (± 0.62%) 187,806k (± 0.01%) ~ 187,768k 187,839k p=0.173 n=6
Parse Time 1.60s (± 0.47%) 1.61s (± 1.02%) ~ 1.59s 1.63s p=0.672 n=6
Bind Time 0.84s (± 0.00%) 0.85s (± 0.61%) +0.01s (+ 0.79%) 0.84s 0.85s p=0.025 n=6
Check Time 10.23s (± 0.49%) 10.42s (± 0.43%) +0.19s (+ 1.86%) 10.37s 10.47s p=0.005 n=6
Emit Time 3.13s (± 0.56%) 3.14s (± 0.63%) ~ 3.11s 3.17s p=0.317 n=6
Total Time 15.79s (± 0.24%) 16.01s (± 0.37%) +0.21s (+ 1.35%) 15.94s 16.09s p=0.005 n=6
Monaco - node (v14.15.1, x64)
Memory used 340,530k (± 0.00%) 340,615k (± 0.00%) +85k (+ 0.03%) 340,602k 340,629k p=0.005 n=6
Parse Time 2.83s (± 0.37%) 2.83s (± 0.48%) ~ 2.81s 2.84s p=0.804 n=6
Bind Time 1.10s (± 0.94%) 1.10s (± 0.68%) ~ 1.09s 1.11s p=0.931 n=6
Check Time 8.15s (± 0.70%) 8.15s (± 0.64%) ~ 8.06s 8.22s p=0.415 n=6
Emit Time 4.68s (± 0.48%) 4.66s (± 0.25%) ~ 4.65s 4.68s p=0.195 n=6
Total Time 16.75s (± 0.37%) 16.74s (± 0.26%) ~ 16.67s 16.79s p=1.000 n=6
TFS - node (v14.15.1, x64)
Memory used 294,887k (± 0.00%) 295,084k (± 0.00%) +197k (+ 0.07%) 295,074k 295,096k p=0.005 n=6
Parse Time 2.40s (± 1.34%) 2.39s (± 0.61%) ~ 2.37s 2.41s p=0.746 n=6
Bind Time 1.07s (± 0.38%) 1.07s (± 0.76%) ~ 1.06s 1.08s p=0.206 n=6
Check Time 7.51s (± 0.34%) 7.48s (± 0.45%) ~ 7.42s 7.52s p=0.089 n=6
Emit Time 4.29s (± 0.68%) 4.26s (± 0.89%) ~ 4.20s 4.30s p=0.172 n=6
Total Time 15.27s (± 0.24%) 15.19s (± 0.17%) -0.07s (- 0.49%) 15.17s 15.24s p=0.012 n=6
material-ui - node (v14.15.1, x64)
Memory used 471,963k (± 0.00%) 471,978k (± 0.01%) ~ 471,907k 472,009k p=0.336 n=6
Parse Time 3.34s (± 0.41%) 3.36s (± 0.40%) +0.02s (+ 0.65%) 3.34s 3.38s p=0.027 n=6
Bind Time 1.00s (± 0.63%) 1.00s (± 0.81%) ~ 0.99s 1.01s p=0.432 n=6
Check Time 19.17s (± 1.15%) 19.04s (± 0.23%) ~ 18.97s 19.09s p=0.199 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 23.51s (± 0.95%) 23.40s (± 0.25%) ~ 23.30s 23.48s p=0.228 n=6
xstate - node (v14.15.1, x64)
Memory used 539,131k (± 0.01%) 539,133k (± 0.00%) ~ 539,117k 539,141k p=0.471 n=6
Parse Time 4.23s (± 0.81%) 4.23s (± 0.71%) ~ 4.19s 4.27s p=1.000 n=6
Bind Time 1.67s (± 0.73%) 1.67s (± 0.31%) ~ 1.67s 1.68s p=1.000 n=6
Check Time 3.18s (± 0.88%) 3.20s (± 0.72%) ~ 3.17s 3.23s p=0.147 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=1.000 n=6
Total Time 9.17s (± 0.34%) 9.19s (± 0.51%) ~ 9.13s 9.26s p=0.468 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 53261 6
Baseline main 6

TSServer

Comparison Report - main..53261
Metric main 53261 Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,390ms (± 0.37%) 2,397ms (± 0.49%) ~ 2,382ms 2,418ms p=0.295 n=6
Req 2 - geterr 5,388ms (± 0.69%) 5,555ms (± 0.60%) +167ms (+ 3.10%) 5,520ms 5,607ms p=0.005 n=6
Req 3 - references 343ms (± 1.13%) 337ms (± 0.39%) ~ 335ms 338ms p=0.050 n=6
Req 4 - navto 280ms (± 0.29%) 282ms (± 0.30%) +1ms (+ 0.42%) 280ms 282ms p=0.045 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 68ms (± 1.72%) 84ms (± 2.43%) +16ms (+23.59%) 80ms 86ms p=0.004 n=6
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,520ms (± 1.19%) 2,521ms (± 0.48%) ~ 2,507ms 2,543ms p=1.000 n=6
Req 2 - geterr 4,055ms (± 1.03%) 4,064ms (± 0.68%) ~ 4,028ms 4,104ms p=0.630 n=6
Req 3 - references 349ms (± 1.59%) 348ms (± 1.53%) ~ 342ms 355ms p=0.571 n=6
Req 4 - navto 295ms (± 1.24%) 291ms (± 0.40%) -4ms (- 1.30%) 289ms 292ms p=0.016 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 82ms (± 2.82%) 81ms (± 5.25%) ~ 76ms 87ms p=0.747 n=6
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,020ms (± 0.80%) 3,011ms (± 0.40%) ~ 2,997ms 3,026ms p=0.810 n=6
Req 2 - geterr 1,588ms (± 1.08%) 1,684ms (± 1.12%) +96ms (+ 6.07%) 1,669ms 1,718ms p=0.005 n=6
Req 3 - references 107ms (± 0.38%) 106ms (± 0.49%) ~ 106ms 107ms p=0.112 n=6
Req 4 - navto 354ms (± 0.34%) 355ms (± 0.15%) ~ 355ms 356ms p=0.142 n=6
Req 5 - completionInfo count 2,861 (± 0.00%) 2,861 (± 0.00%) ~ 2,861 2,861 p=1.000 n=6
Req 5 - completionInfo 387ms (± 0.64%) 381ms (± 2.35%) ~ 364ms 391ms p=0.077 n=6
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,511ms (± 0.98%) 2,492ms (± 0.37%) ~ 2,482ms 2,506ms p=0.148 n=6
Req 2 - geterr 5,813ms (± 0.46%) 5,906ms (± 0.36%) +93ms (+ 1.60%) 5,866ms 5,926ms p=0.005 n=6
Req 3 - references 351ms (± 0.50%) 345ms (± 0.50%) -6ms (- 1.57%) 343ms 348ms p=0.006 n=6
Req 4 - navto 279ms (± 1.48%) 279ms (± 0.27%) ~ 278ms 280ms p=1.000 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 74ms (± 1.01%) 74ms (± 1.11%) ~ 73ms 75ms p=0.306 n=6
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,684ms (± 0.71%) 2,675ms (± 0.97%) ~ 2,651ms 2,722ms p=0.471 n=6
Req 2 - geterr 4,390ms (± 0.17%) 4,429ms (± 0.46%) +38ms (+ 0.87%) 4,402ms 4,459ms p=0.005 n=6
Req 3 - references 358ms (± 0.58%) 362ms (± 0.84%) ~ 358ms 365ms p=0.064 n=6
Req 4 - navto 291ms (± 0.61%) 288ms (± 0.31%) -3ms (- 1.03%) 287ms 289ms p=0.007 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 81ms (± 4.95%) 76ms (± 0.68%) ~ 76ms 77ms p=0.060 n=6
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,166ms (± 0.33%) 3,167ms (± 0.16%) ~ 3,158ms 3,172ms p=0.228 n=6
Req 2 - geterr 1,769ms (± 1.51%) 1,844ms (± 0.88%) +75ms (+ 4.25%) 1,827ms 1,867ms p=0.005 n=6
Req 3 - references 116ms (± 1.61%) 114ms (± 1.57%) ~ 112ms 117ms p=0.139 n=6
Req 4 - navto 341ms (± 0.36%) 343ms (± 0.31%) +2ms (+ 0.54%) 341ms 344ms p=0.033 n=6
Req 5 - completionInfo count 2,861 (± 0.00%) 2,861 (± 0.00%) ~ 2,861 2,861 p=1.000 n=6
Req 5 - completionInfo 396ms (± 0.67%) 402ms (± 5.01%) ~ 390ms 442ms p=0.573 n=6
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,604ms (± 0.75%) 2,611ms (± 0.52%) ~ 2,592ms 2,628ms p=0.689 n=6
Req 2 - geterr 6,202ms (± 0.51%) 6,305ms (± 0.43%) +103ms (+ 1.67%) 6,268ms 6,344ms p=0.005 n=6
Req 3 - references 364ms (± 0.94%) 366ms (± 0.73%) ~ 362ms 369ms p=0.420 n=6
Req 4 - navto 279ms (± 0.84%) 281ms (± 0.74%) ~ 277ms 283ms p=0.293 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 100ms (± 4.47%) 101ms (± 1.49%) ~ 99ms 103ms p=0.514 n=6
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,826ms (± 0.35%) 2,827ms (± 0.48%) ~ 2,815ms 2,844ms p=0.936 n=6
Req 2 - geterr 4,590ms (± 2.55%) 4,665ms (± 2.22%) ~ 4,553ms 4,787ms p=0.065 n=6
Req 3 - references 397ms (± 7.08%) 397ms (± 6.66%) ~ 371ms 426ms p=0.936 n=6
Req 4 - navto 291ms (± 1.61%) 289ms (± 0.22%) ~ 288ms 290ms p=0.459 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 81ms (± 8.87%) 82ms (±10.57%) ~ 74ms 93ms p=0.678 n=6
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,445ms (± 0.27%) 3,450ms (± 0.63%) ~ 3,426ms 3,481ms p=0.689 n=6
Req 2 - geterr 1,899ms (± 1.00%) 1,888ms (± 1.09%) ~ 1,856ms 1,914ms p=0.377 n=6
Req 3 - references 128ms (± 2.93%) 124ms (± 0.94%) ~ 123ms 126ms p=0.052 n=6
Req 4 - navto 371ms (± 0.71%) 370ms (± 0.30%) ~ 368ms 371ms p=1.000 n=6
Req 5 - completionInfo count 2,861 (± 0.00%) 2,861 (± 0.00%) ~ 2,861 2,861 p=1.000 n=6
Req 5 - completionInfo 408ms (± 0.57%) 409ms (± 0.40%) ~ 407ms 411ms p=0.460 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 53261 6
Baseline main 6

Startup

Comparison Report - main..53261
Metric main 53261 Delta Best Worst p-value
tsc-startup - node (v16.17.1, x64)
Execution time 141.76ms (± 0.19%) 141.94ms (± 0.16%) +0.18ms (+ 0.13%) 141.25ms 145.25ms p=0.000 n=600
tsserver-startup - node (v16.17.1, x64)
Execution time 226.41ms (± 0.18%) 227.28ms (± 0.18%) +0.87ms (+ 0.38%) 226.48ms 236.05ms p=0.000 n=600
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 228.34ms (± 0.20%) 229.13ms (± 0.15%) +0.79ms (+ 0.35%) 228.27ms 233.85ms p=0.000 n=600
typescript-startup - node (v16.17.1, x64)
Execution time 209.12ms (± 0.20%) 209.86ms (± 0.17%) +0.74ms (+ 0.35%) 209.03ms 215.39ms p=0.000 n=600
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 53261 6
Baseline main 6

Developer Information:

Download Benchmark

@gabritto
Copy link
Member Author

It's kinda unfortunate that there seems to be a slight perf impact on check time, but I don't immediately see what could be done to improve this, since my previous attempts to either defer the call to checkExpression or to use checkExpressionCached led to bad results, so...

@gabritto gabritto merged commit 8947825 into main Mar 23, 2023
@gabritto gabritto deleted the gabritto/issue52813 branch March 23, 2023 18:46
@gabritto gabritto mentioned this pull request Mar 23, 2023
Closed
DanielRosenwasser added a commit that referenced this pull request Apr 17, 2023
@DanielRosenwasser
Revert "Defer type comparability check for assertions (#53261)"
dd00a34 
This reverts commit 8947825.
@DanielRosenwasser DanielRosenwasser mentioned this pull request Apr 17, 2023
Closed
@gabritto gabritto mentioned this pull request May 12, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@weswigham weswigham weswigham approved these changes

Assignees

@gabritto gabritto

Labels
Author: Team For Milestone Bug PRs that fix a bug with a specific milestone
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Inconsistent errors reported between tsc and VSCode
3 participants
@gabritto @typescript-bot @weswigham