Skip to content

Make type comparison error elaboration consistent #58859

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 9 commits into from
Jul 15, 2024
Merged

Make type comparison error elaboration consistent #58859

merged 9 commits into from
Jul 15, 2024

Conversation

gabritto
Copy link
Member

@gabritto gabritto commented Jun 13, 2024 •
edited
Loading

Updated version of #55234.
Fixes #3276.

Motivated by #57842.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jun 13, 2024
@gabritto
Copy link
Member Author

@typescript-bot test it

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 13, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started ✅ Results
user test this ✅ Started 👀 Results
run dt ✅ Started ✅ Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

Hey @gabritto, the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the user tests with tsc comparing main and refs/pull/58859/merge:

Something interesting changed - please have a look.

Details

effect

tsconfig.json

tsconfig.base.json

lodash

/mnt/ts_downloads/_/m/lodash/tsconfig.json

  • [NEW] error TS2740: Type 'SetCache' is missing the following properties from type 'any[]': length, pop, concat, join, and 34 more.
    • /mnt/ts_downloads/_/m/lodash/node_modules/lodash/_baseUniq.js(39,5)
  • [MISSING] error TS2322: Type 'SetCache' is not assignable to type 'any[]'.
    • /mnt/ts_downloads/_/m/lodash/node_modules/lodash/_baseUniq.js(39,5)

xterm.js

src/tsconfig-library-base.json

@typescript-bot
Copy link
Collaborator

@gabritto
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,729k (± 0.78%) 193,392k (± 0.98%) ~ 192,152k 195,835k p=0.471 n=6
Parse Time 1.95s (± 0.82%) 1.95s (± 0.39%) ~ 1.94s 1.96s p=0.788 n=6
Bind Time 1.07s (± 0.76%) 1.06s (± 1.41%) ~ 1.05s 1.09s p=0.451 n=6
Check Time 13.89s (± 0.40%) 13.92s (± 0.34%) ~ 13.87s 13.98s p=0.422 n=6
Emit Time 4.07s (± 3.10%) 4.02s (± 0.58%) ~ 3.98s 4.05s p=1.000 n=6
Total Time 20.98s (± 0.42%) 20.95s (± 0.17%) ~ 20.89s 21.00s p=0.810 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,104 944,104 ~ ~ ~ p=1.000 n=6
Types 407,044 407,044 ~ ~ ~ p=1.000 n=6
Memory used 1,218,275k (± 0.01%) 1,218,301k (± 0.00%) ~ 1,218,229k 1,218,362k p=0.423 n=6
Parse Time 7.94s (± 0.50%) 7.97s (± 0.64%) ~ 7.90s 8.03s p=0.334 n=6
Bind Time 2.23s (± 0.34%) 2.23s (± 0.44%) ~ 2.22s 2.24s p=0.554 n=6
Check Time 35.93s (± 0.32%) 35.89s (± 0.27%) ~ 35.76s 36.05s p=0.748 n=6
Emit Time 16.23s (± 0.31%) 16.21s (± 0.47%) ~ 16.09s 16.31s p=0.872 n=6
Total Time 62.33s (± 0.28%) 62.31s (± 0.17%) ~ 62.10s 62.40s p=0.747 n=6
mui-docs - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 2,132,301 2,132,301 ~ ~ ~ p=1.000 n=6
Types 926,081 926,081 ~ ~ ~ p=1.000 n=6
Memory used 2,114,644k (± 0.00%) 2,114,763k (± 0.00%) +119k (+ 0.01%) 2,114,728k 2,114,867k p=0.005 n=6
Parse Time 7.86s (± 0.15%) 7.86s (± 0.24%) ~ 7.84s 7.89s p=0.684 n=6
Bind Time 2.77s (± 0.27%) 2.76s (± 0.50%) ~ 2.75s 2.78s p=0.740 n=6
Check Time 84.38s (± 0.32%) 83.94s (± 0.33%) -0.44s (- 0.52%) 83.52s 84.22s p=0.031 n=6
Emit Time 0.16s (± 4.75%) 0.16s (± 4.99%) ~ 0.15s 0.17s p=0.306 n=6
Total Time 95.17s (± 0.28%) 94.72s (± 0.30%) -0.45s (- 0.47%) 94.30s 95.02s p=0.031 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,770 1,230,758 -12 (- 0.00%) ~ ~ p=0.001 n=6
Types 261,222 261,219 -3 (- 0.00%) ~ ~ p=0.001 n=6
Memory used 2,355,243k (± 1.04%) 2,345,705k (± 0.06%) ~ 2,343,313k 2,346,879k p=0.936 n=6
Parse Time 5.99s (± 1.16%) 5.95s (± 1.22%) ~ 5.86s 6.06s p=0.471 n=6
Bind Time 2.24s (± 0.34%) 2.25s (± 0.83%) ~ 2.23s 2.28s p=0.073 n=6
Check Time 39.81s (± 0.20%) 39.74s (± 0.37%) ~ 39.53s 39.96s p=0.378 n=6
Emit Time 3.18s (± 3.90%) 3.10s (± 2.88%) ~ 3.00s 3.26s p=0.230 n=6
Total Time 51.23s (± 0.41%) 51.06s (± 0.51%) ~ 50.87s 51.57s p=0.173 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,770 1,230,758 -12 (- 0.00%) ~ ~ p=0.001 n=6
Types 261,222 261,219 -3 (- 0.00%) ~ ~ p=0.001 n=6
Memory used 2,422,483k (± 0.03%) 2,431,708k (± 1.02%) ~ 2,420,791k 2,482,249k p=0.298 n=6
Parse Time 6.26s (± 0.87%) 6.23s (± 0.74%) ~ 6.17s 6.29s p=0.298 n=6
Bind Time 2.01s (± 0.77%) 1.99s (± 0.94%) -0.02s (- 1.16%) 1.96s 2.01s p=0.048 n=6
Check Time 40.15s (± 0.30%) 40.18s (± 0.43%) ~ 40.05s 40.52s p=0.810 n=6
Emit Time 3.14s (± 2.00%) 3.12s (± 1.47%) ~ 3.04s 3.17s p=0.810 n=6
Total Time 51.58s (± 0.32%) 51.53s (± 0.35%) ~ 51.39s 51.89s p=0.575 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,665 258,664 -1 (- 0.00%) ~ ~ p=0.001 n=6
Types 104,938 104,935 -3 (- 0.00%) ~ ~ p=0.001 n=6
Memory used 428,300k (± 0.01%) 428,268k (± 0.01%) ~ 428,246k 428,322k p=0.078 n=6
Parse Time 3.30s (± 0.68%) 3.31s (± 0.88%) ~ 3.26s 3.35s p=0.572 n=6
Bind Time 1.30s (± 0.69%) 1.29s (± 0.42%) ~ 1.29s 1.30s p=0.341 n=6
Check Time 18.00s (± 0.29%) 17.96s (± 0.16%) ~ 17.91s 18.00s p=0.280 n=6
Emit Time 1.37s (± 1.49%) 1.33s (± 0.41%) -0.03s (- 2.44%) 1.33s 1.34s p=0.008 n=6
Total Time 23.97s (± 0.25%) 23.90s (± 0.16%) ~ 23.86s 23.95s p=0.053 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,414k (± 0.03%) 369,569k (± 0.04%) +154k (+ 0.04%) 369,468k 369,824k p=0.045 n=6
Parse Time 2.76s (± 0.88%) 2.78s (± 0.95%) ~ 2.74s 2.81s p=0.373 n=6
Bind Time 1.60s (± 1.51%) 1.59s (± 0.73%) ~ 1.58s 1.61s p=0.459 n=6
Check Time 15.47s (± 0.45%) 15.51s (± 0.33%) ~ 15.43s 15.57s p=0.227 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.84s (± 0.37%) 19.88s (± 0.33%) ~ 19.77s 19.95s p=0.422 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,853,853 2,853,853 ~ ~ ~ p=1.000 n=6
Types 968,176 968,176 ~ ~ ~ p=1.000 n=6
Memory used 3,020,552k (± 0.00%) 3,020,543k (± 0.00%) ~ 3,020,506k 3,020,572k p=0.873 n=6
Parse Time 16.82s (± 0.33%) 16.76s (± 0.25%) ~ 16.70s 16.81s p=0.109 n=6
Bind Time 5.09s (± 0.32%) 5.09s (± 0.42%) ~ 5.05s 5.11s p=0.805 n=6
Check Time 88.92s (± 0.54%) 89.11s (± 0.69%) ~ 88.16s 89.89s p=0.575 n=6
Emit Time 28.87s (± 1.37%) 28.57s (± 2.39%) ~ 27.61s 29.24s p=0.521 n=6
Total Time 139.70s (± 0.32%) 139.54s (± 0.27%) ~ 138.99s 140.06s p=0.378 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 266,914 266,914 ~ ~ ~ p=1.000 n=6
Types 108,685 108,685 ~ ~ ~ p=1.000 n=6
Memory used 411,295k (± 0.01%) 411,377k (± 0.02%) +82k (+ 0.02%) 411,313k 411,485k p=0.031 n=6
Parse Time 4.71s (± 0.71%) 4.69s (± 0.43%) ~ 4.66s 4.72s p=0.292 n=6
Bind Time 2.07s (± 0.36%) 2.06s (± 0.66%) ~ 2.04s 2.08s p=0.273 n=6
Check Time 20.85s (± 0.55%) 20.86s (± 0.30%) ~ 20.78s 20.92s p=1.000 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.63s (± 0.33%) 27.61s (± 0.19%) ~ 27.54s 27.68s p=0.470 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 523,429 523,429 ~ ~ ~ p=1.000 n=6
Types 177,955 177,955 ~ ~ ~ p=1.000 n=6
Memory used 461,458k (± 0.08%) 461,891k (± 0.01%) ~ 461,827k 461,946k p=0.173 n=6
Parse Time 2.62s (± 0.38%) 2.63s (± 0.61%) ~ 2.61s 2.65s p=0.271 n=6
Bind Time 0.99s (± 0.64%) 0.99s (± 0.41%) ~ 0.98s 0.99s p=0.673 n=6
Check Time 15.25s (± 0.66%) 15.27s (± 0.46%) ~ 15.21s 15.41s p=0.420 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.86s (± 0.52%) 18.89s (± 0.34%) ~ 18.83s 19.01s p=0.261 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the top 400 repos with tsc comparing main and refs/pull/58859/merge:

Everything looks good!

@gabritto
Copy link
Member Author

It doesn't seem like the case of reporting the same assignability error more than once is that frequent, since there were only 3 changes in user tests, none in top 400, and no perf impact.

Two things I want to do, though:

  • Special case type comparison "overflow" cases so that we don't do the comparison again.
  • Maybe only have this consistent assignability elaboration when we know we are in an editor scenario, if that's possible.

@gabritto
Copy link
Member Author

@typescript-bot test top800

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 14, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top800 ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the top 800 repos with tsc comparing main and refs/pull/58859/merge:

Everything looks good!

@jakebailey jakebailey closed this Jul 10, 2024
@jakebailey jakebailey reopened this Jul 10, 2024
@gabritto
Copy link
Member Author

@typescript-bot perf test this
@typescript-bot test top800

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 10, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results
test top800 ✅ Started ✅ Results

@gabritto gabritto changed the title Make assignability error elaboration consistent Make type comparison error elaboration consistent Jul 10, 2024
@gabritto
Copy link
Member Author

@typescript-bot user test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 10, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
user test this ✅ Started 👀 Results

@typescript-bot typescript-bot removed the For Uncommitted Bug PR for untriaged, rejected, closed or missing bug label Jul 10, 2024
@typescript-bot typescript-bot added the For Backlog Bug PRs that fix a backlog bug label Jul 10, 2024
@typescript-bot
Copy link
Collaborator

@gabritto
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,916k (± 1.00%) 192,162k (± 0.03%) ~ 192,077k 192,226k p=0.298 n=6
Parse Time 1.96s (± 0.43%) 1.95s (± 0.70%) ~ 1.93s 1.96s p=0.262 n=6
Bind Time 1.05s (± 0.79%) 1.05s (± 1.11%) ~ 1.04s 1.07s p=0.555 n=6
Check Time 13.85s (± 0.61%) 13.86s (± 0.31%) ~ 13.78s 13.90s p=0.570 n=6
Emit Time 4.03s (± 0.57%) 4.04s (± 1.16%) ~ 3.95s 4.08s p=0.373 n=6
Total Time 20.89s (± 0.46%) 20.90s (± 0.17%) ~ 20.84s 20.95s p=0.688 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,369k (± 0.00%) 1,218,341k (± 0.00%) ~ 1,218,306k 1,218,388k p=0.230 n=6
Parse Time 7.94s (± 0.50%) 7.97s (± 0.65%) ~ 7.93s 8.07s p=0.226 n=6
Bind Time 2.22s (± 0.66%) 2.23s (± 0.57%) ~ 2.22s 2.25s p=0.198 n=6
Check Time 35.77s (± 0.23%) 35.88s (± 0.29%) ~ 35.75s 36.04s p=0.078 n=6
Emit Time 16.21s (± 0.82%) 16.15s (± 0.26%) ~ 16.10s 16.21s p=0.687 n=6
Total Time 62.14s (± 0.13%) 62.23s (± 0.24%) ~ 62.01s 62.43s p=0.296 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,130,296 2,130,296 ~ ~ ~ p=1.000 n=6
Types 927,656 927,656 ~ ~ ~ p=1.000 n=6
Memory used 2,107,135k (± 0.01%) 2,107,141k (± 0.01%) ~ 2,107,047k 2,107,310k p=1.000 n=6
Parse Time 7.82s (± 0.36%) 7.81s (± 0.40%) ~ 7.75s 7.83s p=0.324 n=6
Bind Time 2.74s (± 0.56%) 2.73s (± 0.95%) ~ 2.69s 2.76s p=0.565 n=6
Check Time 83.40s (± 0.35%) 83.55s (± 0.35%) ~ 83.20s 84.07s p=0.688 n=6
Emit Time 0.16s (± 4.75%) 0.16s (± 3.29%) ~ 0.15s 0.16s p=0.784 n=6
Total Time 94.12s (± 0.32%) 94.25s (± 0.31%) ~ 93.94s 94.79s p=0.748 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,784 1,225,809 +25 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,518 261,522 +4 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,338,907k (± 0.02%) 2,339,004k (± 0.05%) ~ 2,338,265k 2,341,142k p=0.575 n=6
Parse Time 7.56s (± 0.66%) 7.54s (± 0.57%) ~ 7.49s 7.62s p=0.688 n=6
Bind Time 2.81s (± 0.83%) 2.80s (± 0.73%) ~ 2.77s 2.83s p=0.935 n=6
Check Time 49.50s (± 0.30%) 49.52s (± 0.28%) ~ 49.41s 49.75s p=0.471 n=6
Emit Time 4.76s (± 1.61%) 4.73s (± 1.26%) ~ 4.66s 4.82s p=0.423 n=6
Total Time 64.63s (± 0.31%) 64.60s (± 0.27%) ~ 64.43s 64.91s p=0.936 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,784 1,225,809 +25 (+ 0.00%) ~ ~ p=0.001 n=6
Types 261,518 261,522 +4 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,412,730k (± 0.02%) 2,413,490k (± 0.02%) +761k (+ 0.03%) 2,412,408k 2,413,863k p=0.045 n=6
Parse Time 7.89s (± 0.48%) 7.83s (± 0.80%) ~ 7.75s 7.91s p=0.128 n=6
Bind Time 2.51s (± 0.59%) 2.53s (± 0.92%) ~ 2.50s 2.57s p=0.222 n=6
Check Time 50.12s (± 0.23%) 50.13s (± 0.25%) ~ 49.92s 50.27s p=0.575 n=6
Emit Time 4.88s (± 1.22%) 4.79s (± 1.00%) -0.10s (- 2.01%) 4.72s 4.84s p=0.019 n=6
Total Time 65.41s (± 0.23%) 65.29s (± 0.23%) ~ 65.02s 65.40s p=0.378 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,268 258,271 +3 (+ 0.00%) ~ ~ p=0.001 n=6
Types 104,775 104,779 +4 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 427,125k (± 0.03%) 427,113k (± 0.01%) ~ 427,043k 427,221k p=0.575 n=6
Parse Time 3.29s (± 0.78%) 3.29s (± 0.61%) ~ 3.26s 3.32s p=0.746 n=6
Bind Time 1.30s (± 1.05%) 1.32s (± 1.04%) ~ 1.29s 1.33s p=0.142 n=6
Check Time 17.77s (± 0.40%) 17.82s (± 0.29%) ~ 17.72s 17.86s p=0.296 n=6
Emit Time 1.64s (± 0.46%) 1.60s (± 0.75%) -0.03s (- 2.14%) 1.59s 1.62s p=0.005 n=6
Total Time 24.00s (± 0.39%) 24.03s (± 0.24%) ~ 23.94s 24.08s p=0.748 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,534k (± 0.04%) 369,523k (± 0.04%) ~ 369,388k 369,814k p=0.936 n=6
Parse Time 2.78s (± 1.58%) 2.77s (± 1.04%) ~ 2.74s 2.81s p=0.684 n=6
Bind Time 1.59s (± 1.08%) 1.59s (± 0.76%) ~ 1.58s 1.61s p=0.459 n=6
Check Time 15.52s (± 0.39%) 15.51s (± 0.41%) ~ 15.41s 15.59s p=1.000 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.89s (± 0.24%) 19.87s (± 0.32%) ~ 19.76s 19.93s p=0.748 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,889,341 2,889,341 ~ ~ ~ p=1.000 n=6
Types 977,546 977,546 ~ ~ ~ p=1.000 n=6
Memory used 3,052,777k (± 0.00%) 3,052,788k (± 0.00%) ~ 3,052,700k 3,052,869k p=0.575 n=6
Parse Time 16.81s (± 0.32%) 16.84s (± 0.49%) ~ 16.71s 16.95s p=0.688 n=6
Bind Time 5.11s (± 0.27%) 5.11s (± 0.53%) ~ 5.08s 5.14s p=1.000 n=6
Check Time 92.35s (± 2.99%) 91.35s (± 2.88%) ~ 89.34s 94.74s p=0.229 n=6
Emit Time 27.01s (± 9.59%) 27.65s (± 8.98%) ~ 24.34s 29.48s p=0.810 n=6
Total Time 141.29s (± 0.16%) 140.95s (± 0.23%) ~ 140.39s 141.32s p=0.066 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,235 267,235 ~ ~ ~ p=1.000 n=6
Types 108,815 108,815 ~ ~ ~ p=1.000 n=6
Memory used 411,783k (± 0.01%) 411,811k (± 0.01%) ~ 411,775k 411,856k p=0.298 n=6
Parse Time 3.19s (± 0.74%) 3.20s (± 0.47%) ~ 3.17s 3.21s p=0.461 n=6
Bind Time 1.40s (± 0.58%) 1.41s (± 0.59%) ~ 1.39s 1.41s p=0.718 n=6
Check Time 14.19s (± 0.29%) 14.22s (± 0.37%) ~ 14.15s 14.29s p=0.296 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.78s (± 0.31%) 18.82s (± 0.28%) ~ 18.75s 18.88s p=0.199 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,958k (± 0.08%) 463,184k (± 0.02%) ~ 463,030k 463,293k p=0.471 n=6
Parse Time 2.66s (± 0.57%) 2.65s (± 0.41%) ~ 2.64s 2.67s p=0.618 n=6
Bind Time 0.98s (± 0.84%) 0.98s (± 0.53%) ~ 0.97s 0.98s p=0.929 n=6
Check Time 15.19s (± 0.35%) 15.20s (± 0.27%) ~ 15.16s 15.27s p=0.810 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.82s (± 0.29%) 18.83s (± 0.24%) ~ 18.78s 18.91s p=0.747 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,295ms (± 0.18%) 2,292ms (± 0.60%) ~ 2,271ms 2,306ms p=0.873 n=6
Req 2 - geterr 5,082ms (± 0.77%) 5,080ms (± 0.29%) ~ 5,054ms 5,093ms p=0.810 n=6
Req 3 - references 265ms (± 0.44%) 265ms (± 0.28%) ~ 264ms 266ms p=0.933 n=6
Req 4 - navto 227ms (± 1.20%) 228ms (± 0.18%) ~ 227ms 228ms p=0.474 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 91ms (± 5.94%) 93ms (± 2.49%) ~ 91ms 97ms p=0.934 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,575ms (± 1.02%) 3,583ms (± 1.53%) ~ 3,525ms 3,657ms p=0.688 n=6
Req 2 - geterr 5,539ms (± 0.59%) 5,548ms (± 0.28%) ~ 5,526ms 5,569ms p=0.810 n=6
Req 3 - references 418ms (± 0.33%) 417ms (± 0.33%) ~ 415ms 419ms p=0.253 n=6
Req 4 - navto 338ms (± 0.71%) 339ms (± 1.03%) ~ 336ms 346ms p=0.319 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 109ms (± 2.59%) 109ms (± 2.78%) ~ 103ms 111ms p=0.858 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,187ms (± 0.69%) 5,205ms (± 0.50%) ~ 5,183ms 5,255ms p=0.689 n=6
Req 2 - geterr 1,125ms (± 1.27%) 1,120ms (± 0.94%) ~ 1,111ms 1,138ms p=0.688 n=6
Req 3 - references 80ms (± 5.27%) 77ms (± 3.11%) ~ 76ms 82ms p=0.325 n=6
Req 4 - navto 456ms (± 0.38%) 458ms (± 0.26%) ~ 457ms 460ms p=0.081 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 842ms (± 1.72%) 834ms (± 1.76%) ~ 819ms 856ms p=0.336 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.04ms (± 0.18%) 156.88ms (± 0.17%) -0.16ms (- 0.10%) 155.74ms 161.19ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 232.00ms (± 0.13%) 232.11ms (± 0.16%) +0.11ms (+ 0.05%) 230.57ms 237.89ms p=0.014 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 226.37ms (± 0.16%) 226.23ms (± 0.14%) -0.14ms (- 0.06%) 224.58ms 228.25ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 225.97ms (± 0.17%) 226.15ms (± 0.20%) +0.18ms (+ 0.08%) 224.42ms 234.03ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the user tests with tsc comparing main and refs/pull/58859/merge:

Something interesting changed - please have a look.

Details

effect

tsconfig.json

tsconfig.build.json

tsconfig.base.json

lodash

/mnt/ts_downloads/_/m/lodash/tsconfig.json

  • [NEW] error TS2740: Type 'SetCache' is missing the following properties from type 'any[]': length, pop, concat, join, and 34 more.
    • /mnt/ts_downloads/_/m/lodash/node_modules/lodash/_baseUniq.js(39,5)
  • [MISSING] error TS2322: Type 'SetCache' is not assignable to type 'any[]'.
    • /mnt/ts_downloads/_/m/lodash/node_modules/lodash/_baseUniq.js(39,5)

xterm.js

src/tsconfig-library-base.json

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the top 800 repos with tsc comparing main and refs/pull/58859/merge:

Everything looks good!

@gabritto gabritto marked this pull request as ready for review July 11, 2024 00:43
weswigham
weswigham approved these changes Jul 11, 2024
View reviewed changes
Copy link
Member

@weswigham weswigham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perf still looks fine, baselines look good, and I already approved the old version of the PR :)

@gabritto gabritto merged commit 0314372 into main Jul 15, 2024
29 checks passed
@gabritto gabritto deleted the gabritto/elaboration branch July 15, 2024 18:11
@Andarist Andarist mentioned this pull request Jul 15, 2024
Closed
@DanielRosenwasser DanielRosenwasser mentioned this pull request Jul 19, 2024
Open
@LukeAbby LukeAbby mentioned this pull request Oct 12, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@weswigham weswigham weswigham approved these changes

Assignees

@gabritto gabritto

Labels
Author: Team For Backlog Bug PRs that fix a backlog bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Diagnostic elaboration is not consistently reported in language service
4 participants
@gabritto @typescript-bot @weswigham @jakebailey