Skip to content

feat(#48863): Split ES5 lib into individual parts #49853

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

iainjreid
Copy link

This PR hopes to address #48863, the scale of the changes is small, and non breaking, but the impact on the tests is pretty huge. More context can be found in the linked issue explaining the changes in more depth.

@ghost
Copy link

ghost commented Jul 10, 2022

CLA assistant check
All CLA requirements met.

@typescript-bot
Copy link
Collaborator

This PR doesn't have any linked issues. Please open an issue that references this PR. From there we can discuss and prioritise.

@typescript-bot typescript-bot added the For Uncommitted Bug PR for untriaged, rejected, closed or missing bug label Jul 10, 2022
@iainjreid
Copy link
Author

Some backstory and a lesson learned whilst implementing these changes...

I originally tried to exclude the new es2009.x library files by passing ["lib.*"] as the exclude argument to tryReadDirectory to reduce the impact on the tests, but I found that doing so actually added more complexity than it would have otherwise solved...

@sandersn
Copy link
Member

@typescript-bot perf test this

(though I'm not sure how well the perf tests measure startup time)

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 18, 2022

Heya @sandersn, I've started to run the perf test suite on this PR at 0cbbb34. You can monitor the build here.

Update: The results are in!

@sandersn sandersn self-assigned this Jul 18, 2022
@sandersn
Copy link
Member

I feel like this is something we need to discuss in a design meeting before reviewing since the code change is fairly simple but also hugely susceptible to going stale. Let's see what performance is like and then I'd vote to close the PR until the design is finalised.

@typescript-bot
Copy link
Collaborator

@sandersn
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..49853
Metric main 49853 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 360,110k (± 0.01%) 361,057k (± 0.02%) +947k (+ 0.26%) 360,903k 361,167k
Parse Time 2.09s (± 0.47%) 2.12s (± 0.59%) +0.03s (+ 1.34%) 2.09s 2.16s
Bind Time 0.89s (± 0.58%) 0.90s (± 0.74%) +0.01s (+ 1.57%) 0.89s 0.92s
Check Time 6.00s (± 0.52%) 5.98s (± 0.43%) -0.02s (- 0.30%) 5.92s 6.04s
Emit Time 6.19s (± 1.12%) 6.09s (± 0.47%) -0.10s (- 1.58%) 6.03s 6.16s
Total Time 15.16s (± 0.59%) 15.09s (± 0.34%) -0.07s (- 0.49%) 14.99s 15.20s
Compiler-Unions - node (v10.16.3, x64)
Memory used 206,921k (± 0.02%) 207,798k (± 0.04%) +877k (+ 0.42%) 207,550k 207,955k
Parse Time 0.83s (± 0.91%) 0.85s (± 0.52%) +0.02s (+ 2.16%) 0.84s 0.86s
Bind Time 0.53s (± 1.83%) 0.54s (± 2.30%) +0.01s (+ 2.08%) 0.52s 0.57s
Check Time 7.10s (± 0.41%) 7.10s (± 0.75%) +0.00s (+ 0.03%) 7.01s 7.21s
Emit Time 2.52s (± 1.12%) 2.52s (± 0.97%) +0.00s (+ 0.12%) 2.46s 2.57s
Total Time 10.99s (± 0.43%) 11.02s (± 0.67%) +0.03s (+ 0.31%) 10.88s 11.17s
Monaco - node (v10.16.3, x64)
Memory used 343,920k (± 0.02%) 343,879k (± 0.01%) -41k (- 0.01%) 343,752k 343,951k
Parse Time 1.59s (± 0.49%) 1.59s (± 0.67%) +0.00s (+ 0.19%) 1.57s 1.62s
Bind Time 0.75s (± 0.77%) 0.75s (± 0.74%) -0.00s (- 0.13%) 0.74s 0.76s
Check Time 5.97s (± 0.49%) 5.98s (± 0.41%) +0.01s (+ 0.17%) 5.92s 6.03s
Emit Time 3.26s (± 0.90%) 3.25s (± 0.39%) -0.01s (- 0.31%) 3.22s 3.29s
Total Time 11.58s (± 0.50%) 11.58s (± 0.30%) -0.00s (- 0.02%) 11.50s 11.68s
TFS - node (v10.16.3, x64)
Memory used 305,135k (± 0.02%) 305,138k (± 0.03%) +3k (+ 0.00%) 304,957k 305,296k
Parse Time 1.28s (± 0.51%) 1.29s (± 0.38%) +0.01s (+ 0.63%) 1.28s 1.30s
Bind Time 0.72s (± 0.69%) 0.71s (± 0.70%) -0.00s (- 0.14%) 0.70s 0.72s
Check Time 5.43s (± 0.39%) 5.39s (± 0.58%) -0.04s (- 0.83%) 5.32s 5.44s
Emit Time 3.44s (± 1.04%) 3.44s (± 0.87%) -0.00s (- 0.09%) 3.38s 3.51s
Total Time 10.86s (± 0.39%) 10.82s (± 0.39%) -0.04s (- 0.39%) 10.76s 10.97s
material-ui - node (v10.16.3, x64)
Memory used 469,418k (± 0.02%) 470,324k (± 0.01%) +906k (+ 0.19%) 470,225k 470,502k
Parse Time 1.83s (± 0.91%) 1.84s (± 0.74%) +0.01s (+ 0.77%) 1.82s 1.87s
Bind Time 0.68s (± 1.55%) 0.70s (± 1.26%) +0.02s (+ 2.34%) 0.67s 0.71s
Check Time 14.45s (± 0.42%) 14.50s (± 0.69%) +0.06s (+ 0.39%) 14.29s 14.77s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.96s (± 0.38%) 17.04s (± 0.65%) +0.09s (+ 0.51%) 16.82s 17.34s
xstate - node (v10.16.3, x64)
Memory used 578,065k (± 0.02%) 588,018k (± 1.66%) +9,953k (+ 1.72%) 581,389k 614,505k
Parse Time 2.60s (± 0.52%) 2.69s (± 0.32%) +0.09s (+ 3.58%) 2.67s 2.71s
Bind Time 1.02s (± 0.73%) 1.04s (± 0.91%) +0.01s (+ 1.37%) 1.01s 1.05s
Check Time 1.57s (± 0.48%) 1.55s (± 0.50%) -0.02s (- 1.02%) 1.54s 1.57s
Emit Time 0.07s (± 3.14%) 0.08s (± 4.79%) +0.01s (+ 7.04%) 0.07s 0.08s
Total Time 5.25s (± 0.38%) 5.35s (± 0.25%) +0.10s (+ 1.86%) 5.33s 5.38s
Angular - node (v12.1.0, x64)
Memory used 337,523k (± 0.10%) 338,525k (± 0.02%) +1,002k (+ 0.30%) 338,368k 338,642k
Parse Time 2.09s (± 0.51%) 2.12s (± 0.96%) +0.03s (+ 1.53%) 2.09s 2.19s
Bind Time 0.85s (± 1.12%) 0.86s (± 0.64%) +0.01s (+ 0.82%) 0.85s 0.87s
Check Time 5.85s (± 0.60%) 5.78s (± 0.54%) -0.07s (- 1.15%) 5.73s 5.85s
Emit Time 6.44s (± 1.32%) 6.39s (± 0.54%) -0.05s (- 0.78%) 6.30s 6.47s
Total Time 15.23s (± 0.72%) 15.15s (± 0.40%) -0.08s (- 0.51%) 15.07s 15.32s
Compiler-Unions - node (v12.1.0, x64)
Memory used 194,229k (± 0.32%) 195,339k (± 0.03%) +1,110k (+ 0.57%) 195,204k 195,509k
Parse Time 0.83s (± 0.72%) 0.85s (± 0.61%) +0.02s (+ 2.78%) 0.84s 0.86s
Bind Time 0.55s (± 1.06%) 0.55s (± 0.62%) +0.01s (+ 1.10%) 0.55s 0.56s
Check Time 6.66s (± 0.46%) 6.69s (± 0.57%) +0.04s (+ 0.53%) 6.58s 6.75s
Emit Time 2.53s (± 0.96%) 2.56s (± 0.71%) +0.03s (+ 1.07%) 2.52s 2.59s
Total Time 10.56s (± 0.50%) 10.65s (± 0.43%) +0.09s (+ 0.85%) 10.51s 10.74s
Monaco - node (v12.1.0, x64)
Memory used 326,816k (± 0.02%) 326,812k (± 0.05%) -5k (- 0.00%) 326,178k 327,053k
Parse Time 1.58s (± 0.97%) 1.57s (± 0.65%) -0.01s (- 0.76%) 1.56s 1.60s
Bind Time 0.74s (± 0.40%) 0.75s (± 0.80%) +0.00s (+ 0.54%) 0.73s 0.76s
Check Time 5.80s (± 0.49%) 5.79s (± 0.47%) -0.02s (- 0.29%) 5.74s 5.85s
Emit Time 3.30s (± 0.70%) 3.32s (± 0.77%) +0.02s (+ 0.55%) 3.27s 3.37s
Total Time 11.43s (± 0.45%) 11.42s (± 0.39%) -0.01s (- 0.06%) 11.35s 11.54s
TFS - node (v12.1.0, x64)
Memory used 289,749k (± 0.02%) 289,722k (± 0.02%) -26k (- 0.01%) 289,563k 289,843k
Parse Time 1.30s (± 1.14%) 1.30s (± 1.02%) -0.01s (- 0.46%) 1.28s 1.33s
Bind Time 0.72s (± 0.62%) 0.71s (± 1.05%) -0.00s (- 0.70%) 0.70s 0.74s
Check Time 5.35s (± 0.47%) 5.35s (± 0.59%) -0.00s (- 0.02%) 5.28s 5.41s
Emit Time 3.54s (± 1.50%) 3.53s (± 1.06%) -0.01s (- 0.23%) 3.41s 3.59s
Total Time 10.91s (± 0.75%) 10.89s (± 0.59%) -0.02s (- 0.17%) 10.71s 10.98s
material-ui - node (v12.1.0, x64)
Memory used 448,411k (± 0.05%) 449,092k (± 0.08%) +681k (+ 0.15%) 448,028k 449,472k
Parse Time 1.83s (± 0.61%) 1.84s (± 0.43%) +0.01s (+ 0.60%) 1.83s 1.87s
Bind Time 0.67s (± 0.83%) 0.68s (± 0.76%) +0.01s (+ 1.34%) 0.67s 0.70s
Check Time 13.05s (± 0.37%) 13.08s (± 1.00%) +0.02s (+ 0.18%) 12.86s 13.36s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.56s (± 0.35%) 15.60s (± 0.84%) +0.05s (+ 0.30%) 15.37s 15.89s
xstate - node (v12.1.0, x64)
Memory used 543,390k (± 0.02%) 549,798k (± 1.30%) +6,408k (+ 1.18%) 546,469k 578,616k
Parse Time 2.54s (± 0.77%) 2.64s (± 0.41%) +0.09s (+ 3.69%) 2.62s 2.67s
Bind Time 1.03s (± 1.01%) 1.06s (± 0.78%) +0.03s (+ 2.72%) 1.04s 1.08s
Check Time 1.50s (± 0.79%) 1.50s (± 0.43%) -0.00s (- 0.00%) 1.49s 1.52s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.15s (± 0.65%) 5.27s (± 0.32%) +0.13s (+ 2.45%) 5.24s 5.32s
Angular - node (v14.15.1, x64)
Memory used 336,000k (± 0.01%) 336,802k (± 0.01%) +801k (+ 0.24%) 336,768k 336,851k
Parse Time 2.05s (± 0.46%) 2.08s (± 0.58%) +0.02s (+ 1.12%) 2.05s 2.10s
Bind Time 0.90s (± 0.75%) 0.90s (± 0.58%) +0.00s (+ 0.45%) 0.89s 0.91s
Check Time 5.79s (± 0.57%) 5.82s (± 0.37%) +0.03s (+ 0.48%) 5.76s 5.86s
Emit Time 6.33s (± 0.58%) 6.40s (± 0.86%) +0.06s (+ 0.96%) 6.31s 6.56s
Total Time 15.07s (± 0.46%) 15.19s (± 0.42%) +0.12s (+ 0.77%) 15.11s 15.36s
Compiler-Unions - node (v14.15.1, x64)
Memory used 193,136k (± 0.02%) 193,900k (± 0.02%) +764k (+ 0.40%) 193,833k 193,987k
Parse Time 0.84s (± 0.70%) 0.86s (± 0.34%) +0.02s (+ 2.13%) 0.86s 0.87s
Bind Time 0.58s (± 1.03%) 0.59s (± 1.01%) +0.01s (+ 1.74%) 0.57s 0.60s
Check Time 6.75s (± 0.62%) 6.71s (± 0.76%) -0.04s (- 0.53%) 6.61s 6.84s
Emit Time 2.49s (± 0.70%) 2.51s (± 1.03%) +0.02s (+ 0.96%) 2.45s 2.56s
Total Time 10.65s (± 0.49%) 10.67s (± 0.67%) +0.02s (+ 0.18%) 10.51s 10.83s
Monaco - node (v14.15.1, x64)
Memory used 325,616k (± 0.01%) 325,641k (± 0.00%) +25k (+ 0.01%) 325,611k 325,669k
Parse Time 1.57s (± 0.64%) 1.58s (± 0.56%) +0.00s (+ 0.25%) 1.56s 1.60s
Bind Time 0.78s (± 0.85%) 0.78s (± 0.74%) -0.00s (- 0.38%) 0.77s 0.79s
Check Time 5.65s (± 0.52%) 5.68s (± 0.55%) +0.03s (+ 0.48%) 5.62s 5.76s
Emit Time 3.37s (± 0.64%) 3.36s (± 0.95%) -0.01s (- 0.24%) 3.30s 3.47s
Total Time 11.38s (± 0.35%) 11.40s (± 0.49%) +0.02s (+ 0.15%) 11.29s 11.55s
TFS - node (v14.15.1, x64)
Memory used 288,797k (± 0.01%) 288,812k (± 0.01%) +14k (+ 0.00%) 288,768k 288,866k
Parse Time 1.35s (± 2.48%) 1.33s (± 1.79%) -0.02s (- 1.56%) 1.28s 1.38s
Bind Time 0.78s (± 4.49%) 0.78s (± 5.08%) -0.01s (- 0.64%) 0.73s 0.85s
Check Time 5.33s (± 0.47%) 5.34s (± 0.31%) +0.01s (+ 0.24%) 5.30s 5.37s
Emit Time 3.55s (± 2.08%) 3.64s (± 1.28%) +0.08s (+ 2.39%) 3.52s 3.72s
Total Time 11.00s (± 0.78%) 11.08s (± 0.55%) +0.07s (+ 0.66%) 10.90s 11.19s
material-ui - node (v14.15.1, x64)
Memory used 446,654k (± 0.01%) 447,433k (± 0.01%) +779k (+ 0.17%) 447,371k 447,498k
Parse Time 1.88s (± 0.52%) 1.90s (± 0.55%) +0.02s (+ 1.17%) 1.88s 1.93s
Bind Time 0.72s (± 1.21%) 0.72s (± 1.39%) +0.01s (+ 0.84%) 0.70s 0.74s
Check Time 13.15s (± 0.53%) 13.10s (± 0.30%) -0.05s (- 0.34%) 13.02s 13.20s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.74s (± 0.47%) 15.73s (± 0.27%) -0.02s (- 0.11%) 15.64s 15.83s
xstate - node (v14.15.1, x64)
Memory used 541,306k (± 0.00%) 544,406k (± 0.00%) +3,099k (+ 0.57%) 544,362k 544,440k
Parse Time 2.61s (± 0.48%) 2.70s (± 0.30%) +0.09s (+ 3.49%) 2.68s 2.71s
Bind Time 1.15s (± 1.23%) 1.18s (± 1.02%) +0.02s (+ 1.99%) 1.16s 1.21s
Check Time 1.54s (± 0.44%) 1.54s (± 0.32%) 0.00s ( 0.00%) 1.54s 1.56s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.38s (± 0.40%) 5.50s (± 0.31%) +0.12s (+ 2.19%) 5.46s 5.54s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-210-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v10.16.3, x64)
  • Compiler-Unions - node (v12.1.0, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v10.16.3, x64)
  • xstate - node (v12.1.0, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 49853 10
Baseline main 10

TSServer

Comparison Report - main..49853
Metric main 49853 Delta Best Worst
xstateTSServer - node (v10.16.3, x64)
Initial load time 2,153.40ms (± 0.49%) 2,214.50ms (± 0.26%) +61.10ms (+ 2.84%) 2,202.00ms 2,230.00ms
Geterr time 771.70ms (± 0.76%) 772.30ms (± 0.86%) +0.60ms (+ 0.08%) 759.00ms 784.00ms
References time 95.00ms (± 0.92%) 93.10ms (± 5.08%) -1.90ms (- 2.00%) 81.00ms 100.00ms
Navto time 252.70ms (± 1.02%) 254.20ms (± 0.39%) +1.50ms (+ 0.59%) 253.00ms 257.00ms
Navto count 256.00 (± 0.00%) 256.00 (± 0.00%) 0.00 ( 0.00%) 256.00 256.00
Completions time 244.40ms (± 1.11%) 244.00ms (± 1.15%) -0.40ms (- 0.16%) 238.00ms 252.00ms
Completions count 3,205.00 (± 0.00%) 3,205.00 (± 0.00%) 0.00 ( 0.00%) 3,205.00 3,205.00
xstateTSServer - node (v12.1.0, x64)
Initial load time 2,088.10ms (± 0.50%) 2,151.60ms (± 0.32%) +63.50ms (+ 3.04%) 2,135.00ms 2,165.00ms
Geterr time 762.80ms (± 0.79%) 756.90ms (± 0.81%) -5.90ms (- 0.77%) 744.00ms 776.00ms
References time 67.00ms (± 1.11%) 66.70ms (± 1.41%) -0.30ms (- 0.45%) 65.00ms 69.00ms
Navto time 240.90ms (± 0.59%) 242.50ms (± 0.48%) +1.60ms (+ 0.66%) 239.00ms 244.00ms
Navto count 256.00 (± 0.00%) 256.00 (± 0.00%) 0.00 ( 0.00%) 256.00 256.00
Completions time 240.20ms (± 0.86%) 241.40ms (± 1.14%) +1.20ms (+ 0.50%) 236.00ms 250.00ms
Completions count 3,205.00 (± 0.00%) 3,205.00 (± 0.00%) 0.00 ( 0.00%) 3,205.00 3,205.00
xstateTSServer - node (v14.15.1, x64)
Initial load time 2,230.20ms (± 0.70%) 2,257.30ms (± 0.67%) +27.10ms (+ 1.22%) 2,234.00ms 2,291.00ms
Geterr time 777.50ms (± 0.44%) 772.40ms (± 0.28%) -5.10ms (- 0.66%) 769.00ms 779.00ms
References time 66.00ms (± 1.88%) 64.60ms (± 1.56%) -1.40ms (- 2.12%) 63.00ms 68.00ms
Navto time 251.20ms (± 0.66%) 254.40ms (± 0.60%) +3.20ms (+ 1.27%) 252.00ms 259.00ms
Navto count 256.00 (± 0.00%) 256.00 (± 0.00%) 0.00 ( 0.00%) 256.00 256.00
Completions time 241.70ms (± 0.55%) 238.70ms (± 0.71%) -3.00ms (- 1.24%) 235.00ms 244.00ms
Completions count 3,205.00 (± 0.00%) 3,205.00 (± 0.00%) 0.00 ( 0.00%) 3,205.00 3,205.00
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-210-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v14.15.1, x64)
Scenarios
  • xstateTSServer - node (v10.16.3, x64)
  • xstateTSServer - node (v12.1.0, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 49853 10
Baseline main 10

Developer Information:

Download Benchmark

@iainjreid
Copy link
Author

@sandersn thanks for the update, I made a comment in my initial issue outlining a liking better solution to the functionality I'm looking for, but I still think that splitting up the ES5 library will be hugely beneficial to the end goal!

@sandersn
Copy link
Member

No significant performance difference that I noticed.

@sandersn
Copy link
Member

sandersn commented Mar 9, 2023

To help with PR housekeeping, I'm going to close this until its bug is ready.

@sandersn sandersn closed this Mar 9, 2023
@iainjreid
Copy link
Author

For future reference, what could I have done differently to get this one pushed through @sandersn? I understand that a lot of files were changed, and reviewing this would have taken some time, but the overall impact was very low, almost negligible.

I'm still keen to champion this improvement if you're accepting contributions from outside collaborators 👍

@sandersn
Copy link
Member

That's a tricky one. The state Awaiting More Feedback on the issue tracker means that we're waiting for more people to chime in with reasons why they want this feature. Sometimes that's to know whether more than one person wants it, sometimes that's to find out the variety of things that could be done, and to decide which should be done.

We get so many bugs and PRs that we generally don't review any outside PR until there's an accepted bug for it.

Finally, for this particular feature: changing the lib, even in obviously non-breaking ways, somehow ends up breaking people. So merging this PR would make us commit to finding and fixing breaks in important libraries before the next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants