Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sort types for display #44614

Closed
wants to merge 6 commits into from
Closed

Sort types for display #44614

wants to merge 6 commits into from

Conversation

sandersn
Copy link
Member

Pushing to a PR for performance testing. This initial cut looks extremely slow.

Fixes #32224

Pushing to a PR for performance testing. This looks extremely slow.
@typescript-bot typescript-bot added Author: Team For Milestone Bug PRs that fix a bug with a specific milestone labels Jun 16, 2021
@sandersn
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 16, 2021

Heya @sandersn, I've started to run the perf test suite on this PR at 72a1808. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@sandersn
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..44614

Metric main 44614 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 344,262k (± 0.02%) 344,643k (± 0.02%) +382k (+ 0.11%) 344,545k 344,753k
Parse Time 1.80s (± 0.46%) 1.79s (± 0.36%) -0.00s (- 0.22%) 1.77s 1.80s
Bind Time 0.84s (± 0.69%) 0.83s (± 0.59%) -0.00s (- 0.36%) 0.83s 0.85s
Check Time 5.21s (± 0.45%) 5.20s (± 0.37%) -0.02s (- 0.29%) 5.15s 5.25s
Emit Time 5.49s (± 0.96%) 5.47s (± 0.52%) -0.03s (- 0.47%) 5.41s 5.53s
Total Time 13.34s (± 0.56%) 13.30s (± 0.25%) -0.04s (- 0.31%) 13.21s 13.37s
Compiler-Unions - node (v10.16.3, x64)
Memory used 200,493k (± 0.03%) 200,949k (± 0.03%) +456k (+ 0.23%) 200,811k 201,030k
Parse Time 0.78s (± 1.00%) 0.77s (± 0.64%) -0.01s (- 0.64%) 0.77s 0.79s
Bind Time 0.53s (± 1.42%) 0.53s (± 1.29%) -0.00s (- 0.19%) 0.51s 0.54s
Check Time 7.55s (± 0.56%) 7.52s (± 0.53%) -0.03s (- 0.40%) 7.43s 7.63s
Emit Time 2.24s (± 0.75%) 2.25s (± 0.91%) +0.01s (+ 0.36%) 2.21s 2.30s
Total Time 11.10s (± 0.42%) 11.07s (± 0.39%) -0.03s (- 0.29%) 10.97s 11.16s
Monaco - node (v10.16.3, x64)
Memory used 340,543k (± 0.02%) 340,787k (± 0.02%) +244k (+ 0.07%) 340,686k 341,061k
Parse Time 1.45s (± 0.68%) 1.45s (± 1.18%) -0.01s (- 0.41%) 1.40s 1.48s
Bind Time 0.74s (± 0.63%) 0.74s (± 0.79%) -0.00s (- 0.41%) 0.73s 0.75s
Check Time 5.36s (± 0.63%) 5.34s (± 0.25%) -0.02s (- 0.41%) 5.31s 5.37s
Emit Time 2.98s (± 1.13%) 2.95s (± 0.86%) -0.03s (- 1.11%) 2.92s 3.03s
Total Time 10.54s (± 0.48%) 10.48s (± 0.32%) -0.06s (- 0.55%) 10.41s 10.57s
TFS - node (v10.16.3, x64)
Memory used 304,171k (± 0.02%) 304,388k (± 0.02%) +217k (+ 0.07%) 304,262k 304,491k
Parse Time 1.18s (± 0.42%) 1.17s (± 0.57%) -0.00s (- 0.17%) 1.15s 1.18s
Bind Time 0.71s (± 0.73%) 0.70s (± 0.79%) -0.01s (- 1.27%) 0.69s 0.71s
Check Time 4.89s (± 0.47%) 4.88s (± 0.29%) -0.01s (- 0.16%) 4.86s 4.92s
Emit Time 3.07s (± 1.56%) 3.06s (± 1.10%) -0.01s (- 0.23%) 3.01s 3.16s
Total Time 9.84s (± 0.60%) 9.82s (± 0.46%) -0.02s (- 0.21%) 9.76s 9.96s
material-ui - node (v10.16.3, x64)
Memory used 473,969k (± 0.02%) 474,058k (± 0.01%) +89k (+ 0.02%) 473,974k 474,180k
Parse Time 1.73s (± 0.35%) 1.72s (± 0.44%) -0.01s (- 0.58%) 1.70s 1.73s
Bind Time 0.66s (± 0.72%) 0.66s (± 0.88%) -0.01s (- 0.90%) 0.64s 0.67s
Check Time 15.03s (± 0.51%) 14.77s (± 0.38%) -0.26s (- 1.73%) 14.66s 14.89s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 17.42s (± 0.47%) 17.14s (± 0.32%) -0.27s (- 1.58%) 17.04s 17.27s
Angular - node (v12.1.0, x64)
Memory used 322,503k (± 0.03%) 322,818k (± 0.02%) +315k (+ 0.10%) 322,686k 322,925k
Parse Time 1.77s (± 0.50%) 1.76s (± 0.68%) -0.00s (- 0.17%) 1.74s 1.80s
Bind Time 0.82s (± 0.83%) 0.82s (± 0.54%) -0.00s (- 0.24%) 0.81s 0.83s
Check Time 5.12s (± 0.46%) 5.08s (± 0.50%) -0.04s (- 0.70%) 5.02s 5.13s
Emit Time 5.68s (± 0.86%) 5.64s (± 0.50%) -0.04s (- 0.62%) 5.58s 5.72s
Total Time 13.39s (± 0.35%) 13.31s (± 0.29%) -0.08s (- 0.57%) 13.24s 13.40s
Compiler-Unions - node (v12.1.0, x64)
Memory used 187,760k (± 0.10%) 188,323k (± 0.09%) +563k (+ 0.30%) 187,648k 188,508k
Parse Time 0.77s (± 0.72%) 0.77s (± 1.18%) +0.00s (+ 0.13%) 0.75s 0.79s
Bind Time 0.53s (± 0.69%) 0.53s (± 0.65%) +0.00s (+ 0.19%) 0.52s 0.53s
Check Time 7.02s (± 0.54%) 7.01s (± 0.59%) -0.00s (- 0.06%) 6.95s 7.16s
Emit Time 2.24s (± 1.03%) 2.27s (± 1.36%) +0.03s (+ 1.25%) 2.22s 2.35s
Total Time 10.55s (± 0.36%) 10.58s (± 0.54%) +0.03s (+ 0.26%) 10.50s 10.72s
Monaco - node (v12.1.0, x64)
Memory used 323,536k (± 0.02%) 323,724k (± 0.02%) +188k (+ 0.06%) 323,607k 323,809k
Parse Time 1.42s (± 0.59%) 1.41s (± 0.53%) -0.01s (- 0.85%) 1.39s 1.42s
Bind Time 0.71s (± 1.05%) 0.71s (± 0.42%) -0.00s (- 0.14%) 0.71s 0.72s
Check Time 5.21s (± 0.37%) 5.19s (± 0.41%) -0.02s (- 0.44%) 5.16s 5.24s
Emit Time 3.00s (± 0.66%) 3.00s (± 0.42%) +0.00s (+ 0.13%) 2.97s 3.03s
Total Time 10.35s (± 0.31%) 10.31s (± 0.28%) -0.04s (- 0.34%) 10.25s 10.38s
TFS - node (v12.1.0, x64)
Memory used 288,651k (± 0.03%) 288,938k (± 0.02%) +287k (+ 0.10%) 288,835k 289,107k
Parse Time 1.19s (± 0.89%) 1.19s (± 0.56%) +0.00s (+ 0.08%) 1.17s 1.20s
Bind Time 0.69s (± 1.21%) 0.69s (± 1.02%) +0.00s (+ 0.44%) 0.67s 0.70s
Check Time 4.78s (± 0.36%) 4.76s (± 0.52%) -0.01s (- 0.29%) 4.73s 4.85s
Emit Time 3.09s (± 0.85%) 3.12s (± 1.12%) +0.03s (+ 0.87%) 3.05s 3.21s
Total Time 9.75s (± 0.36%) 9.77s (± 0.42%) +0.02s (+ 0.21%) 9.66s 9.86s
material-ui - node (v12.1.0, x64)
Memory used 452,770k (± 0.01%) 452,812k (± 0.01%) +42k (+ 0.01%) 452,668k 452,911k
Parse Time 1.71s (± 0.62%) 1.71s (± 0.56%) -0.01s (- 0.58%) 1.69s 1.73s
Bind Time 0.64s (± 1.05%) 0.63s (± 0.94%) -0.00s (- 0.47%) 0.62s 0.65s
Check Time 13.43s (± 0.45%) 13.31s (± 0.28%) -0.12s (- 0.90%) 13.24s 13.41s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.79s (± 0.45%) 15.65s (± 0.27%) -0.13s (- 0.84%) 15.57s 15.75s
Angular - node (v14.15.1, x64)
Memory used 321,215k (± 0.01%) 321,430k (± 0.01%) +215k (+ 0.07%) 321,381k 321,480k
Parse Time 1.79s (± 0.43%) 1.79s (± 0.39%) 0.00s ( 0.00%) 1.78s 1.81s
Bind Time 0.87s (± 1.00%) 0.87s (± 0.86%) -0.00s (- 0.12%) 0.86s 0.89s
Check Time 5.12s (± 0.44%) 5.11s (± 0.52%) -0.01s (- 0.12%) 5.05s 5.18s
Emit Time 5.72s (± 0.50%) 5.67s (± 0.62%) -0.06s (- 0.98%) 5.61s 5.75s
Total Time 13.50s (± 0.28%) 13.44s (± 0.32%) -0.06s (- 0.45%) 13.35s 13.53s
Compiler-Unions - node (v14.15.1, x64)
Memory used 188,899k (± 0.57%) 188,704k (± 0.68%) -196k (- 0.10%) 186,231k 190,483k
Parse Time 0.81s (± 0.92%) 0.80s (± 0.37%) -0.01s (- 0.99%) 0.80s 0.81s
Bind Time 0.56s (± 0.53%) 0.55s (± 0.54%) -0.01s (- 1.08%) 0.55s 0.56s
Check Time 7.16s (± 0.53%) 7.16s (± 0.54%) -0.01s (- 0.10%) 7.10s 7.27s
Emit Time 2.27s (± 1.14%) 2.28s (± 1.36%) +0.00s (+ 0.09%) 2.20s 2.33s
Total Time 10.81s (± 0.47%) 10.79s (± 0.38%) -0.02s (- 0.20%) 10.71s 10.86s
Monaco - node (v14.15.1, x64)
Memory used 322,583k (± 0.01%) 322,715k (± 0.01%) +132k (+ 0.04%) 322,674k 322,752k
Parse Time 1.47s (± 0.53%) 1.47s (± 0.61%) -0.00s (- 0.27%) 1.45s 1.48s
Bind Time 0.74s (± 1.10%) 0.75s (± 1.46%) +0.01s (+ 0.67%) 0.73s 0.78s
Check Time 5.18s (± 0.53%) 5.17s (± 0.55%) -0.01s (- 0.21%) 5.11s 5.23s
Emit Time 3.07s (± 0.52%) 3.05s (± 0.48%) -0.02s (- 0.75%) 3.00s 3.08s
Total Time 10.47s (± 0.29%) 10.44s (± 0.36%) -0.03s (- 0.29%) 10.36s 10.52s
TFS - node (v14.15.1, x64)
Memory used 287,688k (± 0.01%) 287,943k (± 0.01%) +255k (+ 0.09%) 287,910k 287,972k
Parse Time 1.26s (± 1.74%) 1.28s (± 1.97%) +0.02s (+ 1.83%) 1.23s 1.32s
Bind Time 0.71s (± 0.63%) 0.71s (± 0.42%) 0.00s ( 0.00%) 0.71s 0.72s
Check Time 4.80s (± 0.31%) 4.79s (± 0.27%) -0.01s (- 0.25%) 4.76s 4.82s
Emit Time 3.21s (± 0.73%) 3.21s (± 0.57%) +0.00s (+ 0.06%) 3.17s 3.25s
Total Time 9.98s (± 0.27%) 9.99s (± 0.40%) +0.01s (+ 0.11%) 9.92s 10.06s
material-ui - node (v14.15.1, x64)
Memory used 451,067k (± 0.01%) 450,966k (± 0.01%) -101k (- 0.02%) 450,900k 451,050k
Parse Time 1.74s (± 0.60%) 1.74s (± 0.47%) -0.01s (- 0.34%) 1.72s 1.75s
Bind Time 0.69s (± 0.65%) 0.69s (± 0.32%) +0.00s (+ 0.15%) 0.68s 0.69s
Check Time 13.61s (± 0.50%) 13.50s (± 0.45%) -0.10s (- 0.74%) 13.40s 13.72s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.04s (± 0.43%) 15.93s (± 0.35%) -0.11s (- 0.66%) 15.84s 16.13s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-206-generic
Architecturex64
Available Memory16 GB
Available Memory1 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v10.16.3, x64)
  • Compiler-Unions - node (v12.1.0, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v14.15.1, x64)
Benchmark Name Iterations
Current 44614 10
Baseline main 10

Developer Information:

Download Benchmark

@@ -365,6 +365,93 @@ namespace ts {

const argumentsSymbol = createSymbol(SymbolFlags.Property, "arguments" as __String);
const requireSymbol = createSymbol(SymbolFlags.Property, "require" as __String);
const fail = (name: string) => () => Debug.fail("Unit kind expected for " + name)
// boolean, bigint, number, string, symbol [literal], boolean, bigint, number, string, symbol, object, union (recursive?!), ..., unknown, never, null, void, undefined, any
const ordre: Array<[TypeFlags, (t1: Type, t2: Type) => Comparison]> = [
Copy link
Contributor

@a-tarasyuk a-tarasyuk Jun 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

order?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a draft, so this is just the name that fell out of my brain and it happened to be in French. (I often use short, intentionally bad names so I'll be forced to change them later.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it :)

@DanielRosenwasser
Copy link
Member

Do we have declaration emit captured in the perf tests?

#44033

@sandersn
Copy link
Member Author

@amcasey seemed to think not, so on his recommendation I have been testing by compiling tinymce instead. However, it has around 1000 unions, which may not be enough to detect a worst-case slowdown. Maybe I should find a gnarly DT package, read it, then print it out and see how long printing it takes.

@sandersn
Copy link
Member Author

Notes from discussion in our team meeting:

  1. typeToString already puts null/undefined at end, need to dedupe this code
  2. enums should probably maintain order
  3. true | false collapses to boolean
  4. should maintain order of properties dumped via keyof
  5. decide what to do with origin types from denormalised unions

@sandersn
Copy link
Member Author

sandersn commented Jun 22, 2021

Well, this was an interesting experiment, but after finishing the sort code, I found the best performance test is our type baselines. I just ran time gulp runtests-parallel and looked at the amount of CPU time spent. It's 10% slower even skipping the less-important-to-sort types. So I don't think this is a viable approach. Probably the only thing that would work is to sort during union creation.

Note that the type baselines are a worst-case test of performance. Lots of projects wouldn't slow down that much, but really big ones with a lot of types generated across modules would resemble the type baselines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Milestone Bug PRs that fix a bug with a specific milestone
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature request] sort literals in .d.ts
4 participants