Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the performance of requesting completions within a massive array literal #40953

Merged

Conversation

weswigham
Copy link
Member

By binary searching for the first child whose end position is greater than the input position, rather than scanning them all in order (since tokens and nodes are generally sorted by both their pos and end in lists of children). This takes the runtime of the example given in the linked issue (which is the test verbatim) down from "will this ever end, I've waited at least 5 minutes" to a somewhat more reasonable 11s.

Fixes #40100

@typescript-bot typescript-bot assigned weswigham and unassigned weswigham Oct 5, 2020
@typescript-bot typescript-bot added Author: Team For Milestone Bug PRs that fix a bug with a specific milestone labels Oct 5, 2020
@weswigham
Copy link
Member Author

@typescript-bot perf test this - this shouldn't affect perf tests here at all, since this is only used in a LS codepath, which the perf tests don't really do; as such I expect no changes.

@minestarks I assume the crawler is collecting some kind of perf metrics (?) - should we be monitoring it for a before/after type picture with this change? While this definitely helps pathological cases like this one, it does incur some small overhead in more typical cases where nodes have only one or two children; it'd be nice to know if I needed to special case the small-N scenario to still be a linear scan, but I can only really tell if it is necessary with some data.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 5, 2020

Heya @weswigham, I've started to run the perf test suite on this PR at da57f98. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..40953

Metric master 40953 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 349,757k (± 0.02%) 349,753k (± 0.03%) -4k (- 0.00%) 349,515k 349,949k
Parse Time 2.01s (± 0.80%) 2.00s (± 0.58%) -0.00s (- 0.25%) 1.98s 2.04s
Bind Time 0.82s (± 1.07%) 0.82s (± 0.49%) -0.00s (- 0.24%) 0.81s 0.83s
Check Time 4.94s (± 0.54%) 4.93s (± 0.44%) -0.01s (- 0.16%) 4.87s 4.98s
Emit Time 5.19s (± 0.41%) 5.16s (± 0.61%) -0.03s (- 0.54%) 5.11s 5.24s
Total Time 12.96s (± 0.34%) 12.91s (± 0.21%) -0.05s (- 0.37%) 12.86s 12.97s
Monaco - node (v10.16.3, x64)
Memory used 354,351k (± 0.02%) 354,397k (± 0.02%) +46k (+ 0.01%) 354,251k 354,546k
Parse Time 1.57s (± 0.57%) 1.57s (± 0.64%) +0.00s (+ 0.06%) 1.55s 1.59s
Bind Time 0.71s (± 0.56%) 0.71s (± 0.78%) -0.00s (- 0.42%) 0.70s 0.72s
Check Time 5.07s (± 0.54%) 5.11s (± 0.63%) +0.04s (+ 0.69%) 5.03s 5.17s
Emit Time 2.76s (± 0.93%) 2.77s (± 1.04%) +0.00s (+ 0.14%) 2.72s 2.87s
Total Time 10.11s (± 0.37%) 10.15s (± 0.47%) +0.04s (+ 0.37%) 10.05s 10.26s
TFS - node (v10.16.3, x64)
Memory used 307,640k (± 0.04%) 307,592k (± 0.03%) -49k (- 0.02%) 307,438k 307,830k
Parse Time 1.22s (± 0.76%) 1.22s (± 0.67%) -0.00s (- 0.41%) 1.20s 1.24s
Bind Time 0.66s (± 1.37%) 0.67s (± 1.16%) +0.01s (+ 1.21%) 0.64s 0.68s
Check Time 4.57s (± 0.94%) 4.53s (± 0.53%) -0.04s (- 0.81%) 4.47s 4.58s
Emit Time 2.90s (± 1.72%) 2.90s (± 0.60%) +0.01s (+ 0.17%) 2.87s 2.94s
Total Time 9.35s (± 0.59%) 9.32s (± 0.34%) -0.03s (- 0.32%) 9.22s 9.37s
material-ui - node (v10.16.3, x64)
Memory used 489,125k (± 0.02%) 489,069k (± 0.02%) -56k (- 0.01%) 488,954k 489,245k
Parse Time 1.99s (± 0.54%) 1.99s (± 0.34%) -0.00s (- 0.20%) 1.97s 2.00s
Bind Time 0.65s (± 0.90%) 0.65s (± 0.76%) -0.00s (- 0.15%) 0.64s 0.66s
Check Time 13.42s (± 0.51%) 13.41s (± 0.51%) -0.01s (- 0.07%) 13.27s 13.54s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.06s (± 0.45%) 16.05s (± 0.43%) -0.02s (- 0.09%) 15.90s 16.19s
Angular - node (v12.1.0, x64)
Memory used 326,934k (± 0.02%) 326,877k (± 0.02%) -57k (- 0.02%) 326,705k 327,020k
Parse Time 2.00s (± 0.53%) 1.99s (± 0.77%) -0.00s (- 0.25%) 1.97s 2.04s
Bind Time 0.81s (± 1.12%) 0.81s (± 0.37%) -0.00s (- 0.12%) 0.80s 0.81s
Check Time 4.83s (± 0.40%) 4.83s (± 0.51%) -0.01s (- 0.10%) 4.77s 4.90s
Emit Time 5.39s (± 0.84%) 5.37s (± 0.63%) -0.03s (- 0.48%) 5.33s 5.48s
Total Time 13.04s (± 0.52%) 13.00s (± 0.40%) -0.04s (- 0.28%) 12.91s 13.11s
Monaco - node (v12.1.0, x64)
Memory used 336,578k (± 0.02%) 336,637k (± 0.03%) +59k (+ 0.02%) 336,501k 336,913k
Parse Time 1.54s (± 0.72%) 1.53s (± 0.69%) -0.00s (- 0.20%) 1.50s 1.55s
Bind Time 0.69s (± 0.58%) 0.69s (± 0.58%) +0.00s (+ 0.29%) 0.68s 0.70s
Check Time 4.88s (± 0.50%) 4.87s (± 0.32%) -0.01s (- 0.23%) 4.84s 4.90s
Emit Time 2.82s (± 0.91%) 2.81s (± 0.77%) -0.01s (- 0.50%) 2.75s 2.85s
Total Time 9.93s (± 0.39%) 9.90s (± 0.22%) -0.02s (- 0.25%) 9.87s 9.97s
TFS - node (v12.1.0, x64)
Memory used 291,933k (± 0.03%) 291,896k (± 0.02%) -37k (- 0.01%) 291,741k 292,012k
Parse Time 1.23s (± 0.49%) 1.24s (± 0.60%) +0.01s (+ 0.98%) 1.22s 1.25s
Bind Time 0.65s (± 1.38%) 0.64s (± 1.01%) -0.00s (- 0.62%) 0.63s 0.66s
Check Time 4.47s (± 0.30%) 4.46s (± 0.43%) -0.01s (- 0.13%) 4.40s 4.49s
Emit Time 2.92s (± 0.99%) 2.92s (± 0.55%) -0.00s (- 0.10%) 2.86s 2.94s
Total Time 9.26s (± 0.36%) 9.25s (± 0.27%) -0.00s (- 0.03%) 9.17s 9.29s
material-ui - node (v12.1.0, x64)
Memory used 467,034k (± 0.05%) 467,127k (± 0.01%) +93k (+ 0.02%) 467,030k 467,247k
Parse Time 2.01s (± 0.37%) 2.01s (± 0.47%) -0.01s (- 0.30%) 1.99s 2.03s
Bind Time 0.64s (± 1.05%) 0.64s (± 0.58%) -0.00s (- 0.31%) 0.63s 0.64s
Check Time 12.05s (± 0.94%) 11.95s (± 0.37%) -0.10s (- 0.85%) 11.86s 12.03s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.70s (± 0.81%) 14.59s (± 0.26%) -0.11s (- 0.76%) 14.49s 14.66s
Angular - node (v8.9.0, x64)
Memory used 346,442k (± 0.01%) 346,394k (± 0.02%) -48k (- 0.01%) 346,294k 346,529k
Parse Time 2.54s (± 0.51%) 2.54s (± 0.52%) -0.00s (- 0.12%) 2.52s 2.57s
Bind Time 0.85s (± 0.43%) 0.86s (± 0.90%) +0.01s (+ 0.82%) 0.84s 0.88s
Check Time 5.56s (± 0.52%) 5.58s (± 0.68%) +0.02s (+ 0.43%) 5.51s 5.67s
Emit Time 6.15s (± 1.07%) 6.05s (± 1.59%) -0.09s (- 1.53%) 5.80s 6.20s
Total Time 15.11s (± 0.55%) 15.04s (± 0.40%) -0.07s (- 0.45%) 14.89s 15.15s
Monaco - node (v8.9.0, x64)
Memory used 355,704k (± 0.02%) 355,737k (± 0.02%) +32k (+ 0.01%) 355,564k 355,882k
Parse Time 1.89s (± 0.77%) 1.88s (± 0.31%) -0.01s (- 0.48%) 1.86s 1.89s
Bind Time 0.89s (± 0.45%) 0.90s (± 0.58%) +0.01s (+ 0.90%) 0.88s 0.91s
Check Time 5.62s (± 0.37%) 5.62s (± 0.34%) -0.00s (- 0.04%) 5.59s 5.68s
Emit Time 3.29s (± 1.17%) 3.34s (± 0.96%) +0.06s (+ 1.73%) 3.24s 3.40s
Total Time 11.69s (± 0.28%) 11.74s (± 0.37%) +0.05s (+ 0.46%) 11.62s 11.85s
TFS - node (v8.9.0, x64)
Memory used 309,391k (± 0.02%) 309,349k (± 0.02%) -43k (- 0.01%) 309,237k 309,529k
Parse Time 1.55s (± 0.64%) 1.56s (± 0.39%) +0.00s (+ 0.13%) 1.54s 1.57s
Bind Time 0.68s (± 0.73%) 0.68s (± 0.77%) +0.00s (+ 0.44%) 0.67s 0.69s
Check Time 5.31s (± 0.58%) 5.29s (± 0.53%) -0.02s (- 0.28%) 5.23s 5.35s
Emit Time 2.94s (± 0.34%) 2.91s (± 0.81%) -0.03s (- 0.92%) 2.84s 2.97s
Total Time 10.48s (± 0.34%) 10.44s (± 0.37%) -0.04s (- 0.38%) 10.33s 10.52s
material-ui - node (v8.9.0, x64)
Memory used 493,431k (± 0.01%) 493,479k (± 0.01%) +48k (+ 0.01%) 493,414k 493,548k
Parse Time 2.40s (± 0.46%) 2.41s (± 0.34%) +0.00s (+ 0.17%) 2.39s 2.43s
Bind Time 0.81s (± 1.02%) 0.81s (± 1.16%) -0.00s (- 0.49%) 0.79s 0.83s
Check Time 17.95s (± 0.69%) 17.88s (± 0.76%) -0.07s (- 0.41%) 17.46s 18.08s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 21.17s (± 0.56%) 21.10s (± 0.67%) -0.07s (- 0.34%) 20.68s 21.32s
Angular - node (v8.9.0, x86)
Memory used 198,690k (± 0.01%) 198,692k (± 0.04%) +2k (+ 0.00%) 198,523k 198,829k
Parse Time 2.48s (± 1.02%) 2.48s (± 0.62%) +0.00s (+ 0.04%) 2.44s 2.50s
Bind Time 1.00s (± 1.00%) 1.00s (± 0.76%) +0.00s (+ 0.20%) 0.99s 1.02s
Check Time 5.05s (± 0.56%) 5.03s (± 0.75%) -0.01s (- 0.28%) 4.93s 5.12s
Emit Time 5.89s (± 0.83%) 5.89s (± 0.74%) +0.01s (+ 0.10%) 5.76s 5.97s
Total Time 14.42s (± 0.58%) 14.41s (± 0.48%) -0.00s (- 0.03%) 14.25s 14.55s
Monaco - node (v8.9.0, x86)
Memory used 201,493k (± 0.01%) 201,479k (± 0.02%) -15k (- 0.01%) 201,417k 201,559k
Parse Time 1.94s (± 1.05%) 1.91s (± 0.60%) -0.03s (- 1.60%) 1.89s 1.95s
Bind Time 0.71s (± 0.47%) 0.70s (± 0.53%) -0.01s (- 0.70%) 0.70s 0.71s
Check Time 5.47s (± 0.38%) 5.44s (± 0.72%) -0.02s (- 0.44%) 5.36s 5.53s
Emit Time 3.06s (± 0.59%) 3.06s (± 0.41%) -0.00s (- 0.03%) 3.02s 3.08s
Total Time 11.18s (± 0.25%) 11.12s (± 0.42%) -0.06s (- 0.54%) 11.02s 11.22s
TFS - node (v8.9.0, x86)
Memory used 176,867k (± 0.03%) 176,871k (± 0.02%) +4k (+ 0.00%) 176,775k 176,941k
Parse Time 1.60s (± 1.10%) 1.59s (± 0.63%) -0.02s (- 1.12%) 1.57s 1.62s
Bind Time 0.66s (± 2.04%) 0.65s (± 1.05%) -0.01s (- 1.37%) 0.64s 0.67s
Check Time 4.82s (± 0.58%) 4.83s (± 0.67%) +0.00s (+ 0.08%) 4.73s 4.89s
Emit Time 2.80s (± 0.66%) 2.79s (± 1.71%) -0.01s (- 0.50%) 2.71s 2.95s
Total Time 9.88s (± 0.51%) 9.85s (± 0.76%) -0.03s (- 0.33%) 9.68s 10.10s
material-ui - node (v8.9.0, x86)
Memory used 277,878k (± 0.02%) 277,845k (± 0.03%) -32k (- 0.01%) 277,765k 278,072k
Parse Time 2.46s (± 0.43%) 2.47s (± 0.53%) +0.01s (+ 0.49%) 2.44s 2.50s
Bind Time 0.69s (± 1.35%) 0.72s (± 6.63%) +0.03s (+ 4.94%) 0.68s 0.85s
Check Time 16.45s (± 0.73%) 16.45s (± 0.78%) -0.00s (- 0.03%) 16.19s 16.77s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 19.60s (± 0.63%) 19.64s (± 0.66%) +0.04s (+ 0.21%) 19.47s 19.93s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-166-generic
Architecturex64
Available Memory16 GB
Available Memory3 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v8.9.0, x64)
  • material-ui - node (v8.9.0, x86)
Benchmark Name Iterations
Current 40953 10
Baseline master 10

Copy link
Member

@amcasey amcasey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love the idea, didn't carefully review the impl, wonder if we could just use a helper: binarySearchKey(children, position + 1, n => n.end, compareValues)

src/services/utilities.ts Outdated Show resolved Hide resolved
src/services/utilities.ts Outdated Show resolved Hide resolved
@amcasey
Copy link
Member

amcasey commented Oct 6, 2020

Does the fuzzer even have coverage for completion?

@weswigham
Copy link
Member Author

Love the idea, didn't carefully review the impl, wonder if we could just use a helper: binarySearchKey(children, position + 1, n => n.end, compareValues)

Nah, that condition doesn't work - it hangs forever, same as current search, and fails some tests besides (just tried it again, just in case I missed something). As I said in my comment - the compareValues function actually needs to be quite complicated, since it needs to actually look at a sliding window of two elements to know which element satisfies the condition first (and since the callback doesn't pass indexes, I don't even think it can perform a comparison like that). It's easier to just have the core binary search algorithm in-line due to that complexity.

@weswigham
Copy link
Member Author

Ehh, I still rephrased the code to use binarySearchKey, I just had to modify it to pass the index to the keySelector. This should handle all of your comments (assuming you're fine with the modification to binarySearchKey), I think?

@weswigham weswigham requested a review from amcasey October 6, 2020 19:34
@@ -1173,7 +1173,17 @@ namespace ts {
}

const children = n.getChildren(sourceFile);
for (let i = 0; i < children.length; i++) {
const i = binarySearchKey(children, position, (_, i) => i, (middle, _) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A comparer that ignores one argument is definitely not more readable than what you started with.

Copy link
Member Author

@weswigham weswigham Oct 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So... that we call it a "comparer" is itself kinda a mistake on our part. Yes, you can pass a comparer and it does what you might want, but we reliably pass the second argument to binarySearchKey as the second argument to the "comparer" - it never changes. I could choose to not ignore it, but there's no reason to - it's the same constant position value I close over.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it's more of a "selector" callback - it just needs to return if you "select the pivot element", "select the left group", or "select the right group". The Comparison values are just mapped to that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a comment would be helpful here.

@weswigham weswigham merged commit 4dc7e59 into microsoft:master Oct 7, 2020
@weswigham weswigham deleted the improve-completions-large-literal-perf branch October 7, 2020 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Milestone Bug PRs that fix a bug with a specific milestone
Projects
None yet
Development

Successfully merging this pull request may close these issues.

getCompletions in a very large array literal causes tsserver to spin
3 participants