Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve out of order op processing #4749

Merged
merged 4 commits into from
Jan 6, 2021

Conversation

GaryWilber
Copy link
Contributor

Consider the following scenario:

  1. Client loads a summary @ seq 150 and connects to the document
  2. Server sends the client their join op @ seq 160
    • This causes the client to add that op into pending and call fetchMissingDeltas to ask for ops 151 - 159
    • Lets say the storage api is not returning any deltas. At this point the client keeps retrying that call until it gets them
  3. Server sends the client ops 151 - 159 through the WebSocket
  4. Client processes ops 151 - 159

At this point the client has all the ops it needs to be fully connected. However deltaManager did not process op 160, which remains in pending. So the client is stuck waiting for fetchMissingDeltas to complete. This PR makes the client process that pending op so it won't get stuck.

Changes:

  • Process pending ops after successfully processing an inbound op
  • Stop trying to fetch deltas from storage if the delta manager already caught up
  • Replace catchUpCore method with processPendingOps method that only processes pending ops

@GaryWilber GaryWilber requested a review from vladsud January 6, 2021 19:22
if (to !== undefined && this.lastQueuedSequenceNumber >= to) {
// the client caught up while we were trying to fetch ops from storage
// bail out since we no longer need to request these ops
callback([]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think making an empty call is required, or we do it in other places.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The callback invokes this.refreshDelayInfo(this.deltaStorageDelayId);. If we skip this callback, that won't be called. Is that okay?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated it to not pass anything so only refreshDelayInfo is invoked.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's important. I think it's cleaner for fetchMissingDeltas() to call it after it's done.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume it still has to call refreshDelayInfo in every callback too?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

@msfluid-bot
Copy link
Collaborator

msfluid-bot commented Jan 6, 2021

@fluidframework/base-host: +267 Bytes
Metric NameBaseline SizeCompare SizeSize Diff
main.js 166.94 KB 167.2 KB +267 Bytes
Total Size 166.94 KB 167.2 KB +267 Bytes
@fluid-example/bundle-size-tests: No change
Metric NameBaseline SizeCompare SizeSize Diff
container.js 191.32 KB 191.32 KB No change
map.js 45.84 KB 45.84 KB No change
matrix.js 144.52 KB 144.52 KB No change
odspDriver.js 193.22 KB 193.22 KB No change
sharedString.js 158.46 KB 158.46 KB No change
Total Size 733.36 KB 733.36 KB No change

Baseline commit: a252465

Generated by 🚫 dangerJS against 229d845

@github-actions github-actions bot requested a review from vladsud January 6, 2021 19:53
@github-actions github-actions bot requested a review from vladsud January 6, 2021 20:46
@github-actions github-actions bot requested a review from vladsud January 6, 2021 20:48
@GaryWilber GaryWilber merged commit 03889e4 into microsoft:main Jan 6, 2021
@GaryWilber GaryWilber deleted the improve_out_of_order branch January 6, 2021 21:05
GaryWilber added a commit to GaryWilber/FluidFramework that referenced this pull request Jan 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants