-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(NODE-6136): parse cursor responses on demand #4112
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nbbeeken
force-pushed
the
NODE-6136-cursor-response
branch
2 times, most recently
from
May 16, 2024 14:53
df69639
to
e6de40a
Compare
nbbeeken
force-pushed
the
NODE-6136-cursor-response
branch
2 times, most recently
from
June 4, 2024 20:28
31efa9c
to
a37ec31
Compare
nbbeeken
force-pushed
the
NODE-6136-cursor-response
branch
from
June 4, 2024 20:30
a37ec31
to
be871b8
Compare
baileympearson
added
the
Primary Review
In Review with primary reviewer, not yet ready for team's eyes
label
Jun 5, 2024
baileympearson
requested changes
Jun 6, 2024
This was referenced Oct 5, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
What is changing?
ok === 0
)makeWriteConcernResultObject
would delete properties when ok was set to 0 which is never the case for a writeConcernError because the "command succeeded" the "write concern/requirements failed". The delete logic would not run and the unit tests that asserted the properties did not exist only passed because the mock data fed in was inaccurate.writeConcernError
property.Is there new documentation needed for these changes?
No.
What is the motivation for this change?
Performance and enhancing internal capabilities for discrete BSON parsing.
Release Highlight
Cursor responses are now parsed lazily 🦥
MongoDB cursors (find, aggregate, etc.) operate on batches of documents equal to
batchSize
. Each time the driver runs out of documents for the current batch it gets more (getMore
) and returns each document one at a time through APIs likecursor.next()
orfor await (const doc of cursor)
.Prior to this change, the Node.js driver was designed in such a way that the entire BSON response was decoded after it was received. Parsing BSON, just like parsing JSON, is a synchronous blocking operation. This means that throughout a cursor's lifetime invocations of
.next()
that need to fetch a new batch hold up on parsingbatchSize
(default 1000) documents before returning to the user.In an effort to provide more responsiveness, the driver now decodes BSON "on demand". By operating on the layers of data returned by the server, the driver now receives a batch, and only obtains metadata like size, and if there are more documents to iterate after this batch. After that, each document is parsed out of the BSON as the cursor is iterated.
A perfect example of where this comes in handy is our beloved
mongosh
! 💚That
Type "it" for more
message would now print after parsing only the documents displayed rather than after the entire batch is parsed.Double check the following
npm run check:lint
scripttype(NODE-xxxx)[!]: description
feat(NODE-1234)!: rewriting everything in coffeescript