Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(document): add fast path for applying non-nested virtuals to JSON #14543

Merged
merged 1 commit into from
Apr 29, 2024

Conversation

vkarpov15
Copy link
Collaborator

Re: #14394

Summary

#14394 points out that toObject() gets slow when there's a lot of virtuals. Part of that is we always take the "slow path" when applying virtuals by assuming the virtual is nested.

Consider the following script, 200 documents with 800 virtuals each. The below example uses replica set, but not strictly necessary for this example.

'use strict';

const mongoose = require('mongoose');

void async function main() {
  await mongoose.connect('mongodb://127.0.0.1:27017,127.0.0.1:27018/mongoose_test');

  const ChildSchema = new mongoose.Schema({ name: String, parentId: 'ObjectId' });
  const ChildModel = mongoose.model('Child', ChildSchema);

  const ParentSchema = new mongoose.Schema({
    name: String
  });
  ParentSchema.virtual('child1', { ref: 'Child', localField: '_id', foreignField: 'parentId' });
  ParentSchema.virtual('child2', { ref: 'Child', localField: '_id', foreignField: 'parentId' });
  ParentSchema.virtual('child3', { ref: 'Child', localField: '_id', foreignField: 'parentId' });
  ParentSchema.virtual('child4', { ref: 'Child', localField: '_id', foreignField: 'parentId' });
  const ParentModel = mongoose.model('Parent', ParentSchema);
  /*await ParentModel.deleteMany({});
  await ChildModel.deleteMany({});

  const parents = [];
  for (let i = 0; i < 200; ++i) {
    const parent = await ParentModel.create({ name: 'test parent ' + i });
    const children = [];
    console.log(`${i} / 200`);
    for (let j = 0; j < 200; ++j) {
      children.push({ name: 'test child ' + i + '_' + j, parentId: parent._id });
    }
    await ChildModel.create(children);
    parents.push(parent);
  }

  await new Promise(resolve => setTimeout(resolve, 3000));*/

    const docs = await ParentModel.find().populate(['child1', 'child2', 'child3', 'child4']);
  console.log('Num docs', docs.length, docs[0]?.child1?.length);
  for (const doc of docs) {
    doc.name = 'test parent';
  }
  const start = Date.now();
  docs.forEach(doc => doc.toObject({ virtuals: true }));
  console.log('toObject ms:', Date.now() - start);

  console.log('Done');
  process.exit(0);
}();

Without this change:

$ node gh-14394.js 
Num docs 200 200
BulkSave and toObject ms: 2380
Done

With this change:

$ node gh-14394.js 
Num docs 200 200
BulkSave and toObject ms: 1215
Done

Nearly 2x faster. We can take a look and see if we can do better, but this PR seems like low hanging fruit.

Examples

@vkarpov15 vkarpov15 added this to the 8.3.3 milestone Apr 27, 2024
@vkarpov15 vkarpov15 merged commit 286c7d1 into master Apr 29, 2024
34 checks passed
@hasezoey hasezoey deleted the vkarpov15/gh-14394 branch April 30, 2024 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants