Introduce a simpler way to "populate unless populated"

[iliakan]

🚀 Feature Proposal

Is there a method to populate only if the value isn't already there? If not, maybe it's worth adding?

Motivation

Currently, a populate call (both on a Model and an instance) queries the database, even if the value is already populated.

However, it's quite common that objects come from different code paths, and they may have paths populated already. So, performance-wise, we should not do it again.

For a single instance, we can write

if (!user.populated('friends')) {
  await user.populate('friends');
}

...But it's cumbersome to repeat this often.

Also, for an array of users we need to check each item, which makes even longer code compared to a single call, e.g. User.populateMissing(users, 'friends'), which could work just as User.populate(single query to fetch all users), but omit already-populated paths.

Additional thoughts

To say the truth, after many years of development with Mongoose, my experience is that the current behaviour (always querying the database) is much less needed than the lazy one (query only if populated path doesn't exist).

[satyamkale27]

@vkarpov15 I want to work on this feature please assign me.

[vkarpov15]

I agree with the point that if (!user.populated()) is cumbersome. A couple of potential APIs we can support:

1. user.populate('friends', { forceRepopulate: false }) to opt out of automatic repopulating
2. user.populateIfNecessary('friends')

I think I prefer option (1), and likely switch forceRepopulate to be false by default in Mongoose 9. I agree that user.populate('friends') likely shouldn't populate friends if it is already populated - you can use User.populate(user, 'friends') or user.populate('friends', { forceRepopulate: true }).

What do you think @iliakan ?

[iliakan]

@vkarpov15 The option flag looks like a good solution.

The good thing about it is that a global default setting can be added as well.

So people don't need to wait till Mongoose 9, they can set e.g. mongoose.set('forceRepopulate', false) and have this behavior enabled even in Mongoose 8. The next Mongoose can just change this default.

Will the collection variant look like this User.populate(user, 'friends', { forceRepopulate: false })?

[vkarpov15]

I was thinking we would just support forceRepopulate option on user.populate() (document variant) rather than User.populate() (model variant). User.populate() seems a bit more explicit. What do you think?

Agree 💯 on the global option.

[iliakan]

I believe, both Model and instance methods deserve to have the "proper" populate mechanics. Both use cases are equally important.

Also, both instance and Model populate should work in the same way.

[vkarpov15]

@iliakan what do you think of #15044?

[iliakan]

@vkarpov15 looks good to me, I'm not familiar with Mongoose internal population mechanics, but the tests seem ok.

Thank you.