Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Waterline's sails-mongo adapter fails to convert query input to ObjectId when applying certain operators #6979

Open
Rua-Yuki opened this issue Apr 22, 2020 · 2 comments · May be fixed by balderdashy/sails-mongo#484
Labels
has pr There is an open pull request (in this repo or elsewhere) related to this issue. mongo Issue only occurs when using MongoDB orm Related to models, datastores, orm config, Waterline, sails-hook-orm, etc.

Comments

@Rua-Yuki
Copy link

Node version: v8.17.0
Sails version (sails): 1.2.3
ORM hook version (sails-hook-orm): 2.1.1
DB adapter & version: sails-mongo@1.1.0

No additional top-level dependencies present. Confirmed with merely sails, sails-hook-orm and sails-mongo.


There appears to be an issue where the sails-mongo Waterline adapter fails to transform input query filter values to Mongo ObjectId instances where appropriate, instead leaving such values as simple strings.

This occurs in cases where the target field appears to be an appropriate candidate (a primary key) for normalisation to ObjectId and only affects the following operators:

  • <
  • <=
  • >
  • >=

For instance, the following Waterline query:

Post.find({
  id: {
    '>': '5e9a49923a48025cccdaee43',
  },
});

Will ultimately issue this to MongoDB:

{
  _id: {
    '$gt': '5e9a49923a48025cccdaee43'
  }
}

Rather than the expected query of:

{
  _id: {
    '$gt': ObjectId('5e9a49923a48025cccdaee43')
  }
}

This results in an inability to perform a number of useful queries against PK fields (such as paging on document IDs) and feels inconsistent with the behaviour of the other Waterline query operators supported by sails-mongo, as well as other adapters.

I've built a mostly minimal test case at Rua-Yuki/waterline-mongo-objectid-comparison-funk, which also contains more information about this issue.

A quick fix exists in the fpm-git/sails-mongo fork.

@sailsbot
Copy link

@Rua-Yuki Thanks for posting! We'll take a look as soon as possible.

In the mean time, there are a few ways you can help speed things along:

  • look for a workaround. (Even if it's just temporary, sharing your solution can save someone else a lot of time and effort.)
  • tell us why this issue is important to you and your team. What are you trying to accomplish? (Submissions with a little bit of human context tend to be easier to understand and faster to resolve.)
  • make sure you've provided clear instructions on how to reproduce the bug from a clean install.
  • double-check that you've provided all of the requested version and dependency information. (Some of this info might seem irrelevant at first, like which database adapter you're using, but we ask that you include it anyway. Oftentimes an issue is caused by a confluence of unexpected factors, and it can save everybody a ton of time to know all the details up front.)
  • read the code of conduct.
  • if appropriate, ask your business to sponsor your issue. (Open source is our passion, and our core maintainers volunteer many of their nights and weekends working on Sails. But you only get so many nights and weekends in life, and stuff gets done a lot faster when you can work on it during normal daylight hours.)
  • let us know if you are using a 3rd party plugin; whether that's a database adapter, a non-standard view engine, or any other dependency maintained by someone other than our core team. (Besides the name of the 3rd party package, it helps to include the exact version you're using. If you're unsure, check out this list of all the core packages we maintain.)

Please remember: never post in a public forum if you believe you've found a genuine security vulnerability. Instead, disclose it responsibly.

For help with questions about Sails, click here.

@Rua-Yuki
Copy link
Author

(The following is mostly snipped from Rua-Yuki/waterline-mongo-objectid-comparison-funk, for posterity's sake)

Regarding one use case which this issue prevents, we are left unable to page based on document IDs or use said IDs as tie-breakers when paging on fields that may not be unique.

Given the following model:

/**
 * @file Post.js
 */

module.exports = {
  attributes: {
    content: {
      type: 'string',
      required: true,
      maxLength: 280,
      minLength: 1,
    },
  },
};

And a dataset like so:

_id content createdAt updatedAt
ObjectId("5e9a49923a48025cccdaee43") Hello World! 1587169682656 1587169682656
ObjectId("5e9a49923a48025cccdaee44") Testing 123. 1587169682656 1587169682656
ObjectId("5e9a49923a48025cccdaee45") Testing 456. 1587169682656 1587169682656
ObjectId("5e9a49923a48025cccdaee46") Testing 789. 1587169682656 1587169682656

The following query will not return results as expected:

// Try and find the next item after our input.
Post.find({
  id: {
    '>': '5e9a49923a48025cccdaee43',
  },
}).sort('id ASC').limit(1).then((res) => {
  // Logs "Found []"
  sails.log.debug('Found', res);
});

The expected result of this query is to return the item directly after our input ID, if any:

[{
  id: '5e9a49923a48025cccdaee44',
  createdAt: 1587169682656,
  updatedAt: 1587169682656,
  content: 'Testing 123.'
}]

We instead receive only an empty array, regardless of whether a match exists or not:

[]

@johnabrams7 johnabrams7 added mongo Issue only occurs when using MongoDB orm Related to models, datastores, orm config, Waterline, sails-hook-orm, etc. has pr There is an open pull request (in this repo or elsewhere) related to this issue. labels Jul 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
has pr There is an open pull request (in this repo or elsewhere) related to this issue. mongo Issue only occurs when using MongoDB orm Related to models, datastores, orm config, Waterline, sails-hook-orm, etc.
Development

Successfully merging a pull request may close this issue.

3 participants