'Cannot do exclusion on field in inclusion projection' error when field1 excluded in schema & query.select('field2 -_id')

[lorand-horvath]

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the bug has not already been reported

Mongoose version

6.7.2

Node.js version

18.x

MongoDB server version

5.x

Typescript version (if applicable)

No response

Description

I would like to report a long standing bug regarding simultaneous field exclusion in schema and select() query, with -_id not specified as first field.

Steps to Reproduce

Suppose we have field1 excluded from selection/projection in schema:

const schema = new mongoose.Schema({
    field1: {
        type: String,
        select: false
    },
    field2: String
});
const User = mongoose.model('User', schema);

User.find().select('field2 -_id')
    .then(console.log)
    .catch(err => console.log(err.message));

The error returned is Cannot do exclusion on field field1 in inclusion projection.
However, if I first exclude the _id in the select() then the query executes just fine, no error in that case:

User.find().select('-_id field2')

Not sure when this regression was introduced or whether it has ever worked in the first place. @vkarpov15 could you please double check? Thanks for looking into this!

Expected Behavior

Excluding -_id from select() should work irrespective of its position in the list of fields. At the moment it only works when -_id is the first field, then the rest of the included fields.
This issue only occurs when there's another field excluded from select in the schema.

[lorand-horvath]

@vkarpov15 I've just verified and confirm to have the same problem when selecting via the find method's second argument (projection):
User.find({}, {field2: 1, _id: 0}) - throws the error Cannot do exclusion on field field1 in inclusion projection

If I specify _id: 0 as the first, it works fine:
User.find({}, {_id: 0, field2: 1})

[IslandRhythms]

I'll mark this as a bug because I'm not entirely sure, but this could be expected behavior.

A projection must be either inclusive or exclusive. In other words, you must either list the fields to include (which excludes all others), or list the fields to exclude (which implies all other fields are included). The [_id field is the only exception because MongoDB includes it by default](https://docs.mongodb.com/manual/tutorial/project-fields-from-query-results/#suppress-id-field).

Its possible that the way it works it that it sees field2 and thinks "Ok this is an inclusive projection" and then it sees the -_id and goes "wait, can't do that."
Then when you run it the other way: it sees -_id and goes "ok exclusive, but this is _id so I can ignore this" and then sees field2 and goes "ok, inclusive it is".

[lorand-horvath]

@IslandRhythms It's obviously a bug and I think it's caused when merging the schema selection/projection (fields marked with select: false) with the query projection (fields passed to .select() or as second argument to .find() ).

The merging should be done by making sure the position of -_id in either projection doesn't influence the decision whether it will finally be an exclusion or inclusion selection (-_id should be allowed in either, irrespective of its position).