Error when trying to update primitives at the bottom of multidimensional arrays

[Spade-and-Archer]

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the bug has not already been reported

Mongoose version

7.1.0

Node.js version

17.9.1

MongoDB server version

4.4.21

Typescript version (if applicable)

5.0.4

Description

When attempting to modify a multidimensional array of primitives (e.g. [[Number]]), changing the values and attempting a save causes an error:

The positional operator did not find the match needed from the query.
MongoServerError: The positional operator did not find the match needed from the query

Steps to Reproduce

const RecentDataPointSchema = new Schema({
    date: Date,
    value: Number,
})
const  TestingMultiDimArraysModel =  mongoose.model("TestingMultiDimArrays ", new Schema( {
    intArray: [[Number]],
    strArray: [[String]],
    objArray:[[RecentDataPointSchema]]
}));
let testObj = new TestingMultiDimArraysModel({
    intArray: [[1,2], [3,4]],
    strArray: [["a", "b"], ["c", "d"]],
    objArray: [
        [
            {date: new Date(Date.now()), value: 1},
            {date: new Date(Date.now()), value: 2},
        ],
        [
            {date: new Date(Date.now()), value: 3},
            {date: new Date(Date.now()), value: 4},
        ]
    ],
})
await testObj.save();
testObj = await TestingMultiDimArraysModel.findById(testObj._id);

//works
//testObj.getChanges() returns: {"$set":{"objArray.0.0.value":3}}
testObj.objArray[0][0].value = 3;
await testObj.save();
testObj = await TestingMultiDimArraysModel.findById(testObj._id);
expect(testObj.objArray[0][0].value).toBe(3);

//works
//testObj.getChanges() returns {"$set":{"objArray.0.0":{"date":"2023-05-03T00:25:28.100Z","value":3,"_id":"6451a9f83e792f2678d175e4"}}}
//@ts-ignore
testObj.objArray[0][0] = {date: new Date(Date.now()), value: 3};
await testObj.save();
testObj = await TestingMultiDimArraysModel.findById(testObj._id);
expect(testObj.objArray[0][0].value).toBe(3);

//works
//testObj.getChanges() returns: {"$set":{"strArray.0":["j","d"]},"$inc":{"__v":1}}
//@ts-ignore
testObj.strArray[0] = ["j", "d"];
await testObj.save();
testObj = await TestingMultiDimArraysModel.findById(testObj._id);
expect(testObj.strArray[0][0]).toBe("j");

//works
//testObj.getChanges() returns: {"$set":{"intArray.0":[9,9]},"$inc":{"__v":1}}
//@ts-ignore
testObj.intArray[0] = [9, 9];
await testObj.save();
testObj = await TestingMultiDimArraysModel.findById(testObj._id);
expect(testObj.intArray[0][0]).toBe(9);

//fails
//testObj.getChanges() returns: {"$set":{"intArray.$.0":[[null,null],[null,null]]},"$inc":{"__v":1}}
//Upon trying to save, the error message below is produced:
//    The positional operator did not find the match needed from the query.
//    MongoServerError: The positional operator did not find the match needed from the query
testObj.intArray[0][0] = 9;
await testObj.save();
testObj = await TestingMultiDimArraysModel.findById(testObj._id);
expect(testObj.intArray[0][0]).toBe(9);


//fails
//testObj.getChanges() returns: {"$set":{"strArray.$.0":[[null,null],[null,null]]},"$inc":{"__v":1}}
//Upon trying to save, the error message below is produced:
//    The positional operator did not find the match needed from the query.
//    MongoServerError: The positional operator did not find the match needed from the query
testObj.strArray[0][0] = "z";
await testObj.save();
testObj = await TestingMultiDimArraysModel.findById(testObj._id);
expect(testObj.strArray[0][0]).toBe("z");

I believe the problem is something to do with how the update command is generated, it seems to loose all sense of what the array is. Looking at the getChanges() (which I have documented in the code sample) It seems like it's not even sure where in the array it's trying to set the values and it's also trying to put an entire empty shell of the original object shape into a single slot in the array.

One other note: I was having the same problem in version 5 of mongoose. I upgraded versions hoping that would fix it. I don't have as thorough documentation of what was going wrong in version 5, so I can't be confident that nothing has changed about the bug, but based on this I feel confident saying this wasn't caused by a recent change.

Expected Behavior

I expect the document to be updated properly and not error out.

potential fix

I am not an expert on this repo, but my own experimenting showed that changing the arrayPath on the integer array helped substantially. The following code works properly:

 testObj.intArray[1][arrayPathSymbol] = "intArray.1"
//@ts-ignore
testObj.intArray[0][arrayPathSymbol] = "intArray.0"
testObj.intArray[0][0] = 9;

let changes = testObj.getChanges();
await testObj.save();
testObj = await TestingMultiDimArraysModel.findById(testObj._id);
expect(testObj.intArray[0][0]).toBe(9);

For some reason, the array path is being set to intArray.$. On a normal single array the arrayPath defaults to intArray. On the 2d object array, the arrayPath defaults to objArray.0. But for some reason on a 2d integer array, it's intArray.$. I think that it may be set that way for use by other commands like populated(), but _markModified() is just presuming that the path for querying whether there are any items in the list will be the same as the command to set a single item in the list?

[Spade-and-Archer]

I believe that this is a related issue #8926 and that the linked commit 7911d92 can probably be used to solve this. Seems like the major problem is just that these changes were, for some reason, only applied to document arrays. If we apply the same changes to standard arrays, that should be all that needs doing.

Also, I'd be willing to write the pull request if someone can give me some guidance on how these properties are supposed to be used and how I should go about altering them.

[hasezoey]

original issue made in typegoose typegoose/typegoose#828

i could also reproduce this, see https://github.com/typegoose/typegoose-testing/blob/d3173d3a76fa324ba1faaf51adf91d557ef21622/src/test.ts