Under certain circumstances undefined values are recorded in database as null values

[jlchereau]

To reproduce (with mongoose 3.8.33 and 4.07 + mongodb 3.0.4 and nodejs 0.12.7 on windows 8.1):

var mongoose = require('mongoose');

mongoose.connect("mongodb://localhost/TestDB", { server: { auto_reconnect: true, poolSize: 10 } });

var dummySchema = new mongoose.Schema({
    alpha             : { type: String },
    more              : {
        beta              : { type: String },
        gamma             : { type: Number },
        delta             : { type: String }
    }
});
mongoose.model('Dummy', dummySchema);

var dummy = new mongoose.models.Dummy({ alpha: 'blabla', more: { beta: 'blabla' } });
dummy.save(function(error, saved) {
    saved.alpha = 'blabla2';
    saved['more'] = {
        beta              : 'blabla2',
        gamma             : undefined,
        delta             : 'blabla2'
    };
    saved.save(function(error, updated) {
        console.log('done');
    });
});

As you can see below gamma is null instead of being discarded

[jlchereau]

Notes:

• after executing dummy.save (insert), the object is correct (undefined gamma property not recorded) in database but after executing saved.save (update), the value of gamma in database is null. So the defect appears in an update, not in an insert.
• the problem seems to be in mongodb-core : until binary serialization of the query, the value of gamma remains undefined.

[vkarpov15]

Thanks for the detailed report, will investigate within the next couple days.

[vkarpov15]

This is a tricky issue - our bson serializer treats undefined as null, so you will never save something with bson type undefined to mongodb. That's one issue.

The other subtlety is that

    saved['more'] = {
        beta              : 'blabla2',
        gamma             : undefined,
        delta             : 'blabla2'
    }

is not the same as

    saved['more'] = {
        beta              : 'blabla2',
        delta             : 'blabla2'
    }

These are two different objects - the first object has 3 keys, the second has 2. In this case, I think mongoose is doing the right thing by trying to save gamma: undefined in the database, which is technically valid.

[jlchereau]

There is a matter of consistency:

The BSON serializer doe not treat gamma = undefined as gamma = null with:

var dummySchema = new mongoose.Schema({
    alpha            : { type: String },
    beta              : { type: String },
    gamma         : { type: Number },
    delta             : { type: String }
});

or with

var dummySchema = new mongoose.Schema({
    alpha             : { type: String },
    more              : {
        beta              : { type: String },
        gamma             : { type: Number }
        // removed delta
    }
});

and in the case reported, it only occurs with updates, not with inserts.

Generally, mongoose does not record undefined fields in database, and I think it is the right thing to do. Changing undefined values for null values is definitely wrong.

[bonesoul]

so any fix on this?

[ghost]

@bonesoul it was fixed with a driver update almost 3 years ago in c6c0b3c. Are you running into this now? If so, can you please create a new issue with a code sample that demonstrates the problem?

If I run the repro script from this issue on mongoose 5.1.7:

3169.js

#!/usr/bin/env node
'use strict';

var mongoose = require('mongoose');

mongoose.connect('mongodb://localhost/gh-3169');

var dummySchema = new mongoose.Schema({
  alpha: { type: String },
  more: {
    beta: { type: String },
    gamma: { type: Number },
    delta: { type: String }
  }
});
mongoose.model('Dummy', dummySchema);

var dummy = new mongoose.models.Dummy({ alpha: 'blabla', more: { beta: 'blabla' } });

dummy.save(function (error, saved) {
  saved.alpha = 'blabla2';
  saved['more'] = {
    beta: 'blabla2',
    gamma: undefined,
    delta: 'blabla2'
  };
  saved.save(function (error, updated) {
    console.log(updated);
    return mongoose.connection.close();
  });
});

I get:

Output:

issues: ./3169.js
{ more: { beta: 'blabla2', delta: 'blabla2' },
  _id: 5b3612d16c128213694f8a4b,
  alpha: 'blabla2',
  __v: 0 }
issues:

It works correctly.