Failed to copy newResults into _published!

[deejross]

Hello, I saw #2033, and while I get the same error, I think this bug report differs from the other one.

I've been getting this same issue for quite a while and simply disabled oplog tailing for my entire app. I finally got time to figure out which publication was causing this problem (which was preventing users from logging in and logging out):

Meteor.publish('OpenIncidents', function(organizationId) {
    organizationId = Auth.getOrganizationId(this.userId);
    return Incidents.find({end: null, $or: [{organizationId: organizationId}, {'sharedWith.organizationId': organizationId}]});
});

The code first confirms that the organizationId passed is the correct one for the given user (to prevent users from subscribing to an Organization they don't belong to). This line of code is in several of my publications and does not cause any problems.

The second line simply returns Incident objects that are still open (end: null) and belong to the given Organization, or an Organization that has granted permission for them to see it.

There is nearly an identical publication for recently closed Incidents that works just fine:

Meteor.publish('RecentIncidents', function(organizationId) {
    organizationId = Auth.getOrganizationId(this.userId);
    return Incidents.find({end: {$exists: true}, $or: [{organizationId: organizationId}, {'sharedWith.organizationId': organizationId}]}, {sort: {created: -1}, limit: 5});
});

The only difference I see here is I'm looking for closed incidents (end != null) and I'm sorting and limiting. I added sorting and limiting to OpenIncidents with no change. If I replace end: null with end: {$type: 10}, which checks for a null type instead of null value, it no longer errors out, but the query does not return any rows.

I have to add _disableOplog: true for this to work correctly. This is the only publication where I'm specifically checking if a field is null (and it needs to be null, not undefined due to other logic). So I think the oplog driver doesn't like directly checking for null.

[deejross]

I should also mention that I'm running the latest Meteor version as of today: 0.8.1.3

[glasser]

Please read our instructions on submitting bug reports. A code snippet is not a reproduction. https://github.com/meteor/meteor/wiki/Contributing-to-Meteor#reporting-a-bug-in-meteor

[deejross]

There is plenty of detail, and it's very simple to reproduce simply by asking a publication to to check for null in a query. I would understand if this was something that was a complex bug or one that was difficult to reproduce, but closing a bug because I didn't take the time to make a full scale app proof of concept for something that takes less than a minute to check on your own seems very unfriendly and a deterrent to reporting similar easily reproducible bugs. The time I have to work on this project is minimal as it, so this kind of response is really frustrating.

[estark37]

@deejross We know that these replies can be frustrating, but we've found that it's really the best way to deal with the volume of bug reports that we get. For what it's worth, I'm not able to reproduce this by simply publishing a cursor that queries for null:

Foo = new Meteor.Collection("foo");                              

if (Meteor.isClient) {                                                                                           
  Meteor.subscribe("test");                                      
}                                                                

if (Meteor.isServer) {                                           
  Meteor.startup(function () {                                   
    Meteor.publish("test", function () {                         
      return Foo.find({foo: null});                              
    });                                                          
  });                                                            
}                                                                

There are two documents in the collection: { foo: null } and { foo: 'bar' }. On the client, when I run Foo.find({}).fetch(), I see a single document in the local collection, with foo: null, as expected.

We always ask for cloneable reproductions because there are often non-obvious factors that are crucial for reproducing a bug, such as the list of core or Atmosphere packages that you're using in your app. Hope that helps explain it somewhat.

[deejross]

Emily, thank you for trying it on your end, and thank you for the quick response. I will try to carve out some time to make a reproducible app for demonstrating.

[Sewdn]

I've been having this issue this evening.
My suggestion: check your data. The issue was triggered by querying documents with a faulty value in one of the fields used in the query's conditions.

My publication's query returns:
Events.find({'room._id': roomId, endedAt: {$exists: false}, startingAt: {$lt: tomorrow.getTime()}});

My collection had documents with a NaN value in the startingAt field (matching the 2 other criteria).

Hope this helps. Goog luck!

[looshi]

@deejross , I ran into the same issue. I noticed both in your issue, and issue 2033, the mongo $or operator were used in a query against an array, e.g.
$or: [{array: value} ...
I have replicated the issue here :
https://github.com/davluzius/meteor-issue-2033.git

you might be able to check for null against your array search terms to get around the exception. I'm curious to know if you've resolved it on your end or found similar workaround, cause I had a great deal of trouble with this today.

[deejross]

@davluzius Unfortunately, I have not had time to do more in-depth troubleshooting. My only workaround is to disable oplog for the publication causing the issue. But thank you for the information, I'm glad you were able to replicate it.

[glasser]

So, this error occurs if Meteor's minimongo implementation disagrees with Mongo's implementation of a query. In this case, whether or not the selector {'a.b': null} matches a document with a field a which is an empty array.

Around the beginning of the year, we rewrote the minimongo selector compiler for better compatibility with Mongo. We noticed that mongod had rewritten their selector compiler in Mongo 2.6, and decided that we should target the future (more consistent) implementation rather than the past one.

This particular query is one that changed from 2.4 to 2.6. In 2.4, it matches; in 2.6 it does not.

Unfortunately, we are not yet recommending 2.6 for use with Meteor/oplog tailing (mostly a matter of prioritizing other work; hopefully we'll get around to this soon). So in the meantime, you get this error.

A glib answer is: Don't write queries like that, since they are subtle enough that Mongo changed their semantics between versions without even making a big deal about it (eg, this change is certainly not mentioned in their release notes).

We should at least improve the error, though.

[glasser]

Improving the error a bit.