Found a pointer column not in the schema, dropping it.

[nickkuh]

Any idea what might cause this kind of Parse Server debug message?

transform.js Found a pointer column not in the schema, dropping it. CircleProfile circle
transform.js Found a pointer column not in the schema, dropping it. CircleProfile profile

CircleProfile has both circle and profile pointers so I don't know why this wouldn't be found in the scheme.

The issue also only happens periodically. The query that triggers it looks like this:

        var query = new Parse.Query(CircleProfile);
        query.include("circle");
        query.include("circle.activeChallenge");
        query.include("circle.activeChallenge.blocks");
        query.include("circle.scheduledChallenges");
        query.include("circle.scheduledChallenges.template");
        return query.get(requestState.circleProfile.id)

Any ideas?

[gfosco]

Check the _Schema collection for your CircleProfile class. Is there a _p_circle entry?

[nickkuh]

It looks like this:

{
    "_id": "CircleProfile",
    "_metadata": {
        "class_permissions": {
            "get": {
                "*": true
            },
            "find": {
                "*": true
            },
            "update": {
                "*": true
            },
            "create": {
                "*": true
            },
            "delete": {
                "*": true
            },
            "addField": {
                "*": true
            },
            "readUserFields": [],
            "writeUserFields": []
        }
    },
    "circle": "*Circle",
    "profile": "*Profile",
    "updatedAt": "string",
    "createdAt": "string",
    "objectId": "string"
}

The original app was on parse.com - I haven't made any changes to the schema of this class since migration.

Nick

[gfosco]

That appears to be incorrect. Update those keys to _p_circle and _p_profile

[nickkuh]

Ok... as long as that's not going to break my production app.

I don't really understand what's going on here as in that query it appears that sometimes the circle and profile values are populated and sometimes they're not even when values do exist - and it appears to be down to this 'Found a pointer column not in the schema, dropping it. CircleProfile circle' issue.

[nickkuh]

I don't think that editing the schema is the way to go. I've just tried adding a column with the latest version of Parse dashboard.

I added a pointer to Profile for another test class with the key 'testy'. In the resulting mongodb scheme it's listed as:

"testy": "*Profile"

Any more ideas?

Nick

[nickkuh]

I haven't seen this problem occurring before Parse Server 2.2.7 - would one of the changes in 2.2.7 have caused it?

[gfosco]

If the dashboard is not making a _p_ field, then it does sound like a recent change is putting things out of sync... It's clear from MongoTransform.js that _p_ is still expected on pointer columns, as that's the way Parse has always stored pointers in the schema. I would still encourage you to change the circle/profile keys to add _p_ and test that.

[nickkuh]

@gfosco so, I've been running some local tests with the scheme based on your suggestion:

CircleProfile schema before:

{
    "_id" : "CircleProfile",
    "_metadata" : {
        "class_permissions" : {
            "get" : {
                "*" : true
            },
            "find" : {
                "*" : true
            },
            "update" : {
                "*" : true
            },
            "create" : {
                "*" : true
            },
            "delete" : {
                "*" : true
            },
            "addField" : {
                "*" : true
            },
            "readUserFields" : [],
            "writeUserFields" : []
        }
    },
    "circle" : "*Circle",
    "profile" : "*Profile",
    "updatedAt" : "string",
    "createdAt" : "string",
    "objectId" : "string"
}

And here's an example of a CircleProfile document in mongodb:

{
    "_id" : "02he4MDYsD",
    "_p_circle" : "Circle$bERsT2SBO2",
    "_p_profile" : "Profile$RUBAbjznsu",
    "_updated_at" : ISODate("2016-05-01T07:03:40.356Z"),
    "_created_at" : ISODate("2016-05-01T07:01:26.126Z")
}

So you can see that p does exist for the pointers in the actual instances of CircleProfile.

You suggested that I change the scheme for CircleProfile. So I did so as follows:

{
    "_id" : "CircleProfile",
    "_metadata" : {
        "class_permissions" : {
            "get" : {
                "*" : true
            },
            "find" : {
                "*" : true
            },
            "update" : {
                "*" : true
            },
            "create" : {
                "*" : true
            },
            "delete" : {
                "*" : true
            },
            "addField" : {
                "*" : true
            },
            "readUserFields" : [],
            "writeUserFields" : []
        }
    },
    "_p_circle" : "*Circle",
    "_p_profile" : "*Profile",
    "updatedAt" : "string",
    "createdAt" : "string",
    "objectId" : "string"
}

Is that what you meant? What happens is my app just starts failing with:

info: transform.js Found a pointer column not in the schema, dropping it. CircleProfile profile
info: transform.js Found a pointer column not in the schema, dropping it. CircleProfile circle

Could you confirm if I've followed your instructions correctly?

For now I've reverted my app to use 2.2.6 and the issue is currently no longer occurring.

Are there other scenarios where the Parse server app might fail to load the scheme and this kind of issue would occur? I'm keen to use 2.2.7 as it fixes a number of other issues but it seems unstable at the moment.

Thanks for your help

Nick

[steven-supersolid]

On a Parse.com app our _Schema looks like the original schema posted, e.g. a schema with a pointer looks like:

{
    "_id": "CircleProfile",
...
    "circle": "*Circle"
}

parse-server should also follow this convention.

[nickkuh]

@gfosco - any idea if this issue got resolved in 2.2.8?

[drew-gross]

If this is in a migrated app that is old, then the issue could be caused by legacy data formats still in your database. Are you seeing an actual bug in your app, or only the warning?

[nickkuh]

If this is in a migrated app that is old

Yes, it's a migrated app.

Are you seeing an actual bug in your app, or only the warning?

I'm not seeing the bug now as I've reverted my Parse Server to 2.2.6

It started happening in Parse Server 2.2.7

then the issue could be caused by legacy data formats still in your database

Ok - is there some documentation on what the format should be now? I'd prefer to use the latest Parse Server (but only if this issue's resolved).

[drew-gross]

The format looks correct. I'm not sure what would cause this, but without there being an actual bug I don't think we'll prioritize this. You can run multiple Parse Servers with different versions connected to the same database to search for an actual bug, or use a snapshot of your database.

[steven-supersolid]

@nickkuh could you test with 2.2.9 or even better create a test in a PR? Would need to be able to reproduce 100% in the test however.

[iForests]

I've migrated to AWS and running Parse-Server 2.2.10, but the server's health went to "severe" and every requests failed with error code 499. I saw about 5000 lines in a short time of

info: transform.js Found a pointer column not in the schema, dropping it. Inbox user
info: transform.js Found a pointer column not in the schema, dropping it. Inbox post

in nodejs.log at that moment.

Is there anything I can do to solve this issue? Please let me know if I can provide any useful information. Thanks.

[drew-gross]

If you can consistently reproduce the error that would be very helpful. If you can provide a database snapshot, or a series of SDK or cURL calls that create data that exhibits the issue, then an SDK call or cURL that causes that line to be logged, then we should be able to debug the issue.

[iForests]

I can't reproduce it consistently. The server went to "severe" about 3 ~ 4 times a day, but I'm not sure what kinds of queries cause the problem. I'm not familiar with AWS, Node.js and Parse-Server now, so please tell me if I can add something into my code, therefore I can get useful information when the server go to "severe" next time. Thanks.

[steven-supersolid]

We're getting this error too (Still on 2.2.7). It happens very infrequently (~1 in 1 million queries). I'll try and add more debugging to the error message to try and determine why but may not have an answer until next week.

[nickkuh]

Yep - it's not an easy one to reproduce as it doesn't seem to occur when the server first starts but only after it's been online for a while.

It doesn't happen on 2.2.6 so it must have been introduced in 2.2.7

[steven-supersolid]

I added the following debug to MongoTransform.js

            log.info('transform.js', 'typeof schema', typeof schema);
            if (schema) {
              log.info('transform.js', 'typeof schema.getExpectedType', typeof schema.getExpectedType);
              log.info('transform.js', 'schema.data', schema.data);
            }

When the bug occurs, schema.data is not set:

May 19 13:27:43 myApp app/web.2: info: transform.js Found a pointer column not in the schema, dropping it. Class pointerName
May 19 13:27:43 myApp app/web.2: info: transform.js typeof schema object 
May 19 13:27:43 myApp app/web.2: info: transform.js typeof schema.getExpectedType function 
May 19 13:27:43 myApp app/web.2: info: transform.js schema.data

There may be a mistake with the logging as would expect schema.data to at least be an empty object whereas here it seems undefined.

[drew-gross]

yes that may be an issue with the schema being not loaded when this function is called. The stuff I'm working on now should also fix this bug.

[nickkuh]

@drew-gross is this issue resolved in 2.2.11? Thanks!

[drew-gross]

It should be, yes.

[ilanco]

I am running parse server 2.3.1 and seeing this error.

info: transform.js Found a pointer column not in the schema, dropping it. material collection

The material schema is defined as :

{
  "className": "material",
  "fields": {
    "objectId": {
      "type": "String"
    },
    "createdAt": {
      "type": "Date"
    },
    "updatedAt": {
      "type": "Date"
    },
    "ACL": {
      "type": "ACL"
    },
    "author": {
      "type": "Pointer",
      "targetClass": "authors"
    },
    "categories": {
      "type": "Array"
    },
    "content": {
      "type": "String"
    },
    "description": {
      "type": "String"
    },
    "downloads": {
      "type": "Number"
    },
    "featuredImage": {
      "type": "File"
    },
    "mediaType": {
      "type": "String"
    },
    "region": {
      "type": "String"
    },
    "tags": {
      "type": "Array"
    },
    "thumbnail_small": {
      "type": "File"
    },
    "title": {
      "type": "String"
    },
    "uploads": {
      "type": "Object"
    },
    "views": {
      "type": "Number"
    },
    "draft": {
      "type": "Boolean"
    },
    "orderDate": {
      "type": "Date"
    },
    "collections": {
      "type": "Array"
    },
    "hidePdf": {
      "type": "Boolean"
    }
  }
}