Need to manage MongoDB indexes yourself

[davidaik]

From the Parse Server guide-

When using MongoDB with your Parse app, there are some differences with the hosted Parse database:

You need to manage your indexes yourself. Hosted Parse automatically adds indexes based on the incoming query stream.
You need to size up your database as your data grows.

I don't have any experience with MongoDB, so I would like to know if the requirement for manual management of the db will be hard work or could the Parse Server codes be updated to manage the MongoDB automatically? I'm wondering if this is a thing that can be smoothed out.

[natanrolnik]

+1

[wagnercsfilho]

+1

[kingmatusevich]

+1

[talkaboutdesign]

+1 Have zero experience with this. What does this actually mean?

[flessard]

+1

[rebelchris]

+1

[davidaik]

@talkaboutdesign I don't have much experience with databases but iirc indices are very important for the database performance. If the index management is done automatically by properly implemented code, we'll have less server-side stuff to worry about, which is the reason why Parse is so popular in the first place.

[mauran]

+1

[skinp]

Automatic indexing is a very hard problem. At Parse, it required hundreds (if not thousands) of engineering hours to build and a sizable dedicated infrastructure... and we still don't believe it's anywhere near perfect. In all cases, the person querying the DB (the app developer in this case) is always the best person to know which indexes are going to be needed and optimal for a particular query.

The good news is, for anyone migrating their existing app to a different Mongo database using the migration tool, all the existing indexes are also migrated. You will still have to add new ones or remove unused ones yourself though.

That said, I guess this could be added to parse-server eventually. It was most likely too complex for this version. As always, PR welcome...

[davidaik]

@skinp hmm I guess I'm gonna have to learn MongoDB and Node.js. With the shutting down of Parse, if I am to continue using the Parse client sdk, it looks like I'm actually gonna have to participate in the development of this project.

[kingmatusevich]

@davidaik We're all in the same boat here, but we have an opportunity to make Parse something even better than it ever was.

[davidaik]

@kingmatusevich I learnt of the sad news this morning and have been worndering since about whether to keep using the Parse SDK or to move to another service. But now I've decided to stick with it. I'm slowly starting to see Parse Server becoming big and awesome like Wordpress, or at least I hope so.

[rawrmaan]

I'd highly recommend deploying your MongoDB instance on http://www.mongolab.com/ if you care about this feature. They have a feature that automatically suggests indexes to add to your DB based on usage patterns and you can add an index with one click.

[jamesyu]

Yep, if you use Parse Server, you will need to understand MongoDB, and also specifically indexing. Doing this automatically at Parse for each app was a hard problem because we had to solve the general use case, but, managing it for your own app with moderate traffic isn't too difficult (and as mentioned, folks like MongoLab actually suggest indexes for you).

It is no different than using Rails with MySQL. In that case, you would have to manage your indexes in MySQL. You should understand what queries your app is making, and add the proper indexes to ensure those queries are efficient.

[davidaik]

@rawrmaan @jamesyu I was thinking that using MongoLab has the possibility of turning out to be the same as with Parse. What if they decide to shut down their service at some point? I know that in such a case, my data can be migrated easily. But then I think that having complete control over the database will be better now that we have to host the server code ourselves anyway.

[jamesyu]

As you mentioned, migrating from a DBaaS like MongoLab will be much easier than something like Parse, so I wouldn't worry about that so much. However, you can also just host your own MongoDB on your own box or deploy to aws https://s3.amazonaws.com/quickstart-reference/mongodb/latest/doc/MongoDB_on_the_AWS_Cloud.pdf (yes, the AWS instructions are much more daunting, which is why I would recommend a DBaaS at least as a first experience).

[kingmatusevich]

��@davidaik right there with you, except I found out last night.

[davidaik]

@jamesyu thanks for the suggestions. I will likely be using MongoLab, but I'll also be looking into hosting my own MongoDB if my use-case doesn't make it too hard.