Document/Understand how to use new mongo read preference feature.

[acinader]

I want to be able to send some queries to the secondary. My use case is something like: if i know that that a "set of related queries" or a cloud function is just going to read and not write to the db, then I want to send those queries to the secondary.

The thinking being that an app can scale better if read traffic can go to secondaries, cause one can always add more secondaries to handle load.

In looking at the new secondary read feature in parse-server 2.5.0 introduced in #3865 I couldn't really figure out how I could use. I looked at the unit tests and could only see how the read pref was set to SECONDARY in a beforeFind hook, which is by class, not by query, so I'd be setting all queries for that class to the secondary which isn't what i think(?) I want.

It would seem to me that what I'd want to be able to do is something like:

new Parse.Query('Foo').find({ useMasterKey: true, readPreference: 'SECONDARY' }) 

So I figured I'd test that out and here are the changes I needed to make in order to be able to make a Rest Query use the secondary, see #3959

So my questions:

1. Is my premise somehow flawed as to what I want to use the secondary for?
2. is there a way to do this already with the 2.5.0 release and I'm just missing it?
3. does it seem like i'm on the right track?

If it does seem like I am on the right track, I can finish this up and add documentation as well as make any change to the js sdk to support that might be necessary?

[acinader]

@davimacedo @flovilmart feedback welcome....

[davimacedo]

@acinader please find my comments:

1. I agree with you. It is easier to use secondary reads directly from the API. My original ideas was to allow this, but I preferred to split it in two PRs because I knew it would generate some discussion. I was waiting 2.5.0 to be released before sending the next PR. The main concern about secondary reads from API is kinda conceptual. The responsibility for choosing from where the query reads (primary or secondary) should be in the backend (and not in the frontend). So secondary read from API is conceptually wrong but it is far away easier to be done. In my opinion, the upside is bigger than the downside and we should allow. Maybe we can reduce the downside impact by only allowing it using master key and/or maybe by a security flag in Parse Server initialization. It would obligate the backend to provide a cloud function for each query that the frontend would run in the secondary and it would be not so conceptual wrong.

@flovilmart any ideas?

2. It is not so easy, but it would be possible. You'd have to analyse the parameters (where, isCount, isGet, etc) and check if it is a query allowed to run in secondary or not.

3. I think you are. I will review your PR now. I have the required changes in JS SDK already done and I can also send it. I can also contribute in the docs. Finally, after this discussion here, I plan to send another PR in which it will be possible to select general rules like: all counts in secondaries, all geoqueries in secondaries, etc.

Best!

[acinader]

I closed my pr, we can work on your additions: #3963 which look good. I had a few little nits.

I don't think that general rules like: 'run counts on the secondary' will work. here's an example. let's say that you create and save an object with a particular attribute, and then you need to count the total # of those objects. If all counts are on the secondary and there's any lag, you'll get the wrong count -- for some applications that may well be unacceptable. But in other places, where you are not just adding right before counting, going to the secondary is possible and therefore should be done to help spread load.

I think the only real question in the case is how should reading from the secondary be done in the api's. should it be:

a. query.find({readPreference: 'secondary'}) or should it be
b. query.setReadPref('secondary').find()

Note that "b." above mimics exactly the mongo javascript api.

[acinader]

@davimacedo @TylerBrock I added a pr for the start for docs: parse-community/docs#449 can you take a look and add comments on the pr of what else/additional examples could help??

[davimacedo]

About the general rule the idea is to allow the user to select from which counts, geoqueries, etc will read in the parse server initialization (by default it will be primary) and this general rule can always be overriden per query using API or the trigger.

query.find({readPreference: 'secondary'}) is what I have already done.

I will take a look in the docs right now.

I've just fixed the PR with your feedback.

[TylerBrock]

I think @davimacedo mentioned this elsewhere but the thinking was to dip our toe in the water with this one and then potentially expand the scope beyond beforeFind.

EDIT: whoops, should have refreshed the page before adding my feedback, didn't see you all had been discussing already.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.