Update Mongoid to 5.x release (mongodb 3.x + compatibility)

[macdiesel]

All tests passing on mongodb 3.x installed locally. We will need to run a load test on this as the Identify mapping (caching) functionality was dropped from mongoid in the 4.x version.

[smarnach]

@macdiesel Great to see the progress of the Mongo3 upgrade! We would like to run one or our Open edX instances on Mongo3 only. I understand that we'd need to include this PR to make the forums work on Mongo 3. Is this correct? Is there a timeline for reviewing and merging this PR?

[macdiesel]

@smarnach yes you would need this work to use mongodb 3.x+. My concern is that since the caching layer was removed from mongoid we could see a big hit in performance of the application. I have some time schedule to do load testing on Thursday (9/10/2015). This should give us a better handle on what the impact of the removal of identity mapping will be.

[smarnach]

@macdiesel Thanks for the prompt reply! We aren't that concerned about performance since the instance we would be using this for has a relatively small number of users, at least compared to edx.org. I'm watching the JIRA ticket for this, and will be interested in the results. Thanks for doing this!

[macdiesel]

@smarnach ticketing for the load test is here: https://openedx.atlassian.net/browse/PLAT-830

[smarnach]

@macdiesel Thanks for the link. I already added myself to the watchlist of https://openedx.atlassian.net/browse/PLAT-768, which seems to be a duplicate.

[smarnach]

This dependency results in mongoid being installed at exactly the version 4.0.0, instead of the bugfix release version 4.0.2. I can't really imagine this is the actual intention.

[smarnach]

@macdiesel I tested this PR on a local devstack with MongoDB 3.0 installed, both manually and by running the tests. All tests are passing for me as well, and I couldn't find any issues. I also had a quick glance at the code. It looks all reasonable to me, but my Ruby knowledge is rather cursory, so that's probably not worth a lot. :)

I noticed the following comment in two places:

TODO: Pull in protected_attributes gem to fix this functionality

What functionality would be affected by this? Would this be something blocking us from using this patch in a production environment?

Thanks again for doing this!

[macdiesel]

@smarnach I am still working on the load testing and should have a better idea of the release date after the next two days.

As to attr_accessible: http://guides.rubyonrails.org/v3.2.13/security.html#mass-assignment
It's a white list of fields that can be assigned when using the constructor of a model. This is something that I will finish before the final merge, thanks for the reminder, it had slipped my mind.

[mtyaka]

@macdiesel This PR upgrades the i18n dependency to 0.7.0. The enforce_available_locales configuration option default has changed from nil (meaning log warning) to true (raise exception).
That means that trying to access the comments service from an edx-platform installation in a language other than the few available by default raises an error.

It would good to set I18n.enforce_available_locales = false somewhere in app.rb to maintain backwards compatibility.

[mtyaka]

@macdiesel We found out that mongoid 4.x doesn't properly work with mongo 3. We had to upgrade to mongoid 5.0. We're working on it here: edx/cs_comments_service@bbeggs/mongodb-3.0-PLAT-730...open-craft:mongodb-3.0

We'll open a proper PR once we fix some remaining issues and update the specs.

[macdiesel]

@mtyaka Interesting, what kind of issues did you find? One of the reasons I had stopped at mongoid 4.x was because of voteable_mongo. but it seems like you may have found a mongoid 5.x compatible branch?

I'm really interested to hear about what difficulties you ran in to during your testing.

[mtyaka]

@macdiesel Mongoid 4 failed to connect to our mongolab hosted databases that were migrated to mongo 3. It failed with an authentication error. Mongolab only supports the new SCRAM SHA authentication schema which doesn't work with Mongoid 4.x (see mongoid/moped#367).

[macdiesel]

@mtyaka Done, and thank you for the help.

[macdiesel]

First pass load test results can be found here: https://openedx.atlassian.net/wiki/display/PLAT/Mongoid+5.0.0+load+test+results

[macdiesel]

@BenjiLee @feanil @e0d Please take a look, I would like to get the mongoid upgrade merged back in to master.

[BenjiLee]

@macdiesel My only concern is that our loadtests are not simulating the same type of traffic on production. If this were to roll out, is there a reasonable rollback plan?

[e0d]

@macdiesel somehow I missed the @ but I'm in favor of merging. Correct me if I'm wrong, but I don't think the rollback plan is complex. There are no alterations to persisted data, for example.

[macdiesel]

Since there are now data migrations the rollback plan for this would be to simply revert the PR and put the prior AMIs back in the load balancer.

@e0d @BenjiLee thumbs then? good to merge?

[e0d]

I'm 👍 to merge.

[BenjiLee]

👍